Commit graph

902 commits

Author SHA1 Message Date
Steven R. Loomis
6c1447bb4b feat(core,developer): simplify markers 🙀
- clean up removing multi-segment processing
2024-01-31 18:17:17 -06:00
Steven R. Loomis
32d2e6ea2c feat(core,developer): simplify markers 🙀
- maybe don't need NFD table

#10317
2024-01-31 18:15:37 -06:00
Steven R. Loomis
687e50557f feat(core): add some C++ side tests 🙀
- add more test cases from @jahorton

#10317
2024-01-31 18:14:01 -06:00
Steven R. Loomis
600fdcc2c8 feat(core): double markers once again 🙀
- C++ side passes

Fixes: #10516
2024-01-31 18:05:53 -06:00
Steven R. Loomis
2891d19d88 feat(core): double markers once again 🙀
- C++ side

Fixes: #10516
2024-01-31 18:05:53 -06:00
Steven R. Loomis
2c92cfea4b feat(core): reorder processing 🙀
- update per code review
- move quickcheck of map into add_back_markers()

For: #10516
2024-01-31 17:44:21 -06:00
Steven R. Loomis
b429dc3383 feat(core): markers and reorder 🙀
- reorders don't interact with markers, but must transit them
- add updated tests
- simplify reorder_group::apply(): the 'common prefix' discussion
was out-of-date as std::mismatch is handled by the ldml_processor
and the core context.
- add_back_markers() promoted to SPI so it can be called from reordering.

Fixes: #10516
2024-01-31 17:44:21 -06:00
Marc Durdin
9ee5777e48 chore(core): move action apis into keyman_core_api_actions.h
Fixes #10520.

The definitions for `struct km_core_action_item`, `struct
km_core_backspace_item`, `enum km_core_backspace_type`, `enum
km_core_action_type`, `km_core_state_action_items()`,
`km_core_state_queue_action_items()`, `km_core_process_queued_actions()`
are now in keyman_core_api_actions.h rather than in the primary
keyman_core_api.h, as they should not be used by engines in general.

The current users of those APIs are the interactive kmx debugger, and
the IMX integration in Keyman Engine for Windows. Both of these depend
on the old action queue model rather than the actions struct.

In a future version, we may refactor these further to make them apply
directly to the KMX processor, and provide access via an interface to
the KMX processor rather than as a general Core API. Once this change is
made, then Core itself will no longer support action queues at all, and
will expect keyboard processors to fill in an action struct (this is
already done for LDML).
2024-01-31 11:32:09 +07:00
Marc Durdin
f546c44048
Merge pull request #10390 from keymanapp/epic/core/9999-normalization
epic: Keyman Core normalization 🌱
2024-01-31 10:45:51 +11:00
Marc Durdin
4c95fef55a
Merge pull request #10533 from keymanapp/feat/core/10530-actions-deleted-context
feat(core): add `deleted_context` to `km_core_actions` struct 🌱
2024-01-30 11:46:10 +11:00
Marc Durdin
868247e9fe chore(core): address review comments 2024-01-30 07:36:52 +07:00
Steven R. Loomis
5ce6490a1f feat(developer): dev side norm 🙀
- repertoire test needed normalization also

For: #10317
2024-01-29 09:31:48 -06:00
Eberhard Beilharz
2ebca107e0
chore(core): Fix build failures 2024-01-29 12:17:27 +01:00
Eberhard Beilharz
5404c02315
chore(core): Merge remote-tracking branch 'origin/epic/core/9999-normalization' into fix/core/10100-ContextNuances
# Conflicts:
#	core/tests/unit/kmnkbd/state_context_api.cpp
2024-01-29 12:05:17 +01:00
Marc Durdin
8501f2aa6d feat(core): add deleted_context to km_core_actions struct
Fixes #10530.

Adds the deleted_context member to the km_core_actions struct, and
associated unit tests. Simplifies integration by providing the consumer
with all the data they need in order to execute the transform,
specifically around number of delete operations required, without
needing to query the target application context again. The number of
delete operations will vary according to application compliance and
selected encoding; for example a UTF-16 string may require 2
delete-backs for surrogate pairs in the text buffer for a compliant app,
whereas there will be a single delete-back key event for a non-compliant
app.

The deleted_context member should also be used for debug assertions.
2024-01-29 13:21:25 +07:00
Steven R. Loomis
580e372a7e feat(core): ldml marker normalization fix 🙀
- document per code review

For: #10516
2024-01-26 14:45:54 -06:00
Steven R. Loomis
4108ecd298 feat(core): ldml marker normalization fix 🙀
- decomp the 'glued' char before storing in map

For: #10516
2024-01-25 17:45:59 -06:00
Marc Durdin
41cc2f4d1e feat(developer): make context_get and context_length public again
Rolls back the privatisation of the km_core_context_get and
km_core_context_length APIs because the debugger uses them.
2024-01-26 06:37:09 +07:00
Marc Durdin
c5e9a8ce35
Merge branch 'refactor/developer/10214-debugger-app-context' into feat/developer/10416-10458-ldml-debugger-markers 2024-01-26 09:50:06 +11:00
Marc Durdin
4906545e31
Merge branch 'epic/core/9999-normalization' into refactor/core/10431-remove-unused-core-context-apis 2024-01-24 12:14:11 +11:00
Marc Durdin
fa39ae0b22 chore(core): fixup types on unit test 2024-01-23 13:12:47 +07:00
Marc Durdin
39b64be67a refactor(developer): use km_core_state_get_actions for LDML keyboard debugger
Relates to #10416.

Moves from using action queue to action struct for the LDML keyboard
debugger. Does not do the same for the KMX keyboard debugger, because
that relies on action items to do single-step debugging.
2024-01-23 12:42:27 +07:00
Steven R. Loomis
aecf4e4552
Update core/tests/unit/kmnkbd/test_actions_normalize.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-22 16:47:28 -06:00
Eberhard Beilharz
4fa014a098
chore(core): Address code review comments 2024-01-22 11:58:35 +01:00
Steven R. Loomis
c3bae838ec feat(developer): ldml marker normalization in ts 🙀
- use Intl.Segmenter instead.

For: #10317
2024-01-19 18:32:08 -06:00
Steven R. Loomis
f2082d79a6 feat(developer): ldml marker normalization in ts 🙀
- cleanup and move marker_no_index into the shared area

For: #10317
2024-01-19 18:07:51 -06:00
Steven R. Loomis
72371675dd
Merge branch 'master' into feat/developer/10317-dev-nfd-epic-ldml 2024-01-19 18:01:38 -06:00
Steven R. Loomis
2bb4747c2f feat(developer): ldml marker normalization in ts 🙀
- add nfd table from ICU

For: #10317
2024-01-19 16:26:01 -06:00
Steven R. Loomis
384ee3b189 feat(developer): ldml marker normalization in ts 🙀
- C++ build fix

For: #10317
2024-01-19 14:30:19 -06:00
Steven R. Loomis
ac315661ed feat(core): ldml actions: include what you use 🌱
- missed an include

For: #10410
2024-01-19 13:45:12 -06:00
Steven R. Loomis
259471408a feat(core): ldml actions: reinstate divergence test 🌱
- compare context to test context, but SKIPPING markers.

For: #10410
2024-01-19 11:43:33 -06:00
Steven R. Loomis
04a6e164d7 feat(core): ldml actions: updates per code review 🌱
- fix backspace processing
- fix k_102 - we can no longer test for context invalidation here
- fix a test that assumed non-const km_core_actions.output

For: #10410
2024-01-19 11:24:41 -06:00
Steven R. Loomis
6ea83391a7 Merge remote-tracking branch 'origin/epic/core/9999-normalization' into feat/core/10410-ldml-really-use-action-struct 2024-01-19 10:40:42 -06:00
Eberhard Beilharz
a4c940418f
fix(core): Fix nuances when updating context
Fixes #10100.
2024-01-19 15:56:40 +01:00
Marc Durdin
249ed77835
Merge branch 'epic/core/9999-normalization' into chore/merge-master-into-core/9999-normalization 2024-01-19 17:42:40 +11:00
Steven R. Loomis
ade29e0d84 feat(core): ldml marker: assert that all markers were re-added 🙀
For: #10369
2024-01-19 00:13:08 -06:00
Steven R. Loomis
93340649d1
Merge pull request #10425 from keymanapp/feat/core/19410-ldml-use-action-struct
chore(core): ldml shuffle some deck chairs 🌱
2024-01-19 00:05:59 -06:00
Steven R. Loomis
9ff937b4ab
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-18 23:59:29 -06:00
Marc Durdin
583b0836de
Merge pull request #10430 from keymanapp/chore/core/10424-surrogate-pair-tests
chore(core): add surrogate pair tests to test_actions_normalize 🌱
2024-01-19 13:51:16 +11:00
Marc Durdin
c1304a9c19
Merge pull request #10428 from keymanapp/refactor/core/10423-move=context-helpers
refactor(core): move context helpers to a new module 🌱
2024-01-19 13:51:09 +11:00
Marc Durdin
4f909dcc83
Merge pull request #10403 from keymanapp/feat/core/9999-normalize-output
feat(core): infrastructure for normalization of output 🌱
2024-01-19 13:51:00 +11:00
Marc Durdin
ba46d2127d chore(core): address review comments 2024-01-19 09:50:00 +07:00
Marc Durdin
a26e21aba7 chore(core): address review comments 2024-01-19 08:01:45 +07:00
Steven R. Loomis
ae00eda4a0 chore(core): ldml shuffle some deck chairs 🌱
- move the emit_ and context functions into an ldml_event_state object

For #10410
2024-01-18 15:47:15 -06:00
Steven R. Loomis
6e2030b3b8 feat(developer): ldml marker normalization in ts 🙀
- intial parsing

For: #10317
2024-01-18 12:39:19 -06:00
Steven R. Loomis
973cf5d044 feat(core): ldml use action struct in tests 🌱
- clarify warnings in ldml.cpp for assertions we are skipping for now
- remove unused parameter

For: #10410
2024-01-18 08:48:50 -06:00
Steven R. Loomis
8095977bec feat(core): ldml action struct test runner changes 🌱
- change ldml test suite, with TODOs
- backspace validation is different, as we don't know the expected char.

For: #10410
2024-01-18 22:55:06 -06:00
Steven R. Loomis
4c322ddb8a feat(core): ldml action struct 🌱
- update km_core_actions.output to be a const*
- for now, call state->actions().commit() from ldml_event_state::commit()

For: #10410
2024-01-18 22:26:46 -06:00
Steven R. Loomis
20a13fa682 feat(core): ldml attempt to use action struct 🌱
- not quite working

For: #10410
2024-01-18 17:00:04 -06:00
Steven R. Loomis
16d641fc3b feat(developer): ldml marker normalization in ts 🙀
- scaffolding and tests

For: #10317
2024-01-18 05:43:44 -06:00