Commit graph

634 commits

Author SHA1 Message Date
Steven R. Loomis
b429dc3383 feat(core): markers and reorder 🙀
- reorders don't interact with markers, but must transit them
- add updated tests
- simplify reorder_group::apply(): the 'common prefix' discussion
was out-of-date as std::mismatch is handled by the ldml_processor
and the core context.
- add_back_markers() promoted to SPI so it can be called from reordering.

Fixes: #10516
2024-01-31 17:44:21 -06:00
Marc Durdin
9ee5777e48 chore(core): move action apis into keyman_core_api_actions.h
Fixes #10520.

The definitions for `struct km_core_action_item`, `struct
km_core_backspace_item`, `enum km_core_backspace_type`, `enum
km_core_action_type`, `km_core_state_action_items()`,
`km_core_state_queue_action_items()`, `km_core_process_queued_actions()`
are now in keyman_core_api_actions.h rather than in the primary
keyman_core_api.h, as they should not be used by engines in general.

The current users of those APIs are the interactive kmx debugger, and
the IMX integration in Keyman Engine for Windows. Both of these depend
on the old action queue model rather than the actions struct.

In a future version, we may refactor these further to make them apply
directly to the KMX processor, and provide access via an interface to
the KMX processor rather than as a general Core API. Once this change is
made, then Core itself will no longer support action queues at all, and
will expect keyboard processors to fill in an action struct (this is
already done for LDML).
2024-01-31 11:32:09 +07:00
Marc Durdin
f546c44048
Merge pull request #10390 from keymanapp/epic/core/9999-normalization
epic: Keyman Core normalization 🌱
2024-01-31 10:45:51 +11:00
Marc Durdin
4c95fef55a
Merge pull request #10533 from keymanapp/feat/core/10530-actions-deleted-context
feat(core): add `deleted_context` to `km_core_actions` struct 🌱
2024-01-30 11:46:10 +11:00
Eberhard Beilharz
2ebca107e0
chore(core): Fix build failures 2024-01-29 12:17:27 +01:00
Eberhard Beilharz
5404c02315
chore(core): Merge remote-tracking branch 'origin/epic/core/9999-normalization' into fix/core/10100-ContextNuances
# Conflicts:
#	core/tests/unit/kmnkbd/state_context_api.cpp
2024-01-29 12:05:17 +01:00
Marc Durdin
8501f2aa6d feat(core): add deleted_context to km_core_actions struct
Fixes #10530.

Adds the deleted_context member to the km_core_actions struct, and
associated unit tests. Simplifies integration by providing the consumer
with all the data they need in order to execute the transform,
specifically around number of delete operations required, without
needing to query the target application context again. The number of
delete operations will vary according to application compliance and
selected encoding; for example a UTF-16 string may require 2
delete-backs for surrogate pairs in the text buffer for a compliant app,
whereas there will be a single delete-back key event for a non-compliant
app.

The deleted_context member should also be used for debug assertions.
2024-01-29 13:21:25 +07:00
Steven R. Loomis
580e372a7e feat(core): ldml marker normalization fix 🙀
- document per code review

For: #10516
2024-01-26 14:45:54 -06:00
Steven R. Loomis
4108ecd298 feat(core): ldml marker normalization fix 🙀
- decomp the 'glued' char before storing in map

For: #10516
2024-01-25 17:45:59 -06:00
Marc Durdin
41cc2f4d1e feat(developer): make context_get and context_length public again
Rolls back the privatisation of the km_core_context_get and
km_core_context_length APIs because the debugger uses them.
2024-01-26 06:37:09 +07:00
Marc Durdin
4906545e31
Merge branch 'epic/core/9999-normalization' into refactor/core/10431-remove-unused-core-context-apis 2024-01-24 12:14:11 +11:00
Eberhard Beilharz
4fa014a098
chore(core): Address code review comments 2024-01-22 11:58:35 +01:00
Steven R. Loomis
c3bae838ec feat(developer): ldml marker normalization in ts 🙀
- use Intl.Segmenter instead.

For: #10317
2024-01-19 18:32:08 -06:00
Steven R. Loomis
f2082d79a6 feat(developer): ldml marker normalization in ts 🙀
- cleanup and move marker_no_index into the shared area

For: #10317
2024-01-19 18:07:51 -06:00
Steven R. Loomis
72371675dd
Merge branch 'master' into feat/developer/10317-dev-nfd-epic-ldml 2024-01-19 18:01:38 -06:00
Steven R. Loomis
384ee3b189 feat(developer): ldml marker normalization in ts 🙀
- C++ build fix

For: #10317
2024-01-19 14:30:19 -06:00
Steven R. Loomis
04a6e164d7 feat(core): ldml actions: updates per code review 🌱
- fix backspace processing
- fix k_102 - we can no longer test for context invalidation here
- fix a test that assumed non-const km_core_actions.output

For: #10410
2024-01-19 11:24:41 -06:00
Steven R. Loomis
6ea83391a7 Merge remote-tracking branch 'origin/epic/core/9999-normalization' into feat/core/10410-ldml-really-use-action-struct 2024-01-19 10:40:42 -06:00
Eberhard Beilharz
a4c940418f
fix(core): Fix nuances when updating context
Fixes #10100.
2024-01-19 15:56:40 +01:00
Marc Durdin
249ed77835
Merge branch 'epic/core/9999-normalization' into chore/merge-master-into-core/9999-normalization 2024-01-19 17:42:40 +11:00
Steven R. Loomis
ade29e0d84 feat(core): ldml marker: assert that all markers were re-added 🙀
For: #10369
2024-01-19 00:13:08 -06:00
Steven R. Loomis
93340649d1
Merge pull request #10425 from keymanapp/feat/core/19410-ldml-use-action-struct
chore(core): ldml shuffle some deck chairs 🌱
2024-01-19 00:05:59 -06:00
Steven R. Loomis
9ff937b4ab
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-18 23:59:29 -06:00
Marc Durdin
c1304a9c19
Merge pull request #10428 from keymanapp/refactor/core/10423-move=context-helpers
refactor(core): move context helpers to a new module 🌱
2024-01-19 13:51:09 +11:00
Marc Durdin
4f909dcc83
Merge pull request #10403 from keymanapp/feat/core/9999-normalize-output
feat(core): infrastructure for normalization of output 🌱
2024-01-19 13:51:00 +11:00
Marc Durdin
ba46d2127d chore(core): address review comments 2024-01-19 09:50:00 +07:00
Marc Durdin
a26e21aba7 chore(core): address review comments 2024-01-19 08:01:45 +07:00
Steven R. Loomis
ae00eda4a0 chore(core): ldml shuffle some deck chairs 🌱
- move the emit_ and context functions into an ldml_event_state object

For #10410
2024-01-18 15:47:15 -06:00
Steven R. Loomis
6e2030b3b8 feat(developer): ldml marker normalization in ts 🙀
- intial parsing

For: #10317
2024-01-18 12:39:19 -06:00
Steven R. Loomis
8095977bec feat(core): ldml action struct test runner changes 🌱
- change ldml test suite, with TODOs
- backspace validation is different, as we don't know the expected char.

For: #10410
2024-01-18 22:55:06 -06:00
Steven R. Loomis
4c322ddb8a feat(core): ldml action struct 🌱
- update km_core_actions.output to be a const*
- for now, call state->actions().commit() from ldml_event_state::commit()

For: #10410
2024-01-18 22:26:46 -06:00
Steven R. Loomis
20a13fa682 feat(core): ldml attempt to use action struct 🌱
- not quite working

For: #10410
2024-01-18 17:00:04 -06:00
Steven R. Loomis
16d641fc3b feat(developer): ldml marker normalization in ts 🙀
- scaffolding and tests

For: #10317
2024-01-18 05:43:44 -06:00
Steven R. Loomis
64c808c4cc feat(core): ldml marker segments: fix build errs 🙀
- remove unused function

For: #10369
2024-01-18 02:54:55 -06:00
Steven R. Loomis
2c56223f53 feat(core): ldml marker: improve segment algorithm 🙀
- easier to understand loop
- comments

For: #10369
2024-01-18 14:39:10 -06:00
Steven R. Loomis
3ccffe9430 feat(core): ldml marker: refactor, use nfd props 🙀
- split out parse_next_marker()
- use NFD safe boundaries to segment marker interaction

For: #10369
2024-01-18 13:51:40 -06:00
Marc Durdin
83f80c061a refactor(core): remove unused context APIs
Fixes #10431.

The `km_core_` prefix has been removed from internal-only functions, and
these function declarations moved to context.hpp.

The functions have not been moved from km_core_context_api.cpp at this
stage.

Rewrote the function documentation in Javadoc style comments for the
internal use functions.
2024-01-18 12:18:07 +07:00
Marc Durdin
c6acf86629
Merge pull request #10419 from keymanapp/chore/core/10410-ldml-just-say-no-to-action-queues
chore(core): ldml assert if queue functions are called 🌱
2024-01-18 14:47:04 +11:00
Marc Durdin
63574f8513 chore(core): add assertions to context helpers 2024-01-18 09:31:13 +07:00
Marc Durdin
b3261b8dc4 refactor(core): move context helpers to a new module
Fixes #10423.
2024-01-18 09:26:58 +07:00
Marc Durdin
3878394787 chore(core): remove todo comment 2024-01-18 08:53:55 +07:00
Marc Durdin
425ebe36b0 feat(core): synchronize app_context for non-normalizing keyboardprocessors
Fixes #10421.
2024-01-18 08:52:59 +07:00
Marc Durdin
4ab47ea435 chore(core): further debug assertions for actions_normalize 2024-01-18 08:39:53 +07:00
Marc Durdin
823120b0c4 chore(core): simplify input assumptions for actions_normalize
Per discussion in #10422, we can assume that input cached_context is
always NFD. However input actions->output may not start at a
normalization boundary, so we still need to backtrack to a normalization
boundary in order to get our NFC output. But cached_context never need
change.

This makes no change to the algorithm, but tweaks some of the unit tests
to adhere to this input assumption.

Note: we could consider adding a debug assertion that cached_context is
NFD.
2024-01-18 08:25:15 +07:00
Steven R. Loomis
7bc8831267 feat(core): updates to add_back_markers 🙀
- support out-of-order re-adding
- test for updated marker map
2024-01-17 19:04:11 -06:00
Steven R. Loomis
990635eb09 feat(core): WIP attempt at cross segment markers 🙀
#10369
2024-01-17 19:03:43 -06:00
Steven R. Loomis
b6a6b7e6b8 feat(core): ldml fix test case and parser 🙀
For: #10319
2024-01-17 18:32:39 -06:00
Steven R. Loomis
d08e7051cf feat(core): ldml updates per code review 🙀
- don't try to remove irregular sentinel sequences
- update test

For: #10319
2024-01-17 18:34:56 -06:00
Steven R. Loomis
763bc96739 feat(core): ldml updates per code review 🙀
- remove stray change to kmx_processevent.cpp
- fix escaping of syntax chars
- fix comment per review

For: #10319
2024-01-17 18:05:28 -06:00
Steven R. Loomis
792329fba0
chore(core): tag unused args 2024-01-17 17:30:48 -06:00