Commit graph

541 commits

Author SHA1 Message Date
Steven R. Loomis
2c4d8e34e3 fix(core): skip leading trail surrogate char in km_core_state_context_set_if_needed() 2024-04-04 12:02:17 -05:00
Steven R. Loomis
5b1c853557 fix(common,core): support other lookup for modifiers
- the 'other' keyword was incorrectly called 'default'

Fixes: #11072
2024-04-01 23:21:33 -05:00
Steven R. Loomis
ae6f7b5d78 chore(core): support default lookup for modifiers
- fix spec

Fixes: #11072
2024-03-29 12:38:18 -05:00
Steven R. Loomis
bca037cbf3 chore(core): support default lookup for modifiers
- remove default flag from kmx_file.h

Fixes: #11072
2024-03-29 12:07:29 -05:00
Steven R. Loomis
89f5a0e090 fix(core): support default lookup for modifiers
Fixes: #11072
2024-03-28 13:12:01 -05:00
Steven R. Loomis
1691a59394
Merge branch 'feat/developer/11044-error-invalid-var' into feat/core/11072-default-modifier 2024-03-28 12:23:01 -05:00
Steven R. Loomis
313bbd32a7 feat(core): kmx+ scaffolding for modifiers=default
- add a new value, 0x10000 to indicate 'default'

For: #11072
2024-03-28 12:19:41 -05:00
Marc Durdin
5790f634cf
Merge pull request #11101 from keymanapp/fix/core/11067-fix-actions-normalize-utf32
fix(core): fix pointer math in actions_normalize() 🙀
2024-03-28 19:18:15 +11:00
Steven R. Loomis
92a4a2a0a2
Merge pull request #11094 from keymanapp/chore/core/11057-optimize-marker-context
chore(core): optimize ldml_event_state::emit_difference() when no diff 🙀
2024-03-28 00:13:45 -05:00
Steven R. Loomis
7106b662b7
Merge pull request #11100 from keymanapp/fix/core/11067-fix-actions-normalize-output-string
fix(core): actions_normalize() length and dead store fix 🙀
2024-03-28 00:13:21 -05:00
Steven R. Loomis
adc421974c chore(core): fix actions_normalize() pointer math - tests
- fix comments yet again
- add regression test in test_actions_normalize.cpp

For: #11067
2024-03-27 23:55:01 -05:00
Steven R. Loomis
d894db78f6 fix(core): add an assert on code_points_to_delete 2024-03-27 23:25:43 -05:00
Steven R. Loomis
ca34b5f00f fix(core): fix actions_normalize() UChar32 calculation
- UnicodeString is UTF-16, but can be used with UTF-32 boundaries.
- fix calculation of code_points_to_delete

Fixes: #11067
2024-03-27 19:37:08 -05:00
Steven R. Loomis
3f42b0b831 fix(core): fix actions_normalize() use of UnicodeString
- output[0] was used as a boolean, but needed to check output.isEmpty() instead
- dead store to  actions.code_points_to_delete

For: #11067 parts 1 and 2
2024-03-27 19:13:01 -05:00
Steven R. Loomis
8b5723028c chore(core): optimize ldml_event_state::emit_difference() when no difference
#11057
2024-03-27 08:18:03 -05:00
Steven R. Loomis
e8a8590eee fix(core): revert some interim fixes
- ldml_test_source fix moved off to issue #11083
- an optimization in ldml_processor also rolled back
2024-03-27 08:05:39 -05:00
Steven R. Loomis
18c6ecfc08 fix(core): fix 2 marker cases in ldml_event_state::emit_difference 🙀
- handle the case where old and new context strings are the same (i.e. no work to do).
- fix pointer arithmetic error in #10356 - the special case where one marker is being replaced by another. The named regression test didn't actually hit this case.

Fixes: bug(core): 'string too long' #11057
2024-03-25 16:45:01 -05:00
Steven R. Loomis
17a653fa19 chore(core): ldml update out of techpreview 🙀
- updates to support techpreview -> 45
- test changes

Fixes: #10900
2024-03-02 22:40:39 -06:00
Steven R. Loomis
2ee402c2b8
Update core/src/ldml/ldml_transforms.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-02-19 12:37:26 -06:00
Steven R. Loomis
6b229ffa14 fix(core): fix output append logic in any_group::apply_transform
- logic updating intermediate 'output' was incorrect
- update comments
- also replace size() with length() several places

Fixes #10736
2024-02-17 23:50:55 -06:00
Steven R. Loomis
ceb990e713 chore(core): more shuffling
For #10736
2024-02-16 18:27:35 -06:00
Steven R. Loomis
746af68afa chore(core): shuffle transform matcher. re-enable failing case
For #10736
2024-02-16 12:49:19 -06:00
Steven R. Loomis
e7b31113a3 chore(core): remove obsolete TODOs. Temporary change to test.
- TODO: 10734 There's a bug in the matching still. Come back and fix this, but the crasher is fixed.

For #10734
2024-02-15 16:21:53 -06:00
Steven R. Loomis
5edb70f07a fix(core): fix for ldml_transforms.cpp
For #10734
2024-02-15 15:26:30 -06:00
Marc Durdin
64daa6182b
Merge pull request #10618 from keymanapp/fix/core/10615-10616-10617-core-surrogates-and-app-context-markers-and-memory-leak
fix(core): surrogate handling, markers in app context, and memory leak
2024-02-07 14:43:12 +11:00
Marc Durdin
bcc18bfe8d
Merge pull request #10607 from keymanapp/fix/core/10605-app-context-copy-strip-markers
fix(core): strip markers in `actions_update_app_context_nfu()`
2024-02-07 14:43:01 +11:00
Marc Durdin
6c4d5ff7fa
Merge pull request #10585 from keymanapp/fix/core/10582-km_core_state_get_actions-idempotency
fix(core): make `km_core_state_get_actions()` idempotent
2024-02-07 14:42:50 +11:00
Steven R. Loomis
889e9c1581
Merge pull request #10586 from keymanapp/feat/developer/10554-sometimes-no-norm 2024-02-06 21:38:39 -06:00
Marc Durdin
0cc6901085
Merge branch 'fix/core/10605-app-context-copy-strip-markers' into fix/core/10615-10616-10617-core-surrogates-and-app-context-markers-and-memory-leak 2024-02-07 12:33:33 +11:00
Marc Durdin
232df37400
chore: Update core/src/state.cpp 2024-02-07 12:29:45 +11:00
Marc Durdin
c807415563
chore: Apply suggestions from code review
Co-authored-by: Steven R. Loomis <srl295@gmail.com>
2024-02-07 12:20:56 +11:00
Steven R. Loomis
15b7a42829 feat(core): support normalization=disabled 🙀
- clean up process_output:  remove some unnecessary steps,
refactor into 3 functions, clarify
- reorder functions for ldml_state
- remove extraneous assert() clutter when the normalization
functions already assert/DebugLog at the innermost level

#10554
2024-02-06 16:09:18 -06:00
Marc Durdin
c9eb545c05 refactor(core): more renames to satisfy compiler on linux
Shadowed types are not permitted.
2024-02-05 12:20:44 +07:00
Marc Durdin
6d8cae1f2e refactor(core): is_context_changed() into two functions
Refactors `is_context_changed()` into two functions,
`get_context_change()` and `get_context_items_change()`, to eliminate
use of `goto` for cleanup, and more accurately reflect the purpose of
the function. Also renames some variables referencing the return value
from these functions.

These functions would probably benefit from unit tests proving them.
2024-02-05 12:07:41 +07:00
Marc Durdin
fe8eba176c
Merge branch 'fix/core/10605-app-context-copy-strip-markers' into fix/core/10615-10616-10617-core-surrogates-and-app-context-markers-and-memory-leak 2024-02-05 13:45:03 +11:00
Marc Durdin
49eb6f42a9
Merge branch 'fix/core/10582-km_core_state_get_actions-idempotency' into fix/core/10605-app-context-copy-strip-markers 2024-02-05 13:44:56 +11:00
Marc Durdin
70ba1d1c1e
Merge branch 'chore/core/10520-hide-old-action-apis' into fix/core/10582-km_core_state_get_actions-idempotency 2024-02-05 13:44:47 +11:00
Marc Durdin
aa80992fb8
Merge branch 'feat/windows/use-km-core-actions-struct' into chore/core/10520-hide-old-action-apis 2024-02-05 13:44:36 +11:00
Marc Durdin
9ba919c2a9 fix(core): surrogate handling, markers in app context, and memory leak
Fixes #10615 (surrogate handling, km_core_state_context_set_if_needed).
Fixes #10616 (markers in app context, in unit tests)
Fixes #10617 (memory leak in replace_context).

The surrogate handling and markers in app context bugs were quite
tangled, so opted to fix those both together in one commit. The memory
leak was spotted while verifying the changes for surrogate handling.

Surrogates were not handled in is_context_unchanged, and opted to
refactor to work directly with km_core_context_item arrays rather than
continue with the string-based comparison, for simplicity.

While writing the corresponding unit tests, realised that the
app_context checks were incorrect in a number of existing tests, and so
fixed up the existing tests at the same time as writing up the new
tests. The only changes to core for this were to add assertions for the
app_context to ensure that markers are not inadvertently added.
2024-02-03 09:16:16 +07:00
Marc Durdin
9e50f50687 fix(core): strip markers in actions_update_app_context_nfu()
Fixes #10605.
2024-02-02 15:10:05 +07:00
Steven R. Loomis
71443836a2 feat(developer): support normalization=disabled 🙀
- avoid dead store to transforms class

#10554
2024-02-01 19:05:31 -06:00
Steven R. Loomis
7572c9476e feat(developer): support normalization=disabled 🙀
- main core changes and test changes

#10554
2024-02-01 17:32:42 -06:00
Marc Durdin
aad7c0a42b chore(core): fixup typos 2024-02-01 15:05:24 +07:00
Marc Durdin
0b7556870f
chore: Apply suggestions from code review 2024-02-01 18:56:26 +11:00
Marc Durdin
15f2412717 chore(core): build whackamole 2024-02-01 12:50:23 +07:00
Marc Durdin
2abaead178 chore(core): string include 2024-02-01 12:38:58 +07:00
Marc Durdin
56cda3fff4 chore(core): address cross-platform builds 2024-02-01 12:17:10 +07:00
Marc Durdin
1b88ab99be fix(core): make km_core_state_get_actions() idempotent
Fixes #10582.

Returned struct from the `km_core_state_get_actions()` API is now owned
by the state object. This is a change in API contract. The corresponding
`km_core_actions_dispose()` API has been made private, because there is
never any need for API consumers to call it. As this change is happening
in alpha, we won't bump the ABI version.

`state->app_context` is now guaranteed to be in sync with
`state->context` after `km_core_process_event()`, with the actions
struct populated at that point.

The next and final step in this refactor is to remove the actions queue
altogether from the Core and make that a kmn-only concept, but that will
wait until 18.0. The only consumers of the actions queue are the
kmn-specific interactive debugger in Keyman Developer, and kmn-specific
Input Method eXtensions in Keyman Engine for Windows.
2024-02-01 11:28:08 +07:00
Steven R. Loomis
a28cee0fac
Merge branch 'feat/core/10317-double-marker2-epic-ldml' into feat/core/10317-simplify-marker-epic-ldml 2024-01-31 22:25:45 -06:00
Steven R. Loomis
006b955f8a
Merge branch 'feat/core/10317-double-marker-epic-ldml' into feat/core/10317-double-marker2-epic-ldml 2024-01-31 22:24:46 -06:00