Commit graph

271 commits

Author SHA1 Message Date
Steven R. Loomis
6d2ae221af chore(core,developer,resources): ldml update out of techpreview 🙀
- further fixes
2024-03-04 08:21:45 -06:00
Steven R. Loomis
17a653fa19 chore(core): ldml update out of techpreview 🙀
- updates to support techpreview -> 45
- test changes

Fixes: #10900
2024-03-02 22:40:39 -06:00
Steven R. Loomis
209717a24a chore(core,developer): rename unicodeSet to uset in xml
chore(developer): rename unicodeSet to uset 🙀  #10657
2024-02-27 18:11:18 -06:00
Steven R. Loomis
b1d74ce809 chore(core): describe what is being tested in each stage
Fixes: #10505
2024-02-22 16:19:21 -06:00
Steven R. Loomis
993f5252e9 chore(core): describe what is being tested in each stage
Fixes: #10505
2024-02-22 13:15:46 -06:00
Steven R. Loomis
9ef198ff19 chore(core): update sample ldml keyboard
Fixes: #10505
2024-02-20 12:47:25 -06:00
Steven R. Loomis
746af68afa chore(core): shuffle transform matcher. re-enable failing case
For #10736
2024-02-16 12:49:19 -06:00
Steven R. Loomis
503f196e42 chore(core): update test case with TODO for known issue
For #10734
2024-02-15 16:27:26 -06:00
Steven R. Loomis
e7b31113a3 chore(core): remove obsolete TODOs. Temporary change to test.
- TODO: 10734 There's a bug in the matching still. Come back and fix this, but the crasher is fixed.

For #10734
2024-02-15 16:21:53 -06:00
Steven R. Loomis
e099ffa3a1 chore(core): update ldml_test
- includes crasher from #10734

For #10505
2024-02-15 15:26:09 -06:00
Steven R. Loomis
5f01fca679 chore(core): create sample LDML keyboard that exercises effectively all functionality
- test keyboard was wrong

For #10505
2024-02-15 14:26:44 -06:00
Steven R. Loomis
e415f411e4 chore(core): create sample LDML keyboard that exercises effectively all functionality
- add some more functionality

For #10505
2024-02-15 14:26:44 -06:00
Steven R. Loomis
701291a7f7 chore(core): create sample LDML keyboard that exercises effectively all functionality
For #10505
2024-02-15 14:26:44 -06:00
Steven R. Loomis
3a538c103f feat(developer): support normalization=disabled 🙀
- test and code fixes

#10554
2024-02-01 18:40:02 -06:00
Steven R. Loomis
7572c9476e feat(developer): support normalization=disabled 🙀
- main core changes and test changes

#10554
2024-02-01 17:32:42 -06:00
Steven R. Loomis
9120bb4c87 feat(core,developer): normalization=disabled 🙀
- failing tests and initial work

#10554
2024-01-31 22:38:06 -06:00
Steven R. Loomis
687e50557f feat(core): add some C++ side tests 🙀
- add more test cases from @jahorton

#10317
2024-01-31 18:14:01 -06:00
Steven R. Loomis
600fdcc2c8 feat(core): double markers once again 🙀
- C++ side passes

Fixes: #10516
2024-01-31 18:05:53 -06:00
Steven R. Loomis
2891d19d88 feat(core): double markers once again 🙀
- C++ side

Fixes: #10516
2024-01-31 18:05:53 -06:00
Steven R. Loomis
b429dc3383 feat(core): markers and reorder 🙀
- reorders don't interact with markers, but must transit them
- add updated tests
- simplify reorder_group::apply(): the 'common prefix' discussion
was out-of-date as std::mismatch is handled by the ldml_processor
and the core context.
- add_back_markers() promoted to SPI so it can be called from reordering.

Fixes: #10516
2024-01-31 17:44:21 -06:00
Marc Durdin
f546c44048
Merge pull request #10390 from keymanapp/epic/core/9999-normalization
epic: Keyman Core normalization 🌱
2024-01-31 10:45:51 +11:00
Steven R. Loomis
5ce6490a1f feat(developer): dev side norm 🙀
- repertoire test needed normalization also

For: #10317
2024-01-29 09:31:48 -06:00
Steven R. Loomis
4108ecd298 feat(core): ldml marker normalization fix 🙀
- decomp the 'glued' char before storing in map

For: #10516
2024-01-25 17:45:59 -06:00
Marc Durdin
41cc2f4d1e feat(developer): make context_get and context_length public again
Rolls back the privatisation of the km_core_context_get and
km_core_context_length APIs because the debugger uses them.
2024-01-26 06:37:09 +07:00
Marc Durdin
4906545e31
Merge branch 'epic/core/9999-normalization' into refactor/core/10431-remove-unused-core-context-apis 2024-01-24 12:14:11 +11:00
Steven R. Loomis
c3bae838ec feat(developer): ldml marker normalization in ts 🙀
- use Intl.Segmenter instead.

For: #10317
2024-01-19 18:32:08 -06:00
Steven R. Loomis
72371675dd
Merge branch 'master' into feat/developer/10317-dev-nfd-epic-ldml 2024-01-19 18:01:38 -06:00
Steven R. Loomis
2bb4747c2f feat(developer): ldml marker normalization in ts 🙀
- add nfd table from ICU

For: #10317
2024-01-19 16:26:01 -06:00
Steven R. Loomis
259471408a feat(core): ldml actions: reinstate divergence test 🌱
- compare context to test context, but SKIPPING markers.

For: #10410
2024-01-19 11:43:33 -06:00
Steven R. Loomis
04a6e164d7 feat(core): ldml actions: updates per code review 🌱
- fix backspace processing
- fix k_102 - we can no longer test for context invalidation here
- fix a test that assumed non-const km_core_actions.output

For: #10410
2024-01-19 11:24:41 -06:00
Steven R. Loomis
6ea83391a7 Merge remote-tracking branch 'origin/epic/core/9999-normalization' into feat/core/10410-ldml-really-use-action-struct 2024-01-19 10:40:42 -06:00
Marc Durdin
249ed77835
Merge branch 'epic/core/9999-normalization' into chore/merge-master-into-core/9999-normalization 2024-01-19 17:42:40 +11:00
Steven R. Loomis
9ff937b4ab
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-18 23:59:29 -06:00
Marc Durdin
ba46d2127d chore(core): address review comments 2024-01-19 09:50:00 +07:00
Steven R. Loomis
973cf5d044 feat(core): ldml use action struct in tests 🌱
- clarify warnings in ldml.cpp for assertions we are skipping for now
- remove unused parameter

For: #10410
2024-01-18 08:48:50 -06:00
Steven R. Loomis
8095977bec feat(core): ldml action struct test runner changes 🌱
- change ldml test suite, with TODOs
- backspace validation is different, as we don't know the expected char.

For: #10410
2024-01-18 22:55:06 -06:00
Marc Durdin
83f80c061a refactor(core): remove unused context APIs
Fixes #10431.

The `km_core_` prefix has been removed from internal-only functions, and
these function declarations moved to context.hpp.

The functions have not been moved from km_core_context_api.cpp at this
stage.

Rewrote the function documentation in Javadoc style comments for the
internal use functions.
2024-01-18 12:18:07 +07:00
Steven R. Loomis
7bc8831267 feat(core): updates to add_back_markers 🙀
- support out-of-order re-adding
- test for updated marker map
2024-01-17 19:04:11 -06:00
Steven R. Loomis
2e46bfea81 chore: reformat 2024-01-17 19:04:05 -06:00
Steven R. Loomis
990635eb09 feat(core): WIP attempt at cross segment markers 🙀
#10369
2024-01-17 19:03:43 -06:00
Steven R. Loomis
e4e7d9fd27
Merge branch 'feat/core/10319-escaping-you-epic-ldml' into feat/developer/10319-remove-old-escape-epic-ldml 2024-01-17 18:49:50 -06:00
Steven R. Loomis
b6a6b7e6b8 feat(core): ldml fix test case and parser 🙀
For: #10319
2024-01-17 18:32:39 -06:00
Steven R. Loomis
d08e7051cf feat(core): ldml updates per code review 🙀
- don't try to remove irregular sentinel sequences
- update test

For: #10319
2024-01-17 18:34:56 -06:00
Marc Durdin
671973baab refactor(core): split context API from Core primary API
Relates to #9999.
Fixes #10384.

The context API endpoints should no longer be considered as part of the
standard Core API. The only consumers that have a need to access these
APIs are the IMX integration in Engine for Windows, and the Keyman
Developer Debugger.

These symbols are currently used by Developer:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()`
* `km_core_context_set()`
* `km_core_context_clear()`

These symbols are currently used by Windows IMX:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_context_items_dispose()`
* `km_core_context_item_list_size()`
* `km_core_state_get_intermediate_context()`

The following functions and symbols are moving to
keyman_core_api_context.h:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()` function
* `km_core_state_get_intermediate_context()` function
* `km_core_context_set()` function
* `km_core_context_clear()` function
* `km_core_context_get()` function
* `km_core_context_items_from_utf16()` function
* `km_core_context_items_from_utf8()` function
* `km_core_context_items_to_utf8()` function
* `km_core_context_items_to_utf16()` function
* `km_core_context_items_to_utf32()` function
* `km_core_context_items_dispose()` function
* `km_core_context_length()` function
* `km_core_context_append()` function
* `km_core_context_shrink()` function
* `km_core_context_item_list_size()` function
2024-01-17 14:44:25 +11:00
Marc Durdin
e5c2525241 refactor(core): split context API from Core primary API
Relates to #9999.
Fixes #10384.

The context API endpoints should no longer be considered as part of the
standard Core API. The only consumers that have a need to access these
APIs are the IMX integration in Engine for Windows, and the Keyman
Developer Debugger.

These symbols are currently used by Developer:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()`
* `km_core_context_set()`
* `km_core_context_clear()`

These symbols are currently used by Windows IMX:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_context_items_dispose()`
* `km_core_context_item_list_size()`
* `km_core_state_get_intermediate_context()`

The following functions and symbols are moving to
keyman_core_api_context.h:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()` function
* `km_core_state_get_intermediate_context()` function
* `km_core_context_set()` function
* `km_core_context_clear()` function
* `km_core_context_get()` function
* `km_core_context_items_from_utf16()` function
* `km_core_context_items_from_utf8()` function
* `km_core_context_items_to_utf8()` function
* `km_core_context_items_to_utf16()` function
* `km_core_context_items_to_utf32()` function
* `km_core_context_items_dispose()` function
* `km_core_context_length()` function
* `km_core_context_append()` function
* `km_core_context_shrink()` function
* `km_core_context_item_list_size()` function
2024-01-16 12:04:41 +07:00
Steven R. Loomis
eb39bba374 chore(developer,core): change sample and test files to use \u{…} 🙀
- #10389 will require this, but at least change the test files

#10319
2024-01-15 17:27:41 -06:00
Steven R. Loomis
ab67061843 feat(core): 32 bit escapades 🙀
- improved test code and marker parsing

#10319
2024-01-15 14:07:19 -06:00
Marc Durdin
84785fdd9a chore(core): wasm linker flags 2024-01-15 13:23:29 +07:00
Marc Durdin
932ad9d1df chore(core): add unit test deps for wasm 2024-01-15 12:50:41 +07:00
Marc Durdin
036c32af1f feat(core): normalize input context
First half of #9999.

Adds support for normalization (to NFD) of input app context into the
cached context. The keyboard processor will work with the NFD cached
context.

Adds unit tests for the normalization as part of the LDML keyboard
processor test suite.

TODO:
* Comparing modified cached context to app context to determine the
  transform required to send to the app
* Handling illegal unicode and unpaired surrogates on input context
2024-01-15 11:50:16 +07:00