Commit graph

902 commits

Author SHA1 Message Date
Steven R. Loomis
d4873d362a chore(core): split out some files 🙀
- reapply 0d9f543a78 code review comments

#10320
2024-01-11 11:41:40 -06:00
Eberhard Beilharz
9b768c57d7
refactor(linux): Use km_core_state_context_set_if_needed
Also update comment on `km_core_state_context_set_if_needed` to make it
clearer that it's expecting a UTF-16 string.

Closes #10212.
2024-01-10 18:57:41 +01:00
Steven R. Loomis
e06cf99b02
Merge branch 'feat/core/10320-stack-markers-epic-ldml' into chore/core/10320-refactor-epic-ldml 2024-01-08 12:37:51 -06:00
Steven R. Loomis
776a902eb0 feat(core): stacked markers 🙀
- build fix for win

#10320
2024-01-06 13:56:37 -06:00
Steven R. Loomis
f38054f65a chore(core): split out some files 🙀
- markers/normalization really needs its own file
- split out ldml_markers.* from ldml_transforms.*
- split out ldml_utils.hpp for a common assert macro

#10320
2024-01-06 11:56:11 -06:00
Steven R. Loomis
b2b897970d feat(core): stacked markers 🙀
- renamed the single-segment functions to normalize_nfd_markers_segment() since they should only be run on a single segment
- commented out NFC for now, dead code
- add some additional checks/DebugLog around normalization

#10320
2024-01-06 11:33:51 -06:00
Steven R. Loomis
f4be7856da feat(core): stacked markers tests 🙀
- we can finally support single markers with the new structure
2024-01-06 10:17:10 -06:00
Steven R. Loomis
52fe9073a0 chore(core): stacked markers tests 🙀
- first we break

#10320
2024-01-05 18:52:43 -06:00
Steven R. Loomis
91b761b027 feat(core): fixup markers as regex 🙀
- move prepend_marker out of line
- replace bool for_regex with an enum
- cleanup another instance of \b -> \u0008

#9121
2024-01-02 17:21:04 -06:00
Steven R. Loomis
a6151e681b
Merge branch 'chore/core/ldml-test-robust-epic-ldml' into feat/developer/9121-reject-bad-regex-epic-ldml 2024-01-02 17:04:37 -06:00
Steven R. Loomis
84c218ff85
Merge branch 'feat/core/9121-more-regex-epic-ldml' into chore/core/ldml-test-robust-epic-ldml 2024-01-02 17:03:07 -06:00
Steven R. Loomis
0c3d1b513e
Merge branch 'feat/core/9468-stacked-markers-epic-ldml' into feat/core/9121-more-regex-epic-ldml 2024-01-02 17:03:00 -06:00
Steven R. Loomis
32ba632af9
Update core/tests/unit/ldml/test_transforms.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-02 17:02:35 -06:00
Steven R. Loomis
02ba64d857 feat(core): fixup markers as regex 🙀
- needed to remove/re-add the “≈≈[\u0001-\uD7FE]” format  (aka LDML_MARKER_ANY_INDEX)

#9121
2023-12-30 21:09:42 -06:00
Steven R. Loomis
80ba5aa0ce fix(developer): escape markers as regex 🙀
- escape markers to prevent the n'th marker from having some other meaning
2023-12-30 11:18:51 -06:00
Steven R. Loomis
763f7abeb0 chore(core): failing test with markers 🙀
- marker overrun! the 40th marker becomes a parenthesis, breaking regex

feat(core): ldml proper regex implementation 🙀🙀 #9121
2023-12-28 18:57:33 -06:00
Steven R. Loomis
561cecb0de fix(core): fix ldml test runner loops 🙀
- more easily handle test events
2023-12-28 17:57:40 -06:00
Steven R. Loomis
ae96008c11 feat(core): ldml more regex tests 🙀
- match in a set
- quantifiers

For: #9121
2023-12-28 17:38:21 -06:00
Steven R. Loomis
ef2f8395b4 feat(core): better reporting in ldml tests 🙀
- more err reporting out of inner functions such as key not found
- helper functions for setting FAIL and SKIP actions with messages
- colorization

For: #9121
2023-12-28 17:38:21 -06:00
Steven R. Loomis
53201263d9 fix(core): use marker-safe normalization in between transform stages 🙀
- the intermediate stages of transforms also need to use marker-safe normalization
- re-enable a test that was failing previously due to this

For: #9468
2023-12-22 17:51:18 -06:00
Steven R. Loomis
9d1599d557 chore(core): add some more marker tests 🙀
- one disabled for now

For #9468
2023-12-21 11:36:35 -06:00
Steven R. Loomis
12aa9a5b69 chore(core): add some more marker tests 🙀
For #9468
2023-12-20 19:23:29 -06:00
Steven R. Loomis
0b3e7a94e8 chore(core): uncomment some more tests 🙀
For #9468
2023-12-20 18:50:21 -06:00
Steven R. Loomis
094f5ad492
Merge branch 'master' into feat/developer/9451-key-not-found-epic-ldml 2023-12-19 10:14:11 -06:00
rc-swag
1cb288144c chore(core): Merge branch 'master' into fix/core/clear-core-context-invalidate 2023-12-19 09:13:43 +10:00
rc-swag
deceabd5c3
fix(windows): review comments and suggestions
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2023-12-18 20:39:16 +10:00
rc-swag
2f737981f7 fix(core): add expected_context to keyboard unit tests
The unit tests needed to be updated to handle the cases
where the context is invalidated by a keystroke. However, they
still need to be able to verify the final string the application
should recieve. To do this a expected context token has been added
to the unit test format.
2023-12-15 11:20:08 +10:00
Steven R. Loomis
767d216936
Merge branch 'master' into feat/developer/9451-key-not-found-epic-ldml 2023-12-14 15:18:06 -06:00
rc-swag
437f938368 fix(core): fix baseline test cases k_000 and k_0008
Test k_000___null_keyboard has been updated as the [RALT K_B] should
cause the context to be reset.

Test k_008___vkey_input__ctrl_alt_2_ has been updated as the [LCTRL LALT
K_B] does not match a keyboard rule the context is deemed invalid and should be
reset.

It could be a consideration to modify the text_store object to better
reflect the state of a applications context however this would mean
we would need two expected results one for the application and one
for the context
2023-12-14 12:25:32 +10:00
Steven R. Loomis
74985f4576 fix(core): ldml: fix bad usage of unique_ptr 🙀
- needed to declare the pointer properly

Fixes: bug(core): Build failures in LDML code on newer Ubuntu versions #10249
2023-12-13 13:06:27 -06:00
Steven R. Loomis
0d3458deef chore(core): comments in key-not-found 🙀
- typo fix
- clarify default behavior for missing keys

For: #9451
2023-12-12 18:43:00 -06:00
Steven R. Loomis
ff5427f258
Merge pull request #10222 from keymanapp/feat/core/9121-regex2-epic-ldml
fix(core): improve memory allocation issues in calling into icu 🙀
2023-12-12 16:21:05 -06:00
Steven R. Loomis
6871cf03b3
Merge pull request #10198 from keymanapp/fix/core/8069-todo-ldml-epic-ldml
fix(core): remove a TODO-LDML 🙀
2023-12-12 16:20:20 -06:00
Steven R. Loomis
26048702d2
Merge pull request #10090 from keymanapp/fix/core/9451-key-not-found-epic-ldml
feat(core): ldml improve key-not-found 🙀
2023-12-12 16:19:53 -06:00
Steven R. Loomis
46b8d0668a
Merge branch 'fix/core/8069-todo-ldml-epic-ldml' into feat/core/9121-regex2-epic-ldml 2023-12-12 08:58:29 -06:00
Steven R. Loomis
c975b61c9b
Merge branch 'fix/core/9451-key-not-found-epic-ldml' into fix/core/8069-todo-ldml-epic-ldml 2023-12-12 08:58:11 -06:00
Steven R. Loomis
1cd97fadb9 feat(core): ldml improve key-not-found 🙀
- move vkey_to_contextreset  table into a common spot in core
- assert that vk < 0x100 to avoid running off the table

For: #9451
2023-12-12 08:56:33 -06:00
Steven R. Loomis
7735fc045a fix(core): improve memory allocation issues in calling into icu 🙀
- typo from  code review

For: #9121

Co-authored-by: Joshua A. Horton <joshua_horton@sil.org>
2023-12-12 08:18:45 -06:00
rc-swag
4a511bde1e fix(core): clear core context on invalidate cache 2023-12-12 23:49:04 +10:00
Steven R. Loomis
6506bfd89d fix(core): improve memory allocation issues in calling into icu 🙀
For: #9121
2023-12-11 16:53:41 -06:00
Steven R. Loomis
cadf7a1e14 fix(core): remove a TODO-LDML 🙀
Fixes: #8069

(all of the rest are in other tickets)
2023-12-08 18:48:01 -06:00
Steven R. Loomis
1e32a2a963 feat(core): ldml improve key-not-found 🙀
- copy exception table for reset from VKScanCodes.cpp
- update test

For: #9451
2023-12-07 17:15:44 -06:00
Steven R. Loomis
3c95c85b7c
Merge branch 'master' into fix/core/9451-key-not-found-epic-ldml 2023-12-07 14:00:59 -06:00
Steven R. Loomis
2071d25745 feat(core): ldml_processor fix for markers 🙀
- don't skip markers when calling context_to_string()! Oops.
- update docs on ldml_processor::remove_text()
- update remove_text() to handle markers in the context string.

This is really: #9468
2023-12-07 19:54:16 +00:00
Steven R. Loomis
392c535270 feat(core): ldml improve key-not-found 🙀
- also affects markers, feat(core): normalization per spec for transforms/etc 🙀  #9468
- keep markers in nfd context string
- fix ldml test harness to handle context reset
- update test case
- still issues with overproduction of markers in the context

For: #9451
2023-12-06 18:33:51 -06:00
Marc Durdin
802f22578a chore(core): address review comments 2023-12-04 13:37:59 +07:00
Marc Durdin
b6daac0bdb feat(core): action struct to action items conversion
Adds state->set_actions(). This sets the Core's action list to match the
contents of the action struct. Note that markers are not supported and
backspace expected_values will be empty, as this information is not
available. As the intended consumer of the action struct does not need
to know this information, this should be adequate.
2023-12-01 12:27:49 +07:00
rc-swag
51a462cff2 chore(windows): Merge branch 'master' into chore/windows/10052/remove-cached-context 2023-11-30 17:26:09 +10:00
rc-swag
d5963e05b6 fix: check for null termination
Determine the best way to check for null termination all compilers
2023-11-30 14:31:18 +10:00
rc-swag
35064073c1 chore(windows): Merge branch 'master' into chore/windows/10052/remove-cached-context 2023-11-30 13:58:19 +10:00