Steven R. Loomis
c3bae838ec
feat(developer): ldml marker normalization in ts 🙀
...
- use Intl.Segmenter instead.
For: #10317
2024-01-19 18:32:08 -06:00
Steven R. Loomis
72371675dd
Merge branch 'master' into feat/developer/10317-dev-nfd-epic-ldml
2024-01-19 18:01:38 -06:00
Steven R. Loomis
2bb4747c2f
feat(developer): ldml marker normalization in ts 🙀
...
- add nfd table from ICU
For: #10317
2024-01-19 16:26:01 -06:00
Steven R. Loomis
9ff937b4ab
Apply suggestions from code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-18 23:59:29 -06:00
Steven R. Loomis
7bc8831267
feat(core): updates to add_back_markers 🙀
...
- support out-of-order re-adding
- test for updated marker map
2024-01-17 19:04:11 -06:00
Steven R. Loomis
2e46bfea81
chore: reformat
2024-01-17 19:04:05 -06:00
Steven R. Loomis
990635eb09
feat(core): WIP attempt at cross segment markers 🙀
...
#10369
2024-01-17 19:03:43 -06:00
Steven R. Loomis
e4e7d9fd27
Merge branch 'feat/core/10319-escaping-you-epic-ldml' into feat/developer/10319-remove-old-escape-epic-ldml
2024-01-17 18:49:50 -06:00
Steven R. Loomis
b6a6b7e6b8
feat(core): ldml fix test case and parser 🙀
...
For: #10319
2024-01-17 18:32:39 -06:00
Steven R. Loomis
d08e7051cf
feat(core): ldml updates per code review 🙀
...
- don't try to remove irregular sentinel sequences
- update test
For: #10319
2024-01-17 18:34:56 -06:00
Steven R. Loomis
eb39bba374
chore(developer,core): change sample and test files to use \u{…} 🙀
...
- #10389 will require this, but at least change the test files
#10319
2024-01-15 17:27:41 -06:00
Steven R. Loomis
ab67061843
feat(core): 32 bit escapades 🙀
...
- improved test code and marker parsing
#10319
2024-01-15 14:07:19 -06:00
Steven R. Loomis
c5d0e0525e
feat(developer): 32 bit escapades 🙀
...
- escaping working, matching not yet
#10319
2024-01-13 12:55:51 -06:00
Steven R. Loomis
e06cf99b02
Merge branch 'feat/core/10320-stack-markers-epic-ldml' into chore/core/10320-refactor-epic-ldml
2024-01-08 12:37:51 -06:00
Steven R. Loomis
776a902eb0
feat(core): stacked markers 🙀
...
- build fix for win
#10320
2024-01-06 13:56:37 -06:00
Steven R. Loomis
f38054f65a
chore(core): split out some files 🙀
...
- markers/normalization really needs its own file
- split out ldml_markers.* from ldml_transforms.*
- split out ldml_utils.hpp for a common assert macro
#10320
2024-01-06 11:56:11 -06:00
Steven R. Loomis
b2b897970d
feat(core): stacked markers 🙀
...
- renamed the single-segment functions to normalize_nfd_markers_segment() since they should only be run on a single segment
- commented out NFC for now, dead code
- add some additional checks/DebugLog around normalization
#10320
2024-01-06 11:33:51 -06:00
Steven R. Loomis
f4be7856da
feat(core): stacked markers tests 🙀
...
- we can finally support single markers with the new structure
2024-01-06 10:17:10 -06:00
Steven R. Loomis
52fe9073a0
chore(core): stacked markers tests 🙀
...
- first we break
#10320
2024-01-05 18:52:43 -06:00
Steven R. Loomis
91b761b027
feat(core): fixup markers as regex 🙀
...
- move prepend_marker out of line
- replace bool for_regex with an enum
- cleanup another instance of \b -> \u0008
#9121
2024-01-02 17:21:04 -06:00
Steven R. Loomis
a6151e681b
Merge branch 'chore/core/ldml-test-robust-epic-ldml' into feat/developer/9121-reject-bad-regex-epic-ldml
2024-01-02 17:04:37 -06:00
Steven R. Loomis
84c218ff85
Merge branch 'feat/core/9121-more-regex-epic-ldml' into chore/core/ldml-test-robust-epic-ldml
2024-01-02 17:03:07 -06:00
Steven R. Loomis
0c3d1b513e
Merge branch 'feat/core/9468-stacked-markers-epic-ldml' into feat/core/9121-more-regex-epic-ldml
2024-01-02 17:03:00 -06:00
Steven R. Loomis
32ba632af9
Update core/tests/unit/ldml/test_transforms.cpp
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-02 17:02:35 -06:00
Steven R. Loomis
02ba64d857
feat(core): fixup markers as regex 🙀
...
- needed to remove/re-add the “≈≈[\u0001-\uD7FE]” format (aka LDML_MARKER_ANY_INDEX)
#9121
2023-12-30 21:09:42 -06:00
Steven R. Loomis
80ba5aa0ce
fix(developer): escape markers as regex 🙀
...
- escape markers to prevent the n'th marker from having some other meaning
2023-12-30 11:18:51 -06:00
Steven R. Loomis
763f7abeb0
chore(core): failing test with markers 🙀
...
- marker overrun! the 40th marker becomes a parenthesis, breaking regex
feat(core): ldml proper regex implementation 🙀 🙀 #9121
2023-12-28 18:57:33 -06:00
Steven R. Loomis
561cecb0de
fix(core): fix ldml test runner loops 🙀
...
- more easily handle test events
2023-12-28 17:57:40 -06:00
Steven R. Loomis
ae96008c11
feat(core): ldml more regex tests 🙀
...
- match in a set
- quantifiers
For: #9121
2023-12-28 17:38:21 -06:00
Steven R. Loomis
ef2f8395b4
feat(core): better reporting in ldml tests 🙀
...
- more err reporting out of inner functions such as key not found
- helper functions for setting FAIL and SKIP actions with messages
- colorization
For: #9121
2023-12-28 17:38:21 -06:00
Steven R. Loomis
53201263d9
fix(core): use marker-safe normalization in between transform stages 🙀
...
- the intermediate stages of transforms also need to use marker-safe normalization
- re-enable a test that was failing previously due to this
For: #9468
2023-12-22 17:51:18 -06:00
Steven R. Loomis
9d1599d557
chore(core): add some more marker tests 🙀
...
- one disabled for now
For #9468
2023-12-21 11:36:35 -06:00
Steven R. Loomis
12aa9a5b69
chore(core): add some more marker tests 🙀
...
For #9468
2023-12-20 19:23:29 -06:00
Steven R. Loomis
0b3e7a94e8
chore(core): uncomment some more tests 🙀
...
For #9468
2023-12-20 18:50:21 -06:00
Steven R. Loomis
0d3458deef
chore(core): comments in key-not-found 🙀
...
- typo fix
- clarify default behavior for missing keys
For: #9451
2023-12-12 18:43:00 -06:00
Steven R. Loomis
1e32a2a963
feat(core): ldml improve key-not-found 🙀
...
- copy exception table for reset from VKScanCodes.cpp
- update test
For: #9451
2023-12-07 17:15:44 -06:00
Steven R. Loomis
392c535270
feat(core): ldml improve key-not-found 🙀
...
- also affects markers, feat(core): normalization per spec for transforms/etc 🙀 #9468
- keep markers in nfd context string
- fix ldml test harness to handle context reset
- update test case
- still issues with overproduction of markers in the context
For: #9451
2023-12-06 18:33:51 -06:00
Steven R. Loomis
11334ed7fa
feat(core): ldml improve key-not-found 🙀
...
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no
For: #9451
2023-11-28 18:07:46 -06:00
Steven R. Loomis
a9b2dc9ede
Merge branch 'fix/developer/9438-vis-kbd-crash-epic-ldml' into fix/core-9961-optional-startContext-epic-ldml
2023-11-21 18:23:04 -06:00
Steven R. Loomis
037454b86e
Merge branch 'master' into feat/core/9707-tertiary-reorder-epic-ldml
2023-11-21 16:43:57 -06:00
Steven R. Loomis
d82f64fadc
Merge branch 'master' into feat/core/9450-backspace-epic-ldml
2023-11-21 15:01:51 -06:00
Steven R. Loomis
f524519b3f
feat(developer): ldml fix testcase processing 🙀
...
- KM_CORE_BT_UNKNOWN should only be used with an empty context
For: #9468 but related to #9450
2023-11-20 16:57:47 -06:00
Steven R. Loomis
86e8a4f609
Merge branch 'fix/developer/9438-vis-kbd-crash-epic-ldml' into fix/core-9961-optional-startContext-epic-ldml
2023-11-16 16:17:12 -06:00
Steven R. Loomis
350eb83ab3
feat(core): ldml typedef for reorder weights 🙀
...
For: #9707
2023-11-16 16:13:48 -06:00
Steven R. Loomis
6bc72ff294
fix(core): dx: ldml: startContext should be optional in ldml test data 🙀
...
- per spec
- update tests to remove these
Fixes : #9961
2023-11-16 15:51:56 -06:00
Steven R. Loomis
3cd6c7630c
feat(core): ldml tertiary reorder 🙀
...
- handle case where there's no tertiary base before the first char
- separate some element API tests from the actual tertiary test. Tests were failing because of now-correct implementation.
Fixes : #9707
2023-11-16 13:55:49 -06:00
Steven R. Loomis
30d951ef67
feat(core): ldml dx: launch ldml unit tests before keyboard tests 🙀
...
For: #9707
2023-11-16 13:43:12 -06:00
Steven R. Loomis
f7566c7356
feat(core): ldml tertiary reorder 🙀
...
- implemented tertiary reordering
- reinstated bengali reordering!
- fixed reordering to be longest-first (by sorting the list before applying)
- updated debug logging
Fixes : #9707
2023-11-16 12:31:08 -06:00
Steven R. Loomis
85a5afafc9
feat(core): ldml tertiary reorder 🙀
...
- failing test cases, hooray! In XML and C++
For: #9707
2023-11-16 12:31:08 -06:00
Steven R. Loomis
d0c8ea5c88
feat(core): ldml backspace transform 🙀
...
- refactor keypress some (needs more)
- copypasta implementation of backspace transform
For: #9450
2023-11-16 12:25:15 -06:00