Commit graph

271 commits

Author SHA1 Message Date
Marc Durdin
cd69405960 feat(core): add app_context and failing tests
Adds the application context member of state and a corresponding,
failing unit test, which starts the process of verifying the handling
of the application context.
2024-01-15 11:50:15 +07:00
Steven R. Loomis
c5d0e0525e feat(developer): 32 bit escapades 🙀
- escaping working, matching not yet

#10319
2024-01-13 12:55:51 -06:00
Steven R. Loomis
e06cf99b02
Merge branch 'feat/core/10320-stack-markers-epic-ldml' into chore/core/10320-refactor-epic-ldml 2024-01-08 12:37:51 -06:00
Steven R. Loomis
776a902eb0 feat(core): stacked markers 🙀
- build fix for win

#10320
2024-01-06 13:56:37 -06:00
Steven R. Loomis
f38054f65a chore(core): split out some files 🙀
- markers/normalization really needs its own file
- split out ldml_markers.* from ldml_transforms.*
- split out ldml_utils.hpp for a common assert macro

#10320
2024-01-06 11:56:11 -06:00
Steven R. Loomis
b2b897970d feat(core): stacked markers 🙀
- renamed the single-segment functions to normalize_nfd_markers_segment() since they should only be run on a single segment
- commented out NFC for now, dead code
- add some additional checks/DebugLog around normalization

#10320
2024-01-06 11:33:51 -06:00
Steven R. Loomis
f4be7856da feat(core): stacked markers tests 🙀
- we can finally support single markers with the new structure
2024-01-06 10:17:10 -06:00
Steven R. Loomis
52fe9073a0 chore(core): stacked markers tests 🙀
- first we break

#10320
2024-01-05 18:52:43 -06:00
Steven R. Loomis
91b761b027 feat(core): fixup markers as regex 🙀
- move prepend_marker out of line
- replace bool for_regex with an enum
- cleanup another instance of \b -> \u0008

#9121
2024-01-02 17:21:04 -06:00
Steven R. Loomis
a6151e681b
Merge branch 'chore/core/ldml-test-robust-epic-ldml' into feat/developer/9121-reject-bad-regex-epic-ldml 2024-01-02 17:04:37 -06:00
Steven R. Loomis
84c218ff85
Merge branch 'feat/core/9121-more-regex-epic-ldml' into chore/core/ldml-test-robust-epic-ldml 2024-01-02 17:03:07 -06:00
Steven R. Loomis
0c3d1b513e
Merge branch 'feat/core/9468-stacked-markers-epic-ldml' into feat/core/9121-more-regex-epic-ldml 2024-01-02 17:03:00 -06:00
Steven R. Loomis
32ba632af9
Update core/tests/unit/ldml/test_transforms.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-01-02 17:02:35 -06:00
Steven R. Loomis
02ba64d857 feat(core): fixup markers as regex 🙀
- needed to remove/re-add the “≈≈[\u0001-\uD7FE]” format  (aka LDML_MARKER_ANY_INDEX)

#9121
2023-12-30 21:09:42 -06:00
Steven R. Loomis
80ba5aa0ce fix(developer): escape markers as regex 🙀
- escape markers to prevent the n'th marker from having some other meaning
2023-12-30 11:18:51 -06:00
Steven R. Loomis
763f7abeb0 chore(core): failing test with markers 🙀
- marker overrun! the 40th marker becomes a parenthesis, breaking regex

feat(core): ldml proper regex implementation 🙀🙀 #9121
2023-12-28 18:57:33 -06:00
Steven R. Loomis
561cecb0de fix(core): fix ldml test runner loops 🙀
- more easily handle test events
2023-12-28 17:57:40 -06:00
Steven R. Loomis
ae96008c11 feat(core): ldml more regex tests 🙀
- match in a set
- quantifiers

For: #9121
2023-12-28 17:38:21 -06:00
Steven R. Loomis
ef2f8395b4 feat(core): better reporting in ldml tests 🙀
- more err reporting out of inner functions such as key not found
- helper functions for setting FAIL and SKIP actions with messages
- colorization

For: #9121
2023-12-28 17:38:21 -06:00
Steven R. Loomis
53201263d9 fix(core): use marker-safe normalization in between transform stages 🙀
- the intermediate stages of transforms also need to use marker-safe normalization
- re-enable a test that was failing previously due to this

For: #9468
2023-12-22 17:51:18 -06:00
Steven R. Loomis
9d1599d557 chore(core): add some more marker tests 🙀
- one disabled for now

For #9468
2023-12-21 11:36:35 -06:00
Steven R. Loomis
12aa9a5b69 chore(core): add some more marker tests 🙀
For #9468
2023-12-20 19:23:29 -06:00
Steven R. Loomis
0b3e7a94e8 chore(core): uncomment some more tests 🙀
For #9468
2023-12-20 18:50:21 -06:00
Steven R. Loomis
0d3458deef chore(core): comments in key-not-found 🙀
- typo fix
- clarify default behavior for missing keys

For: #9451
2023-12-12 18:43:00 -06:00
Steven R. Loomis
1e32a2a963 feat(core): ldml improve key-not-found 🙀
- copy exception table for reset from VKScanCodes.cpp
- update test

For: #9451
2023-12-07 17:15:44 -06:00
Steven R. Loomis
392c535270 feat(core): ldml improve key-not-found 🙀
- also affects markers, feat(core): normalization per spec for transforms/etc 🙀  #9468
- keep markers in nfd context string
- fix ldml test harness to handle context reset
- update test case
- still issues with overproduction of markers in the context

For: #9451
2023-12-06 18:33:51 -06:00
Steven R. Loomis
11334ed7fa feat(core): ldml improve key-not-found 🙀
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no

For: #9451
2023-11-28 18:07:46 -06:00
Steven R. Loomis
a9b2dc9ede
Merge branch 'fix/developer/9438-vis-kbd-crash-epic-ldml' into fix/core-9961-optional-startContext-epic-ldml 2023-11-21 18:23:04 -06:00
Steven R. Loomis
037454b86e
Merge branch 'master' into feat/core/9707-tertiary-reorder-epic-ldml 2023-11-21 16:43:57 -06:00
Steven R. Loomis
d82f64fadc
Merge branch 'master' into feat/core/9450-backspace-epic-ldml 2023-11-21 15:01:51 -06:00
Steven R. Loomis
f524519b3f feat(developer): ldml fix testcase processing 🙀
- KM_CORE_BT_UNKNOWN should only be used with an empty context

For: #9468  but related to #9450
2023-11-20 16:57:47 -06:00
Steven R. Loomis
86e8a4f609
Merge branch 'fix/developer/9438-vis-kbd-crash-epic-ldml' into fix/core-9961-optional-startContext-epic-ldml 2023-11-16 16:17:12 -06:00
Steven R. Loomis
350eb83ab3 feat(core): ldml typedef for reorder weights 🙀
For: #9707
2023-11-16 16:13:48 -06:00
Steven R. Loomis
6bc72ff294 fix(core): dx: ldml: startContext should be optional in ldml test data 🙀
- per spec
- update tests to remove these

Fixes: #9961
2023-11-16 15:51:56 -06:00
Steven R. Loomis
3cd6c7630c feat(core): ldml tertiary reorder 🙀
- handle case where there's no tertiary base before the first char
- separate some element API tests from the actual tertiary test.  Tests were failing because of now-correct implementation.

Fixes: #9707
2023-11-16 13:55:49 -06:00
Steven R. Loomis
30d951ef67 feat(core): ldml dx: launch ldml unit tests before keyboard tests 🙀
For: #9707
2023-11-16 13:43:12 -06:00
Steven R. Loomis
f7566c7356 feat(core): ldml tertiary reorder 🙀
- implemented tertiary reordering
- reinstated bengali reordering!
- fixed reordering to be longest-first (by sorting the list before applying)
- updated debug logging

Fixes: #9707
2023-11-16 12:31:08 -06:00
Steven R. Loomis
85a5afafc9 feat(core): ldml tertiary reorder 🙀
- failing test cases, hooray! In XML and C++

For: #9707
2023-11-16 12:31:08 -06:00
Steven R. Loomis
d0c8ea5c88 feat(core): ldml backspace transform 🙀
- refactor keypress some (needs more)
- copypasta implementation of backspace transform

For: #9450
2023-11-16 12:25:15 -06:00
Steven R. Loomis
e28f35903a feat(core): marker normalization 🙀
- test fix

For: #9468
2023-11-15 17:29:18 -06:00
Steven R. Loomis
5a04a812fa feat(core): marker normalization 🙀
- temporarily remove a backspace test until #9450 is fixed.

For: #9468
2023-11-15 17:09:26 -06:00
Steven R. Loomis
07a4821336 feat(core): marker normalization 🙀
- a little further
- couple places where "it wasn't plugged in"
- adding some LDML-TODOs - marker creep
- fixed one unnecessary alloc/dealloc

For: #9468
2023-11-15 17:00:04 -06:00
Steven R. Loomis
2f843d98f3 feat(core): marker normalization 🙀
- go back to NFD for the context, for now
- anticipating when the privatecontext is NFD but the public context is NFC
- also update the test cases

For: #9468
2023-11-14 16:36:15 -06:00
Steven R. Loomis
7b481945e6 feat(core): test fix 🙀
- fix a cast
- fix some test cases

For: #9468
2023-11-14 07:28:57 -06:00
Steven R. Loomis
78c2ac4aec feat(core): ldml marker normalization 🙀
- refactor out backspace processing into a function.
- for now, just drop any markers in the context when we're lopping off the end

For: #9468
2023-11-14 07:28:41 -06:00
Steven R. Loomis
72830edfad Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-11-14 07:28:41 -06:00
Steven R. Loomis
a69feb583f feat(core): ldml marker normalization 🙀
- km::kbp is soooo last month!
- test_transforms can run NFD with markers, with some caveats.

For: #9468
2023-11-14 07:28:10 -06:00
Steven R. Loomis
015a738226 feat(core): ldml marker normalization 🙀
- add a new remove_markers(std::u32string) function
- add test cases for text utils
- update (failing) test cases for transform
- improve documentation of append process
- support KM_CORE_BT_UNKNOWN in ldml test
- remove_markers with a map
- update normalize test

For: #9468
2023-11-14 07:26:47 -06:00
Steven R. Loomis
255960ecae feat(core): ldml dx: dump vkey and modifier 🙀
For: #9468
2023-11-14 07:26:47 -06:00
Steven R. Loomis
efc8ef022f feat(core): ldml normalization 🙀
- some failing marker tests

For: #9468
2023-11-14 07:26:47 -06:00