Steven R. Loomis
ae00eda4a0
chore(core): ldml shuffle some deck chairs 🌱
...
- move the emit_ and context functions into an ldml_event_state object
For #10410
2024-01-18 15:47:15 -06:00
Steven R. Loomis
792329fba0
chore(core): tag unused args
2024-01-17 17:30:48 -06:00
Steven R. Loomis
f484615450
chore(core): ldml don't use queue action functions 🌱
2024-01-17 16:16:56 -06:00
Steven R. Loomis
f38054f65a
chore(core): split out some files 🙀
...
- markers/normalization really needs its own file
- split out ldml_markers.* from ldml_transforms.*
- split out ldml_utils.hpp for a common assert macro
#10320
2024-01-06 11:56:11 -06:00
Steven R. Loomis
b2b897970d
feat(core): stacked markers 🙀
...
- renamed the single-segment functions to normalize_nfd_markers_segment() since they should only be run on a single segment
- commented out NFC for now, dead code
- add some additional checks/DebugLog around normalization
#10320
2024-01-06 11:33:51 -06:00
Steven R. Loomis
0d3458deef
chore(core): comments in key-not-found 🙀
...
- typo fix
- clarify default behavior for missing keys
For: #9451
2023-12-12 18:43:00 -06:00
Steven R. Loomis
c975b61c9b
Merge branch 'fix/core/9451-key-not-found-epic-ldml' into fix/core/8069-todo-ldml-epic-ldml
2023-12-12 08:58:11 -06:00
Steven R. Loomis
1cd97fadb9
feat(core): ldml improve key-not-found 🙀
...
- move vkey_to_contextreset table into a common spot in core
- assert that vk < 0x100 to avoid running off the table
For: #9451
2023-12-12 08:56:33 -06:00
Steven R. Loomis
cadf7a1e14
fix(core): remove a TODO-LDML 🙀
...
Fixes : #8069
(all of the rest are in other tickets)
2023-12-08 18:48:01 -06:00
Steven R. Loomis
1e32a2a963
feat(core): ldml improve key-not-found 🙀
...
- copy exception table for reset from VKScanCodes.cpp
- update test
For: #9451
2023-12-07 17:15:44 -06:00
Steven R. Loomis
2071d25745
feat(core): ldml_processor fix for markers 🙀
...
- don't skip markers when calling context_to_string()! Oops.
- update docs on ldml_processor::remove_text()
- update remove_text() to handle markers in the context string.
This is really: #9468
2023-12-07 19:54:16 +00:00
Steven R. Loomis
392c535270
feat(core): ldml improve key-not-found 🙀
...
- also affects markers, feat(core): normalization per spec for transforms/etc 🙀 #9468
- keep markers in nfd context string
- fix ldml test harness to handle context reset
- update test case
- still issues with overproduction of markers in the context
For: #9451
2023-12-06 18:33:51 -06:00
Steven R. Loomis
4d22b177af
feat(core): ldml improve key-not-found 🙀
...
- unused nit fix
For: #9451
2023-11-29 14:34:33 -06:00
Steven R. Loomis
84cc59c174
feat(core): ldml improve key-not-found 🙀
...
- restructure keyup/keydown code some
- new utility function emit_invalidate_passthrough_keystroke()
For: #9451
2023-11-29 14:29:26 -06:00
Steven R. Loomis
11334ed7fa
feat(core): ldml improve key-not-found 🙀
...
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no
For: #9451
2023-11-28 18:07:46 -06:00
Steven R. Loomis
e9e5665cb8
feat(core): ldml backspace transform 🙀
...
- refactor transform/normalization inner routine
For: #9450
2023-11-16 12:27:23 -06:00
Steven R. Loomis
d0c8ea5c88
feat(core): ldml backspace transform 🙀
...
- refactor keypress some (needs more)
- copypasta implementation of backspace transform
For: #9450
2023-11-16 12:25:15 -06:00
Steven R. Loomis
2f843d98f3
feat(core): marker normalization 🙀
...
- go back to NFD for the context, for now
- anticipating when the privatecontext is NFD but the public context is NFC
- also update the test cases
For: #9468
2023-11-14 16:36:15 -06:00
Steven R. Loomis
78c2ac4aec
feat(core): ldml marker normalization 🙀
...
- refactor out backspace processing into a function.
- for now, just drop any markers in the context when we're lopping off the end
For: #9468
2023-11-14 07:28:41 -06:00
Steven R. Loomis
a69feb583f
feat(core): ldml marker normalization 🙀
...
- km::kbp is soooo last month!
- test_transforms can run NFD with markers, with some caveats.
For: #9468
2023-11-14 07:28:10 -06:00
Steven R. Loomis
015a738226
feat(core): ldml marker normalization 🙀
...
- add a new remove_markers(std::u32string) function
- add test cases for text utils
- update (failing) test cases for transform
- improve documentation of append process
- support KM_CORE_BT_UNKNOWN in ldml test
- remove_markers with a map
- update normalize test
For: #9468
2023-11-14 07:26:47 -06:00
Eberhard Beilharz
f4d8529789
chore(linux): rename keyboardprocessor_ldml.* to keyman_core_ldml
2023-10-20 17:30:10 +02:00
Eberhard Beilharz
7369b22599
chore(linux): Merge with chore/linux/9733_RenameLibknmkbp
...
# Conflicts:
# core/src/debuglog.h
# core/src/ldml/ldml_processor.cpp
# core/src/ldml/ldml_transforms.cpp
# core/src/ldml/ldml_transforms.hpp
# core/tests/unit/ldml/ldml.cpp
2023-10-20 12:52:38 +02:00
Eberhard Beilharz
604fbc1a75
chore(linux): Rename namespace kbp to core
2023-10-19 10:39:55 +02:00
Steven R. Loomis
9dbf70c94e
fix(core): ldml more updates to normalization per review 🙀
...
- verify marker string as it's popped off
- add a missing pragma to debuglog.h
- move UErrorCode out of ldml_processor, change normalization functions to return true on success
For: #9468
2023-10-18 17:47:56 -05:00
Steven R. Loomis
93e1c18af2
fix(core): ldml update to normalization per review 🙀
...
For: #9468
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-10-16 20:22:52 -05:00
Steven R. Loomis
a577bd2f53
feat(core): ldml normalization 🙀
...
- more progress in the normalization pipeline. Trying to keep it from leaking.
- normalize output to NFC.
- normalize json test data to NFC (both context and expected).
- we do NOT try to normalize the 'embedded' strings currently.
- yet more test data fixes (turns out u+00e0 ≠ u+00e8)
For: #9468
2023-10-12 16:57:37 -05:00
Steven R. Loomis
e64f59613a
feat(core): ldml normalization 🙀
...
- split out 'process_key_string'
For: #9468
2023-10-12 10:10:51 -05:00
Steven R. Loomis
daa0523dde
feat(core): ldml normalization 🙀
...
- 0 steps forward, 4294967294 steps back
For: #9468
2023-10-10 19:01:14 -05:00
Marc Durdin
db76e2541d
chore: rename KM_KBP_ to KM_CORE_
2023-10-09 14:53:22 +07:00
Marc Durdin
ba5f18defd
chore: rename km_kbp_ to km_core_
2023-10-09 14:53:13 +07:00
Steven R. Loomis
9f033278d4
feat(core): marker validation 🙀
...
- validate marker number when going in or out of the context
#9119
2023-08-11 15:09:54 -05:00
Steven R. Loomis
2bf22c440d
feat(core): marker fixes per review comments 🙀
...
- simplify and improve context processing
For: #9119
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 13:30:29 -05:00
Steven R. Loomis
66ee68d2b3
Apply suggestions from code review
...
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 09:36:16 -05:00
Steven R. Loomis
c23303ee23
Apply suggestions from code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-08-09 12:52:28 -05:00
Steven R. Loomis
f0c81687f6
feat(core): marker fixes per review comments 🙀
...
- move prepend_marker() to inline
- clear the string in context_to_string()
For: #9119
2023-08-09 12:51:35 -05:00
Steven R. Loomis
290bcede2f
feat(core): improve documentation on transform/marker processing loop 🙀
...
For: #9119
2023-08-04 16:12:02 -05:00
Steven R. Loomis
1480e4d934
feat(core): improve emit_text() marker processing loop 🙀
...
For: #9119
2023-08-04 16:03:13 -05:00
Steven R. Loomis
fbb874fbfe
feat(core): warning fix 🙀
...
For: #9119
2023-08-04 15:29:19 -05:00
Steven R. Loomis
a9badff183
feat(core): remove reference to CODE_DEADKEY 🙀
...
For: #9119
2023-08-04 15:21:38 -05:00
Steven R. Loomis
b80b43e1e9
feat(core): re-refactor transform processing logic 🙀
...
For: #9119
2023-08-04 15:20:00 -05:00
Steven R. Loomis
8f4cf8c24e
feat(core): refactor sentinel 🙀
...
- and break all the things
For: #9119
2023-08-03 18:52:53 -05:00
Steven R. Loomis
20d1aad679
feat(core): refactor char emit 🙀
...
- and add an assert that UC_SENTINEL doesn't go into the text stream
For: #9119
2023-08-03 17:39:09 -05:00
Steven R. Loomis
cd3116d7cb
feat(core): improve processing logic 🙀
...
- improve the ctxt managing logic a little bit.
- need to handle markers as markers, though.
#9119
2023-08-03 15:20:37 -05:00
Steven R. Loomis
c7881d99b2
Update core/src/ldml/ldml_processor.cpp
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-28 21:55:39 -05:00
Steven R. Loomis
875d59987a
chore(core): update documentation in transform logic and processor 🙀
...
- drop some unneeded variables also, should be no semantic change
For: #7375
2023-07-26 16:16:57 -05:00
Steven R. Loomis
0a2aa6c5ca
Apply suggestions from code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-25 16:30:56 -05:00
Steven R. Loomis
bc378dcac6
feat(core): move transform processing to u32 🙀
...
- merge code paths…
#7375
2023-07-18 19:16:23 -05:00
Steven R. Loomis
83bb7e6f12
Merge branch 'master' into feat/developer-9050-utf32-epic-ldml
2023-06-23 17:31:43 -05:00
Steven R. Loomis
de90644d7a
feat(core,common,developer): support UTF-32 single chars 🙀
...
- C++ side now
- updated tests
- refactored escaping code and squashed bugs
- still todo: basic.xml
For: #9050
2023-06-23 16:55:05 -05:00