Commit graph

101 commits

Author SHA1 Message Date
Steven R. Loomis
ae00eda4a0 chore(core): ldml shuffle some deck chairs 🌱
- move the emit_ and context functions into an ldml_event_state object

For #10410
2024-01-18 15:47:15 -06:00
Steven R. Loomis
792329fba0
chore(core): tag unused args 2024-01-17 17:30:48 -06:00
Steven R. Loomis
f484615450 chore(core): ldml don't use queue action functions 🌱 2024-01-17 16:16:56 -06:00
Steven R. Loomis
f38054f65a chore(core): split out some files 🙀
- markers/normalization really needs its own file
- split out ldml_markers.* from ldml_transforms.*
- split out ldml_utils.hpp for a common assert macro

#10320
2024-01-06 11:56:11 -06:00
Steven R. Loomis
b2b897970d feat(core): stacked markers 🙀
- renamed the single-segment functions to normalize_nfd_markers_segment() since they should only be run on a single segment
- commented out NFC for now, dead code
- add some additional checks/DebugLog around normalization

#10320
2024-01-06 11:33:51 -06:00
Steven R. Loomis
0d3458deef chore(core): comments in key-not-found 🙀
- typo fix
- clarify default behavior for missing keys

For: #9451
2023-12-12 18:43:00 -06:00
Steven R. Loomis
c975b61c9b
Merge branch 'fix/core/9451-key-not-found-epic-ldml' into fix/core/8069-todo-ldml-epic-ldml 2023-12-12 08:58:11 -06:00
Steven R. Loomis
1cd97fadb9 feat(core): ldml improve key-not-found 🙀
- move vkey_to_contextreset  table into a common spot in core
- assert that vk < 0x100 to avoid running off the table

For: #9451
2023-12-12 08:56:33 -06:00
Steven R. Loomis
cadf7a1e14 fix(core): remove a TODO-LDML 🙀
Fixes: #8069

(all of the rest are in other tickets)
2023-12-08 18:48:01 -06:00
Steven R. Loomis
1e32a2a963 feat(core): ldml improve key-not-found 🙀
- copy exception table for reset from VKScanCodes.cpp
- update test

For: #9451
2023-12-07 17:15:44 -06:00
Steven R. Loomis
2071d25745 feat(core): ldml_processor fix for markers 🙀
- don't skip markers when calling context_to_string()! Oops.
- update docs on ldml_processor::remove_text()
- update remove_text() to handle markers in the context string.

This is really: #9468
2023-12-07 19:54:16 +00:00
Steven R. Loomis
392c535270 feat(core): ldml improve key-not-found 🙀
- also affects markers, feat(core): normalization per spec for transforms/etc 🙀  #9468
- keep markers in nfd context string
- fix ldml test harness to handle context reset
- update test case
- still issues with overproduction of markers in the context

For: #9451
2023-12-06 18:33:51 -06:00
Steven R. Loomis
4d22b177af feat(core): ldml improve key-not-found 🙀
- unused nit fix

For: #9451
2023-11-29 14:34:33 -06:00
Steven R. Loomis
84cc59c174 feat(core): ldml improve key-not-found 🙀
- restructure keyup/keydown code some
- new utility function emit_invalidate_passthrough_keystroke()

For: #9451
2023-11-29 14:29:26 -06:00
Steven R. Loomis
11334ed7fa feat(core): ldml improve key-not-found 🙀
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no

For: #9451
2023-11-28 18:07:46 -06:00
Steven R. Loomis
e9e5665cb8 feat(core): ldml backspace transform 🙀
- refactor transform/normalization inner routine

For: #9450
2023-11-16 12:27:23 -06:00
Steven R. Loomis
d0c8ea5c88 feat(core): ldml backspace transform 🙀
- refactor keypress some (needs more)
- copypasta implementation of backspace transform

For: #9450
2023-11-16 12:25:15 -06:00
Steven R. Loomis
2f843d98f3 feat(core): marker normalization 🙀
- go back to NFD for the context, for now
- anticipating when the privatecontext is NFD but the public context is NFC
- also update the test cases

For: #9468
2023-11-14 16:36:15 -06:00
Steven R. Loomis
78c2ac4aec feat(core): ldml marker normalization 🙀
- refactor out backspace processing into a function.
- for now, just drop any markers in the context when we're lopping off the end

For: #9468
2023-11-14 07:28:41 -06:00
Steven R. Loomis
a69feb583f feat(core): ldml marker normalization 🙀
- km::kbp is soooo last month!
- test_transforms can run NFD with markers, with some caveats.

For: #9468
2023-11-14 07:28:10 -06:00
Steven R. Loomis
015a738226 feat(core): ldml marker normalization 🙀
- add a new remove_markers(std::u32string) function
- add test cases for text utils
- update (failing) test cases for transform
- improve documentation of append process
- support KM_CORE_BT_UNKNOWN in ldml test
- remove_markers with a map
- update normalize test

For: #9468
2023-11-14 07:26:47 -06:00
Eberhard Beilharz
f4d8529789
chore(linux): rename keyboardprocessor_ldml.* to keyman_core_ldml 2023-10-20 17:30:10 +02:00
Eberhard Beilharz
7369b22599
chore(linux): Merge with chore/linux/9733_RenameLibknmkbp
# Conflicts:
#	core/src/debuglog.h
#	core/src/ldml/ldml_processor.cpp
#	core/src/ldml/ldml_transforms.cpp
#	core/src/ldml/ldml_transforms.hpp
#	core/tests/unit/ldml/ldml.cpp
2023-10-20 12:52:38 +02:00
Eberhard Beilharz
604fbc1a75
chore(linux): Rename namespace kbp to core 2023-10-19 10:39:55 +02:00
Steven R. Loomis
9dbf70c94e fix(core): ldml more updates to normalization per review 🙀
- verify marker string as it's popped off
- add a missing pragma to debuglog.h
- move UErrorCode out of ldml_processor, change normalization functions to return true on success

For: #9468
2023-10-18 17:47:56 -05:00
Steven R. Loomis
93e1c18af2 fix(core): ldml update to normalization per review 🙀
For: #9468

Co-authored-by: Marc Durdin <marc@durdin.net>
2023-10-16 20:22:52 -05:00
Steven R. Loomis
a577bd2f53 feat(core): ldml normalization 🙀
- more progress in the normalization pipeline. Trying to keep it from leaking.
- normalize output to NFC.
- normalize json test data to NFC (both context and expected).
- we do NOT try to normalize the 'embedded' strings currently.
- yet more test data fixes (turns out u+00e0 ≠ u+00e8)

For: #9468
2023-10-12 16:57:37 -05:00
Steven R. Loomis
e64f59613a feat(core): ldml normalization 🙀
- split out 'process_key_string'

For: #9468
2023-10-12 10:10:51 -05:00
Steven R. Loomis
daa0523dde feat(core): ldml normalization 🙀
- 0 steps forward, 4294967294 steps back

For: #9468
2023-10-10 19:01:14 -05:00
Marc Durdin
db76e2541d chore: rename KM_KBP_ to KM_CORE_ 2023-10-09 14:53:22 +07:00
Marc Durdin
ba5f18defd chore: rename km_kbp_ to km_core_ 2023-10-09 14:53:13 +07:00
Steven R. Loomis
9f033278d4 feat(core): marker validation 🙀
- validate marker number when going in or out of the context

#9119
2023-08-11 15:09:54 -05:00
Steven R. Loomis
2bf22c440d feat(core): marker fixes per review comments 🙀
- simplify and improve context processing

For: #9119

Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 13:30:29 -05:00
Steven R. Loomis
66ee68d2b3
Apply suggestions from code review
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 09:36:16 -05:00
Steven R. Loomis
c23303ee23
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-08-09 12:52:28 -05:00
Steven R. Loomis
f0c81687f6 feat(core): marker fixes per review comments 🙀
- move prepend_marker() to inline
- clear the string in context_to_string()

For: #9119
2023-08-09 12:51:35 -05:00
Steven R. Loomis
290bcede2f feat(core): improve documentation on transform/marker processing loop 🙀
For: #9119
2023-08-04 16:12:02 -05:00
Steven R. Loomis
1480e4d934 feat(core): improve emit_text() marker processing loop 🙀
For: #9119
2023-08-04 16:03:13 -05:00
Steven R. Loomis
fbb874fbfe feat(core): warning fix 🙀
For: #9119
2023-08-04 15:29:19 -05:00
Steven R. Loomis
a9badff183 feat(core): remove reference to CODE_DEADKEY 🙀
For: #9119
2023-08-04 15:21:38 -05:00
Steven R. Loomis
b80b43e1e9 feat(core): re-refactor transform processing logic 🙀
For: #9119
2023-08-04 15:20:00 -05:00
Steven R. Loomis
8f4cf8c24e feat(core): refactor sentinel 🙀
- and break all the things

For: #9119
2023-08-03 18:52:53 -05:00
Steven R. Loomis
20d1aad679 feat(core): refactor char emit 🙀
- and add an assert that UC_SENTINEL doesn't go into the text stream

For: #9119
2023-08-03 17:39:09 -05:00
Steven R. Loomis
cd3116d7cb feat(core): improve processing logic 🙀
- improve the ctxt managing logic a little bit.
- need to handle markers as markers, though.

#9119
2023-08-03 15:20:37 -05:00
Steven R. Loomis
c7881d99b2
Update core/src/ldml/ldml_processor.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-28 21:55:39 -05:00
Steven R. Loomis
875d59987a chore(core): update documentation in transform logic and processor 🙀
- drop some unneeded variables also, should be no semantic change

For: #7375
2023-07-26 16:16:57 -05:00
Steven R. Loomis
0a2aa6c5ca
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-25 16:30:56 -05:00
Steven R. Loomis
bc378dcac6 feat(core): move transform processing to u32 🙀
- merge code paths…

#7375
2023-07-18 19:16:23 -05:00
Steven R. Loomis
83bb7e6f12
Merge branch 'master' into feat/developer-9050-utf32-epic-ldml 2023-06-23 17:31:43 -05:00
Steven R. Loomis
de90644d7a feat(core,common,developer): support UTF-32 single chars 🙀
- C++ side now
- updated tests
- refactored escaping code and squashed bugs
- still todo: basic.xml

For: #9050
2023-06-23 16:55:05 -05:00