Steven R. Loomis
84cc59c174
feat(core): ldml improve key-not-found 🙀
...
- restructure keyup/keydown code some
- new utility function emit_invalidate_passthrough_keystroke()
For: #9451
2023-11-29 14:29:26 -06:00
Steven R. Loomis
11334ed7fa
feat(core): ldml improve key-not-found 🙀
...
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no
For: #9451
2023-11-28 18:07:46 -06:00
Steven R. Loomis
e9e5665cb8
feat(core): ldml backspace transform 🙀
...
- refactor transform/normalization inner routine
For: #9450
2023-11-16 12:27:23 -06:00
Steven R. Loomis
d0c8ea5c88
feat(core): ldml backspace transform 🙀
...
- refactor keypress some (needs more)
- copypasta implementation of backspace transform
For: #9450
2023-11-16 12:25:15 -06:00
Steven R. Loomis
2f843d98f3
feat(core): marker normalization 🙀
...
- go back to NFD for the context, for now
- anticipating when the privatecontext is NFD but the public context is NFC
- also update the test cases
For: #9468
2023-11-14 16:36:15 -06:00
Steven R. Loomis
78c2ac4aec
feat(core): ldml marker normalization 🙀
...
- refactor out backspace processing into a function.
- for now, just drop any markers in the context when we're lopping off the end
For: #9468
2023-11-14 07:28:41 -06:00
Steven R. Loomis
a69feb583f
feat(core): ldml marker normalization 🙀
...
- km::kbp is soooo last month!
- test_transforms can run NFD with markers, with some caveats.
For: #9468
2023-11-14 07:28:10 -06:00
Steven R. Loomis
015a738226
feat(core): ldml marker normalization 🙀
...
- add a new remove_markers(std::u32string) function
- add test cases for text utils
- update (failing) test cases for transform
- improve documentation of append process
- support KM_CORE_BT_UNKNOWN in ldml test
- remove_markers with a map
- update normalize test
For: #9468
2023-11-14 07:26:47 -06:00
Eberhard Beilharz
f4d8529789
chore(linux): rename keyboardprocessor_ldml.* to keyman_core_ldml
2023-10-20 17:30:10 +02:00
Eberhard Beilharz
7369b22599
chore(linux): Merge with chore/linux/9733_RenameLibknmkbp
...
# Conflicts:
# core/src/debuglog.h
# core/src/ldml/ldml_processor.cpp
# core/src/ldml/ldml_transforms.cpp
# core/src/ldml/ldml_transforms.hpp
# core/tests/unit/ldml/ldml.cpp
2023-10-20 12:52:38 +02:00
Eberhard Beilharz
604fbc1a75
chore(linux): Rename namespace kbp to core
2023-10-19 10:39:55 +02:00
Steven R. Loomis
9dbf70c94e
fix(core): ldml more updates to normalization per review 🙀
...
- verify marker string as it's popped off
- add a missing pragma to debuglog.h
- move UErrorCode out of ldml_processor, change normalization functions to return true on success
For: #9468
2023-10-18 17:47:56 -05:00
Steven R. Loomis
93e1c18af2
fix(core): ldml update to normalization per review 🙀
...
For: #9468
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-10-16 20:22:52 -05:00
Steven R. Loomis
a577bd2f53
feat(core): ldml normalization 🙀
...
- more progress in the normalization pipeline. Trying to keep it from leaking.
- normalize output to NFC.
- normalize json test data to NFC (both context and expected).
- we do NOT try to normalize the 'embedded' strings currently.
- yet more test data fixes (turns out u+00e0 ≠ u+00e8)
For: #9468
2023-10-12 16:57:37 -05:00
Steven R. Loomis
e64f59613a
feat(core): ldml normalization 🙀
...
- split out 'process_key_string'
For: #9468
2023-10-12 10:10:51 -05:00
Steven R. Loomis
daa0523dde
feat(core): ldml normalization 🙀
...
- 0 steps forward, 4294967294 steps back
For: #9468
2023-10-10 19:01:14 -05:00
Marc Durdin
db76e2541d
chore: rename KM_KBP_ to KM_CORE_
2023-10-09 14:53:22 +07:00
Marc Durdin
ba5f18defd
chore: rename km_kbp_ to km_core_
2023-10-09 14:53:13 +07:00
Steven R. Loomis
9f033278d4
feat(core): marker validation 🙀
...
- validate marker number when going in or out of the context
#9119
2023-08-11 15:09:54 -05:00
Steven R. Loomis
2bf22c440d
feat(core): marker fixes per review comments 🙀
...
- simplify and improve context processing
For: #9119
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 13:30:29 -05:00
Steven R. Loomis
66ee68d2b3
Apply suggestions from code review
...
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 09:36:16 -05:00
Steven R. Loomis
c23303ee23
Apply suggestions from code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-08-09 12:52:28 -05:00
Steven R. Loomis
f0c81687f6
feat(core): marker fixes per review comments 🙀
...
- move prepend_marker() to inline
- clear the string in context_to_string()
For: #9119
2023-08-09 12:51:35 -05:00
Steven R. Loomis
290bcede2f
feat(core): improve documentation on transform/marker processing loop 🙀
...
For: #9119
2023-08-04 16:12:02 -05:00
Steven R. Loomis
1480e4d934
feat(core): improve emit_text() marker processing loop 🙀
...
For: #9119
2023-08-04 16:03:13 -05:00
Steven R. Loomis
fbb874fbfe
feat(core): warning fix 🙀
...
For: #9119
2023-08-04 15:29:19 -05:00
Steven R. Loomis
a9badff183
feat(core): remove reference to CODE_DEADKEY 🙀
...
For: #9119
2023-08-04 15:21:38 -05:00
Steven R. Loomis
b80b43e1e9
feat(core): re-refactor transform processing logic 🙀
...
For: #9119
2023-08-04 15:20:00 -05:00
Steven R. Loomis
8f4cf8c24e
feat(core): refactor sentinel 🙀
...
- and break all the things
For: #9119
2023-08-03 18:52:53 -05:00
Steven R. Loomis
20d1aad679
feat(core): refactor char emit 🙀
...
- and add an assert that UC_SENTINEL doesn't go into the text stream
For: #9119
2023-08-03 17:39:09 -05:00
Steven R. Loomis
cd3116d7cb
feat(core): improve processing logic 🙀
...
- improve the ctxt managing logic a little bit.
- need to handle markers as markers, though.
#9119
2023-08-03 15:20:37 -05:00
Steven R. Loomis
c7881d99b2
Update core/src/ldml/ldml_processor.cpp
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-28 21:55:39 -05:00
Steven R. Loomis
875d59987a
chore(core): update documentation in transform logic and processor 🙀
...
- drop some unneeded variables also, should be no semantic change
For: #7375
2023-07-26 16:16:57 -05:00
Steven R. Loomis
0a2aa6c5ca
Apply suggestions from code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-25 16:30:56 -05:00
Steven R. Loomis
bc378dcac6
feat(core): move transform processing to u32 🙀
...
- merge code paths…
#7375
2023-07-18 19:16:23 -05:00
Steven R. Loomis
83bb7e6f12
Merge branch 'master' into feat/developer-9050-utf32-epic-ldml
2023-06-23 17:31:43 -05:00
Steven R. Loomis
de90644d7a
feat(core,common,developer): support UTF-32 single chars 🙀
...
- C++ side now
- updated tests
- refactored escaping code and squashed bugs
- still todo: basic.xml
For: #9050
2023-06-23 16:55:05 -05:00
Steven R. Loomis
d7e0c286e4
Update core/src/ldml/ldml_processor.cpp
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-06-23 10:32:33 -05:00
Steven R. Loomis
9ce4f0eaa8
fix(core): fix ldml enter 🙀
...
- pass unknown keys back to OS
#7377
2023-06-22 18:43:34 -05:00
Steven R. Loomis
6e97c40f26
fix(core): fix ldml backspace 🙀
...
- don't pop state->context() if we know it is empty
#7377
2023-06-22 18:09:53 -05:00
Steven R. Loomis
877070d07e
fix(core): fix ldml transforms 🙀
...
- document transform calling code in ldml_transform
- remove obsolete portions
- optimize keystroke code when no transforms are present
2023-06-22 17:55:18 -05:00
Steven R. Loomis
903fd7e82b
feat(core): transform fixes 🙀
...
- refactor matching per review comments
- move global fcn into class statics
#7377
2023-06-19 18:27:36 -05:00
Steven R. Loomis
c74be813b1
Apply suggestions from code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-06-16 20:10:50 -05:00
Steven R. Loomis
c90c7aaa64
feat(core): non-regex implementation of tran 🙀
...
- some scaffolding for loading transforms
- add transform_test
- simple test, simple impl - non-regex, simple strings
- scaffolding for bksp
- bksp transform test, but disabled for now
For: #7377
2023-06-14 10:14:33 -05:00
Steven R. Loomis
46b4b7f5cc
feat(core): fixups to C++ side 🙀
...
- builds, passes tests for now
For: #7377
2023-06-06 18:38:44 -05:00
Steven R. Loomis
e1e5eea469
feat(core): ldml rename key2 to keys 🙀
...
- So in summary, since before #7532 basically keys was restructured, and the old 'keys' was moved into 'keys.kmap'.
#8069
2023-02-20 17:40:34 -06:00
Steven R. Loomis
8ea87aa828
feat(common): ldml: merge keys into key2 🙀
2023-02-16 12:29:19 -06:00
Steven R. Loomis
7c02ab85af
feat(core): ldml C++ feature support 🙀
...
- make ldml.cpp print FAILED on failure especially because of https://github.com/microsoft/vscode-cpptools/issues/487
- add vkey validation (#7135 )
- add disp validation
- fix: a latent bug in disp! disp.count was === vkey.count, oops. Fix and test cases.
For: #7985
2023-02-14 09:23:03 -06:00
Steven R. Loomis
57d4dfea75
Apply suggestions from code review
...
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-01-07 22:46:41 -06:00
Steven R. Loomis
2f44780e87
fix(core): ldml: add asserts to C++ 🙀
...
- found a logic bug in kmx_plus.cpp
Fixes #7984
2023-01-06 16:54:24 -06:00