Commit graph

88 commits

Author SHA1 Message Date
Steven R. Loomis
84cc59c174 feat(core): ldml improve key-not-found 🙀
- restructure keyup/keydown code some
- new utility function emit_invalidate_passthrough_keystroke()

For: #9451
2023-11-29 14:29:26 -06:00
Steven R. Loomis
11334ed7fa feat(core): ldml improve key-not-found 🙀
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no

For: #9451
2023-11-28 18:07:46 -06:00
Steven R. Loomis
e9e5665cb8 feat(core): ldml backspace transform 🙀
- refactor transform/normalization inner routine

For: #9450
2023-11-16 12:27:23 -06:00
Steven R. Loomis
d0c8ea5c88 feat(core): ldml backspace transform 🙀
- refactor keypress some (needs more)
- copypasta implementation of backspace transform

For: #9450
2023-11-16 12:25:15 -06:00
Steven R. Loomis
2f843d98f3 feat(core): marker normalization 🙀
- go back to NFD for the context, for now
- anticipating when the privatecontext is NFD but the public context is NFC
- also update the test cases

For: #9468
2023-11-14 16:36:15 -06:00
Steven R. Loomis
78c2ac4aec feat(core): ldml marker normalization 🙀
- refactor out backspace processing into a function.
- for now, just drop any markers in the context when we're lopping off the end

For: #9468
2023-11-14 07:28:41 -06:00
Steven R. Loomis
a69feb583f feat(core): ldml marker normalization 🙀
- km::kbp is soooo last month!
- test_transforms can run NFD with markers, with some caveats.

For: #9468
2023-11-14 07:28:10 -06:00
Steven R. Loomis
015a738226 feat(core): ldml marker normalization 🙀
- add a new remove_markers(std::u32string) function
- add test cases for text utils
- update (failing) test cases for transform
- improve documentation of append process
- support KM_CORE_BT_UNKNOWN in ldml test
- remove_markers with a map
- update normalize test

For: #9468
2023-11-14 07:26:47 -06:00
Eberhard Beilharz
f4d8529789
chore(linux): rename keyboardprocessor_ldml.* to keyman_core_ldml 2023-10-20 17:30:10 +02:00
Eberhard Beilharz
7369b22599
chore(linux): Merge with chore/linux/9733_RenameLibknmkbp
# Conflicts:
#	core/src/debuglog.h
#	core/src/ldml/ldml_processor.cpp
#	core/src/ldml/ldml_transforms.cpp
#	core/src/ldml/ldml_transforms.hpp
#	core/tests/unit/ldml/ldml.cpp
2023-10-20 12:52:38 +02:00
Eberhard Beilharz
604fbc1a75
chore(linux): Rename namespace kbp to core 2023-10-19 10:39:55 +02:00
Steven R. Loomis
9dbf70c94e fix(core): ldml more updates to normalization per review 🙀
- verify marker string as it's popped off
- add a missing pragma to debuglog.h
- move UErrorCode out of ldml_processor, change normalization functions to return true on success

For: #9468
2023-10-18 17:47:56 -05:00
Steven R. Loomis
93e1c18af2 fix(core): ldml update to normalization per review 🙀
For: #9468

Co-authored-by: Marc Durdin <marc@durdin.net>
2023-10-16 20:22:52 -05:00
Steven R. Loomis
a577bd2f53 feat(core): ldml normalization 🙀
- more progress in the normalization pipeline. Trying to keep it from leaking.
- normalize output to NFC.
- normalize json test data to NFC (both context and expected).
- we do NOT try to normalize the 'embedded' strings currently.
- yet more test data fixes (turns out u+00e0 ≠ u+00e8)

For: #9468
2023-10-12 16:57:37 -05:00
Steven R. Loomis
e64f59613a feat(core): ldml normalization 🙀
- split out 'process_key_string'

For: #9468
2023-10-12 10:10:51 -05:00
Steven R. Loomis
daa0523dde feat(core): ldml normalization 🙀
- 0 steps forward, 4294967294 steps back

For: #9468
2023-10-10 19:01:14 -05:00
Marc Durdin
db76e2541d chore: rename KM_KBP_ to KM_CORE_ 2023-10-09 14:53:22 +07:00
Marc Durdin
ba5f18defd chore: rename km_kbp_ to km_core_ 2023-10-09 14:53:13 +07:00
Steven R. Loomis
9f033278d4 feat(core): marker validation 🙀
- validate marker number when going in or out of the context

#9119
2023-08-11 15:09:54 -05:00
Steven R. Loomis
2bf22c440d feat(core): marker fixes per review comments 🙀
- simplify and improve context processing

For: #9119

Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 13:30:29 -05:00
Steven R. Loomis
66ee68d2b3
Apply suggestions from code review
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-08-10 09:36:16 -05:00
Steven R. Loomis
c23303ee23
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-08-09 12:52:28 -05:00
Steven R. Loomis
f0c81687f6 feat(core): marker fixes per review comments 🙀
- move prepend_marker() to inline
- clear the string in context_to_string()

For: #9119
2023-08-09 12:51:35 -05:00
Steven R. Loomis
290bcede2f feat(core): improve documentation on transform/marker processing loop 🙀
For: #9119
2023-08-04 16:12:02 -05:00
Steven R. Loomis
1480e4d934 feat(core): improve emit_text() marker processing loop 🙀
For: #9119
2023-08-04 16:03:13 -05:00
Steven R. Loomis
fbb874fbfe feat(core): warning fix 🙀
For: #9119
2023-08-04 15:29:19 -05:00
Steven R. Loomis
a9badff183 feat(core): remove reference to CODE_DEADKEY 🙀
For: #9119
2023-08-04 15:21:38 -05:00
Steven R. Loomis
b80b43e1e9 feat(core): re-refactor transform processing logic 🙀
For: #9119
2023-08-04 15:20:00 -05:00
Steven R. Loomis
8f4cf8c24e feat(core): refactor sentinel 🙀
- and break all the things

For: #9119
2023-08-03 18:52:53 -05:00
Steven R. Loomis
20d1aad679 feat(core): refactor char emit 🙀
- and add an assert that UC_SENTINEL doesn't go into the text stream

For: #9119
2023-08-03 17:39:09 -05:00
Steven R. Loomis
cd3116d7cb feat(core): improve processing logic 🙀
- improve the ctxt managing logic a little bit.
- need to handle markers as markers, though.

#9119
2023-08-03 15:20:37 -05:00
Steven R. Loomis
c7881d99b2
Update core/src/ldml/ldml_processor.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-28 21:55:39 -05:00
Steven R. Loomis
875d59987a chore(core): update documentation in transform logic and processor 🙀
- drop some unneeded variables also, should be no semantic change

For: #7375
2023-07-26 16:16:57 -05:00
Steven R. Loomis
0a2aa6c5ca
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-07-25 16:30:56 -05:00
Steven R. Loomis
bc378dcac6 feat(core): move transform processing to u32 🙀
- merge code paths…

#7375
2023-07-18 19:16:23 -05:00
Steven R. Loomis
83bb7e6f12
Merge branch 'master' into feat/developer-9050-utf32-epic-ldml 2023-06-23 17:31:43 -05:00
Steven R. Loomis
de90644d7a feat(core,common,developer): support UTF-32 single chars 🙀
- C++ side now
- updated tests
- refactored escaping code and squashed bugs
- still todo: basic.xml

For: #9050
2023-06-23 16:55:05 -05:00
Steven R. Loomis
d7e0c286e4
Update core/src/ldml/ldml_processor.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-06-23 10:32:33 -05:00
Steven R. Loomis
9ce4f0eaa8 fix(core): fix ldml enter 🙀
- pass unknown keys back to OS

#7377
2023-06-22 18:43:34 -05:00
Steven R. Loomis
6e97c40f26 fix(core): fix ldml backspace 🙀
- don't pop state->context() if we know it is empty

#7377
2023-06-22 18:09:53 -05:00
Steven R. Loomis
877070d07e fix(core): fix ldml transforms 🙀
- document transform calling code in ldml_transform
- remove obsolete portions
- optimize keystroke code when no transforms are present
2023-06-22 17:55:18 -05:00
Steven R. Loomis
903fd7e82b feat(core): transform fixes 🙀
- refactor matching per review comments
- move global fcn into class statics

#7377
2023-06-19 18:27:36 -05:00
Steven R. Loomis
c74be813b1
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-06-16 20:10:50 -05:00
Steven R. Loomis
c90c7aaa64 feat(core): non-regex implementation of tran 🙀
- some scaffolding for loading transforms
- add transform_test
- simple test, simple impl - non-regex, simple strings
- scaffolding for bksp
- bksp transform test, but disabled for now

For: #7377
2023-06-14 10:14:33 -05:00
Steven R. Loomis
46b4b7f5cc feat(core): fixups to C++ side 🙀
- builds, passes tests for now

For: #7377
2023-06-06 18:38:44 -05:00
Steven R. Loomis
e1e5eea469 feat(core): ldml rename key2 to keys 🙀
- So in summary, since before #7532 basically keys was restructured, and the old 'keys' was moved into 'keys.kmap'.

#8069
2023-02-20 17:40:34 -06:00
Steven R. Loomis
8ea87aa828 feat(common): ldml: merge keys into key2 🙀 2023-02-16 12:29:19 -06:00
Steven R. Loomis
7c02ab85af feat(core): ldml C++ feature support 🙀
- make ldml.cpp print FAILED on failure especially because of https://github.com/microsoft/vscode-cpptools/issues/487
- add vkey validation (#7135)
- add disp validation
- fix: a latent bug in disp! disp.count was === vkey.count, oops. Fix and test cases.

For: #7985
2023-02-14 09:23:03 -06:00
Steven R. Loomis
57d4dfea75
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-01-07 22:46:41 -06:00
Steven R. Loomis
2f44780e87 fix(core): ldml: add asserts to C++ 🙀
- found a logic bug in kmx_plus.cpp

Fixes #7984
2023-01-06 16:54:24 -06:00