Commit graph

126 commits

Author SHA1 Message Date
Steven R. Loomis
6d2ae221af chore(core,developer,resources): ldml update out of techpreview 🙀
- further fixes
2024-03-04 08:21:45 -06:00
Steven R. Loomis
17a653fa19 chore(core): ldml update out of techpreview 🙀
- updates to support techpreview -> 45
- test changes

Fixes: #10900
2024-03-02 22:40:39 -06:00
Steven R. Loomis
209717a24a chore(core,developer): rename unicodeSet to uset in xml
chore(developer): rename unicodeSet to uset 🙀  #10657
2024-02-27 18:11:18 -06:00
Steven R. Loomis
b1d74ce809 chore(core): describe what is being tested in each stage
Fixes: #10505
2024-02-22 16:19:21 -06:00
Steven R. Loomis
993f5252e9 chore(core): describe what is being tested in each stage
Fixes: #10505
2024-02-22 13:15:46 -06:00
Steven R. Loomis
9ef198ff19 chore(core): update sample ldml keyboard
Fixes: #10505
2024-02-20 12:47:25 -06:00
Steven R. Loomis
746af68afa chore(core): shuffle transform matcher. re-enable failing case
For #10736
2024-02-16 12:49:19 -06:00
Steven R. Loomis
503f196e42 chore(core): update test case with TODO for known issue
For #10734
2024-02-15 16:27:26 -06:00
Steven R. Loomis
e7b31113a3 chore(core): remove obsolete TODOs. Temporary change to test.
- TODO: 10734 There's a bug in the matching still. Come back and fix this, but the crasher is fixed.

For #10734
2024-02-15 16:21:53 -06:00
Steven R. Loomis
e099ffa3a1 chore(core): update ldml_test
- includes crasher from #10734

For #10505
2024-02-15 15:26:09 -06:00
Steven R. Loomis
5f01fca679 chore(core): create sample LDML keyboard that exercises effectively all functionality
- test keyboard was wrong

For #10505
2024-02-15 14:26:44 -06:00
Steven R. Loomis
e415f411e4 chore(core): create sample LDML keyboard that exercises effectively all functionality
- add some more functionality

For #10505
2024-02-15 14:26:44 -06:00
Steven R. Loomis
701291a7f7 chore(core): create sample LDML keyboard that exercises effectively all functionality
For #10505
2024-02-15 14:26:44 -06:00
Steven R. Loomis
3a538c103f feat(developer): support normalization=disabled 🙀
- test and code fixes

#10554
2024-02-01 18:40:02 -06:00
Steven R. Loomis
7572c9476e feat(developer): support normalization=disabled 🙀
- main core changes and test changes

#10554
2024-02-01 17:32:42 -06:00
Steven R. Loomis
9120bb4c87 feat(core,developer): normalization=disabled 🙀
- failing tests and initial work

#10554
2024-01-31 22:38:06 -06:00
Steven R. Loomis
b429dc3383 feat(core): markers and reorder 🙀
- reorders don't interact with markers, but must transit them
- add updated tests
- simplify reorder_group::apply(): the 'common prefix' discussion
was out-of-date as std::mismatch is handled by the ldml_processor
and the core context.
- add_back_markers() promoted to SPI so it can be called from reordering.

Fixes: #10516
2024-01-31 17:44:21 -06:00
Marc Durdin
f546c44048
Merge pull request #10390 from keymanapp/epic/core/9999-normalization
epic: Keyman Core normalization 🌱
2024-01-31 10:45:51 +11:00
Steven R. Loomis
4108ecd298 feat(core): ldml marker normalization fix 🙀
- decomp the 'glued' char before storing in map

For: #10516
2024-01-25 17:45:59 -06:00
Steven R. Loomis
04a6e164d7 feat(core): ldml actions: updates per code review 🌱
- fix backspace processing
- fix k_102 - we can no longer test for context invalidation here
- fix a test that assumed non-const km_core_actions.output

For: #10410
2024-01-19 11:24:41 -06:00
Steven R. Loomis
eb39bba374 chore(developer,core): change sample and test files to use \u{…} 🙀
- #10389 will require this, but at least change the test files

#10319
2024-01-15 17:27:41 -06:00
Steven R. Loomis
b2b897970d feat(core): stacked markers 🙀
- renamed the single-segment functions to normalize_nfd_markers_segment() since they should only be run on a single segment
- commented out NFC for now, dead code
- add some additional checks/DebugLog around normalization

#10320
2024-01-06 11:33:51 -06:00
Steven R. Loomis
763f7abeb0 chore(core): failing test with markers 🙀
- marker overrun! the 40th marker becomes a parenthesis, breaking regex

feat(core): ldml proper regex implementation 🙀🙀 #9121
2023-12-28 18:57:33 -06:00
Steven R. Loomis
ae96008c11 feat(core): ldml more regex tests 🙀
- match in a set
- quantifiers

For: #9121
2023-12-28 17:38:21 -06:00
Steven R. Loomis
53201263d9 fix(core): use marker-safe normalization in between transform stages 🙀
- the intermediate stages of transforms also need to use marker-safe normalization
- re-enable a test that was failing previously due to this

For: #9468
2023-12-22 17:51:18 -06:00
Steven R. Loomis
9d1599d557 chore(core): add some more marker tests 🙀
- one disabled for now

For #9468
2023-12-21 11:36:35 -06:00
Steven R. Loomis
12aa9a5b69 chore(core): add some more marker tests 🙀
For #9468
2023-12-20 19:23:29 -06:00
Steven R. Loomis
0b3e7a94e8 chore(core): uncomment some more tests 🙀
For #9468
2023-12-20 18:50:21 -06:00
Steven R. Loomis
1e32a2a963 feat(core): ldml improve key-not-found 🙀
- copy exception table for reset from VKScanCodes.cpp
- update test

For: #9451
2023-12-07 17:15:44 -06:00
Steven R. Loomis
392c535270 feat(core): ldml improve key-not-found 🙀
- also affects markers, feat(core): normalization per spec for transforms/etc 🙀  #9468
- keep markers in nfd context string
- fix ldml test harness to handle context reset
- update test case
- still issues with overproduction of markers in the context

For: #9451
2023-12-06 18:33:51 -06:00
Steven R. Loomis
11334ed7fa feat(core): ldml improve key-not-found 🙀
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no

For: #9451
2023-11-28 18:07:46 -06:00
Steven R. Loomis
86e8a4f609
Merge branch 'fix/developer/9438-vis-kbd-crash-epic-ldml' into fix/core-9961-optional-startContext-epic-ldml 2023-11-16 16:17:12 -06:00
Steven R. Loomis
6bc72ff294 fix(core): dx: ldml: startContext should be optional in ldml test data 🙀
- per spec
- update tests to remove these

Fixes: #9961
2023-11-16 15:51:56 -06:00
Steven R. Loomis
85a5afafc9 feat(core): ldml tertiary reorder 🙀
- failing test cases, hooray! In XML and C++

For: #9707
2023-11-16 12:31:08 -06:00
Steven R. Loomis
d0c8ea5c88 feat(core): ldml backspace transform 🙀
- refactor keypress some (needs more)
- copypasta implementation of backspace transform

For: #9450
2023-11-16 12:25:15 -06:00
Steven R. Loomis
e28f35903a feat(core): marker normalization 🙀
- test fix

For: #9468
2023-11-15 17:29:18 -06:00
Steven R. Loomis
5a04a812fa feat(core): marker normalization 🙀
- temporarily remove a backspace test until #9450 is fixed.

For: #9468
2023-11-15 17:09:26 -06:00
Steven R. Loomis
7b481945e6 feat(core): test fix 🙀
- fix a cast
- fix some test cases

For: #9468
2023-11-14 07:28:57 -06:00
Steven R. Loomis
015a738226 feat(core): ldml marker normalization 🙀
- add a new remove_markers(std::u32string) function
- add test cases for text utils
- update (failing) test cases for transform
- improve documentation of append process
- support KM_CORE_BT_UNKNOWN in ldml test
- remove_markers with a map
- update normalize test

For: #9468
2023-11-14 07:26:47 -06:00
Steven R. Loomis
efc8ef022f feat(core): ldml normalization 🙀
- some failing marker tests

For: #9468
2023-11-14 07:26:47 -06:00
Steven R. Loomis
09411eba33 feat(common): ldml update to WIP cldr data 🙀
- bn-bengali with a basic test
- fr-optimise
- note: disabling bengali reorders for now, not working properly due to tertiary reordering feat(core): tertiary reorders 🙀  #9707

For: #9916
2023-11-02 17:44:14 -05:00
Steven R. Loomis
a45dc5c345 chore(developer): ldml update to prefinal v44 🙀
-  fix binary size issue
- upstream: issue in abnt2 sample keyboard

For feat(developer): ldml re-sync with CLDR v44 spec 🙀  #9838
2023-10-26 18:21:56 -05:00
Steven R. Loomis
4d358cabda chore(developer): ldml update to prefinal v44 🙀
-  utilities for extracting key ids
-  more key validation and munging
-  touch layout
-  fix basic.txt

For feat(developer): ldml re-sync with CLDR v44 spec 🙀  #9838
2023-10-26 12:57:07 -05:00
Steven R. Loomis
870cf76049 chore(developer,core): ldml update to prefinal v44 🙀
- more data files

For feat(developer): ldml re-sync with CLDR v44 spec 🙀  #9838
2023-10-24 14:56:09 -05:00
Steven R. Loomis
5dbda001fc chore(common,core): ldml update to prefinal v44 🙀
- common passes tests
- updated data files

For feat(developer): ldml re-sync with CLDR v44 spec 🙀  #9838
2023-10-24 14:47:22 -05:00
Eberhard Beilharz
7369b22599
chore(linux): Merge with chore/linux/9733_RenameLibknmkbp
# Conflicts:
#	core/src/debuglog.h
#	core/src/ldml/ldml_processor.cpp
#	core/src/ldml/ldml_transforms.cpp
#	core/src/ldml/ldml_transforms.hpp
#	core/tests/unit/ldml/ldml.cpp
2023-10-20 12:52:38 +02:00
Eberhard Beilharz
604fbc1a75
chore(linux): Rename namespace kbp to core 2023-10-19 10:39:55 +02:00
Steven R. Loomis
55bf0642a5 feat(core): add backspace test 🙀
- test needs to be "t" not "ta"

For: #9760
2023-10-13 16:56:00 -05:00
Steven R. Loomis
95ad1c47e4 feat(core): ldml normalization 🙀
- fix unnecessary test churn.

For: #9468
2023-10-12 17:46:41 -05:00
Steven R. Loomis
84b4836bbf
Merge branch 'master' into feat/core/9468-normalization-epic-ldml 2023-10-12 16:59:18 -05:00