Commit graph

447 commits

Author SHA1 Message Date
Steven R. Loomis
f156a7264d feat(core): generator in core/src for util_normalize_table.h
- temporary header file generated by wasm during build
- built using icu
- test_unicode verifies the contents and synchronization with running ICU.
- this is used by util_normalize to provide normalization properties under wasm without needing to include ICU.

Fixes: #9467
2024-06-05 15:47:20 -05:00
Steven R. Loomis
7fbeea85cc Merge remote-tracking branch 'upstream/master' into feat/core/9467-devolve-norm-to-js3-epic-ldml 2024-06-04 11:15:31 -05:00
Steven R. Loomis
53a6638a70 feat(core): generate and use static table in wasm for NFD boundary
- add core/tools build tree with custom targets
- add to core/build.sh to generate nfd_table.h
- test_unicode to validate Unicode version and compare NFD to actual ICU
- currently, linear search of the table.
2024-06-04 11:15:17 -05:00
Steven R. Loomis
31dedb7ca3 test(core): comments in get_block_unicode_ver() 2024-05-31 16:14:05 -05:00
Steven R. Loomis
403d342e10 test(core): clarify output of test_unicode
- group output by functional area

#10183
2024-05-27 16:42:11 -05:00
Marc Durdin
6abf577f21
Merge branch 'feat/core/9467-devolve-regex-to-js-epic-ldml' into feat/core/9467-devolve-norm-to-js2-epic-ldml 2024-05-25 05:41:01 +07:00
Marc Durdin
d37b64be5e
Merge branch 'chore/common/10183-unicode-version' into feat/core/9467-devolve-regex-to-js-epic-ldml 2024-05-25 05:40:42 +07:00
Steven R. Loomis
a80a0a7dcb feat(core): move more normalization logic into JS
- add a normalize_nfd() which takes a single codepoint
- temporarily keep ICU in actions_normalize.cpp and ldml_transforms.cpp
- expand wasm opts in unit tests
2024-05-24 10:42:57 -05:00
Steven R. Loomis
465c4bfdc0 feat(core): add a KMN_NO_ICU internal switch to start being able to turn off ICU
- always set to 0 for now (keep ICU around)
- set KMN_IN_LDML_TESTS in tests to keep ICU there for test and comparison
- add core_icu.cpp and put some utils there.

#9467
2024-05-24 09:24:53 -05:00
Steven R. Loomis
e48f6c39e1 test(core): update per review comments
Fixes: #10183

Co-authored-by: Marc Durdin <marc@durdin.net>
2024-05-23 22:19:36 -05:00
Steven R. Loomis
43cd828e9d feat(core): change normalize_nfd() to use JS native call instead of ICU
- does not hit all ICU usage yet

#9467
2024-05-23 17:09:00 -05:00
Steven R. Loomis
d8b5c25954 chore(core): refactor to move most normalization calls into util_normalize.cpp
- fold ldml_utils.hpp into core_icu.h
- refactor to use km::core::util::normalize_nfd() functions where simple to do so

For #9467
2024-05-23 14:47:38 -05:00
Steven R. Loomis
f3a80f50a6 test(core): build fix for test_unicode
- incorporate change from #11483

Fixes: #10183
2024-05-23 11:26:16 -05:00
Steven R. Loomis
e5ee26aaea fix(core): add a test to verify ICU and Unicode version
- load version data from node.js, Blocks.txt, and ICU4C
- support wasm: copy package.json, nodeversions.json and Blocks.txt into the keyboard area so that they can be mounted under wasm

also:
- rename 'fallback' macro to KMN_FALLBACK to not conflict with hedley in utfcodec.hpp
- fix ambiguous path type in tests

Fixes: #10183
2024-05-23 09:31:15 -05:00
Marc Durdin
9465c3e0b4 refactor(common): move enabled declaration into test_color.cpp
Moves variable declaration for `bool enabled` from .h to .cpp and fixes
up builds. Fixes `#include` in ldml_test_source.cpp which was inside a
namespace.

Fixes: #10426
2024-05-20 15:42:41 +07:00
Steven R. Loomis
d3c9fbbadc chore(core): update core to C++17 fixes
- remove some tangled templates in jsonpp
- remove one unneeded include of sstream, and rewrite another as a string conversion.

Fixes: #8800
2024-05-13 17:50:20 -05:00
Steven R. Loomis
02413c0251 chore(core): update core to C++17
- use utfcodec.hpp's convert<>() instead of codecvt

Fixes: #8800
2024-05-08 16:41:30 -05:00
Steven R. Loomis
6706ae0957 chore(core): km_core_cp -> km_core_cu
- km_core_cp represents a 16 bit code unit, not a code point.

Fixes: #11033
2024-05-02 17:39:15 -05:00
Marc Durdin
93d4e60e63
Merge pull request #11254 from keymanapp/fix/core/10955-double-marker-on-delete
fix(core): ldml backspace processing should delete all markers 🙀
2024-04-26 18:36:08 +10:00
Marc Durdin
a5765d0cc9
Merge pull request #11172 from keymanapp/fix/core/10955-reset-on-frame
fix(core): reset on frame keys 🙀
2024-04-26 18:35:46 +10:00
Steven R. Loomis
44f0dea3cd
Merge branch 'fix/core/10955-reset-on-frame' into fix/core/10955-double-marker-on-delete 2024-04-25 16:59:30 -05:00
Steven R. Loomis
fa1fd75157 fix(core): ldml fix for multiple marker deletion
- current code only deletes a single marker and falls through
- update the ldml test code, get rid of 'expected character' backspace logic (now that we have context object)
- update test cases
2024-04-25 16:59:20 -05:00
Steven R. Loomis
f516538b31 chore(core): outdent test file per review comment 2024-04-24 08:39:35 -05:00
Steven R. Loomis
ba8dcad949
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-04-24 08:22:41 -05:00
Steven R. Loomis
ebc914cfa0 chore(core): data update from CLDR v45
- fix the name of the test azerty keyboard
2024-04-19 17:34:27 -05:00
Steven R. Loomis
7fb97efa43 fix(developer): escape plus and asterisk sign
- additional tests for asterisk and plus

Fixes: #11233
2024-04-19 16:37:16 -05:00
Steven R. Loomis
f5f406b74a fix(developer): escape plus sign
- asterisk also

Fixes: #11233
2024-04-19 15:16:46 -05:00
Steven R. Loomis
5d7361a051 fix(developer): escape plus sign
- improve test case in core

Fixes: #11233
2024-04-19 15:05:58 -05:00
Steven R. Loomis
f0ff05a9a9 fix(developer): escape plus sign
Fixes: #11233
2024-04-19 13:58:31 -05:00
Steven R. Loomis
111a50ccf3 fix(core): dx: denoise test 2024-04-05 18:27:41 -05:00
Steven R. Loomis
a455c5b95b fix(core): update core to handle reset context
- updates to debug_api test for rearranged items
- try to rearrange the action queue to match prior behavior

Fixes: #10955
2024-04-05 16:31:23 -05:00
Steven R. Loomis
55483e4ccc fix(core): update ldml test source to handle reset
- correct invalidate logic

Fixes: #10955
2024-04-05 14:42:52 -05:00
Steven R. Loomis
784a994f12 fix(core): update ldml test source to handle reset
Fixes: #10955
2024-04-05 13:22:47 -05:00
Steven R. Loomis
f1b0d8c837 fix(core): core to automatically reset context if a frame key pressed
- move the reset table into its own cpp
- update function signatures per review comments

Fixes: #10955
2024-04-05 11:58:47 -05:00
Steven R. Loomis
2298f11796
Merge branch 'chore/core/11112-lone-surrogate' into fix/core/10955-reset-on-frame 2024-04-05 11:15:03 -05:00
Steven R. Loomis
e43aa3c737 fix(core): skip leading trail surrogate char
- update tests per review
2024-04-05 11:07:07 -05:00
Steven R. Loomis
c91095f0ca fix(core): core to automatically reset context if a frame key pressed
- update ldml test with an exception for k_102_keytest
2024-04-04 17:12:26 -05:00
Steven R. Loomis
c86ab582a7 chore(core): reinstate context reset test in ldml
- reverts 04a6e164d7 for k_102_keytest.xml
2024-04-04 12:14:54 -05:00
Steven R. Loomis
2c4d8e34e3 fix(core): skip leading trail surrogate char in km_core_state_context_set_if_needed() 2024-04-04 12:02:17 -05:00
Steven R. Loomis
8d2fe35288
Merge pull request #11119 from keymanapp/chore/core/dx-better-embedded-test-msg
chore(core): dx better err message on embedded test vkeys 🙀
2024-04-02 16:39:15 -05:00
Steven R. Loomis
5b1c853557 fix(common,core): support other lookup for modifiers
- the 'other' keyword was incorrectly called 'default'

Fixes: #11072
2024-04-01 23:21:33 -05:00
Steven R. Loomis
054d9fd07c chore(core): dx even better err message on embedded test vkeys
- report on vk-not-found
- separate message for excess string
2024-03-28 13:21:30 -05:00
Steven R. Loomis
0f0019983e
Merge branch 'feat/core/11072-default-modifier' into chore/core/dx-better-embedded-test-msg 2024-03-28 13:12:16 -05:00
Steven R. Loomis
89f5a0e090 fix(core): support default lookup for modifiers
Fixes: #11072
2024-03-28 13:12:01 -05:00
Steven R. Loomis
4881d1aab0 chore(core): dx better err message on embedded test vkeys 2024-03-28 12:41:42 -05:00
Marc Durdin
5790f634cf
Merge pull request #11101 from keymanapp/fix/core/11067-fix-actions-normalize-utf32
fix(core): fix pointer math in actions_normalize() 🙀
2024-03-28 19:18:15 +11:00
Steven R. Loomis
1ac5bdfaa1
Merge pull request #11071 from keymanapp/fix/core/11057-string-too-long
fix(core): calculate offset correctly when replacing marker in transform (fixes crash)🙀
2024-03-28 00:12:18 -05:00
Steven R. Loomis
5f836d72f0
Merge pull request #11059 from keymanapp/fix/core/11045-assert-marker
fix(core,developer): variable/marker substitution in sets and strings🙀
2024-03-28 00:11:28 -05:00
Steven R. Loomis
adc421974c chore(core): fix actions_normalize() pointer math - tests
- fix comments yet again
- add regression test in test_actions_normalize.cpp

For: #11067
2024-03-27 23:55:01 -05:00
Steven R. Loomis
e8a8590eee fix(core): revert some interim fixes
- ldml_test_source fix moved off to issue #11083
- an optimization in ldml_processor also rolled back
2024-03-27 08:05:39 -05:00