Commit graph

1001 commits

Author SHA1 Message Date
Steven R. Loomis
263d0e32f2 chore(core): update comments and remove a raw numeric literal
- per review comments

Fixes: #9467

Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2024-06-13 12:17:55 -05:00
Steven R. Loomis
f156a7264d feat(core): generator in core/src for util_normalize_table.h
- temporary header file generated by wasm during build
- built using icu
- test_unicode verifies the contents and synchronization with running ICU.
- this is used by util_normalize to provide normalization properties under wasm without needing to include ICU.

Fixes: #9467
2024-06-05 15:47:20 -05:00
Steven R. Loomis
fdb2e95d20 feat(core): build improvements for --update-unicode option
Fixes: #9467
2024-06-04 12:26:00 -05:00
Steven R. Loomis
e9673867fa feat(core): speedup NFD boundary table
- use RLE encoding, thanks @mcdurdin
- much smaller table and faster lookup

Fixes: #9467
2024-06-04 12:23:16 -05:00
Steven R. Loomis
7fbeea85cc Merge remote-tracking branch 'upstream/master' into feat/core/9467-devolve-norm-to-js3-epic-ldml 2024-06-04 11:15:31 -05:00
Steven R. Loomis
53a6638a70 feat(core): generate and use static table in wasm for NFD boundary
- add core/tools build tree with custom targets
- add to core/build.sh to generate nfd_table.h
- test_unicode to validate Unicode version and compare NFD to actual ICU
- currently, linear search of the table.
2024-06-04 11:15:17 -05:00
Steven R. Loomis
31dedb7ca3 test(core): comments in get_block_unicode_ver() 2024-05-31 16:14:05 -05:00
Steven R. Loomis
403d342e10 test(core): clarify output of test_unicode
- group output by functional area

#10183
2024-05-27 16:42:11 -05:00
Steven R. Loomis
5d420248eb feat(core): move more normalization logic into JS
- major redo of actions_normalize - into UTF-32 and not using ICU directly
- add some utilities: u32len, u32dup, context_items_from_utf32

#9467
2024-05-24 18:20:30 -05:00
Marc Durdin
6abf577f21
Merge branch 'feat/core/9467-devolve-regex-to-js-epic-ldml' into feat/core/9467-devolve-norm-to-js2-epic-ldml 2024-05-25 05:41:01 +07:00
Marc Durdin
d37b64be5e
Merge branch 'chore/common/10183-unicode-version' into feat/core/9467-devolve-regex-to-js-epic-ldml 2024-05-25 05:40:42 +07:00
Steven R. Loomis
a80a0a7dcb feat(core): move more normalization logic into JS
- add a normalize_nfd() which takes a single codepoint
- temporarily keep ICU in actions_normalize.cpp and ldml_transforms.cpp
- expand wasm opts in unit tests
2024-05-24 10:42:57 -05:00
Steven R. Loomis
465c4bfdc0 feat(core): add a KMN_NO_ICU internal switch to start being able to turn off ICU
- always set to 0 for now (keep ICU around)
- set KMN_IN_LDML_TESTS in tests to keep ICU there for test and comparison
- add core_icu.cpp and put some utils there.

#9467
2024-05-24 09:24:53 -05:00
Steven R. Loomis
e48f6c39e1 test(core): update per review comments
Fixes: #10183

Co-authored-by: Marc Durdin <marc@durdin.net>
2024-05-23 22:19:36 -05:00
Steven R. Loomis
43cd828e9d feat(core): change normalize_nfd() to use JS native call instead of ICU
- does not hit all ICU usage yet

#9467
2024-05-23 17:09:00 -05:00
Steven R. Loomis
d8b5c25954 chore(core): refactor to move most normalization calls into util_normalize.cpp
- fold ldml_utils.hpp into core_icu.h
- refactor to use km::core::util::normalize_nfd() functions where simple to do so

For #9467
2024-05-23 14:47:38 -05:00
Steven R. Loomis
f3a80f50a6 test(core): build fix for test_unicode
- incorporate change from #11483

Fixes: #10183
2024-05-23 11:26:16 -05:00
Steven R. Loomis
e5ee26aaea fix(core): add a test to verify ICU and Unicode version
- load version data from node.js, Blocks.txt, and ICU4C
- support wasm: copy package.json, nodeversions.json and Blocks.txt into the keyboard area so that they can be mounted under wasm

also:
- rename 'fallback' macro to KMN_FALLBACK to not conflict with hedley in utfcodec.hpp
- fix ambiguous path type in tests

Fixes: #10183
2024-05-23 09:31:15 -05:00
Marc Durdin
1e66b6ed0e
Merge pull request #11421 from keymanapp/chore/developer/11317-build.sh
chore(developer): build.sh for developer
2024-05-23 14:22:54 +07:00
Marc Durdin
9465c3e0b4 refactor(common): move enabled declaration into test_color.cpp
Moves variable declaration for `bool enabled` from .h to .cpp and fixes
up builds. Fixes `#include` in ldml_test_source.cpp which was inside a
namespace.

Fixes: #10426
2024-05-20 15:42:41 +07:00
Marc Durdin
ee0fc0ce14 chore(developer): cleanup npm publish
Use a new flag `--npm-publish` in conjunction with `publish` action,
so that the default will always be to `npm pack` if the new flag is not
specified. This flag is also guarded in the actual npm publish code to
ensure that it can only run in the appropriate CI alpha/beta/stable
environment, and not in local or test.

This then also removes the separate `pack` action.

Also removes legacy boilerplate from a number of build scripts.
2024-05-20 06:34:57 +07:00
Steven R. Loomis
d3c9fbbadc chore(core): update core to C++17 fixes
- remove some tangled templates in jsonpp
- remove one unneeded include of sstream, and rewrite another as a string conversion.

Fixes: #8800
2024-05-13 17:50:20 -05:00
Steven R. Loomis
02413c0251 chore(core): update core to C++17
- use utfcodec.hpp's convert<>() instead of codecvt

Fixes: #8800
2024-05-08 16:41:30 -05:00
Steven R. Loomis
1fe76bcd26
Merge branch 'master' into chore/core/11033-points-not-units 2024-05-07 09:26:07 -05:00
Steven R. Loomis
5e8b4a1a17 chre(linux): bump libkeymancore version
- also minor typo in docs
2024-05-07 08:55:48 -05:00
Steven R. Loomis
6706ae0957 chore(core): km_core_cp -> km_core_cu
- km_core_cp represents a 16 bit code unit, not a code point.

Fixes: #11033
2024-05-02 17:39:15 -05:00
Marc Durdin
cd8121ef65 chore(common): builder scripts now use /resources/build/builder.inc.sh
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).

This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
2024-05-02 15:54:31 +07:00
Marc Durdin
0f390d47cc chore(common): maintenance on build scripts - cd
Fixes #11324.

* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
2024-05-02 15:54:29 +07:00
Marc Durdin
93d4e60e63
Merge pull request #11254 from keymanapp/fix/core/10955-double-marker-on-delete
fix(core): ldml backspace processing should delete all markers 🙀
2024-04-26 18:36:08 +10:00
Marc Durdin
a5765d0cc9
Merge pull request #11172 from keymanapp/fix/core/10955-reset-on-frame
fix(core): reset on frame keys 🙀
2024-04-26 18:35:46 +10:00
Steven R. Loomis
44f0dea3cd
Merge branch 'fix/core/10955-reset-on-frame' into fix/core/10955-double-marker-on-delete 2024-04-25 16:59:30 -05:00
Steven R. Loomis
fa1fd75157 fix(core): ldml fix for multiple marker deletion
- current code only deletes a single marker and falls through
- update the ldml test code, get rid of 'expected character' backspace logic (now that we have context object)
- update test cases
2024-04-25 16:59:20 -05:00
Steven R. Loomis
cd867ba801 fix(core): update vkey_should_invalidate_context per comment
Fixes #10955
2024-04-25 08:58:47 -05:00
Steven R. Loomis
f516538b31 chore(core): outdent test file per review comment 2024-04-24 08:39:35 -05:00
Steven R. Loomis
ba8dcad949
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-04-24 08:22:41 -05:00
Steven R. Loomis
ebc914cfa0 chore(core): data update from CLDR v45
- fix the name of the test azerty keyboard
2024-04-19 17:34:27 -05:00
Steven R. Loomis
7fb97efa43 fix(developer): escape plus and asterisk sign
- additional tests for asterisk and plus

Fixes: #11233
2024-04-19 16:37:16 -05:00
Steven R. Loomis
f5f406b74a fix(developer): escape plus sign
- asterisk also

Fixes: #11233
2024-04-19 15:16:46 -05:00
Steven R. Loomis
5d7361a051 fix(developer): escape plus sign
- improve test case in core

Fixes: #11233
2024-04-19 15:05:58 -05:00
Steven R. Loomis
f0ff05a9a9 fix(developer): escape plus sign
Fixes: #11233
2024-04-19 13:58:31 -05:00
Steven R. Loomis
f77bbb2418 fix(core): update state_should_invalidate_context
- don't invalidate on keyup
- don't check context()->empty()
- clean up structure and comments

Fixes: #10955
2024-04-18 10:34:21 -05:00
Steven R. Loomis
dc965f4e1a fix(core): ldml backspace processing should delete all markers
- current code would only delete a single marker and fall through
- should loop consuming all markers until a non-marker char is hit
2024-04-18 08:57:19 -05:00
Steven R. Loomis
4f2ef499ea fix(core): update core to handle reset context
- clear the app context as well as the cached context

Fixes: #10955
2024-04-17 10:31:33 -05:00
Steven R. Loomis
37530acb14 fix(core): update core to handle reset context
- fix range check vk to use a static instead of c++17 function
2024-04-15 20:32:49 -05:00
Steven R. Loomis
4f8c9eb983 fix(core): update core to handle reset context
- range check vk
- split modifier test out to separate function
2024-04-15 10:49:41 -05:00
Steven R. Loomis
bd10290af3 fix(core): update core to handle reset context
- fixup comment
2024-04-11 23:34:20 -05:00
Steven R. Loomis
46239c6a96
Update core/src/state.hpp
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2024-04-11 22:27:02 -05:00
Steven R. Loomis
9ca4e99fb7 fix(core): update core to handle reset context
- mark event flags as unused for now

Fixes: #10955
2024-04-08 09:35:35 -05:00
Steven R. Loomis
51c93acb38 fix(core): update core to handle reset context
- it was the check for deletion that was unneeded

Fixes: #10955
2024-04-05 18:33:37 -05:00
Steven R. Loomis
111a50ccf3 fix(core): dx: denoise test 2024-04-05 18:27:41 -05:00