Commit graph

1143 commits

Author SHA1 Message Date
Eberhard Beilharz
0958a2da23
fix(core): permanently disable logging
This change disables logging at compile time to work around #12661.
Logging can be enabled in the debugger, or by re-compiling with
`g_debug_KeymanLog` set to TRUE.

Related: #12661
Cherry-pick: #12674
2024-11-27 16:36:25 +01:00
Steven R. Loomis
c306147e6b
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-27 08:59:59 -06:00
Eberhard Beilharz
3dc3f040de
feat(core): implement loading KMX from blob
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a

Part-of: #11293
Part-of: #8093
2024-11-27 15:30:05 +01:00
Eberhard Beilharz
718881a027
chore(core): rename test files
This change rename the test files for Core according to the discussion
at the Keyman conference in November 2024.
2024-11-26 17:47:11 +01:00
Steven R. Loomis
f9862c3bdf feat(core): test improvements for get_key_list()
- just compare the key list to the key2.kmap table
- check the keylist for all LdmlTestSource instances - no syntax needed

Fixes: #12298
2024-11-26 09:36:11 -06:00
Steven R. Loomis
6d1035a4ad feat(core): test improvements for get_key_list()
- move kmxplus processing into the base LdmlTestSource class
- add a function to traverse the layer list looking for keys to add
- The @@keylist keyword only has one example from each modifier set

Fixes: #12298
2024-11-25 12:54:02 -06:00
Steven R. Loomis
2e5b6019e1 feat(core): get_key_list needs to include all keys
Fixes: #12298
2024-11-23 12:51:24 -08:00
Steven R. Loomis
741d3057a5
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-07 22:59:32 -06:00
Steven R. Loomis
382b511dce feat(core): ldml: implementation for testing get_key_list()
- ldml::vkeys class updated to keep a set<> of keys
- fix the test case to not leak!

Fixes: #12298
2024-11-07 09:34:30 -06:00
Marc Durdin
fbab6eb8cf chore(core): move API docs from help.keyman.com
Reorganizes /core/doc to /core/docs and moves internal files
accordingly.

Depends-on: keymanapp/help.keyman.com#1684
2024-11-07 08:28:33 +07:00
Steven R. Loomis
dd11a25b27 feat(core): ldml: scaffolding for testing get_key_list()
- move some test utils to statics
- add a new test action type and `@@key-list` keyword
- print warning on unhanded @-commands

Fixes: #12298
2024-11-06 09:30:18 -06:00
Darcy Wong
e62d05cff7 fix(core): Disable test_unicode check for ICU version 2024-09-11 20:56:50 +07:00
Marc Durdin
96f288a83d fix(core): properly support 'other' modifier state with uint32_t type
While the modifier state property in core's API is 16-bit, internally
ldml_processor supports the modifier flag LDML_KEYS_MOD_OTHER with a
value of `0x10000`, which requires widening the value (we match the
32-bit size of the KMX_DWORD value from KMX+).

Note: this is not yet well unit-tested.

Relates-to: #11072
Fixes: #12057
2024-08-24 12:26:52 +08:00
Eberhard Beilharz
a98ee7040b
Merge pull request #12235 from keymanapp/fix/core/emcc_detect
fix(core): look for `emcc` instead of `emcc.py`
2024-08-20 22:41:46 +02:00
Eberhard Beilharz
ab3722ecfa
fix(core): look for emcc instead of emcc.py on Linux and Mac
`emcc.py` is not marked as executable in emcripten's git repo so the
build failed when trying to locate emscripten. However, it turns out
that `emcc` is marked as executable, so we use that instead.

On Windows however, we still need to use `emcc.py` because otherwise
Meson won't detect it as valid compiler.
2024-08-20 15:52:52 +02:00
Steven R. Loomis
7c0f51c486 fix(core): set mac build version for meson cli build to 10.13
- add as option to compiler
- Match version in #11302

Fixes: #11423
2024-08-19 10:22:10 -05:00
Marc Durdin
bad54e4fed
Merge pull request #12171 from keymanapp/refactor/core/make-utfcodec-common
refactor(core): move utfcodec to common
2024-08-14 19:10:16 +10:00
Marc Durdin
613bdb6c19 refactor(core): move utfcodec to common
Refactor to support codecvt cleanup work in kmcmplib.
2024-08-13 15:04:07 +02:00
Marc Durdin
859c88a6de docs: add .kmx specification
Fixes: #11407
2024-08-12 17:04:24 +07:00
Marc Durdin
05edc8a7ce
Merge branch 'master' into refactor/windows/cleanup-engine-logging 2024-07-23 17:29:55 +10:00
Eberhard Beilharz
88f6c95803
fix(core): allow to successfully build on Ubuntu 24.04
Without this change trying to build on Ubuntu 24.04 fails with
"error: 'intptr_t' has not been declared in '::'". I'm not sure
why that is happening, but this is an easy fix.
2024-07-04 13:07:10 +02:00
Marc Durdin
1be73a9c2e refactor(windows): clean up logging
* Remove unused parameters from SendDebugMessage functions
* Add SendDebugEntry and SendDebugExit functions for tracking
  function entry/exit
* Add indenting and function names to log entries
* Remove unused debug functions
* Eliminate now-unused hwnd parameter in initialization functions
* Replace Log,LogEntry,LogExit functions with SendDebug equivalents in
  kmtip

Many functions now have SendDebugEntry/SendDebugExit (or
return_SendDebugExit) pairs. It is important to SendDebugExit on all
returns from a function to keep the log indent depth consistent. In some
cases I chose not to add these logging calls, e.g. on frequently called
functions such as the message hooks.
2024-07-03 21:28:37 +10:00
Steven R. Loomis
a16608e229
Merge branch 'master' into test/core/10968-minimum-ldml-test 2024-06-17 08:24:11 -05:00
Steven R. Loomis
38c6fe1ba3
Merge pull request #11778 from keymanapp/feat/core/9467-remove-icu-for-wasm-epic-ldml
feat(core): remove ICU from core under wasm 🙀
2024-06-17 08:22:38 -05:00
Steven R. Loomis
e580043538
Merge pull request #11777 from keymanapp/feat/core/9467-devolve-regex-to-js2-epic-ldml
feat(core): devolve regex to javascript 🙀
2024-06-17 08:22:22 -05:00
Marc Durdin
d6992097fd
Merge pull request #11795 from keymanapp/fix/core/11794-serialize-core-tests-temporary-mitigation
fix(core): serialize tests for core/wasm on mac agents
2024-06-17 16:26:27 +10:00
Marc Durdin
db53ac1343 fix(core): serialize tests for core/wasm on mac agents
Mitigates: #11794
2024-06-16 16:42:38 +10:00
Steven R. Loomis
ed80095ab9
Merge branch 'master' into test/core/10968-minimum-ldml-test 2024-06-14 09:33:29 -05:00
Steven R. Loomis
2300571ad8 test(core): fix a type conversion issue in the minimal test
- need a null string for either narrow or wide chars

Fixes: #10968
2024-06-14 09:05:50 -05:00
Steven R. Loomis
ca34550a8d feat(core): update util_regex per review comments
- remove some TODOs that were obsolete
- copy/clarify/expand comments between the ICU and non-ICU sides

Fixes: #9467
2024-06-14 08:55:15 -05:00
Steven R. Loomis
3c82343218 test(core): Add a minimal test that exercises the core API
- this test doesn't actually run well, it compiles but fails with an assertion.
- the purpose is to exercise the linker.

Fixes: #10968
2024-06-13 16:56:36 -05:00
Steven R. Loomis
b2e3215b0b
Merge branch 'feat/core/9467-devolve-regex-to-js2-epic-ldml' into feat/core/9467-remove-icu-for-wasm-epic-ldml 2024-06-13 14:29:18 -05:00
Steven R. Loomis
ff3c9736f4
Merge branch 'master' into feat/core/9467-devolve-regex-to-js2-epic-ldml 2024-06-13 14:29:01 -05:00
Steven R. Loomis
edd42d1dcb
Merge pull request #11541 from keymanapp/feat/core/9467-devolve-norm-to-js3-epic-ldml
feat(core): devolve normalization to js 🙀
2024-06-13 14:28:45 -05:00
Steven R. Loomis
a744afc34f feat(core): remove ICU from core under wasm
Fixes: #9467
2024-06-13 14:20:58 -05:00
Steven R. Loomis
1a61a589b3 feat(core): devolve regex to js for wasm
- assert was wrong!

Fixes: #9467
2024-06-13 13:07:29 -05:00
Steven R. Loomis
263d0e32f2 chore(core): update comments and remove a raw numeric literal
- per review comments

Fixes: #9467

Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2024-06-13 12:17:55 -05:00
Marc Durdin
577986bb50
Merge pull request #11523 from keymanapp/chore/common/8399-cleanup-meson-warnings
chore(common): cleanup meson deprecations and warnings
2024-06-08 10:46:41 +10:00
Steven R. Loomis
acd3cc5847 feat(core): devolve regex to JS
- use JS regex code
- almost there - some perhaps escaping differences

Fixes: #9467
2024-06-07 18:11:20 -05:00
Steven R. Loomis
c4b01f7715 test(core): detailed unit test for util_regex
Fixes: #9467
2024-06-07 16:27:45 -05:00
Steven R. Loomis
55a025c767 refactor(core): devolve regex to js for wasm
- new module, core/src/util_regex.hpp
- no wasm implementationyet

Fixes: #9467
2024-06-05 16:53:43 -05:00
Steven R. Loomis
f156a7264d feat(core): generator in core/src for util_normalize_table.h
- temporary header file generated by wasm during build
- built using icu
- test_unicode verifies the contents and synchronization with running ICU.
- this is used by util_normalize to provide normalization properties under wasm without needing to include ICU.

Fixes: #9467
2024-06-05 15:47:20 -05:00
Steven R. Loomis
fdb2e95d20 feat(core): build improvements for --update-unicode option
Fixes: #9467
2024-06-04 12:26:00 -05:00
Steven R. Loomis
e9673867fa feat(core): speedup NFD boundary table
- use RLE encoding, thanks @mcdurdin
- much smaller table and faster lookup

Fixes: #9467
2024-06-04 12:23:16 -05:00
Steven R. Loomis
7fbeea85cc Merge remote-tracking branch 'upstream/master' into feat/core/9467-devolve-norm-to-js3-epic-ldml 2024-06-04 11:15:31 -05:00
Steven R. Loomis
53a6638a70 feat(core): generate and use static table in wasm for NFD boundary
- add core/tools build tree with custom targets
- add to core/build.sh to generate nfd_table.h
- test_unicode to validate Unicode version and compare NFD to actual ICU
- currently, linear search of the table.
2024-06-04 11:15:17 -05:00
Steven R. Loomis
31dedb7ca3 test(core): comments in get_block_unicode_ver() 2024-05-31 16:14:05 -05:00
Steven R. Loomis
403d342e10 test(core): clarify output of test_unicode
- group output by functional area

#10183
2024-05-27 16:42:11 -05:00
Marc Durdin
bd01d4b72a
Merge branch 'master' into chore/common/8399-cleanup-meson-warnings 2024-05-27 20:26:54 +07:00
Steven R. Loomis
5d420248eb feat(core): move more normalization logic into JS
- major redo of actions_normalize - into UTF-32 and not using ICU directly
- add some utilities: u32len, u32dup, context_items_from_utf32

#9467
2024-05-24 18:20:30 -05:00