Commit graph

621 commits

Author SHA1 Message Date
Steven R. Loomis
9989217f9e feat(core): improve kmxplus validation
- restructured validation calls to be more consistent

Fixes: #9446
2025-03-17 12:09:04 -05:00
Steven R. Loomis
0011a2d10d feat(core): improve kmxplus validation
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test

Fixes: #9446

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-17 11:26:57 -05:00
Steven R. Loomis
f1b3555114 feat(core,docs): improve kmxplus validation
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections

Fixes: #9446
2025-03-14 15:37:23 -05:00
Steven R. Loomis
cd002441e5 feat(core): check non-extended keys for bad unicode
- add test case
- turn this into not be an assert, so we can test it

Fixes: #9446
2025-03-14 10:04:21 -05:00
Steven R. Loomis
2e7b5b1417 feat(core): check non-extended keys for bad unicode
- also, use U+FFFD in core if it's a bad char

Fixes: #9446
2025-03-14 09:31:38 -05:00
Steven R. Loomis
b2c3a46283
feat(core) Update core/src/ldml/ldml_processor.cpp
map bad chars to U+0020

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-14 09:23:37 -05:00
Steven R. Loomis
2e50282087
Merge branch 'master' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-14 09:19:03 -05:00
Steven R. Loomis
d9602cede1 feat(core): assert that no noncharacter emitted from ldml_processor
- not counting markers, which have a special path

Fixes: #9446
2025-03-13 17:22:37 -05:00
Steven R. Loomis
1132fe5845 feat(core): tests for check on load for valid kmx_plus unicode
- turn off some asserts- makes untestable
(there are asserts at 'higher levels' such as loading the entire kmx+)
- add a test with a synthesized COMP_KMXPLUS_STRS - a valid and an invalid one

Fixes: #9446
2025-03-13 17:15:21 -05:00
Steven R. Loomis
fba5f72844 feat(core): tests for check on load for valid kmx_plus unicode
Fixes: #9446
2025-03-13 16:44:31 -05:00
Steven R. Loomis
2c2bfe7f15 feat(core): check on load for valid kmx_plus unicode
- error out if failure

Fixes: #9446
2025-03-12 17:45:47 -05:00
Marc Durdin
577cbd3a07 fix(core): unalign usetCount and rangeCount
Fixes: #13419
2025-03-07 12:50:22 +07:00
Marc Durdin
b973bcc0d9 fix(core): include nul in context offset calculations in kmx processor
Two separate bugs addressed, with `index()` references and with
`context()` references -- both have the same root cause, of not taking
`nul` at the start of the context into account (as `nul` is not included
in the `m_miniContext` member, being a non-character). We already fixed
this issue for `if()` quite a long time ago, and some of the same
patterns can be with `m_miniContextIfLen` for example.

Fixes: #13304
Fixes: #13316
2025-02-21 17:17:19 +07:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Steven R. Loomis
de3397b476
Merge pull request #12644 from keymanapp/fix/core/12298-get-key-list
fix(core): implement ldml_processor::get_key_list() 🙀
2024-12-09 08:14:44 -06:00
Steven R. Loomis
feba97979a fix(core): update get_key_list to account for other combinatorics
- yes, expand 'other' to all possible combinations
- use ALT  and CTRL instead of RALT,LALT and RCTRL,LCTRL in the key list (reduce expansions up to 4x)

Fixes: #12298
2024-12-06 12:24:53 -06:00
Steven R. Loomis
da7c864f54 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-12-04 13:58:24 -06:00
Marc Durdin
4c3b2132ee chore(core): remove km_core_keyboard_load API
Fixes: #12497
2024-12-04 13:38:07 +07:00
Marc Durdin
9a01ac166d chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-12-01 16:17:21 +07:00
Steven R. Loomis
9e2b8d0d8c feat(core): improvements for get_key_list()
- reintroduce example keycaps

Fixes: #12298
2024-11-29 18:28:05 -06:00
Steven R. Loomis
7f3bd30961 feat(core): improvements for get_key_list()
- expand OTHER and ALT / CTRL appropriately
- add KM_CORE_MODIFIER_NONE=0
- disable test of get_key_list() for now

Fixes: #12298
2024-11-29 17:25:31 -06:00
Steven R. Loomis
d92dc175c2 chore(core): update to ldmL_test_source.cpp 2024-11-29 15:47:30 -06:00
Steven R. Loomis
c9643642a8 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-11-29 15:33:53 -06:00
Eberhard Beilharz
b8c120dd5c
Merge pull request #12721 from keymanapp/chore/core/cherry-pick/load-from-blob-api
feat(core,linux,developer,windows): implement loading KMX from blob 🍒
2024-11-29 12:40:51 +01:00
Marc Durdin
36ddd793a5 chore(core): ifdef variables only used in assertions 2024-11-28 10:58:51 +07:00
Marc Durdin
bcecedcb04 chore(developer): ifdef variables only used in assertions 2024-11-28 10:36:13 +07:00
Marc Durdin
b9b4ad4996 chore(developer): ifdef variables only used in assertions 2024-11-28 10:27:40 +07:00
Marc Durdin
e2b8222a6b chore(developer): ifdef variables only used in assertions 2024-11-28 10:12:28 +07:00
Eberhard Beilharz
0958a2da23
fix(core): permanently disable logging
This change disables logging at compile time to work around #12661.
Logging can be enabled in the debugger, or by re-compiling with
`g_debug_KeymanLog` set to TRUE.

Related: #12661
Cherry-pick: #12674
2024-11-27 16:36:25 +01:00
Eberhard Beilharz
3dc3f040de
feat(core): implement loading KMX from blob
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a

Part-of: #11293
Part-of: #8093
2024-11-27 15:30:05 +01:00
Steven R. Loomis
6d1035a4ad feat(core): test improvements for get_key_list()
- move kmxplus processing into the base LdmlTestSource class
- add a function to traverse the layer list looking for keys to add
- The @@keylist keyword only has one example from each modifier set

Fixes: #12298
2024-11-25 12:54:02 -06:00
Steven R. Loomis
2e5b6019e1 feat(core): get_key_list needs to include all keys
Fixes: #12298
2024-11-23 12:51:24 -08:00
Steven R. Loomis
741d3057a5
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-07 22:59:32 -06:00
Steven R. Loomis
382b511dce feat(core): ldml: implementation for testing get_key_list()
- ldml::vkeys class updated to keep a set<> of keys
- fix the test case to not leak!

Fixes: #12298
2024-11-07 09:34:30 -06:00
Marc Durdin
fbab6eb8cf chore(core): move API docs from help.keyman.com
Reorganizes /core/doc to /core/docs and moves internal files
accordingly.

Depends-on: keymanapp/help.keyman.com#1684
2024-11-07 08:28:33 +07:00
Marc Durdin
96f288a83d fix(core): properly support 'other' modifier state with uint32_t type
While the modifier state property in core's API is 16-bit, internally
ldml_processor supports the modifier flag LDML_KEYS_MOD_OTHER with a
value of `0x10000`, which requires widening the value (we match the
32-bit size of the KMX_DWORD value from KMX+).

Note: this is not yet well unit-tested.

Relates-to: #11072
Fixes: #12057
2024-08-24 12:26:52 +08:00
Marc Durdin
bad54e4fed
Merge pull request #12171 from keymanapp/refactor/core/make-utfcodec-common
refactor(core): move utfcodec to common
2024-08-14 19:10:16 +10:00
Marc Durdin
613bdb6c19 refactor(core): move utfcodec to common
Refactor to support codecvt cleanup work in kmcmplib.
2024-08-13 15:04:07 +02:00
Marc Durdin
859c88a6de docs: add .kmx specification
Fixes: #11407
2024-08-12 17:04:24 +07:00
Marc Durdin
1be73a9c2e refactor(windows): clean up logging
* Remove unused parameters from SendDebugMessage functions
* Add SendDebugEntry and SendDebugExit functions for tracking
  function entry/exit
* Add indenting and function names to log entries
* Remove unused debug functions
* Eliminate now-unused hwnd parameter in initialization functions
* Replace Log,LogEntry,LogExit functions with SendDebug equivalents in
  kmtip

Many functions now have SendDebugEntry/SendDebugExit (or
return_SendDebugExit) pairs. It is important to SendDebugExit on all
returns from a function to keep the log indent depth consistent. In some
cases I chose not to add these logging calls, e.g. on frequently called
functions such as the message hooks.
2024-07-03 21:28:37 +10:00
Steven R. Loomis
38c6fe1ba3
Merge pull request #11778 from keymanapp/feat/core/9467-remove-icu-for-wasm-epic-ldml
feat(core): remove ICU from core under wasm 🙀
2024-06-17 08:22:38 -05:00
Steven R. Loomis
ca34550a8d feat(core): update util_regex per review comments
- remove some TODOs that were obsolete
- copy/clarify/expand comments between the ICU and non-ICU sides

Fixes: #9467
2024-06-14 08:55:15 -05:00
Steven R. Loomis
b2e3215b0b
Merge branch 'feat/core/9467-devolve-regex-to-js2-epic-ldml' into feat/core/9467-remove-icu-for-wasm-epic-ldml 2024-06-13 14:29:18 -05:00
Steven R. Loomis
ff3c9736f4
Merge branch 'master' into feat/core/9467-devolve-regex-to-js2-epic-ldml 2024-06-13 14:29:01 -05:00
Steven R. Loomis
a744afc34f feat(core): remove ICU from core under wasm
Fixes: #9467
2024-06-13 14:20:58 -05:00
Steven R. Loomis
1a61a589b3 feat(core): devolve regex to js for wasm
- assert was wrong!

Fixes: #9467
2024-06-13 13:07:29 -05:00
Steven R. Loomis
263d0e32f2 chore(core): update comments and remove a raw numeric literal
- per review comments

Fixes: #9467

Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2024-06-13 12:17:55 -05:00
Steven R. Loomis
acd3cc5847 feat(core): devolve regex to JS
- use JS regex code
- almost there - some perhaps escaping differences

Fixes: #9467
2024-06-07 18:11:20 -05:00
Steven R. Loomis
55a025c767 refactor(core): devolve regex to js for wasm
- new module, core/src/util_regex.hpp
- no wasm implementationyet

Fixes: #9467
2024-06-05 16:53:43 -05:00
Steven R. Loomis
f156a7264d feat(core): generator in core/src for util_normalize_table.h
- temporary header file generated by wasm during build
- built using icu
- test_unicode verifies the contents and synchronization with running ICU.
- this is used by util_normalize to provide normalization properties under wasm without needing to include ICU.

Fixes: #9467
2024-06-05 15:47:20 -05:00