Commit graph

634 commits

Author SHA1 Message Date
Eberhard Beilharz
aa36cfe961
fix(core): address code review comments
Some checks failed
Keyman Build Summary / Summarize build status checks (push) Has been cancelled
2026-03-24 12:43:11 +01:00
Eberhard Beilharz
9836070299
fix(core): clarify and extend comment 2026-03-02 17:43:34 +01:00
Eberhard Beilharz
67468c4dce
fix(core): address code review comments
Co-authored-by: Marc Durdin <marc@durdin.net>
2026-03-02 17:28:19 +01:00
Eberhard Beilharz
382b5d45e3
fix(core): fix keydown/up handling for LDML keyboards
For LDML keyboards this change fixes the `emit_key` flag so that it has
the same value for the KeyDown and the KeyUp event. This fixes some
problems with stuck keys. Previously we would set `emit_key=TRUE` on
KeyDown but `emit_key=FALSE` on KeyUp for frame keys. This caused Linux
to never see the KeyUp event, resulting in a stuck key.

Also add unit tests that verifies that the actions that we get after
calling `km_core_process_event` are what we expect.

Fixes: #15569
Fixes: #15550
2026-03-02 17:27:36 +01:00
Eberhard Beilharz
2bd6cfd735
refactor(core): rename fields to start with underscore
Test-bot: skip
2026-03-02 17:27:15 +01:00
Eberhard Beilharz
d8a5f694da
refactor(core): fix whitespace
Build-bot: skip
Test-bot: skip
2026-03-02 17:26:14 +01:00
Eberhard Beilharz
b405bf5842
refactor(core): simplify loop for removing text
Instead of using an iterator to loop over the context items, incrementing
a counter on each iteration, and then finally removing the calculated
number of context items from the list, this change loops through the list
and looks at the last context item, removing it if necessary.

Follows: #15596
Test-bot: skip
2026-02-18 17:34:02 +01:00
Eberhard Beilharz
32bb1c8502
fix(core): fix iterator in backspace handling
This change replaces the reverse iterator loop that holds a stale iterator
across `pop_back()` calls with a pattern that directly accesses
`context.back()` on each iteration. This avoids undefined behavior from
iterator invalidation when mutating the list or vector.

While so far the previous code didn't show problems, it might still access
released memory depending on the implementation. The documentation for
`pop_back()` says "References and iterators to the erased element are
invalidated", so the previous implementation was clearly wrong.

Test-bot: skip
2026-02-18 15:58:45 +01:00
Marc Durdin
74d1bc8fc2 chore(core): address review comments
Co-authored-by: Darcy Wong <darcy_wong@sil.org>
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2026-02-01 06:47:49 +11:00
Marc Durdin
766c6992ed fix(core): normalization segment should end on NFC boundary, not NFD
When normalizing, we need to stop processing on an NFC boundary, not an
NFD boundary, to support normalizations such as in Bengali, where
appending `U+09D7` to a context of `U+0995 U+09C7` should result in
`U+0995 U+09CC`.

The specification is unclear on this; see https://unicode-org.atlassian.net/browse/CLDR-19218

This also updates the ldml keyboard unit test suite to support running
in full NFC mode (used in all Engine implementations) as well retaining
the NFD mode (now only used by the debugger).

Side note: the Bengali normalization failure case was picked up by the
improvements to the unit test suite, proving once again that good tests
are so valuable.

Fixes: #15491
Fixes: #15505
Follows: #15488
Relates-to: CLDR-19218
2026-01-28 14:32:25 +11:00
Marc Durdin
033e3eeb3f docs(core): add comments post review
Co-authored-by: srl295@gmail.com
2026-01-27 10:50:26 +11:00
Marc Durdin
f7cd0c41ba fix(core): handle backspace decomposition
Ensure that when a single backspace decomposes the last NFC character in
the app context, the remainder of the 'cluster' is preserved, matching
the implication of the CLDR keyboard specification.

This addresses the behavior in #15487 where the cached context became
out of sync with the app context after deleting an entire NFC cluster
such as ê, which caused a loop ending up with the entire context being
deleted, at which point the loop exited with a fail-safe.

Note that the LDML keyboard tests (ldml.cpp) do not currently exercise
the normalization code; this is a gap that should be addressed to ensure
that we are testing final application behavior.

Fixes: #15487
2026-01-25 07:24:15 +11:00
Marc Durdin
55d6efe482 fix(developer): prevent ANSI keyboards crashing debugger
Note: #11909 has some additional future cleanup we could do, but these
code paths are not currently accessible, so in the interest of moving
forward, am leaving them for now.

Fixes: #11909
Test-bot: skip
2025-10-24 14:45:17 +02:00
Steven R. Loomis
9989217f9e feat(core): improve kmxplus validation
- restructured validation calls to be more consistent

Fixes: #9446
2025-03-17 12:09:04 -05:00
Steven R. Loomis
0011a2d10d feat(core): improve kmxplus validation
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test

Fixes: #9446

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-17 11:26:57 -05:00
Steven R. Loomis
f1b3555114 feat(core,docs): improve kmxplus validation
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections

Fixes: #9446
2025-03-14 15:37:23 -05:00
Steven R. Loomis
cd002441e5 feat(core): check non-extended keys for bad unicode
- add test case
- turn this into not be an assert, so we can test it

Fixes: #9446
2025-03-14 10:04:21 -05:00
Steven R. Loomis
2e7b5b1417 feat(core): check non-extended keys for bad unicode
- also, use U+FFFD in core if it's a bad char

Fixes: #9446
2025-03-14 09:31:38 -05:00
Steven R. Loomis
b2c3a46283
feat(core) Update core/src/ldml/ldml_processor.cpp
map bad chars to U+0020

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-14 09:23:37 -05:00
Steven R. Loomis
2e50282087
Merge branch 'master' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-14 09:19:03 -05:00
Steven R. Loomis
d9602cede1 feat(core): assert that no noncharacter emitted from ldml_processor
- not counting markers, which have a special path

Fixes: #9446
2025-03-13 17:22:37 -05:00
Steven R. Loomis
1132fe5845 feat(core): tests for check on load for valid kmx_plus unicode
- turn off some asserts- makes untestable
(there are asserts at 'higher levels' such as loading the entire kmx+)
- add a test with a synthesized COMP_KMXPLUS_STRS - a valid and an invalid one

Fixes: #9446
2025-03-13 17:15:21 -05:00
Steven R. Loomis
fba5f72844 feat(core): tests for check on load for valid kmx_plus unicode
Fixes: #9446
2025-03-13 16:44:31 -05:00
Steven R. Loomis
2c2bfe7f15 feat(core): check on load for valid kmx_plus unicode
- error out if failure

Fixes: #9446
2025-03-12 17:45:47 -05:00
Marc Durdin
577cbd3a07 fix(core): unalign usetCount and rangeCount
Fixes: #13419
2025-03-07 12:50:22 +07:00
Marc Durdin
b973bcc0d9 fix(core): include nul in context offset calculations in kmx processor
Two separate bugs addressed, with `index()` references and with
`context()` references -- both have the same root cause, of not taking
`nul` at the start of the context into account (as `nul` is not included
in the `m_miniContext` member, being a non-character). We already fixed
this issue for `if()` quite a long time ago, and some of the same
patterns can be with `m_miniContextIfLen` for example.

Fixes: #13304
Fixes: #13316
2025-02-21 17:17:19 +07:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Steven R. Loomis
de3397b476
Merge pull request #12644 from keymanapp/fix/core/12298-get-key-list
fix(core): implement ldml_processor::get_key_list() 🙀
2024-12-09 08:14:44 -06:00
Steven R. Loomis
feba97979a fix(core): update get_key_list to account for other combinatorics
- yes, expand 'other' to all possible combinations
- use ALT  and CTRL instead of RALT,LALT and RCTRL,LCTRL in the key list (reduce expansions up to 4x)

Fixes: #12298
2024-12-06 12:24:53 -06:00
Steven R. Loomis
da7c864f54 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-12-04 13:58:24 -06:00
Marc Durdin
4c3b2132ee chore(core): remove km_core_keyboard_load API
Fixes: #12497
2024-12-04 13:38:07 +07:00
Marc Durdin
9a01ac166d chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-12-01 16:17:21 +07:00
Steven R. Loomis
9e2b8d0d8c feat(core): improvements for get_key_list()
- reintroduce example keycaps

Fixes: #12298
2024-11-29 18:28:05 -06:00
Steven R. Loomis
7f3bd30961 feat(core): improvements for get_key_list()
- expand OTHER and ALT / CTRL appropriately
- add KM_CORE_MODIFIER_NONE=0
- disable test of get_key_list() for now

Fixes: #12298
2024-11-29 17:25:31 -06:00
Steven R. Loomis
d92dc175c2 chore(core): update to ldmL_test_source.cpp 2024-11-29 15:47:30 -06:00
Steven R. Loomis
c9643642a8 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-11-29 15:33:53 -06:00
Eberhard Beilharz
b8c120dd5c
Merge pull request #12721 from keymanapp/chore/core/cherry-pick/load-from-blob-api
feat(core,linux,developer,windows): implement loading KMX from blob 🍒
2024-11-29 12:40:51 +01:00
Marc Durdin
36ddd793a5 chore(core): ifdef variables only used in assertions 2024-11-28 10:58:51 +07:00
Marc Durdin
bcecedcb04 chore(developer): ifdef variables only used in assertions 2024-11-28 10:36:13 +07:00
Marc Durdin
b9b4ad4996 chore(developer): ifdef variables only used in assertions 2024-11-28 10:27:40 +07:00
Marc Durdin
e2b8222a6b chore(developer): ifdef variables only used in assertions 2024-11-28 10:12:28 +07:00
Eberhard Beilharz
0958a2da23
fix(core): permanently disable logging
This change disables logging at compile time to work around #12661.
Logging can be enabled in the debugger, or by re-compiling with
`g_debug_KeymanLog` set to TRUE.

Related: #12661
Cherry-pick: #12674
2024-11-27 16:36:25 +01:00
Eberhard Beilharz
3dc3f040de
feat(core): implement loading KMX from blob
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a

Part-of: #11293
Part-of: #8093
2024-11-27 15:30:05 +01:00
Steven R. Loomis
6d1035a4ad feat(core): test improvements for get_key_list()
- move kmxplus processing into the base LdmlTestSource class
- add a function to traverse the layer list looking for keys to add
- The @@keylist keyword only has one example from each modifier set

Fixes: #12298
2024-11-25 12:54:02 -06:00
Steven R. Loomis
2e5b6019e1 feat(core): get_key_list needs to include all keys
Fixes: #12298
2024-11-23 12:51:24 -08:00
Steven R. Loomis
741d3057a5
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-07 22:59:32 -06:00
Steven R. Loomis
382b511dce feat(core): ldml: implementation for testing get_key_list()
- ldml::vkeys class updated to keep a set<> of keys
- fix the test case to not leak!

Fixes: #12298
2024-11-07 09:34:30 -06:00
Marc Durdin
fbab6eb8cf chore(core): move API docs from help.keyman.com
Reorganizes /core/doc to /core/docs and moves internal files
accordingly.

Depends-on: keymanapp/help.keyman.com#1684
2024-11-07 08:28:33 +07:00
Marc Durdin
96f288a83d fix(core): properly support 'other' modifier state with uint32_t type
While the modifier state property in core's API is 16-bit, internally
ldml_processor supports the modifier flag LDML_KEYS_MOD_OTHER with a
value of `0x10000`, which requires widening the value (we match the
32-bit size of the KMX_DWORD value from KMX+).

Note: this is not yet well unit-tested.

Relates-to: #11072
Fixes: #12057
2024-08-24 12:26:52 +08:00
Marc Durdin
bad54e4fed
Merge pull request #12171 from keymanapp/refactor/core/make-utfcodec-common
refactor(core): move utfcodec to common
2024-08-14 19:10:16 +10:00