Commit graph

514 commits

Author SHA1 Message Date
Eberhard Beilharz
67468c4dce
fix(core): address code review comments
Co-authored-by: Marc Durdin <marc@durdin.net>
2026-03-02 17:28:19 +01:00
Eberhard Beilharz
382b5d45e3
fix(core): fix keydown/up handling for LDML keyboards
For LDML keyboards this change fixes the `emit_key` flag so that it has
the same value for the KeyDown and the KeyUp event. This fixes some
problems with stuck keys. Previously we would set `emit_key=TRUE` on
KeyDown but `emit_key=FALSE` on KeyUp for frame keys. This caused Linux
to never see the KeyUp event, resulting in a stuck key.

Also add unit tests that verifies that the actions that we get after
calling `km_core_process_event` are what we expect.

Fixes: #15569
Fixes: #15550
2026-03-02 17:27:36 +01:00
Eberhard Beilharz
d8a5f694da
refactor(core): fix whitespace
Build-bot: skip
Test-bot: skip
2026-03-02 17:26:14 +01:00
Eberhard Beilharz
2237db95f6
test(core): add LDML baseline test with minimal keyboard
This replaces and enables the commented `k_000_null_keyboard` test which
didn't work because keys that are not on any layer don't produce output.
This instead defines a minimal keyboard with just two keys and then tests
typing a key that is on the keyboard followed by a key not on the keyboard.

Test-bot: skip
2026-02-20 12:09:08 +01:00
Eberhard Beilharz
df2b675202
fix(core): changed ldml tests to use context list instead of vector
Production code uses a list of context items, so this change modifies
the tests to also use a list instead of a vector to more closely match
production code.

Addresses code review comment.

Test-bot: skip
2026-02-18 16:08:59 +01:00
Eberhard Beilharz
32bb1c8502
fix(core): fix iterator in backspace handling
This change replaces the reverse iterator loop that holds a stale iterator
across `pop_back()` calls with a pattern that directly accesses
`context.back()` on each iteration. This avoids undefined behavior from
iterator invalidation when mutating the list or vector.

While so far the previous code didn't show problems, it might still access
released memory depending on the implementation. The documentation for
`pop_back()` says "References and iterators to the erased element are
invalidated", so the previous implementation was clearly wrong.

Test-bot: skip
2026-02-18 15:58:45 +01:00
Marc Durdin
74d1bc8fc2 chore(core): address review comments
Co-authored-by: Darcy Wong <darcy_wong@sil.org>
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2026-02-01 06:47:49 +11:00
Marc Durdin
766c6992ed fix(core): normalization segment should end on NFC boundary, not NFD
When normalizing, we need to stop processing on an NFC boundary, not an
NFD boundary, to support normalizations such as in Bengali, where
appending `U+09D7` to a context of `U+0995 U+09C7` should result in
`U+0995 U+09CC`.

The specification is unclear on this; see https://unicode-org.atlassian.net/browse/CLDR-19218

This also updates the ldml keyboard unit test suite to support running
in full NFC mode (used in all Engine implementations) as well retaining
the NFD mode (now only used by the debugger).

Side note: the Bengali normalization failure case was picked up by the
improvements to the unit test suite, proving once again that good tests
are so valuable.

Fixes: #15491
Fixes: #15505
Follows: #15488
Relates-to: CLDR-19218
2026-01-28 14:32:25 +11:00
Marc Durdin
f7cd0c41ba fix(core): handle backspace decomposition
Ensure that when a single backspace decomposes the last NFC character in
the app context, the remainder of the 'cluster' is preserved, matching
the implication of the CLDR keyboard specification.

This addresses the behavior in #15487 where the cached context became
out of sync with the app context after deleting an entire NFC cluster
such as ê, which caused a loop ending up with the entire context being
deleted, at which point the loop exited with a fail-safe.

Note that the LDML keyboard tests (ldml.cpp) do not currently exercise
the normalization code; this is a gap that should be addressed to ensure
that we are testing final application behavior.

Fixes: #15487
2026-01-25 07:24:15 +11:00
Eberhard Beilharz
acebacc2b6
maint(core): explain (wrong) meson warning
Build-bot: skip
Test-bot: skip
2025-07-17 17:03:47 +02:00
Marc Durdin
7a56017f5b maint(core): batch build kmx in tests for performance
Build all the .kmx files used for kmn and ldml unit tests in batch
rather than one at a time. The ldml-test-data builds remain unbatched
for now (and are visibly slow!).

Fixes: #13495
2025-04-30 06:02:59 +07:00
Steven R. Loomis
443681999d feat(core): improve kmxplus validation tests
- update per review notes

Fixes: #9446
2025-03-18 11:08:44 -05:00
Steven R. Loomis
0011a2d10d feat(core): improve kmxplus validation
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test

Fixes: #9446

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-17 11:26:57 -05:00
Steven R. Loomis
f1b3555114 feat(core,docs): improve kmxplus validation
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections

Fixes: #9446
2025-03-14 15:37:23 -05:00
Steven R. Loomis
cd002441e5 feat(core): check non-extended keys for bad unicode
- add test case
- turn this into not be an assert, so we can test it

Fixes: #9446
2025-03-14 10:04:21 -05:00
Steven R. Loomis
2e50282087
Merge branch 'master' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-14 09:19:03 -05:00
Steven R. Loomis
a0c2daf25b
Merge pull request #13494 from keymanapp/chore/core/9446-kmx-plus-tests-gtest
feat(core): move kmx_plus.tests.cpp to GTest 🙀
2025-03-14 09:18:34 -05:00
Steven R. Loomis
b696e34001 feat(core): move kmx_plus.tests.cpp to GTest
- msvc global to UTF-8
2025-03-13 22:47:39 -05:00
Steven R. Loomis
289151c990 feat(core): move kmx_plus.tests.cpp to GTest
- set kmx_plus_tests.cpp to UTF-8
2025-03-13 17:34:37 -05:00
Steven R. Loomis
1132fe5845 feat(core): tests for check on load for valid kmx_plus unicode
- turn off some asserts- makes untestable
(there are asserts at 'higher levels' such as loading the entire kmx+)
- add a test with a synthesized COMP_KMXPLUS_STRS - a valid and an invalid one

Fixes: #9446
2025-03-13 17:15:21 -05:00
Steven R. Loomis
fba5f72844 feat(core): tests for check on load for valid kmx_plus unicode
Fixes: #9446
2025-03-13 16:44:31 -05:00
Steven R. Loomis
1306cf8760 feat(core): move kmx_plus.tests.cpp to GTest
- also update test_assert.h to vector to GTest if present
2025-03-13 15:05:55 -05:00
Marc Durdin
03001bd971 chore(common): add unit test for nul and context() in context part of rule
The compiler and Core appear to be working correctly in this situation;
this test verifies Core's behaviour.

Fixes: #13299
2025-03-04 11:22:35 +01:00
Marc Durdin
2b9b8eb419 chore(core): add unit tests to validate nul with index and context
Add 4 test keyboards to validate `nul` and `if` used in conjunction with
`index` and `context` and corresponding references in core unit tests.

Add a script to rebuild baseline keyboards using a copy of kmcomp.exe
16.0.138; this is setup and tested only on Windows (YMMV on WINE, etc).

Add the 4 additional baseline test keyboards to kmcmplib unit tests for
build consistency between kmcomp 16 and kmcmplib 18 (all pass).

Note: two of the new Core tests currently fail. This is expected, see
issue #13304.

Fixes: #13303
2025-02-21 13:06:29 +07:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Eberhard Beilharz
9c8ad2e783
fix(core): fix compile error
Compiling the tests failed in the container.
2025-01-08 17:08:57 +01:00
Eberhard Beilharz
e069c3ad2a
chore(core): remove meson warnings for wasm builds
Replace deprecated function calls. Also add `strip` command to wasm
build defaults.
2024-12-12 19:48:13 +01:00
Steven R. Loomis
de3397b476
Merge pull request #12644 from keymanapp/fix/core/12298-get-key-list
fix(core): implement ldml_processor::get_key_list() 🙀
2024-12-09 08:14:44 -06:00
Steven R. Loomis
feba97979a fix(core): update get_key_list to account for other combinatorics
- yes, expand 'other' to all possible combinations
- use ALT  and CTRL instead of RALT,LALT and RCTRL,LCTRL in the key list (reduce expansions up to 4x)

Fixes: #12298
2024-12-06 12:24:53 -06:00
Marc Durdin
5ec2ce7863 chore: address review comments 2024-12-05 05:35:27 +07:00
Steven R. Loomis
da7c864f54 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-12-04 13:58:24 -06:00
Marc Durdin
4c3b2132ee chore(core): remove km_core_keyboard_load API
Fixes: #12497
2024-12-04 13:38:07 +07:00
Marc Durdin
f97a19db91 chore(core): build parameters 2024-12-02 07:21:30 +07:00
Marc Durdin
9a01ac166d chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-12-01 16:17:21 +07:00
Steven R. Loomis
9e2b8d0d8c feat(core): improvements for get_key_list()
- reintroduce example keycaps

Fixes: #12298
2024-11-29 18:28:05 -06:00
Steven R. Loomis
7f3bd30961 feat(core): improvements for get_key_list()
- expand OTHER and ALT / CTRL appropriately
- add KM_CORE_MODIFIER_NONE=0
- disable test of get_key_list() for now

Fixes: #12298
2024-11-29 17:25:31 -06:00
Steven R. Loomis
d92dc175c2 chore(core): update to ldmL_test_source.cpp 2024-11-29 15:47:30 -06:00
Steven R. Loomis
29db34f252 chore(core): update to ldmL_test_source.cpp 2024-11-29 15:40:48 -06:00
Steven R. Loomis
c9643642a8 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-11-29 15:33:53 -06:00
Steven R. Loomis
2d81b195f0 feat(core): update k_004_tinyshift
- add a ctrl and an other layer

Fixes: #12298
2024-11-28 12:48:32 -06:00
Marc Durdin
b0e87cb85a chore(core): update assertion unit test for release builds 2024-11-28 12:24:54 +07:00
Marc Durdin
e640caaa9c chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-11-28 09:53:45 +07:00
Marc Durdin
d71cb56ca7 fix(core): rename assert() to test_assert() in unit tests
Had a real yak shave this morning with disabling assertions in release
builds in our C/C++ code. It turns out that our unit tests use
`assert()` which we intended to use from `test_assert.h`, but in some
cases `cassert` or `assert.h` had been #included after `test_assert.h`,
overriding our special `assert()` macro. The chain of includes is
somewhat hard to puzzle out -- it's often buried several levels deep.
This meant that a release build would drop all test assertions, meaning
most tests passed, unsurprisingly, as there were no assertions left to
fail ... but some tests failed with crashes because we optimized out
important lines such as `assert(some_important_function())`.

I was quite unhappy with this fragility, so I have opted to rename
`assert()` to `test_assert()` in all of our home-grown C/C++ unit tests,
which further highlighted unit tests which were only using the C/C++
`assert()` and not ours, so then had to figure out which unit test
executables needed to have `test_assert` added, and then ... then ...
discovered a bug in `test_color.h`, where we were #including
`io.h`/`unistd.h` inside a `namespace console_color {}` block, which
just happened to be the first ref to those beautiful headers, and thus
(because `#pragma once`) meant that useful little functions like
`access()` were no longer accessible to us in the global namespace.

I have also audited Every Single Call to `assert()` to verify that we do
not do Important Work inside the parentheses, and, apart from those
offending unit tests, now resolved with `test_assert()`, it looks like
all is good.

I would like to present one very well-shaved yak in this commit.

Fixes: #12619
2024-11-28 09:46:01 +07:00
Steven R. Loomis
c306147e6b
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-27 08:59:59 -06:00
Eberhard Beilharz
3dc3f040de
feat(core): implement loading KMX from blob
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a

Part-of: #11293
Part-of: #8093
2024-11-27 15:30:05 +01:00
Eberhard Beilharz
718881a027
chore(core): rename test files
This change rename the test files for Core according to the discussion
at the Keyman conference in November 2024.
2024-11-26 17:47:11 +01:00
Steven R. Loomis
f9862c3bdf feat(core): test improvements for get_key_list()
- just compare the key list to the key2.kmap table
- check the keylist for all LdmlTestSource instances - no syntax needed

Fixes: #12298
2024-11-26 09:36:11 -06:00
Steven R. Loomis
6d1035a4ad feat(core): test improvements for get_key_list()
- move kmxplus processing into the base LdmlTestSource class
- add a function to traverse the layer list looking for keys to add
- The @@keylist keyword only has one example from each modifier set

Fixes: #12298
2024-11-25 12:54:02 -06:00
Steven R. Loomis
741d3057a5
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-07 22:59:32 -06:00
Steven R. Loomis
382b511dce feat(core): ldml: implementation for testing get_key_list()
- ldml::vkeys class updated to keep a set<> of keys
- fix the test case to not leak!

Fixes: #12298
2024-11-07 09:34:30 -06:00