Commit graph

504 commits

Author SHA1 Message Date
Marc Durdin
7a56017f5b maint(core): batch build kmx in tests for performance
Build all the .kmx files used for kmn and ldml unit tests in batch
rather than one at a time. The ldml-test-data builds remain unbatched
for now (and are visibly slow!).

Fixes: #13495
2025-04-30 06:02:59 +07:00
Steven R. Loomis
443681999d feat(core): improve kmxplus validation tests
- update per review notes

Fixes: #9446
2025-03-18 11:08:44 -05:00
Steven R. Loomis
0011a2d10d feat(core): improve kmxplus validation
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test

Fixes: #9446

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-17 11:26:57 -05:00
Steven R. Loomis
f1b3555114 feat(core,docs): improve kmxplus validation
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections

Fixes: #9446
2025-03-14 15:37:23 -05:00
Steven R. Loomis
cd002441e5 feat(core): check non-extended keys for bad unicode
- add test case
- turn this into not be an assert, so we can test it

Fixes: #9446
2025-03-14 10:04:21 -05:00
Steven R. Loomis
2e50282087
Merge branch 'master' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-14 09:19:03 -05:00
Steven R. Loomis
a0c2daf25b
Merge pull request #13494 from keymanapp/chore/core/9446-kmx-plus-tests-gtest
feat(core): move kmx_plus.tests.cpp to GTest 🙀
2025-03-14 09:18:34 -05:00
Steven R. Loomis
b696e34001 feat(core): move kmx_plus.tests.cpp to GTest
- msvc global to UTF-8
2025-03-13 22:47:39 -05:00
Steven R. Loomis
289151c990 feat(core): move kmx_plus.tests.cpp to GTest
- set kmx_plus_tests.cpp to UTF-8
2025-03-13 17:34:37 -05:00
Steven R. Loomis
1132fe5845 feat(core): tests for check on load for valid kmx_plus unicode
- turn off some asserts- makes untestable
(there are asserts at 'higher levels' such as loading the entire kmx+)
- add a test with a synthesized COMP_KMXPLUS_STRS - a valid and an invalid one

Fixes: #9446
2025-03-13 17:15:21 -05:00
Steven R. Loomis
fba5f72844 feat(core): tests for check on load for valid kmx_plus unicode
Fixes: #9446
2025-03-13 16:44:31 -05:00
Steven R. Loomis
1306cf8760 feat(core): move kmx_plus.tests.cpp to GTest
- also update test_assert.h to vector to GTest if present
2025-03-13 15:05:55 -05:00
Marc Durdin
03001bd971 chore(common): add unit test for nul and context() in context part of rule
The compiler and Core appear to be working correctly in this situation;
this test verifies Core's behaviour.

Fixes: #13299
2025-03-04 11:22:35 +01:00
Marc Durdin
2b9b8eb419 chore(core): add unit tests to validate nul with index and context
Add 4 test keyboards to validate `nul` and `if` used in conjunction with
`index` and `context` and corresponding references in core unit tests.

Add a script to rebuild baseline keyboards using a copy of kmcomp.exe
16.0.138; this is setup and tested only on Windows (YMMV on WINE, etc).

Add the 4 additional baseline test keyboards to kmcmplib unit tests for
build consistency between kmcomp 16 and kmcmplib 18 (all pass).

Note: two of the new Core tests currently fail. This is expected, see
issue #13304.

Fixes: #13303
2025-02-21 13:06:29 +07:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Eberhard Beilharz
9c8ad2e783
fix(core): fix compile error
Compiling the tests failed in the container.
2025-01-08 17:08:57 +01:00
Eberhard Beilharz
e069c3ad2a
chore(core): remove meson warnings for wasm builds
Replace deprecated function calls. Also add `strip` command to wasm
build defaults.
2024-12-12 19:48:13 +01:00
Steven R. Loomis
de3397b476
Merge pull request #12644 from keymanapp/fix/core/12298-get-key-list
fix(core): implement ldml_processor::get_key_list() 🙀
2024-12-09 08:14:44 -06:00
Steven R. Loomis
feba97979a fix(core): update get_key_list to account for other combinatorics
- yes, expand 'other' to all possible combinations
- use ALT  and CTRL instead of RALT,LALT and RCTRL,LCTRL in the key list (reduce expansions up to 4x)

Fixes: #12298
2024-12-06 12:24:53 -06:00
Marc Durdin
5ec2ce7863 chore: address review comments 2024-12-05 05:35:27 +07:00
Steven R. Loomis
da7c864f54 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-12-04 13:58:24 -06:00
Marc Durdin
4c3b2132ee chore(core): remove km_core_keyboard_load API
Fixes: #12497
2024-12-04 13:38:07 +07:00
Marc Durdin
f97a19db91 chore(core): build parameters 2024-12-02 07:21:30 +07:00
Marc Durdin
9a01ac166d chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-12-01 16:17:21 +07:00
Steven R. Loomis
9e2b8d0d8c feat(core): improvements for get_key_list()
- reintroduce example keycaps

Fixes: #12298
2024-11-29 18:28:05 -06:00
Steven R. Loomis
7f3bd30961 feat(core): improvements for get_key_list()
- expand OTHER and ALT / CTRL appropriately
- add KM_CORE_MODIFIER_NONE=0
- disable test of get_key_list() for now

Fixes: #12298
2024-11-29 17:25:31 -06:00
Steven R. Loomis
d92dc175c2 chore(core): update to ldmL_test_source.cpp 2024-11-29 15:47:30 -06:00
Steven R. Loomis
29db34f252 chore(core): update to ldmL_test_source.cpp 2024-11-29 15:40:48 -06:00
Steven R. Loomis
c9643642a8 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-11-29 15:33:53 -06:00
Steven R. Loomis
2d81b195f0 feat(core): update k_004_tinyshift
- add a ctrl and an other layer

Fixes: #12298
2024-11-28 12:48:32 -06:00
Marc Durdin
b0e87cb85a chore(core): update assertion unit test for release builds 2024-11-28 12:24:54 +07:00
Marc Durdin
e640caaa9c chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-11-28 09:53:45 +07:00
Marc Durdin
d71cb56ca7 fix(core): rename assert() to test_assert() in unit tests
Had a real yak shave this morning with disabling assertions in release
builds in our C/C++ code. It turns out that our unit tests use
`assert()` which we intended to use from `test_assert.h`, but in some
cases `cassert` or `assert.h` had been #included after `test_assert.h`,
overriding our special `assert()` macro. The chain of includes is
somewhat hard to puzzle out -- it's often buried several levels deep.
This meant that a release build would drop all test assertions, meaning
most tests passed, unsurprisingly, as there were no assertions left to
fail ... but some tests failed with crashes because we optimized out
important lines such as `assert(some_important_function())`.

I was quite unhappy with this fragility, so I have opted to rename
`assert()` to `test_assert()` in all of our home-grown C/C++ unit tests,
which further highlighted unit tests which were only using the C/C++
`assert()` and not ours, so then had to figure out which unit test
executables needed to have `test_assert` added, and then ... then ...
discovered a bug in `test_color.h`, where we were #including
`io.h`/`unistd.h` inside a `namespace console_color {}` block, which
just happened to be the first ref to those beautiful headers, and thus
(because `#pragma once`) meant that useful little functions like
`access()` were no longer accessible to us in the global namespace.

I have also audited Every Single Call to `assert()` to verify that we do
not do Important Work inside the parentheses, and, apart from those
offending unit tests, now resolved with `test_assert()`, it looks like
all is good.

I would like to present one very well-shaved yak in this commit.

Fixes: #12619
2024-11-28 09:46:01 +07:00
Steven R. Loomis
c306147e6b
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-27 08:59:59 -06:00
Eberhard Beilharz
3dc3f040de
feat(core): implement loading KMX from blob
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a

Part-of: #11293
Part-of: #8093
2024-11-27 15:30:05 +01:00
Eberhard Beilharz
718881a027
chore(core): rename test files
This change rename the test files for Core according to the discussion
at the Keyman conference in November 2024.
2024-11-26 17:47:11 +01:00
Steven R. Loomis
f9862c3bdf feat(core): test improvements for get_key_list()
- just compare the key list to the key2.kmap table
- check the keylist for all LdmlTestSource instances - no syntax needed

Fixes: #12298
2024-11-26 09:36:11 -06:00
Steven R. Loomis
6d1035a4ad feat(core): test improvements for get_key_list()
- move kmxplus processing into the base LdmlTestSource class
- add a function to traverse the layer list looking for keys to add
- The @@keylist keyword only has one example from each modifier set

Fixes: #12298
2024-11-25 12:54:02 -06:00
Steven R. Loomis
741d3057a5
Merge branch 'master' into fix/core/12298-get-key-list 2024-11-07 22:59:32 -06:00
Steven R. Loomis
382b511dce feat(core): ldml: implementation for testing get_key_list()
- ldml::vkeys class updated to keep a set<> of keys
- fix the test case to not leak!

Fixes: #12298
2024-11-07 09:34:30 -06:00
Marc Durdin
fbab6eb8cf chore(core): move API docs from help.keyman.com
Reorganizes /core/doc to /core/docs and moves internal files
accordingly.

Depends-on: keymanapp/help.keyman.com#1684
2024-11-07 08:28:33 +07:00
Steven R. Loomis
dd11a25b27 feat(core): ldml: scaffolding for testing get_key_list()
- move some test utils to statics
- add a new test action type and `@@key-list` keyword
- print warning on unhanded @-commands

Fixes: #12298
2024-11-06 09:30:18 -06:00
Darcy Wong
e62d05cff7 fix(core): Disable test_unicode check for ICU version 2024-09-11 20:56:50 +07:00
Marc Durdin
613bdb6c19 refactor(core): move utfcodec to common
Refactor to support codecvt cleanup work in kmcmplib.
2024-08-13 15:04:07 +02:00
Eberhard Beilharz
88f6c95803
fix(core): allow to successfully build on Ubuntu 24.04
Without this change trying to build on Ubuntu 24.04 fails with
"error: 'intptr_t' has not been declared in '::'". I'm not sure
why that is happening, but this is an easy fix.
2024-07-04 13:07:10 +02:00
Steven R. Loomis
a16608e229
Merge branch 'master' into test/core/10968-minimum-ldml-test 2024-06-17 08:24:11 -05:00
Steven R. Loomis
ed80095ab9
Merge branch 'master' into test/core/10968-minimum-ldml-test 2024-06-14 09:33:29 -05:00
Steven R. Loomis
2300571ad8 test(core): fix a type conversion issue in the minimal test
- need a null string for either narrow or wide chars

Fixes: #10968
2024-06-14 09:05:50 -05:00
Steven R. Loomis
3c82343218 test(core): Add a minimal test that exercises the core API
- this test doesn't actually run well, it compiles but fails with an assertion.
- the purpose is to exercise the linker.

Fixes: #10968
2024-06-13 16:56:36 -05:00
Steven R. Loomis
ff3c9736f4
Merge branch 'master' into feat/core/9467-devolve-regex-to-js2-epic-ldml 2024-06-13 14:29:01 -05:00