Commit graph

1147 commits

Author SHA1 Message Date
Marc Durdin
ffec6a21f3 chore: Merge remote-tracking branch 'origin/epic/web-core' into chore/merge-dev-refactors-into-web-core 2025-04-03 12:19:28 +07:00
Marc Durdin
ac6eb0b767 chore: use explicit node: prefix for node imports
This makes it easier for us to enumerate modules that have node
dependencies. There may be others I have not yet picked up.
2025-04-03 08:01:08 +07:00
Marc Durdin
7c42887ead
Merge branch 'epic/web-core' into chore/merge-master-into-web-core 2025-03-31 12:13:48 +07:00
Marc Durdin
a52437c407 chore: Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:38:08 +07:00
Eberhard Beilharz
1f8f979f61
Update core/include/keyman/keyman_core_api_vkeys.h
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-27 15:39:00 +01:00
Eberhard Beilharz
3e3a0dbce1
refactor(linux): mark undocumented keycodes as internal
Several keys that are defined in `winuser.h` were added after Keyman
defined its vkeys and so are not documented in the Keyman docs. This
change adds underscores before and after to mark them as being internal.
It's still useful to have them in the source code because it makes the
mapping between Linux keys and Keyman vkeys easier.

Addresses code review comment.
2025-03-26 16:49:31 +01:00
Eberhard Beilharz
a645fea903
fix(linux): improve key mapping to allow Core to reset context
This change tries to map all keys to corresponding Core VKeys so that
Core sees them and can reset the context if necessary. Previously we
had 0 for most of the keys, which meant that we didn't send them to
Core but instead returned right away. While this doesn't directly fix
the failing user tests described in #12968, it fixes the original tests
in #11172.

Closes: #12968
2025-03-21 18:15:46 +01:00
Steven R. Loomis
443681999d feat(core): improve kmxplus validation tests
- update per review notes

Fixes: #9446
2025-03-18 11:08:44 -05:00
Steven R. Loomis
9989217f9e feat(core): improve kmxplus validation
- restructured validation calls to be more consistent

Fixes: #9446
2025-03-17 12:09:04 -05:00
Steven R. Loomis
0011a2d10d feat(core): improve kmxplus validation
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test

Fixes: #9446

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-17 11:26:57 -05:00
Steven R. Loomis
f1b3555114 feat(core,docs): improve kmxplus validation
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections

Fixes: #9446
2025-03-14 15:37:23 -05:00
Steven R. Loomis
cd002441e5 feat(core): check non-extended keys for bad unicode
- add test case
- turn this into not be an assert, so we can test it

Fixes: #9446
2025-03-14 10:04:21 -05:00
Steven R. Loomis
2e7b5b1417 feat(core): check non-extended keys for bad unicode
- also, use U+FFFD in core if it's a bad char

Fixes: #9446
2025-03-14 09:31:38 -05:00
Steven R. Loomis
b2c3a46283
feat(core) Update core/src/ldml/ldml_processor.cpp
map bad chars to U+0020

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-14 09:23:37 -05:00
Steven R. Loomis
2e50282087
Merge branch 'master' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-14 09:19:03 -05:00
Steven R. Loomis
a0c2daf25b
Merge pull request #13494 from keymanapp/chore/core/9446-kmx-plus-tests-gtest
feat(core): move kmx_plus.tests.cpp to GTest 🙀
2025-03-14 09:18:34 -05:00
Marc Durdin
068da83e50 chore: Merge branch 'epic/web-core' into chore/merge-master-into-web-core 2025-03-14 11:05:17 +07:00
Marc Durdin
c35e0dac5f chore: Merge branch 'epic/web-core' into chore/merge-master-into-web-core 2025-03-14 11:02:37 +07:00
Steven R. Loomis
b696e34001 feat(core): move kmx_plus.tests.cpp to GTest
- msvc global to UTF-8
2025-03-13 22:47:39 -05:00
Steven R. Loomis
289151c990 feat(core): move kmx_plus.tests.cpp to GTest
- set kmx_plus_tests.cpp to UTF-8
2025-03-13 17:34:37 -05:00
Steven R. Loomis
d9602cede1 feat(core): assert that no noncharacter emitted from ldml_processor
- not counting markers, which have a special path

Fixes: #9446
2025-03-13 17:22:37 -05:00
Steven R. Loomis
1132fe5845 feat(core): tests for check on load for valid kmx_plus unicode
- turn off some asserts- makes untestable
(there are asserts at 'higher levels' such as loading the entire kmx+)
- add a test with a synthesized COMP_KMXPLUS_STRS - a valid and an invalid one

Fixes: #9446
2025-03-13 17:15:21 -05:00
Steven R. Loomis
fba5f72844 feat(core): tests for check on load for valid kmx_plus unicode
Fixes: #9446
2025-03-13 16:44:31 -05:00
Steven R. Loomis
05f3e3ca52 Merge branch 'chore/core/9446-kmx-plus-tests-gtest' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-13 15:06:31 -05:00
Steven R. Loomis
1306cf8760 feat(core): move kmx_plus.tests.cpp to GTest
- also update test_assert.h to vector to GTest if present
2025-03-13 15:05:55 -05:00
Marc Durdin
27e53011af
Merge pull request #13450 from keymanapp/fix/core/13419-uset-unaligned
fix(core): unalign `usetCount` and `rangeCount`
2025-03-13 13:19:04 +07:00
Steven R. Loomis
2c2bfe7f15 feat(core): check on load for valid kmx_plus unicode
- error out if failure

Fixes: #9446
2025-03-12 17:45:47 -05:00
Marc Durdin
3cdaa2fee0 chore(core): add clean actions for :mac and :win targets
Fixes: #13381
2025-03-10 07:53:32 +07:00
Marc Durdin
577cbd3a07 fix(core): unalign usetCount and rangeCount
Fixes: #13419
2025-03-07 12:50:22 +07:00
Eberhard Beilharz
5c7030ffa2
refactor(core): address code review comment 2025-03-05 18:05:35 +01:00
Marc Durdin
03001bd971 chore(common): add unit test for nul and context() in context part of rule
The compiler and Core appear to be working correctly in this situation;
this test verifies Core's behaviour.

Fixes: #13299
2025-03-04 11:22:35 +01:00
Eberhard Beilharz
b2cccdd996
feat(core): expose context and actions APIs to WASM 2025-03-03 11:29:17 +01:00
Eberhard Beilharz
c297c66a64
feat(core): expose state and event API to WASM 2025-03-03 11:28:57 +01:00
Eberhard Beilharz
fe3e02cf9e
feat(web): implement getting properties of KMXKeyboard 2025-03-01 17:18:31 +01:00
Marc Durdin
d83454ab77 chore(windows): ensure dependencies for keyman32 exclude core:wasm
Fixes: #13374
2025-02-28 17:52:27 +01:00
Marc Durdin
b973bcc0d9 fix(core): include nul in context offset calculations in kmx processor
Two separate bugs addressed, with `index()` references and with
`context()` references -- both have the same root cause, of not taking
`nul` at the start of the context into account (as `nul` is not included
in the `m_miniContext` member, being a non-character). We already fixed
this issue for `if()` quite a long time ago, and some of the same
patterns can be with `m_miniContextIfLen` for example.

Fixes: #13304
Fixes: #13316
2025-02-21 17:17:19 +07:00
Marc Durdin
2b9b8eb419 chore(core): add unit tests to validate nul with index and context
Add 4 test keyboards to validate `nul` and `if` used in conjunction with
`index` and `context` and corresponding references in core unit tests.

Add a script to rebuild baseline keyboards using a copy of kmcomp.exe
16.0.138; this is setup and tested only on Windows (YMMV on WINE, etc).

Add the 4 additional baseline test keyboards to kmcmplib unit tests for
build consistency between kmcomp 16 and kmcmplib 18 (all pass).

Note: two of the new Core tests currently fail. This is expected, see
issue #13304.

Fixes: #13303
2025-02-21 13:06:29 +07:00
Marc Durdin
130e697d40
Merge pull request #13230 from keymanapp/chore/merge-master-into-web-core
chore: merge master into web-core
2025-02-14 05:30:04 +07:00
Steven R. Loomis
b54ea40770 chore(developer): update comments per review
- note that 45,46,47 are the same data and so aliased

Fixes: #11307
2025-01-30 23:13:36 -06:00
Steven R. Loomis
b741a88ca0 chore(core,developer): treat CLDR v45, 46, 46.1, 47 as 46
Fixes: #11307
2025-01-30 15:42:30 -06:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Eberhard Beilharz
fbfa3b5cc2
feat(core): add unit tests for the WASM version of Core API
Note that this requires adding an additional `km-core-node.mjs`
module (since the tests are run with node). We can't create one that
would work for both web and node - if we do trying to load the `km-core.js`
fails with `km-core.mjs: Could not resolve import "module".`
2025-01-20 18:22:56 +01:00
Joshua Horton
0d0c339f67
Merge branch 'epic/web-core' into chore/merge-master-into-web-core 2025-01-17 11:37:51 +07:00
Eberhard Beilharz
b84b651590
Merge pull request #11397 from keymanapp/chore/linux/cicontainer
chore: Add docker images for building the different platforms
2025-01-16 12:31:29 +01:00
Eberhard Beilharz
5a0cb9bff5
Merge pull request #12823 from keymanapp/feat/web/loadkmx
feat(web): load .kmx keyboard from blob 🎼
2025-01-15 10:41:36 +01:00
Eberhard Beilharz
9c8ad2e783
fix(core): fix compile error
Compiling the tests failed in the container.
2025-01-08 17:08:57 +01:00
Eberhard Beilharz
2e5722705b
feat(web): address code review comments 2025-01-06 15:12:09 +01:00
Eberhard Beilharz
69cb2fda59
fix(core): use a specific flag for adjusting alignments
This change adds a new flag `KMX_REQUIRES_REALIGNMENT` instead of the
previous `KMX_64BIT` for the cases where we have to re-align the KMX data
at pointer boundaries. Renamed because Emscripten also needs the
re-alignment even though it's 32-bit.

Addresses code review comments.
2025-01-06 14:29:57 +01:00
Eberhard Beilharz
a072171ec8
feat(web): load .kmx keyboard from blob
This change adds the web side of loading a .kmx keyboard from a
blob. It also replaces the `CoreProcessor` class with `CoreFactory`
that allows to directly use the methods defined in WASM without
having to add another wrapper for each method.
2024-12-20 18:01:42 +01:00
Eberhard Beilharz
6bef09f68b
fix(core): work around alignment problem when using Emscripten
The data we read from the KMX file is not in places 32-bit aligned, e.g.
`dpGroupArray` will often start at a 16-bit boundary. Emscripten doesn't
like this and will abort the program if `SAFE_HEAP=1` is defined.

This change works around the problem by calling `CopyKeyboard` and thus
expanding the structure.

See #12844.
2024-12-20 18:01:41 +01:00