Commit graph

1119 commits

Author SHA1 Message Date
Marc Durdin
f3fb3ac87b maint(common): move build-utils-ci.inc.sh to ci/ci-publish.inc.sh
Removes a number of unused functions from the file, renames remaining
functions to match our builder script prefix patterns, removes unused
imports from various build scripts (e.g. Android).

Fixes: #14453
2025-08-05 12:54:56 +10:00
Marc Durdin
2726050601 maint(common): rename and move node-related script functions into node.inc.sh
Consolidates the node-related script functions into node.inc.sh, as part
of cleaning up the build scripts and making them easier to maintain into
the future.

Fixes: #14447
2025-08-02 13:34:29 +10:00
Marc Durdin
f394245636 maint(common): consolidate builder scripts
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.

Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh

Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
  android/build.sh.
* Renames functions in builder-basic.inc.sh

More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.

Fixes: #14065
Build-bot: build all
Test-bot: skip
2025-08-02 08:11:24 +10:00
Marc Durdin
65158a0526 maint(common): rename shellHelperFunctions.sh to utils.inc.sh
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.

Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
2025-08-02 06:58:40 +10:00
Marc Durdin
808b6557d6 maint(common): use cpp_std=c++17 in meson project default_options
We attempted to set the C++ version in standard.meson.build, but this
does not work well as the option name varies by platform, and it also
triggers a warning in meson. This fix moves the setting into each
meson.build project() default_options. As there are only a handful
of these, the maintenance burden is not high.

Fixes: #14432
Relates-to: #14355
Build-bot: build
Test-bot: skip
2025-08-01 10:23:57 +10:00
Eberhard Beilharz
51ca604b47
chore(common): Merge remote-tracking branch 'origin/master' into maint/common/14109_isOsFuncs 2025-07-21 09:20:12 +02:00
Eberhard Beilharz
faa521d82c
maint(common): set C++17 in standard.meson.build
Note that the ICU subprojects still explicitly specify C++17 since they
are maintained outside of Keyman and so shouldn't be tied to the Keyman
compiler version requirement.

Fixes: #12526
Test-bot: skip
2025-07-18 14:04:11 +02:00
Eberhard Beilharz
acebacc2b6
maint(core): explain (wrong) meson warning
Build-bot: skip
Test-bot: skip
2025-07-17 17:03:47 +02:00
Eberhard Beilharz
3b136fcec9
maint(common): consolidate functions to determine OS
- add `builder_is_windows`, `builder_is_macos` and `builder_is_linux`
  functions
- add/move `BUILDER_OS` env variable to `resources/builder.inc.sh`
- use the new functions and env variable

Fixes: #14109
Test-bot: skip
2025-07-09 18:33:09 +02:00
Steven R. Loomis
4d9551ae07
Merge pull request #14050 from keymanapp/feat/developer/13939-ln-layrcompiler
feat(developer): line numbers for layr compiler
2025-05-27 19:00:41 -05:00
Steven R. Loomis
b681885eca feat(developer): add more tests to layr compiler
- check for duplicate minDeviceWidth
- check for invalid minDeviceWidth

Fixes #13939
2025-05-26 12:30:59 -05:00
Marc Durdin
9f15e7c7ec maint: add empty 'scripts' tag to package.json to silence warning
Also removes './' from front of gosh.js to address another npm cleanup
(from `npm pkg fix`).

Fixes: #13833
2025-05-25 06:35:28 +07:00
Eberhard Beilharz
ee93d0d27a
maint(linux): add --no-werror option to not report warnings as errors
By default we add the `--werror` option to meson builds. This might cause
problems when compiling with a different compiler, so this option adds
the `--no-werror` option to our build scripts. This enables integrators
to compile with other compiler versions.

Fixes: #13970
Test-bot: skip
2025-05-20 09:01:22 +02:00
Marc Durdin
43cf66a9d4
Merge pull request #13796 from keymanapp/maint/meson-versioning-improvements
maint(common): cleanup meson versioning
2025-05-02 13:39:47 +07:00
Marc Durdin
9d2db4faf3 maint(common): consolidate standard.meson.build configuration
Use a shared function standard_meson_build to prep the
resources/meson.build folder for consistency. A future cleanup would be
to use meson-utils.inc.sh for all meson build scripts.
2025-04-30 09:16:43 +07:00
Marc Durdin
f8917d2454 maint(common): cleanup meson versioning
* Adds `VERSION_WITH_TAG` variable to standard.meson.build
* Consistently uses `file('VERSION.md')` to retrieve version number

Note that VERSION_WITH_TAG is not yet used but should be used for
version tagged files later.
2025-04-30 08:23:05 +07:00
Marc Durdin
7a56017f5b maint(core): batch build kmx in tests for performance
Build all the .kmx files used for kmn and ldml unit tests in batch
rather than one at a time. The ldml-test-data builds remain unbatched
for now (and are visibly slow!).

Fixes: #13495
2025-04-30 06:02:59 +07:00
Marc Durdin
ac6eb0b767 chore: use explicit node: prefix for node imports
This makes it easier for us to enumerate modules that have node
dependencies. There may be others I have not yet picked up.
2025-04-03 08:01:08 +07:00
Marc Durdin
a52437c407 chore: Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:38:08 +07:00
Eberhard Beilharz
1f8f979f61
Update core/include/keyman/keyman_core_api_vkeys.h
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-27 15:39:00 +01:00
Eberhard Beilharz
3e3a0dbce1
refactor(linux): mark undocumented keycodes as internal
Several keys that are defined in `winuser.h` were added after Keyman
defined its vkeys and so are not documented in the Keyman docs. This
change adds underscores before and after to mark them as being internal.
It's still useful to have them in the source code because it makes the
mapping between Linux keys and Keyman vkeys easier.

Addresses code review comment.
2025-03-26 16:49:31 +01:00
Eberhard Beilharz
a645fea903
fix(linux): improve key mapping to allow Core to reset context
This change tries to map all keys to corresponding Core VKeys so that
Core sees them and can reset the context if necessary. Previously we
had 0 for most of the keys, which meant that we didn't send them to
Core but instead returned right away. While this doesn't directly fix
the failing user tests described in #12968, it fixes the original tests
in #11172.

Closes: #12968
2025-03-21 18:15:46 +01:00
Steven R. Loomis
443681999d feat(core): improve kmxplus validation tests
- update per review notes

Fixes: #9446
2025-03-18 11:08:44 -05:00
Steven R. Loomis
9989217f9e feat(core): improve kmxplus validation
- restructured validation calls to be more consistent

Fixes: #9446
2025-03-17 12:09:04 -05:00
Steven R. Loomis
0011a2d10d feat(core): improve kmxplus validation
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test

Fixes: #9446

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-17 11:26:57 -05:00
Steven R. Loomis
f1b3555114 feat(core,docs): improve kmxplus validation
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections

Fixes: #9446
2025-03-14 15:37:23 -05:00
Steven R. Loomis
cd002441e5 feat(core): check non-extended keys for bad unicode
- add test case
- turn this into not be an assert, so we can test it

Fixes: #9446
2025-03-14 10:04:21 -05:00
Steven R. Loomis
2e7b5b1417 feat(core): check non-extended keys for bad unicode
- also, use U+FFFD in core if it's a bad char

Fixes: #9446
2025-03-14 09:31:38 -05:00
Steven R. Loomis
b2c3a46283
feat(core) Update core/src/ldml/ldml_processor.cpp
map bad chars to U+0020

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-14 09:23:37 -05:00
Steven R. Loomis
2e50282087
Merge branch 'master' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-14 09:19:03 -05:00
Steven R. Loomis
a0c2daf25b
Merge pull request #13494 from keymanapp/chore/core/9446-kmx-plus-tests-gtest
feat(core): move kmx_plus.tests.cpp to GTest 🙀
2025-03-14 09:18:34 -05:00
Steven R. Loomis
b696e34001 feat(core): move kmx_plus.tests.cpp to GTest
- msvc global to UTF-8
2025-03-13 22:47:39 -05:00
Steven R. Loomis
289151c990 feat(core): move kmx_plus.tests.cpp to GTest
- set kmx_plus_tests.cpp to UTF-8
2025-03-13 17:34:37 -05:00
Steven R. Loomis
d9602cede1 feat(core): assert that no noncharacter emitted from ldml_processor
- not counting markers, which have a special path

Fixes: #9446
2025-03-13 17:22:37 -05:00
Steven R. Loomis
1132fe5845 feat(core): tests for check on load for valid kmx_plus unicode
- turn off some asserts- makes untestable
(there are asserts at 'higher levels' such as loading the entire kmx+)
- add a test with a synthesized COMP_KMXPLUS_STRS - a valid and an invalid one

Fixes: #9446
2025-03-13 17:15:21 -05:00
Steven R. Loomis
fba5f72844 feat(core): tests for check on load for valid kmx_plus unicode
Fixes: #9446
2025-03-13 16:44:31 -05:00
Steven R. Loomis
05f3e3ca52 Merge branch 'chore/core/9446-kmx-plus-tests-gtest' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-13 15:06:31 -05:00
Steven R. Loomis
1306cf8760 feat(core): move kmx_plus.tests.cpp to GTest
- also update test_assert.h to vector to GTest if present
2025-03-13 15:05:55 -05:00
Marc Durdin
27e53011af
Merge pull request #13450 from keymanapp/fix/core/13419-uset-unaligned
fix(core): unalign `usetCount` and `rangeCount`
2025-03-13 13:19:04 +07:00
Steven R. Loomis
2c2bfe7f15 feat(core): check on load for valid kmx_plus unicode
- error out if failure

Fixes: #9446
2025-03-12 17:45:47 -05:00
Marc Durdin
3cdaa2fee0 chore(core): add clean actions for :mac and :win targets
Fixes: #13381
2025-03-10 07:53:32 +07:00
Marc Durdin
577cbd3a07 fix(core): unalign usetCount and rangeCount
Fixes: #13419
2025-03-07 12:50:22 +07:00
Marc Durdin
03001bd971 chore(common): add unit test for nul and context() in context part of rule
The compiler and Core appear to be working correctly in this situation;
this test verifies Core's behaviour.

Fixes: #13299
2025-03-04 11:22:35 +01:00
Marc Durdin
d83454ab77 chore(windows): ensure dependencies for keyman32 exclude core:wasm
Fixes: #13374
2025-02-28 17:52:27 +01:00
Marc Durdin
b973bcc0d9 fix(core): include nul in context offset calculations in kmx processor
Two separate bugs addressed, with `index()` references and with
`context()` references -- both have the same root cause, of not taking
`nul` at the start of the context into account (as `nul` is not included
in the `m_miniContext` member, being a non-character). We already fixed
this issue for `if()` quite a long time ago, and some of the same
patterns can be with `m_miniContextIfLen` for example.

Fixes: #13304
Fixes: #13316
2025-02-21 17:17:19 +07:00
Marc Durdin
2b9b8eb419 chore(core): add unit tests to validate nul with index and context
Add 4 test keyboards to validate `nul` and `if` used in conjunction with
`index` and `context` and corresponding references in core unit tests.

Add a script to rebuild baseline keyboards using a copy of kmcomp.exe
16.0.138; this is setup and tested only on Windows (YMMV on WINE, etc).

Add the 4 additional baseline test keyboards to kmcmplib unit tests for
build consistency between kmcomp 16 and kmcmplib 18 (all pass).

Note: two of the new Core tests currently fail. This is expected, see
issue #13304.

Fixes: #13303
2025-02-21 13:06:29 +07:00
Steven R. Loomis
b54ea40770 chore(developer): update comments per review
- note that 45,46,47 are the same data and so aliased

Fixes: #11307
2025-01-30 23:13:36 -06:00
Steven R. Loomis
b741a88ca0 chore(core,developer): treat CLDR v45, 46, 46.1, 47 as 46
Fixes: #11307
2025-01-30 15:42:30 -06:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Eberhard Beilharz
b84b651590
Merge pull request #11397 from keymanapp/chore/linux/cicontainer
chore: Add docker images for building the different platforms
2025-01-16 12:31:29 +01:00