When normalizing, we need to stop processing on an NFC boundary, not an
NFD boundary, to support normalizations such as in Bengali, where
appending `U+09D7` to a context of `U+0995 U+09C7` should result in
`U+0995 U+09CC`.
The specification is unclear on this; see https://unicode-org.atlassian.net/browse/CLDR-19218
This also updates the ldml keyboard unit test suite to support running
in full NFC mode (used in all Engine implementations) as well retaining
the NFD mode (now only used by the debugger).
Side note: the Bengali normalization failure case was picked up by the
improvements to the unit test suite, proving once again that good tests
are so valuable.
Fixes: #15491Fixes: #15505
Follows: #15488
Relates-to: CLDR-19218
Ensure that when a single backspace decomposes the last NFC character in
the app context, the remainder of the 'cluster' is preserved, matching
the implication of the CLDR keyboard specification.
This addresses the behavior in #15487 where the cached context became
out of sync with the app context after deleting an entire NFC cluster
such as ê, which caused a loop ending up with the entire context being
deleted, at which point the loop exited with a fail-safe.
Note that the LDML keyboard tests (ldml.cpp) do not currently exercise
the normalization code; this is a gap that should be addressed to ensure
that we are testing final application behavior.
Fixes: #15487
Add a GitHub action to test Keyman Core on Windows / ARM64. We do not
have a TeamCity ARM64 agent for this. There are a number of config
challenges in making this work as many deps do not yet support ARM64;
however the changes are isolated to the .yml.
Test-bot: skip
With VS2022 the limitations for the -hostarch flag not being available
on arm environment and the -arch only having x86 and x64 has been
removed. There is now single command that works for 3 builds on either
host architecture.
Fixes: 15239
Test-bot: skip
Add a host check for build.bat and run the architecture tests
that match the host. for the meson builds from build.h
for now just skip the arm64 tests.
Meson 1.1 changed the name of the options file from `meson_options.txt`
to `meson.options`. Currently we're stuck with Meson 1.0 as minimum
Meson version because we still support Ubuntu 22.04 Jammy, but on some
machines a newer version of meson gets installed which causes the
options to be ignored.
The first attempt to simply symlink or copy `meson_options.txt` to
`meson.options` failed because git on Windows doesn't support symlinks
by default, and Meson complains if it finds two options files (#15127).
Fortunately not having an options file is only a problem for Core where
we have one option in the options file that defaults to true. The solution
implemented in this change is to always pass the option to Meson.
Test-bot: skip
While not documented as such, it appears that the repository record is
required with trusted publishing of npm packages.
Test-bot: skip
Build-bot: skip
Update the build.bat to handle configuring the build
environment with vsdevcmd correctly for arm64 2019 doesn't
work very well so have to have pretty weird if statement.
also checked in some missed updates to function name change
Due to recent changes in NPM package publishing security requirements,
we have to move from TeamCity build to a GitHub Action to publish our
NPM packages, so we can take advantage of trusted publishing. This
change also consolidates and centralizes the npm publishing into
resources/build/ci/npm-publish.sh, which removes a lot of boilerplate
from each of the build.sh scripts, and ensures consistency.
Packages will be `npm pack`ed on PR and test builds, and published in
release builds.
Ref: https://docs.npmjs.com/trusted-publishers
Ref: https://github.blog/changelog/2025-09-29-strengthening-npm-security-important-changes-to-authentication-and-token-management/Fixes: #14963
Test-bot: skip
Build-bot: release:developer
Note: #11909 has some additional future cleanup we could do, but these
code paths are not currently accessible, so in the interest of moving
forward, am leaving them for now.
Fixes: #11909
Test-bot: skip
Removes a number of unused functions from the file, renames remaining
functions to match our builder script prefix patterns, removes unused
imports from various build scripts (e.g. Android).
Fixes: #14453
Consolidates the node-related script functions into node.inc.sh, as part
of cleaning up the build scripts and making them easier to maintain into
the future.
Fixes: #14447
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.
Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh
Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
android/build.sh.
* Renames functions in builder-basic.inc.sh
More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.
Fixes: #14065
Build-bot: build all
Test-bot: skip
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.
Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
We attempted to set the C++ version in standard.meson.build, but this
does not work well as the option name varies by platform, and it also
triggers a warning in meson. This fix moves the setting into each
meson.build project() default_options. As there are only a handful
of these, the maintenance burden is not high.
Fixes: #14432
Relates-to: #14355
Build-bot: build
Test-bot: skip
Note that the ICU subprojects still explicitly specify C++17 since they
are maintained outside of Keyman and so shouldn't be tied to the Keyman
compiler version requirement.
Fixes: #12526
Test-bot: skip
- add `builder_is_windows`, `builder_is_macos` and `builder_is_linux`
functions
- add/move `BUILDER_OS` env variable to `resources/builder.inc.sh`
- use the new functions and env variable
Fixes: #14109
Test-bot: skip
By default we add the `--werror` option to meson builds. This might cause
problems when compiling with a different compiler, so this option adds
the `--no-werror` option to our build scripts. This enables integrators
to compile with other compiler versions.
Fixes: #13970
Test-bot: skip
Use a shared function standard_meson_build to prep the
resources/meson.build folder for consistency. A future cleanup would be
to use meson-utils.inc.sh for all meson build scripts.
* Adds `VERSION_WITH_TAG` variable to standard.meson.build
* Consistently uses `file('VERSION.md')` to retrieve version number
Note that VERSION_WITH_TAG is not yet used but should be used for
version tagged files later.
Build all the .kmx files used for kmn and ldml unit tests in batch
rather than one at a time. The ldml-test-data builds remain unbatched
for now (and are visibly slow!).
Fixes: #13495
Several keys that are defined in `winuser.h` were added after Keyman
defined its vkeys and so are not documented in the Keyman docs. This
change adds underscores before and after to mark them as being internal.
It's still useful to have them in the source code because it makes the
mapping between Linux keys and Keyman vkeys easier.
Addresses code review comment.
This change tries to map all keys to corresponding Core VKeys so that
Core sees them and can reset the context if necessary. Previously we
had 0 for most of the keys, which meant that we didn't send them to
Core but instead returned right away. While this doesn't directly fix
the failing user tests described in #12968, it fixes the original tests
in #11172.
Closes: #12968
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test
Fixes: #9446
Co-authored-by: Marc Durdin <marc@durdin.net>
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections
Fixes: #9446
