Add support for embedding .keyman-touch-layout into .kmx and complete
support and tests for embedding .kvk. Fixup a number of related
functions.
Add special-key-caps.ts to developer-utils -- in future this will become
the primary source for developer for these key caps, but this cannot be
done until Developer Server and Touch Layout Editor are updated with
Typescript in client code.
Add a KMX+ file reader and tests, including a round-trip builder:reader
test.
Move basic-17.txt, basic-19.txt, and basic.xml to common test resources
folder so they can be used by unit tests in common and developer.
Test-bot: skip
Address a review comment from #15665 and cleanup additional references
that I missed the first time around, along with the `fs.readFileSync`
`Uint8Array` cast.
Follows: #15665
Test-bot: skip
Use RichEdit's `ITextDocument` interface to retrieve the selection
anchor/caret information, and avoid the side-effects that can arise with
the hacky `EM_GETSEL` / `EM_SETSEL` pattern we used previously.
Changing the selection with `EM_SETSEL` in order to find the anchor
point causes notification messages to be generated that can arrive at an
unexpected time in some text selection scenarios, which ended up with us
having a saved selection in the debugger pointing to the wrong text
range.
The name `Anchor` was a misnomer, because it was actually returning the
caret position, not the anchor for the selection, which is the far side
of the selection from the anchor!
Fixes: #11706
Fixes: KEYMAN-DEVELOPER-18A
Simplify usage of `TestCompilerCallbacks` by making it responsible for
the `beforeEach` and `afterEach` incantations itself. There are a couple
of more complex usages of `TestCompilerCallbacks` which have been
excluded from this change on purpose.
Renamed `testCallbacks` to `callbacks` in the one place where it was
different.
Hoist @types/node and @types/mocha because inconsistent versions of
@types/mocha were causing compiler errors with this change.
Also added a cast to `fs.readFileSync` to `Uint8Array` to eliminate
compiler warnings/errors in test files.
Fixes: #15654
Test-bot: skip
Implement the conversion from .kvk into KMX+ format, and embed the
result into the .kmx file. This change does not yet support &displaymap.
Move some virtual key modifier state helpers out of
visual-keyboard-compiler.ts and into shared module
modifier-key-constants.ts.
This change also flags the need to update the package compiler to
rewrite the font metadata into the KMX+ tables.
Test-bot: skip
The disp and layr sections have new v19 layouts, to support the Keyman
OSK requirements for epic/embed-osk-in-kmx. This supports writing the
new versions of these sections in Developer and loading them into Core,
doing transformations where necessary so that Core always works with v19
structures after load.
This does not implement the transformations required to support the OSK
APIs; that will be implemented in a follow-up PR. Nor is support for
writing the OSK data from source .kvks and .keyman-touch-layout
supported; this is just the scaffolding for supporting the structures in
the KMX+ data.
Test-bot: skip
The name `form` better represents the semantic meaning of the
structure, and reduces confusion with the `list` section and other uses
of `list` in KMX+.
Test-bot: skip
To reduce confusion when 'Follow Keyboard Version' is set, clear the
cached package version data from UI and .kps source when saving the
package or making changes to it.
Fixes: #11892
Test-bot: skip
Adds breadcrumbs for debug memo to try and narrow down sequence of
events leading to assertion failure with backspace.
Relates-to: #11706
Test-bot: skip
While not documented as such, it appears that the repository record is
required with trusted publishing of npm packages.
Test-bot: skip
Build-bot: skip
I am not entirely clear why the coverage dropped, but I suspect it's due
to code paths that are not activated in node 24. For now, just dropping
coverage threshold to 40%.
Due to recent changes in NPM package publishing security requirements,
we have to move from TeamCity build to a GitHub Action to publish our
NPM packages, so we can take advantage of trusted publishing. This
change also consolidates and centralizes the npm publishing into
resources/build/ci/npm-publish.sh, which removes a lot of boilerplate
from each of the build.sh scripts, and ensures consistency.
Packages will be `npm pack`ed on PR and test builds, and published in
release builds.
Ref: https://docs.npmjs.com/trusted-publishers
Ref: https://github.blog/changelog/2025-09-29-strengthening-npm-security-important-changes-to-authentication-and-token-management/Fixes: #14963
Test-bot: skip
Build-bot: release:developer
Make the target version parameter for kmc-ldml into a generic build
parameter, so we can use it for any compiler. Add support and tests
to kmc-kmn, kmcmplib, kmc-ldml, kmc.
Support not yet in place for lexical models, or packages. However,
packages will be picking up the min version from the embedded keyboards.
Lexical models need work on minimum version support before this can be
implemented there.
This substantial refactor reorganizes the header data for sections in
the Core LDML processor. The change was substantial because of
assumptions made about the binary layout of sections. In order to make
the code easier to maintain, safer, and more resilient to future
changes, I opted to make a consistent helper for each section, and copy
header data so that it could be transparently reused.
The principal changes are:
1. Support the version field in the COMP_KMXPLUS_HEADER struct, and
split it into COMP_KMXPLUS_HEADER_17 and COMP_KMXPLUS_HEADER_19
versions.
2. Establish a corresponding COMP_KMXPLUS_XXXX_Helper clas for each
section.
3. Refactor the majority of rawData accesses into using helper
functions, which reduces direct pointer manipulation and adds extra
boundary checks.
A special-case exists for BKSP - it is identical to TRAN, except for
its section ident. In order to avoid a complicated pattern for handling
it, I have special-cased it in one place, adding an overloaded
`get_section_from_sect` function for COMP_KMXPLUS_BKSP.
An opportunity exists to refactor a bit further - reduce direct access
to the binary data (through the COMP_KMXPLUS_XXXX structs, and instead
promote use of the COMP_KMXPLUS_XXXX_Helper classes). This indirection
would reduce duplication of data access and make it cleaner when we
start doing multi-version code.
Next commit will add unit tests for v19 format files.
Add support for header.version to KMX+ files, with differentiation
between v17 and v19 in the compiler, and associated tests. This does not
yet make the target version accessible to authors using `kmc`, but the
interfaces from `kmc-ldml` are available.
A v17 KMX+ file will have:
* comp_header.version = VERSION_17
* initial 'sect' section in KMX+ data
* no sect.header.version field in any section
A v19+ KMX+ file will have:
* comp_header.version = VERSION_19
* initial 'sec2' section in KMX+ data
* sect.header.version field for all sections
* each section 4 bytes larger to accommodate
* 'sec2' must have sect.header.version = KMXPlusVersion.Version19
* currently all other sections have sect.header.version = Version17
Consolidate the header structure in KMX+ to make it easier to add a
version value to the header for v19+ KMX+ files; also DRYs out the
header definitions a little.
The compiler has always been very ambiguous on which characters were
accepted in group and store names, even to the point of accepting
things like comma in a store name, which would then make it impossible
to reference in an `index` statement!
This commit clarifies the allowable characters in an identifier. While
it would have been possible to use UAX#31 for this, that would have
extended the requirements for this change substantially, and may have
caused us more trouble with legacy keyboards. Given kmcmplib is
end-of-life (see epic/ng-compiler), I have chosen a lower friction
approach. There are certainly other characters that could be excluded,
but in general I have chosen to exclude only those that will definitely
be problematic.
The set of allowable characters for deadkeys has actually been expanded
in this release to match the store and group name rules.
It is expected that there may be some impacted keyboards, but addressing
this change will be relatively straightforward, so I consider this to be
an acceptable back-compatibility trade-off, see
https://github.com/keymanapp/keyman/wiki/Principles-of-Keyman-Code-Changes#4-source-backward-compatibility-keyboard-model-and-package-source-file-formats-should-be-backward-compatibleFixes: #14604
Test-bot: skip
Build-bot: skip build:developer
Removes a number of unused functions from the file, renames remaining
functions to match our builder script prefix patterns, removes unused
imports from various build scripts (e.g. Android).
Fixes: #14453
Consolidates the node-related script functions into node.inc.sh, as part
of cleaning up the build scripts and making them easier to maintain into
the future.
Fixes: #14447
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.
Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh
Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
android/build.sh.
* Renames functions in builder-basic.inc.sh
More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.
Fixes: #14065
Build-bot: build all
Test-bot: skip
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.
Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
Some of the npm audit warnings left after #13897 require an updated `mocha` version to remedy. As we only really use `mocha` to do unit-testing, it should be safe to update if all of our existing tests pass with the new version.
Test-bot: skip
- added new DelegatingCompilerCallbacks class
- added ResolvingCompilerCallbacks which does event resolution
- added the XML file name metadata to *every object* after XML parse.
Fixes: #10622
