This comes out of a design philosophy review on what we include when we
embed OSK data into KMX.
We will now avoid embedding font name into the OSK (and hence .kmx)
altogether, and leave that metadata to the packaging data. Reasons:
1. The font information is specified in the .kps, so we have to do a
patchup on the .kmx during packaging if we want to embed the info
into the OSK.
2. The referenced font must be supplied separately anyway (via .kmp,
@font-face, or system supplied, etc), so including the font facename
in the keyboard is not really all that helpful.
3. Philosophically, the font is really a presentation level factor
(aside from displaymap considerations). Keeping it together with
future theming and styling choices, rather than the key layout data,
seems appropriate.
4. This makes fewer places where font data is referenced -- in fact, to
just one place: in the .kps/.kmp for LDML keyboards, which is great.
This also simplifies some aspects of the embed-osk-in-kmx work, removing
the need to patch the .kmx after the build, and eliminates the smelly
kmx-plus-osk-token.ts file.
A corresponding change has been made to the design document referenced
in #14857.
Test-bot: skip
Add a KMX+ file reader and tests, including a round-trip builder:reader
test.
Move basic-17.txt, basic-19.txt, and basic.xml to common test resources
folder so they can be used by unit tests in common and developer.
Test-bot: skip
This documents the state of `km_core_actions.emit_keystroke` for
different keys pressed. Also some cleanup in other docs.
This documents the state after merging #15609 (for ldml keyboards) and
NN (for kmn keyboards).
Follows: #15609
Build-bot: skip
Test-bot: skip
Implement the conversion from .kvk into KMX+ format, and embed the
result into the .kmx file. This change does not yet support &displaymap.
Move some virtual key modifier state helpers out of
visual-keyboard-compiler.ts and into shared module
modifier-key-constants.ts.
This change also flags the need to update the package compiler to
rewrite the font metadata into the KMX+ tables.
Test-bot: skip
The disp and layr sections have new v19 layouts, to support the Keyman
OSK requirements for epic/embed-osk-in-kmx. This supports writing the
new versions of these sections in Developer and loading them into Core,
doing transformations where necessary so that Core always works with v19
structures after load.
This does not implement the transformations required to support the OSK
APIs; that will be implemented in a follow-up PR. Nor is support for
writing the OSK data from source .kvks and .keyman-touch-layout
supported; this is just the scaffolding for supporting the structures in
the KMX+ data.
Test-bot: skip
The name `form` better represents the semantic meaning of the
structure, and reduces confusion with the `list` section and other uses
of `list` in KMX+.
Test-bot: skip
While not documented as such, it appears that the repository record is
required with trusted publishing of npm packages.
Test-bot: skip
Build-bot: skip
Due to recent changes in NPM package publishing security requirements,
we have to move from TeamCity build to a GitHub Action to publish our
NPM packages, so we can take advantage of trusted publishing. This
change also consolidates and centralizes the npm publishing into
resources/build/ci/npm-publish.sh, which removes a lot of boilerplate
from each of the build.sh scripts, and ensures consistency.
Packages will be `npm pack`ed on PR and test builds, and published in
release builds.
Ref: https://docs.npmjs.com/trusted-publishers
Ref: https://github.blog/changelog/2025-09-29-strengthening-npm-security-important-changes-to-authentication-and-token-management/Fixes: #14963
Test-bot: skip
Build-bot: release:developer
This substantial refactor reorganizes the header data for sections in
the Core LDML processor. The change was substantial because of
assumptions made about the binary layout of sections. In order to make
the code easier to maintain, safer, and more resilient to future
changes, I opted to make a consistent helper for each section, and copy
header data so that it could be transparently reused.
The principal changes are:
1. Support the version field in the COMP_KMXPLUS_HEADER struct, and
split it into COMP_KMXPLUS_HEADER_17 and COMP_KMXPLUS_HEADER_19
versions.
2. Establish a corresponding COMP_KMXPLUS_XXXX_Helper clas for each
section.
3. Refactor the majority of rawData accesses into using helper
functions, which reduces direct pointer manipulation and adds extra
boundary checks.
A special-case exists for BKSP - it is identical to TRAN, except for
its section ident. In order to avoid a complicated pattern for handling
it, I have special-cased it in one place, adding an overloaded
`get_section_from_sect` function for COMP_KMXPLUS_BKSP.
An opportunity exists to refactor a bit further - reduce direct access
to the binary data (through the COMP_KMXPLUS_XXXX structs, and instead
promote use of the COMP_KMXPLUS_XXXX_Helper classes). This indirection
would reduce duplication of data access and make it cleaner when we
start doing multi-version code.
Next commit will add unit tests for v19 format files.
Add support for header.version to KMX+ files, with differentiation
between v17 and v19 in the compiler, and associated tests. This does not
yet make the target version accessible to authors using `kmc`, but the
interfaces from `kmc-ldml` are available.
A v17 KMX+ file will have:
* comp_header.version = VERSION_17
* initial 'sect' section in KMX+ data
* no sect.header.version field in any section
A v19+ KMX+ file will have:
* comp_header.version = VERSION_19
* initial 'sec2' section in KMX+ data
* sect.header.version field for all sections
* each section 4 bytes larger to accommodate
* 'sec2' must have sect.header.version = KMXPlusVersion.Version19
* currently all other sections have sect.header.version = Version17
Removes a number of unused functions from the file, renames remaining
functions to match our builder script prefix patterns, removes unused
imports from various build scripts (e.g. Android).
Fixes: #14453
Consolidates the node-related script functions into node.inc.sh, as part
of cleaning up the build scripts and making them easier to maintain into
the future.
Fixes: #14447
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.
Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh
Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
android/build.sh.
* Renames functions in builder-basic.inc.sh
More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.
Fixes: #14065
Build-bot: build all
Test-bot: skip
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.
Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
Several keys that are defined in `winuser.h` were added after Keyman
defined its vkeys and so are not documented in the Keyman docs. This
change adds underscores before and after to mark them as being internal.
It's still useful to have them in the source code because it makes the
mapping between Linux keys and Keyman vkeys easier.
Addresses code review comment.
This change tries to map all keys to corresponding Core VKeys so that
Core sees them and can reset the context if necessary. Previously we
had 0 for most of the keys, which meant that we didn't send them to
Core but instead returned right away. While this doesn't directly fix
the failing user tests described in #12968, it fixes the original tests
in #11172.
Closes: #12968
- split keyboard loading into loading KMX file into blob and then
loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
that loads a file in Core; unit tests have some more places that
load files.
- introduce GTest and add unit tests for loading from blob
Cherry-picked from `epic/web-core` branch.
Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a
Part-of: #11293
Part-of: #8093
Use a new flag `--npm-publish` in conjunction with `publish` action,
so that the default will always be to `npm pack` if the new flag is not
specified. This flag is also guarded in the actual npm publish code to
ensure that it can only run in the appropriate CI alpha/beta/stable
environment, and not in local or test.
This then also removes the separate `pack` action.
Also removes legacy boilerplate from a number of build scripts.
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).
This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
Fixes#11324.
* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
Refreshes all of the public API header documentation so we have a single
source for the documentation -- the keyman_core_api.h header. A lot of
reformatting but very little change in terms of content.
This is converted into Markdown by api-header-extractor, and the results
can be copied to help.keyman.com.
Will only do the main keyman_core_api.h for v17 release, in interests of
time. The other headers are primarily internal only and have many
deprecated functions also.
Will also leave the full automation of the documentation generation to a
future PR. The script will be there but it won't run during build for
now.
Fixes#10582.
Returned struct from the `km_core_state_get_actions()` API is now owned
by the state object. This is a change in API contract. The corresponding
`km_core_actions_dispose()` API has been made private, because there is
never any need for API consumers to call it. As this change is happening
in alpha, we won't bump the ABI version.
`state->app_context` is now guaranteed to be in sync with
`state->context` after `km_core_process_event()`, with the actions
struct populated at that point.
The next and final step in this refactor is to remove the actions queue
altogether from the Core and make that a kmn-only concept, but that will
wait until 18.0. The only consumers of the actions queue are the
kmn-specific interactive debugger in Keyman Developer, and kmn-specific
Input Method eXtensions in Keyman Engine for Windows.
