This probably arose with the move to kmcmplib in WASM, and as it was
never unit tested, we missed it. A simple signed vs unsigned issue.
Fixes: #14411
Test-bot: skip
Adjust two incorrect rules that caused backslash to be treated as an
escape in strings in syntax highlighting in .kmn language.
Fixes: #14988
Test-bot: skip
Build-bot: skip
Make the target version parameter for kmc-ldml into a generic build
parameter, so we can use it for any compiler. Add support and tests
to kmc-kmn, kmcmplib, kmc-ldml, kmc.
Support not yet in place for lexical models, or packages. However,
packages will be picking up the min version from the embedded keyboards.
Lexical models need work on minimum version support before this can be
implemented there.
This substantial refactor reorganizes the header data for sections in
the Core LDML processor. The change was substantial because of
assumptions made about the binary layout of sections. In order to make
the code easier to maintain, safer, and more resilient to future
changes, I opted to make a consistent helper for each section, and copy
header data so that it could be transparently reused.
The principal changes are:
1. Support the version field in the COMP_KMXPLUS_HEADER struct, and
split it into COMP_KMXPLUS_HEADER_17 and COMP_KMXPLUS_HEADER_19
versions.
2. Establish a corresponding COMP_KMXPLUS_XXXX_Helper clas for each
section.
3. Refactor the majority of rawData accesses into using helper
functions, which reduces direct pointer manipulation and adds extra
boundary checks.
A special-case exists for BKSP - it is identical to TRAN, except for
its section ident. In order to avoid a complicated pattern for handling
it, I have special-cased it in one place, adding an overloaded
`get_section_from_sect` function for COMP_KMXPLUS_BKSP.
An opportunity exists to refactor a bit further - reduce direct access
to the binary data (through the COMP_KMXPLUS_XXXX structs, and instead
promote use of the COMP_KMXPLUS_XXXX_Helper classes). This indirection
would reduce duplication of data access and make it cleaner when we
start doing multi-version code.
Next commit will add unit tests for v19 format files.
Add support for header.version to KMX+ files, with differentiation
between v17 and v19 in the compiler, and associated tests. This does not
yet make the target version accessible to authors using `kmc`, but the
interfaces from `kmc-ldml` are available.
A v17 KMX+ file will have:
* comp_header.version = VERSION_17
* initial 'sect' section in KMX+ data
* no sect.header.version field in any section
A v19+ KMX+ file will have:
* comp_header.version = VERSION_19
* initial 'sec2' section in KMX+ data
* sect.header.version field for all sections
* each section 4 bytes larger to accommodate
* 'sec2' must have sect.header.version = KMXPlusVersion.Version19
* currently all other sections have sect.header.version = Version17
Consolidate the header structure in KMX+ to make it easier to add a
version value to the header for v19+ KMX+ files; also DRYs out the
header definitions a little.
Special key caps such as '*ZWNJ*' are defined in a number of places in
the source. In theory, we could DRY these definitions out, but that
would add a number of unhelpful dependencies or take considerable time
to implement. So, for now I opted to write a unit test to compare the
definitions as found in the following files, treating web's definition
as primary:
* web/src/engine/osk/src/specialCharacters.ts
* developer/src/tike/xml/layoutbuilder/constants.js
* developer/src/kmc-kmn/kmw-compiler/constants.ts
* developer/docs/help/reference/file-types/keyman-touch-layout.md
* core/include/ldml/keyman_core_ldml.ts
Note that the keyman_core_ldml.ts file changes are not included in this
commit, and hence the unit test for it is currently skipped. This will
be enabled in a subsequent commit along with other KMX+ changes to
support epic/embed-osk-in-kmx.
Test-bot: skip
These two files were largely the same, with legacy_kmx_file.h trailing
kmx_file.h in features. The primary difference was that
legacy_kmx_file.h still had Windows-specific types, but the types in
km_types.h made this a drop-in replacement.
Test-bot: skip
sil_yi was impacted by the changes in #14746, as it had the group name
'Unicode Group', which is now illegal, so the compiler fails to build
the keyboard at the referenced commit. Easiest workaround currently is
to remove it from the set of compared keyboards.
The compiler has always been very ambiguous on which characters were
accepted in group and store names, even to the point of accepting
things like comma in a store name, which would then make it impossible
to reference in an `index` statement!
This commit clarifies the allowable characters in an identifier. While
it would have been possible to use UAX#31 for this, that would have
extended the requirements for this change substantially, and may have
caused us more trouble with legacy keyboards. Given kmcmplib is
end-of-life (see epic/ng-compiler), I have chosen a lower friction
approach. There are certainly other characters that could be excluded,
but in general I have chosen to exclude only those that will definitely
be problematic.
The set of allowable characters for deadkeys has actually been expanded
in this release to match the store and group name rules.
It is expected that there may be some impacted keyboards, but addressing
this change will be relatively straightforward, so I consider this to be
an acceptable back-compatibility trade-off, see
https://github.com/keymanapp/keyman/wiki/Principles-of-Keyman-Code-Changes#4-source-backward-compatibility-keyboard-model-and-package-source-file-formats-should-be-backward-compatibleFixes: #14604
Test-bot: skip
Build-bot: skip build:developer
Add a new FormValidation class to help make a consistent declarative
field validation pattern for Keyman Developer forms, and apply that
initially to the New Keyboard Project Parameters dialog. This pattern
replaces the earlier EnableControls pattern; ideally we will replace
validation in other forms with similar validation requirements with this
over time.
Fixes: #14169
