Fixes#7273.
Adds 4 kmx system stores to allow existing configuration tools to load
and process KMXPlus files without modification:
* `TSS_NAME` (first `<name>`)
* `TSS_COMPILEDVERSION` (only if `options.addCompilerVersion == true`)
* `TSS_KEYBOARDVERSION` (from `<version number>`)
* `TSS_TARGETS` (currently always 'desktop', will later include more
targets)
In order for `TSS_COMPILEDVERSION` to be handled correctly, needed to
plumb in the new `CompilerOptions` interface. This could be a separate
PR, but the changes are not huge, so opted to append the changes here.
Updates related tests and fixtures. Refactored some of the e2e helpers
into helpers/index.ts for use with the metadata-compiler tests.
Refactoring of build.sh to use the builder_ functions. The script should
now be a lot cleaner. Usage is available with `core/build.sh --help`
Targets are :x86, :x64, :wasm, :arch (linux/mac).
Supports additional requirements of Linux build (--target-path,
--configure).
By default will attempt to build all available targets.
Actions are: clean, configure, build, test, install, uninstall.
The install and uninstall actions only work for linux and mac targets at
present.
Fixes#7235.
Encapsulates all strings in the in-memory compile so that we can binary
sort them for build. This means that finalization of strs and elem is
no longer needed -- it's part of setup of build_strs and build_elem --
so that is factored away. Renamed alloc_string and alloc_element_string.
It may be worth renaming some of the ELEM_STRING objects / interfaces as
currently they are a little confusing.
Fixes#7237.
Full clean, configure and build will be required for Core, Developer
after this change.
kmc is much less unwieldy a name than kmldmlc, and becomes our platform
for future compiler integration. Start of implementation for #5283 also.
Fixes#7216.
The warning message 0x209A 'The rule will never be matched because its
key code is never fired.' was being generated multiple times for a
single line because the `JavaScript_Key` function it is generated by is
used for various purposes.
This PR keeps a cache of reported key rules to ensure that the message
is reported only once for a given key rule, and also improves the
reporting to clarify which specific key is unreachable, which makes it
easier to diagnose when using `any(k)` style messages, for example:
```
lao_phonetic.kmn (237): Warning: 209A The rule will never be matched for key 'ñ' because its key code is never fired.
```
This also reduces the warning to a hint, as this should not be a
blocking issue for a keyboard, rather just a place the keyboard author
can tidy up.
Adds compilers and minimal unit tests for transform sections bksp,
finl, ordr, and tran, and cleans up problems in the corresponding
builders.
Adds a utility script for building test fixtures for manual analysis:
./build.sh build-fixtures
Refactors constants to give us design-time type safety once more.
This refactor work simply splits the existing functions into separate
files to prevent kmx-plus-builder.ts getting too big.
The functions now have no side-effects. Some of them mutate input
parameters, but they are fairly self-contained, which should make
unit tests simpler. Note that `alloc_string` is used extensively, but
works on the input parameter `sect_strs`.
Adds definitions for LDML keyboard transform elements `transforms`,
`backspaces`, and `reorders`, and the binary format sections `bksp`,
`elem`, `finl`, `ordr`, and `tran`.
Minimally updates the compiler so that it will build, but no other
changes to the compiler.
Compiler and Typescript updates to support building these sections will
come in the next PR.
The purpose of these tests is to ensure that our slightly WET constant
definitions for compiler errors are consistent in their usage:
* that names are consistent between the message functions and
constants
* that constants have unique codes
* that constants have the right error mask for their type
This means we don't have to worry about DRYing out the constants, which
would probably require us to have a preprocessing step, which has its
own negatives.
Also, added more reporting to e2e test in case of failures.
Uses our `K_xxx` constants for the CLDR VKey Enum mapping. Renames
us-virtual-keys to virtual-key-constants, better reflecting its purpose,
and adds a unit test to verify that the codes in the CLDR VKey Enum
mapping correspond to our expected Keyman virtual key codes.
vkeyMaps table compiler, unit test, and fixes to json schema. Adds a
vkeyMaps entry to the basic e2e test.
us-virtual-keys.ts has a new mapping for CLDR VKey Enums, but I will
tweak this in an upcoming commit to use constants instead of hard-coded
values.
Just wanted to rename this to make it clearer. Also added a prefix to
each message generator to indicate the type of message, as otherwise it
is not obvious within the compiler which messages are going to cause the
compiler to fail.
Note that the message generator function prefixes are Title Cased
whereas the message identifier constant prefixes are CAPS. I guess this
could be confusing but I think it works okay in practice.
Just reorders all work around the various sections to match the binary
format order -- that is, 'sect' first, then all other sections
alphabetically.
This also means the compiler now emits the sections in the expected
order.
No other changes to functionality.
Updates compiler, headers and documentation for the 'name' section,
which will always be present as the LDML keyboard spec requires at least
one name in the `<names>` element.
Adds an `index` command as suggested by #7186.
> Future enhancement suggestion: include ... index() for block indices
> within these repeated blocks. Would be very useful for string tables
> in the example ref.
Fixes#7122.
The compiler was updating currentLine when checking for unreachable
rules, but not restoring it afterwards. This led to mismatches in the
debugger and in compiler warnings.
This makes it easier to write a binary file which contains offsets and
size values. Using this for unit tests for the LDML keyboard compiler.
Example kmldmlc basic.txt is updated and included.
Future enhancement suggestion: include count() for repeated blocks and
index() for block indices within these repeated blocks. Would be very
useful for string tables in the example ref.
This is reasonably random, but wanted to add another test for content
rather than structure, just to verify that the DTD-XSD-jsonschema
translation is doing the right thing according to what we expect.
Looking good.
As we do not appear to have access to a good DTD validation library in
Node.js, we will instead convert the DTD to a JSON schema file and use
that for validation.
I have not verified every aspect of the JSON schema at this point, but
will incrementally do so as I implement further aspects of the compiler.
This updates all the testing to use the schema. As the schema validation
is slow, I have increased the 'normal' time for tests.
Note: bumps target to es2020 for better Intl.* support. Verified as
supported by Node 16+ and modern browsers.
Adds support for multiple locales and basic canonicalization. Does not
support normalization of locales.
