This requires a significant refactor to the compiler interfaces to allow
for asynchronous filesystem and network access. For now, only kmc-copy
uses the asynchronous interfaces, but over time we may try and
consolidate them where other compiler modules find them useful. This
adds a big set of unit test fixtures to avoid network instability in
tests (the fixtures can be regenerated by setting environment variable
`TEST_SAVE_FIXTURES`).
Relates to kmc-copy project.
Moves the .kps file reader out of kmc-package and adds corresponding
kps file writer unit in developer-utils. Also moves related test and
error message definition.
- because of the form to="" in ldml, we need to distinguish attributes and sub-elements in the ldml xml parsing
- use an attributePrefix, and fixup the object tree afterwards
Fixes: #12208
- remove vendored xml2js
- add fast-xml-parser@4.5.0
- Rewire the KeymanXMLReader / KeymanXMLWriter to use fast-xml-parser
Four small changes to the test data on the generated .kvks:
- allow " for XML generation (" is legal)
- use the same XML prologue as the actual .kvks files.
- treat <key></key> as <key/> since we can't distinguish
(removed a "_": "" entry)
- remove whitespace because we now can ("_": "\n \n \n") on non-leaf nodes.
This is ignored by the kvks code, but was difficult to maintain roundtrip.
Fixes: #12208
Subsystems changed:
- ldml keyboard reader (main and test)
- kpj
- kvks
- kmp compiler
test: made the test-xml-utils less verbose about the pathnames
Fixes: #12208
The visual keyboard compiler was never finished in 17.0. This rewrites
it to:
1. Use the kmxplus data rather than reading from xml directly
2. Fill in `visualkeyboard.header.kbdname`
3. Support modifiers
4. Handle encoded characters like `\u{1234}`
5. Handle string variables like `${one}`*
Additional unit tests have been added to verify the behavior of the
visual keyboard compiler in more detail.
TODO-LDML: string variables appear to have a secondary bug -- they seem
to be returning the string 'undefined'. I have disabled the related
tests and will examine this separately, and enable those tests once
fixed.
TODO-LDML: we should probably add a compiler warning + unit test for
`<layers formId="us"><layer id="base">`, because this pattern does not
make sense: when using non-touch forms, the `<layer>` element should use
`modifiers` attribute, and correspondingly, `modifiers` attribute should
_not_ be used when `formId` is `touch`.
Other fixes:
1. The LDML XML reader was relying on its input being a Node.js `Buffer`
even though it was declared `Uint8Array`, as it implicitly used
`Buffer.toString()` to do text conversion. (`Buffer` subclasses from
`Uint8Array`). This breaks when using `Uint8Array` directly and means
we had an implicit dependency on Node.js. See also #12331.
2. XML errors were not captured in the LDML XML reader. See also #12331.
3. The unused and unfinished touch-layout-compiler.ts and
keymanweb-compiler.ts have been removed along with corresponding unit
tests and fixtures. These are replaced by Core implementations; see
#12291.
Fixes: #12395
The default import path is relative to ldml-keyboard-xml-reader.js. The
bundling of kmc with Keyman Developer reduces the directory tree depth,
so we move the import path to be a subdirectory of the immediate parent
directory, rather than two levels up, so that
`C:\Program Files\Keyman\Keyman Developer\kmc` can refer to
`C:\Program Files\Keyman\Keyman Developer\import` when kmc is bundled.
Fixes: #12279
With the full move to ES Modules, we no longer need to include
`references` in tsconfig.json, as we can rely on package.json and
build.sh dependency management. Note however that `tsc -b` may not work
to build dependencies -- they need to be built using `build.sh`, which
calculates which dependencies need building.
kmc-keyboard-info was missing a dependency link to kmc-package in
build.sh, correcting this at the same time.
Relates-to: #12027
