Fixes#9154.
Improves performance in both kmcmplib and in kmc-kmn/kmw-compiler. After
these changes are applied, vietnamese_telex keyboard builds in around 6
seconds on my machine (down from over 2 minutes). This is certainly only
the surface of performance improvements we could make.
Addresses excessive memory reallocations in kmcmplib, by reallocating
store and key arrays in large chunks of 100 items rather than
reallocating on each new item added. This resulted in a 250x speed-up on
functions such as `AddStore` in WASM build.
Some careful modulus arithmetic was needed for resizing the key array,
which can be grown in larger increments.
Resolved excessive zeroing out of destination buffer in u16ncpy, which
was being called with an 8kb destination buffer on every line of the
file.
In the kmw side, vast majority of the performance cost was in copying of
buffers while loading a .kmx into memory, rather than creating views
into the memory. Essentially replaced this pattern:
```
return this.rString.fromBuffer(source.slice(offset));
```
with:
```
const data = new Uint8Array(source.buffer, source.byteOffset + offset);
return this.rString.fromBuffer(data);
```
Fixes#9160.
To match the behaviour of kmcomp, kmc should add/delete the following
fields when writing out a kmw touch layout:
* `platform.displayUnderlying` flag is always emitted by kmcomp, so kmc
should always emit it too.
* `platform.font` and `platform.fontsize` should be eliminated if empty
string
* `key.id` should be eliminated if empty string
Fixes#9145.
* `WARN_KeyboardVersionsDoNotMatch` is now only raised when
`FollowKeyboardVersion` is set.
* `WARN_KeyboardVersionsDoNotMatchPackageVersion` has been removed,
because it did not really make sense; if 'FollowKeyboardVersion' is
set, it could not be raised, and otherwise, the author may wish to
have separate keyboard + package versions anyway.
* Note that the 0x0013 compiler message allocation for
`WARN_KeyboardVersionsDoNotMatchPackageVersion` has been left alone;
as this was only used in pre-release, we can probably re-task it in
the future.
Fixes#9146.
For model packages, 304016 (ERROR_ModelMustHaveAtLeastOneLanguage)
remains as an error (this is a new error for kmc, kmcomp did not
validate model metadata). For keyboard packages, a new warning is
introduced to match the existing kmcomp behaviour, 20401B
(WARN_KeyboardShouldHaveAtLeastOneLanguage).
Fixes#9150.
Legacy .kmn keyboards can refer to a binary .kvk file. kmc needs to be
able to load these as well.
Note: there was a short period of time in which .kvk files were either
source or binary format. We moved to .kvk always being binary, and .kvks
always being source (xml), and so IMO we don't need to include support
for mismatched file formats.
Fixes#9140.
Fixes#9148.
Keyman Developer 9.0 .kpj files included a lot of additional state
metadata. We need a schema which validates these files, as they are
otherwise valid to load (we'll never save them any more). Rather than
add all the extra metadata to what is otherwise a fairly clean schema,
we'll provide a legacy .schema.json.
In the future, we may be able to merge these schemas, as we move towards
the .kpj 2.0 format which doesn't list files. Ideally, all three formats
(Keyman Developer 9.0 schema, call it legacy, 1.0 schema for Keyman
Developer 10.0+ which has Options and Files listed, 2.0 schema for
capturing just project settings for a folder) will be supported by a
single schema file.
- update element string constructors to support UnicodeString and the segmented array from ElementParser
- update compiler machinery to pass USetParser in the DependencySections
- test cases
#7377
- detect and reject nested square brackets, for now
- implement segmentation of reorder element strings,
including type identification (escape, codepoint, uset)
- tests
For #7377
