Fixes#9150.
Legacy .kmn keyboards can refer to a binary .kvk file. kmc needs to be
able to load these as well.
Note: there was a short period of time in which .kvk files were either
source or binary format. We moved to .kvk always being binary, and .kvks
always being source (xml), and so IMO we don't need to include support
for mismatched file formats.
Fixes#9140.
Fixes#9148.
Keyman Developer 9.0 .kpj files included a lot of additional state
metadata. We need a schema which validates these files, as they are
otherwise valid to load (we'll never save them any more). Rather than
add all the extra metadata to what is otherwise a fairly clean schema,
we'll provide a legacy .schema.json.
In the future, we may be able to merge these schemas, as we move towards
the .kpj 2.0 format which doesn't list files. Ideally, all three formats
(Keyman Developer 9.0 schema, call it legacy, 1.0 schema for Keyman
Developer 10.0+ which has Options and Files listed, 2.0 schema for
capturing just project settings for a folder) will be supported by a
single schema file.
- update element string constructors to support UnicodeString and the segmented array from ElementParser
- update compiler machinery to pass USetParser in the DependencySections
- test cases
#7377
- detect and reject nested square brackets, for now
- implement segmentation of reorder element strings,
including type identification (escape, codepoint, uset)
- tests
For #7377
Fixes#9111.
We have existing packages which have only .js in them, for touch-only
keyboards (mostly legacy but still...), so we need to support the freaky
Javascript regex search which we did in the past for extracting
metadata. While this is not necessarily going to work with hand-crafted
Javascript keyboards, it should work with all kmc- and kmcomp-generated
keyboards, so it will suffice to support these legacy packages.
In the future, we will be giving a hint when a package includes a .js
but not a .kmx, gradually upgrading this to a warning and finally an
error as we attempt to phase out .js-based keyboards in preference for
.kmx keyboards.
Running kmc on a set of keyboards would sporadically fail after some
time. It turns out this was related to memory allocation, specifically,
resizing the WASM module's memory buffer caused `TypedArray` views into
the memory to be reset!
By default, when a `Uint8Array` is created from an `ArrayBuffer` (e.g.
`Module.HEAP8.buffer`), it is a dynamic view into that buffer. This
module buffer can be dynamically reallocated at any time, which can
happen when allocating memory in WASM code (so the change will look
_really_ weird in a stack trace). Thus, to ensure we don't trip over
ourselves, we need to copy the buffer. Fortunately, creating a
`Uint8Array` from a `Uint8Array` copies the data, and is pretty quick.
When we read .kmx files, they have no alignment guarantees, so we need
to tell the compiler to generate unaligned-safe code for accesses to
`COMP_` structure members, because typically we point into the buffer
and read at any offset.
A .kmx file will generally be 2-byte aligned, as there are no structures
or data types with smaller than 2 byte widths. There is no requirement
that this be the case per the .kmx spec, though. So we use a 1 byte
alignment attribute for the compiler, so it will generate safe code when
reading these structs.
Note that we are assuming that `COMP_KEYBOARD` is aligned because it is
always the start of the file, so will be at the start of any buffer
which will automatically be aligned correctly.
We should probably review some of our design decisions and structs for
KMXPlus given this.
Picked this up while running `-fsanitize=address` for trying to track
down another bug.
This fixes a buffer overrun (normally invisible) in kmcmplib, where
`xstrchr` was being called, which takes a string as its second
parameter, but being passed a single character. This was incorrect
behaviour for two reasons:
1. Passing a single character doesn't have null termination (big bug!)
2. We were not wanting xstring semantics on the token being parsed here
Replaced with `u16chr`, which does do what we want. Note the
casting because `u16chr` only has a `const` version of its input/output
at present.
Removed `xstrchr` because it is not used anywhere else in kmcmplib.
The kmcmplib compiler source used `delete` instead of `delete[]` in a
number of locations where the corresponding allocation was an array.
This doesn't appear to matter on arrays of primitives on most platforms
that we are targeting, but it _is_ undefined behaviour so we should
correct it.
https://stackoverflow.com/a/2425749/1836776
