Commit graph

1985 commits

Author SHA1 Message Date
Marc Durdin
b6c4e21c1d chore(developer): remove name from meta section
Fixes #7214.
2022-09-09 08:57:52 +10:00
Marc Durdin
70cff601d6
Merge pull request #7213 from keymanapp/feat/developer/kmldmlc-test-coverage
feat(developer): add nyc to kmldmlc for test coverage 🙀
2022-09-09 05:22:05 +10:00
Marc Durdin
0def7dd97d
Merge pull request #7212 from keymanapp/feat/developer/kmldmlc-transform-sections
feat(developer): kmldmlc compiler for transforms 🙀
2022-09-09 05:21:54 +10:00
Marc Durdin
5a14f27d07
Merge pull request #7211 from keymanapp/refactor/developer/kmldmlc-split-builder-functions
refactor(developer): split kmxplus builder functions into separate files 🙀
2022-09-09 05:21:44 +10:00
Sabine
fab2670c9e Merge branch 'refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION' of https://github.com/keymanapp/keyman into refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION
# Conflicts:
#	developer/src/kmcmpdll/CheckForDuplicates.cpp
#	developer/src/kmcmpdll/Compiler.cpp
#	developer/src/kmcmpdll/NamedCodeConstants.cpp
conflicts solved
2022-09-08 17:01:40 +02:00
Sabine
9296e57398 chore(developer): tidied up code 2022-09-08 16:55:26 +02:00
Sabine
667c2fb89d chore(developer): removed all commentaries _S2 2022-09-08 16:18:32 +02:00
Sabine
129bc242bd chore(developer): tidied up code 2022-09-08 15:37:03 +02:00
Sabine
a3c2956f47 chore(developer): tiedied up code 2022-09-08 15:20:28 +02:00
Sabine
9fc1c63783 Merge branch 'refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION' of https://github.com/keymanapp/keyman into refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION
# Conflicts:
#	developer/src/kmcmpdll/Compiler.cpp
solved
2022-09-07 19:00:07 +02:00
Sabine
b9ba055e31 chore(developer): tidied up code 2022-09-07 18:39:34 +02:00
Sabine
271ca512b9 chore(developer): tidied up code 2022-09-07 17:37:56 +02:00
Sabine
bfc1f681da chore(developer): CheckFNConsuistency: called u16printf with f´different par. 2022-09-07 15:44:03 +02:00
Marc Durdin
ec5a83a3fa
Merge pull request #7204 from keymanapp/feat/core/ldml-transform-table-specs-and-constants
feat(core): ldml keyboard transform elements 🙀
2022-09-07 11:28:42 +10:00
Marc Durdin
89331b62c4
Merge pull request #7203 from keymanapp/chore/developer/add-test-for-messages-and-improve-reporting-on-e2e
chore(developer): add test for compiler messages and cleanup e2e test 🙀
2022-09-07 11:28:35 +10:00
Marc Durdin
6ba737de9f
Merge pull request #7202 from keymanapp/refactor/developer/kmldmlc-cleanup-vkey-constants
refactor(developer): cleanup virtual key constants 🙀
2022-09-07 11:28:29 +10:00
Marc Durdin
f96e3493af fix(developer): hide key-sizes when in desktop layout in touch layout editor
Fixes #7028.

Note that the desktop layout is not currently used by KeymanWeb. The
designer has a number of additional issues, as the .keyman-touch-layout
format is not well suited to describing a fixed hardware layout, but
fixing this is outside the scope of this issue.
2022-09-07 11:13:22 +10:00
Marc Durdin
c98fb89b86 fix(developer): suppress repeated warnings about unreachable code
Fixes #7216.

The warning message 0x209A 'The rule will never be matched because its
key code is never fired.' was being generated multiple times for a
single line because the `JavaScript_Key` function it is generated by is
used for various purposes.

This PR keeps a cache of reported key rules to ensure that the message
is reported only once for a given key rule, and also improves the
reporting to clarify which specific key is unreachable, which makes it
easier to diagnose when using `any(k)` style messages, for example:

```
lao_phonetic.kmn (237): Warning: 209A The rule will never be matched for key 'ñ' because its key code is never fired.
```

This also reduces the warning to a hint, as this should not be a
blocking issue for a keyboard, rather just a place the keyboard author
can tidy up.
2022-09-07 06:02:46 +10:00
Sabine
c3be2ba6da Merge branch 'refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION' of https://github.com/keymanapp/keyman into refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION
# Conflicts:
#	developer/src/kmcmpdll/CheckFilenameConsistency.cpp
#	developer/src/kmcmpdll/CheckFilenameConsistency.h
chore(developer): solved conflicts
2022-09-06 18:46:40 +02:00
Sabine
2c6726974a chore(developer): Win-functions in FileNameConsistency 2022-09-06 17:44:18 +02:00
Sabine
143ad3f206 chore(developer): Win-functions in FileNameConsistency 2022-09-06 17:23:09 +02:00
Marc Durdin
447b9afead feat(developer): add nyc for test coverage to kmldmlc 2022-09-06 11:52:22 +10:00
Marc Durdin
8e5ce35f4d feat(developer): kmldmlc compiler and tests for transforms
Adds compilers and minimal unit tests for transform sections bksp,
finl, ordr, and tran, and cleans up problems in the corresponding
builders.

Adds a utility script for building test fixtures for manual analysis:
  ./build.sh build-fixtures

Refactors constants to give us design-time type safety once more.
2022-09-06 10:15:31 +10:00
Marc Durdin
14cbab10a2 feat(developer): add builders for transform sections
Note: basic.txt changed becuase I fixed the build order for sections.
2022-09-05 16:13:15 +10:00
Marc Durdin
917b39e7da refactor(developer): split kmxplus builder functions into separate files
This refactor work simply splits the existing functions into separate
files to prevent kmx-plus-builder.ts getting too big.

The functions now have no side-effects. Some of them mutate input
parameters, but they are fairly self-contained, which should make
unit tests simpler. Note that `alloc_string` is used extensively, but
works on the input parameter `sect_strs`.
2022-09-05 14:02:04 +10:00
Marc Durdin
9f5f8e976e feat(core): ldml keyboard transform elements
Adds definitions for LDML keyboard transform elements `transforms`,
`backspaces`, and `reorders`, and the binary format sections `bksp`,
`elem`, `finl`, `ordr`, and `tran`.

Minimally updates the compiler so that it will build, but no other
changes to the compiler.

Compiler and Typescript updates to support building these sections will
come in the next PR.
2022-09-05 13:11:51 +10:00
Marc Durdin
a708b2f85d chore(developer): add test for compiler messages and cleanup e2e test
The purpose of these tests is to ensure that our slightly WET constant
definitions for compiler errors are consistent in their usage:

  * that names are consistent between the message functions and
    constants
  * that constants have unique codes
  * that constants have the right error mask for their type

This means we don't have to worry about DRYing out the constants, which
would probably require us to have a preprocessing step, which has its
own negatives.

Also, added more reporting to e2e test in case of failures.
2022-09-05 10:43:01 +10:00
Marc Durdin
93551c36b7 chore: fixup import 2022-09-05 10:39:32 +10:00
Marc Durdin
645bca194a refactor(developer): cleanup virtual key constants
Uses our `K_xxx` constants for the CLDR VKey Enum mapping. Renames
us-virtual-keys to virtual-key-constants, better reflecting its purpose,
and adds a unit test to verify that the codes in the CLDR VKey Enum
mapping correspond to our expected Keyman virtual key codes.
2022-09-05 10:35:09 +10:00
Marc Durdin
3524b41b04
Update developer/src/kmldmlc/test/fixtures/sections/vkey/redundant.xml 2022-09-05 10:23:40 +10:00
Marc Durdin
cc0ddffd61 feat(developer): add vkey support
vkeyMaps table compiler, unit test, and fixes to json schema. Adds a
vkeyMaps entry to the basic e2e test.

us-virtual-keys.ts has a new mapping for CLDR VKey Enums, but I will
tweak this in an upcoming commit to use constants instead of hard-coded
values.
2022-09-05 10:14:10 +10:00
Marc Durdin
c08dbfbf9f feat(developer): add unit tests for 'name' compiler 2022-09-04 12:40:02 +10:00
Marc Durdin
b2b62733e6 refactor(developer): kmldmlc - rename CompilerErrors to CompilerMessages
Just wanted to rename this to make it clearer. Also added a prefix to
each message generator to indicate the type of message, as otherwise it
is not obvious within the compiler which messages are going to cause the
compiler to fail.

Note that the message generator function prefixes are Title Cased
whereas the message identifier constant prefixes are CAPS. I guess this
could be confusing but I think it works okay in practice.
2022-09-04 12:28:25 +10:00
Marc Durdin
2c4e3e109a feat(developer): add compiler hint for non-minimal loca
If a loca entry is either formatted wrongly (e.g. wrong case), or is
not minimal, then the compiler will now report a hint.
2022-09-04 12:21:15 +10:00
Marc Durdin
878287ef46 chore(developer): reorder functions for clarity 2022-09-04 06:20:07 +10:00
Marc Durdin
ea6bd8f09f chore(core): order sections according to binary format
Just reorders all work around the various sections to match the binary
format order -- that is, 'sect' first, then all other sections
alphabetically.

This also means the compiler now emits the sections in the expected
order.

No other changes to functionality.
2022-09-04 06:10:34 +10:00
Marc Durdin
b17805df3a feat(core): add 'name' section
Updates compiler, headers and documentation for the 'name' section,
which will always be present as the LDML keyboard spec requires at least
one name in the `<names>` element.
2022-09-04 05:45:43 +10:00
Marc Durdin
ab999ea0ba chore(common): cleanup basic.txt a bit more
Also uses `index()` to get count of strings automatically.

Reformat basic.txt to make it more readable as it will get less
'basic' over time!
2022-09-03 06:36:41 +10:00
Marc Durdin
738401cdb5 feat(common): hextobin index command
Adds an `index` command as suggested by #7186.

> Future enhancement suggestion: include ... index() for block indices
> within these repeated blocks. Would be very useful for string tables
> in the example ref.
2022-09-03 06:28:22 +10:00
Marc Durdin
aef38f12df chore(core): define zero length string vs null string for kmxplus 2022-09-03 06:11:24 +10:00
Marc Durdin
75139ed919 fix(developer): compiler mismatch on currentLine
Fixes #7122.

The compiler was updating currentLine when checking for unreachable
rules, but not restoring it afterwards. This led to mismatches in the
debugger and in compiler warnings.
2022-09-02 17:15:27 +10:00
Marc Durdin
702db90a71 feat(core): add version field to meta
Also adds semver package for validating the version field.
2022-09-02 14:13:10 +10:00
Marc Durdin
17371c3801 feat(common): add size and offset calculators to hextobin
This makes it easier to write a binary file which contains offsets and
size values. Using this for unit tests for the LDML keyboard compiler.

Example kmldmlc basic.txt is updated and included.

Future enhancement suggestion: include count() for repeated blocks and
index() for block indices within these repeated blocks. Would be very
useful for string tables in the example ref.
2022-09-02 11:37:59 +10:00
Marc Durdin
918d261f99 feat(developer): add conformsTo validity unit test
This is reasonably random, but wanted to add another test for content
rather than structure, just to verify that the DTD-XSD-jsonschema
translation is doing the right thing according to what we expect.

Looking good.
2022-09-02 08:01:48 +10:00
Marc Durdin
25b2d255aa feat(developer): use json schema to verify source file
As we do not appear to have access to a good DTD validation library in
Node.js, we will instead convert the DTD to a JSON schema file and use
that for validation.

I have not verified every aspect of the JSON schema at this point, but
will incrementally do so as I implement further aspects of the compiler.

This updates all the testing to use the schema. As the schema validation
is slow, I have increased the 'normal' time for tests.
2022-09-01 16:56:50 +10:00
Marc Durdin
14094c8979 feat(developer): add kmlmc and kmldmlc to make test
Also removes `-test` from normal `make build` for kmlmc, moving that
responsibility to `make test` (but uses `-tdd` to avoid rebuild).
2022-09-01 15:28:41 +10:00
Marc Durdin
7d38e26b09
chore: Apply suggestions from code review 2022-08-31 14:46:27 +10:00
Marc Durdin
1c0f68bb75 feat(developer): loca minimization + cleanup kmldmlc error list
Yeah two for the price of one:

- minimize locales at compile time
- cleans up the list of errors to make it easier to maintain and use
2022-08-31 14:30:03 +10:00
Marc Durdin
17c221f4ce chore: remove comment 2022-08-31 10:21:03 +10:00
Marc Durdin
b58bb36d80 feat(developer): add loca tests
Note: bumps target to es2020 for better Intl.* support. Verified as
supported by Node 16+ and modern browsers.

Adds support for multiple locales and basic canonicalization. Does not
support normalization of locales.
2022-08-31 10:18:57 +10:00