Commit graph

253 commits

Author SHA1 Message Date
Marc Durdin
b58bb36d80 feat(developer): add loca tests
Note: bumps target to es2020 for better Intl.* support. Verified as
supported by Node 16+ and modern browsers.

Adds support for multiple locales and basic canonicalization. Does not
support normalization of locales.
2022-08-31 10:18:57 +10:00
Marc Durdin
5d44d2ecc7 chore(developer): Merge branch 'chore/merge-master-into-feature-ldml' into feat/developer/kmldmlc-meta-tests-and-cleanup 2022-08-31 10:10:03 +10:00
Marc Durdin
8fc7921ed4 chore(developer): cleanup todo comments 2022-08-30 13:57:16 +10:00
Marc Durdin
995d731ccb feat(developer): add unit tests for meta compiler
First, a little bit of refactoring to make it cleaner to do unit test
for each section.

Then add some global error values.

Throw in a helper function or two.

Dust off the edges of the CompilerCallbacks object.

Duct tape together validation of the normalization attribute.

And ... eventually we get a nice clean-like unit test for the meta
compiler, with three pretty little fixtures and as-neat-as-you-like-it
assertions.
2022-08-30 13:52:36 +10:00
Marc Durdin
db11553c9a refactor(developer): each section compiler is a pure function
Well, almost. There are still callback side-effects for messages. These
may in future be returned as an array as well, eliminating the need for
a message callback.

But restructures each section compiler so that it is self-contained and
responsible only for writing its own section. Also tightens up the types
for the sections constants.

Last, but not least, moves responsibility for writing the output .kmx
out of compiler.ts and into its callers -- removing one dependency from
the compiler. Reading input files still needs to be done in the
compiler, as there may be multiple input files with `<import>` that we
cannot know about until we start parsing xml.
2022-08-30 11:13:11 +10:00
Marc Durdin
c0147dc6ea refactor(developer): separate files for kmldmlc sections
@keymanapp-test-bot skip
2022-08-30 07:16:53 +10:00
Marc Durdin
8dc68edc6c
Merge pull request #7140 from keymanapp/feat/developer/unit-tests-for-kmldmlc
feat(developer): unit test framework for kmldmlc 🙀
2022-08-29 15:16:41 -05:00
Marc Durdin
c85b1da333
Merge pull request #7129 from keymanapp/feat/core/use-ldml-keyboard-compiler-for-tests
feat(core): use kmldmlc to build test keyboards 🙀
2022-08-29 15:15:55 -05:00
Marc Durdin
5dc5b9325c
Merge pull request #7000 from keymanapp/chore/developer/keymanweb-compiler-remove-unused-dependencies
chore(developer): remove unused dependencies from KeymanWeb compiler 🛩
2022-08-29 14:48:46 -05:00
Marc Durdin
a72146e42d feat(developer): unit test framework for kmldmlc 2022-08-29 17:21:05 +10:00
Marc Durdin
b31a8755c0
Merge pull request #7127 from keymanapp/feat/developer/ldml-compiler-xml-loader-and-compiler
feat(developer): LDML keyboard XML Loader and Compiler 🙀
2022-08-27 14:59:55 -05:00
Marc Durdin
496ea0641a
chore: add comment for BUILDER_SECT
Co-authored-by: Steven R. Loomis <srl295@gmail.com>
2022-08-27 09:03:29 +10:00
Marc Durdin
b50d03b7d6 chore(developer): chmod build.sh 2022-08-26 15:26:36 +10:00
Marc Durdin
daf6a7970e feat(core): use kmldmlc to build test keyboards
Removes .kmx from repo and builds them from the source .xml files using
kmldmlc (building that if necessary also with a poor fella's dependency
check for now), and generally a lot of cleanup of the meson.build files
for the ldml tests.

Moves keyboards to a subfolder of the ldml unit test folder to keep them
neat and tidy and avoid embedding huge files into ldml.js for test of
WASM.

Note: 000_null_keyboard and 002_null_invalid do build successfully, but
the core ldml keyboardprocessor currently fails on them, because they
are missing keys and vkey sections.
2022-08-26 15:12:38 +10:00
Marc Durdin
dc172cb981 chore(developer): robustness in ldml keyboard compiler 2022-08-26 09:06:12 +10:00
Marc Durdin
3c120ebae7 feat(developer): LDML keyboard XML Loader and Compiler
Implements basic LDML keyboard xml loader and compiler.
2022-08-26 08:51:15 +10:00
Marc Durdin
3865870114 chore(developer): add padding to loca and vkey 2022-08-26 05:23:11 +10:00
Marc Durdin
7644ce668f chore(developer): cleanup for kmx-builder 2022-08-26 04:44:00 +10:00
Marc Durdin
420dc1c10c feat(developer): emit kmxplus format 2022-08-26 04:40:25 +10:00
Marc Durdin
23d9aad0a0 chore(developer): cleanup 2022-08-25 15:12:00 +10:00
Marc Durdin
aed8f0d11a feat(developer): Write kmxplus section
Adds support for writing kmxplus section of a kmx file. Extends KMXFile
to a KMXPlusFile subclass. Adds KMXPlusBuilder which is called from
KMXBuilder if KMXBuilder finds that the file is a KMXPlusFile.

This means that KMXPlusBuilder generates a buffer that could be stored
independently of its parent .kmx file transport, if that is later
considered desirable.

Adds new constants for sizes of various table components, which makes
building the file much simpler.
2022-08-25 12:26:21 +10:00
Marc Durdin
6aeb20e77c
Revert "Revert "feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀"" 2022-08-24 14:32:22 +10:00
Marc Durdin
8ebae9515e
Revert "feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀" 2022-08-24 14:26:44 +10:00
Marc Durdin
5e296c98af feat(core): turn @keymanapp/ldml-keyboard-constants into a module
Splits the building of keyboardprocessor_ldml.h into a separate
tsconfig.build.json, and designates keyboardprocessor_ldml.ts as a new
Typescript/Node module @keymanapp/ldml-keyboard-constants.

Includes an example usage in the (currently unused) kmx-plus.ts.
2022-08-24 14:10:20 +10:00
Marc Durdin
a3e672e808 feat(developer): LDML compiler kmx-builder
Generates a valid .kmx file (legacy structures only) from an
intermediate in-memory structure. Included for completeness, although it
is anticipated that LDML keyboards will not have any groups, though they
will probably have some metadata stores.
2022-08-24 13:43:09 +10:00
Marc Durdin
d4553f674a chore(developer): rename structures 2022-08-24 09:42:08 +10:00
Marc Durdin
4018e528d9 feat(developer): groundwork - kmx file structures
Defines kmx binary structures and wireframe builder to emit a .kmx file.

Currently the builder will emit a roughly correct header. Next steps are
to build a .kmx file with stores, groups and keys, to match existing KMX
compiler behaviour. Once that is done, will return to generation of
KMXPlus data structures.

Compiler is designed to work independent of file system so it should run
inside browser or node environment. kmldmlc.ts is node-specific, but all
other modules should be multi-platform.
2022-08-24 09:32:30 +10:00
Marc Durdin
9ce6892112
Merge pull request #7101 from keymanapp/feat/developer/ldml-keyboard-ts-structures
feat(developer): LDML keyboard compiler - infrastructure 🙀
2022-08-23 18:29:49 -05:00
Marc Durdin
8cba43f989
chore: Apply suggestions from code review
Co-authored-by: Darcy Wong <darcy_wong@sil.org>
Co-authored-by: Steven R. Loomis <srl295@gmail.com>
2022-08-24 09:29:03 +10:00
Marc Durdin
e6ff47f2b3 chore: Merge branch 'master' into feature-ldml 2022-08-24 04:21:21 +10:00
Marc Durdin
cc8875dc9e chore(developer): cleanup comments 2022-08-24 03:54:25 +10:00
Marc Durdin
c108e5ecd6 feat(developer): add kmldmlc to developer installer 2022-08-24 03:29:45 +10:00
Marc Durdin
ab6f91c587 feat(developer): bundling of kmldmlc
Adds basic bundling of kmldmlc to Keyman Developer. Bundling script
built. Yet to come: .wxs integration.
2022-08-23 11:20:54 +10:00
Marc Durdin
1a0b67a242 feat(developer): LDML keyboard compiler - infrastructure 2022-08-23 10:27:34 +10:00
Steven R. Loomis
56aeee35c7 feat(core): add event_flags to process_event #7046
- pascal fix
2022-08-15 12:21:42 -05:00
Marc Durdin
f7189387b3
Merge pull request #7013 from keymanapp/fix/developer/keymanweb-readonly-groups-mismatches
fix(developer): compiler emitting garbage for readonly groups 🎾
2022-08-15 04:40:51 -05:00
Marc Durdin
1a001d6761
Merge pull request #7033 from keymanapp/fix/developer/7032-crash-compiling-ansi-for-web
fix(developer): prevent crash attempting to compile ansi keyboard 🎾
2022-08-15 02:16:02 -05:00
Steven R. Loomis
e8dff5cf4e feat(core): add event_flags to process_event #7046
- use a constant, KM_KBP_EVENT_FLAG_DEFAULT for 0
- update pascal and python
- update linux and windows
2022-08-12 09:58:58 -05:00
Marc Durdin
d54dd049d2 chore(common): add VERSION_160 to headers
Fixes #7044.

Groundwork for KMXPlus file format for LDML keyboard support.
2022-08-07 07:45:57 -05:00
Marc Durdin
e5bd4e0be3 fix(developer): prevent crash attempting to compile ansi keyboard
Fixes #7032.
2022-08-03 12:57:34 -05:00
Marc Durdin
f9240d76a2 fix(developer): stack overflow when compiling non-web keyboard
Fixes #7030.

Prevents a stack overflow / loop when the web target is removed from a
keyboard and it has already been tested on web, and the user presses the
Compile button.
2022-08-03 12:53:03 -05:00
Marc Durdin
4cddfa55c9
Merge pull request #7015 from keymanapp/fix/developer/server-rebuild-model-consistency
fix(developer): rebuilding a model would not refresh it in server 🎾
2022-08-01 10:21:42 +10:00
Marc Durdin
7ba0dc218c
Merge pull request #7003 from keymanapp/fix/developer/increase-max-file-size-for-server
fix(developer): increase maximum file size for server 🎾
2022-08-01 10:21:30 +10:00
Marc Durdin
5138ba3ec7
Merge pull request #7008 from keymanapp/feat/developer/7007-surface-languageUsesCasing-in-ui
feat(developer): add languageUsesCasing to Model Editor
2022-08-01 06:21:16 +10:00
Marc Durdin
79f3272774 fix(developer): rebuilding a model would not refresh it in server
When rebuilding a model, Keyman Developer would query for the
registered model, but a missing simplifyId call meant that it never
matched the model name (due to `.` vs `_`).
2022-07-29 12:24:16 +10:00
Marc Durdin
5c06d59a5e fix(developer): compiler emitting garbage for readonly groups
Picked up while reviewing code for conversion to C++. The compiler
could, in some circumstances emit garbage code for readonly groups
because we weren't testing all scenarios correctly. In practice, this
would have been rare as readonly groups don't emit characters, but it
should be fixed!
2022-07-29 11:22:30 +10:00
Marc Durdin
c1118d908c
Merge branch 'master' into chore/common/builder-script-target-and-option-support 2022-07-29 10:37:18 +10:00
Marc Durdin
4d80760ad9 feat(developer): add languageUsesCasing to Model Editor
Fixes #7007.
2022-07-28 14:55:10 +10:00
Marc Durdin
94f7b135b6 fix(developer): increase maximum file size for server 2022-07-28 10:28:00 +10:00
Marc Durdin
9b7d44398d chore(developer): Coinit required for test suite
Removing unused dependencies also removed the coinit call. Re-adding coinit.
2022-07-28 09:24:18 +10:00