Commit graph

902 commits

Author SHA1 Message Date
Marc Durdin
895ce930a0
Merge pull request #6993 from keymanapp/chore/core/rename-json-source
chore(core): rename json.hpp to jsonpp.hpp 🛩
2022-08-29 14:48:32 -05:00
Marc Durdin
a72146e42d feat(developer): unit test framework for kmldmlc 2022-08-29 17:21:05 +10:00
Marc Durdin
670229937a chore: move hextobin to common 2022-08-29 15:07:43 +10:00
Marc Durdin
10b875da57 chore(core): chmod build.sh 2022-08-29 07:47:49 +10:00
Marc Durdin
3d08348256 feat(core): hextobin tool
The hextobin tool is used to help us construct invalid .kmx files
without having to either hack these into the compiler, or write them
with a hex editor. It feels a little bit of overkill, but this makes it
easy to (a) work with our intended file structures, and (b) review any
changes to these invalid keyboard tests.

Note: I did do a cursory search for a hex2bin or hextobin that we could
easily use but did not find a good simple cross-platform tool that we
could consume (`xxd -r` would work but AFAICT constrains our format to a
dump which is much harder to work with and read -- I wanted to be able
to annotate with file structures.)

For an example, see ik_000_null_invalid.txt.

This means that building the invalid keyboards for tests requires node,
but that should be fine given we need node to build the valid keyboards
anyway.

I've put hextobin under core/tools but it could just as well live under
common/tools in the future, if we find it useful for other project unit
tests.
2022-08-29 06:21:16 +10:00
Marc Durdin
9945055c00 chore(core): move and re-enable invalid ldml keyboard tests 2022-08-28 05:10:21 +10:00
Marc Durdin
0057b7d95c chore(core): emcc off path for linux 2022-08-27 14:53:06 +10:00
Marc Durdin
bc02ba984a chore(core): skip ldml keyboard tests if node not available 2022-08-27 08:47:34 +10:00
Steven R. Loomis
791d253528 feat(core): ldml: improve reading 🙀
- read keys into a std::map instead of on the fly
- remove retention of entire kmx file
- remove some functions from kmx_plus.h

#5015
2022-08-26 16:47:20 -05:00
Steven R. Loomis
3de77cda92 feat(core): ldml: improve reading 🙀
- read keys into a std::map instead of on the fly
- no longer retain copy of entire KMX file in read
2022-08-26 16:43:27 -05:00
Steven R. Loomis
892492a634 feat(core): ldml: improve validation 🙀
- clearer asserts
- validate length of data

#5015
2022-08-26 15:45:48 -05:00
Marc Durdin
44d7d5ca88 chore(core): always build deps 2022-08-27 06:23:33 +10:00
Steven R. Loomis
aeeae1c4f4 feat(core): ldml: improve read side 🙀
- don't reparse structs each time
- move validation into constructor
2022-08-26 13:15:05 -05:00
Marc Durdin
3ff5c4c46e chore(core): fix typo 2022-08-26 15:58:25 +10:00
Marc Durdin
a3b1a96960 chore(core): disable failing tests for now 2022-08-26 15:48:19 +10:00
Marc Durdin
4820f99503 chore(core): also build keyman-version as a prereq 2022-08-26 15:41:44 +10:00
Marc Durdin
daf6a7970e feat(core): use kmldmlc to build test keyboards
Removes .kmx from repo and builds them from the source .xml files using
kmldmlc (building that if necessary also with a poor fella's dependency
check for now), and generally a lot of cleanup of the meson.build files
for the ldml tests.

Moves keyboards to a subfolder of the ldml unit test folder to keep them
neat and tidy and avoid embedding huge files into ldml.js for test of
WASM.

Note: 000_null_keyboard and 002_null_invalid do build successfully, but
the core ldml keyboardprocessor currently fails on them, because they
are missing keys and vkey sections.
2022-08-26 15:12:38 +10:00
Marc Durdin
9756c28102 chore(developer): Merge branch 'feature-ldml' into feat/developer/ldml-compiler-kmxplus-builder 2022-08-26 05:17:05 +10:00
Marc Durdin
144025e29f
Merge pull request #7113 from keymanapp/revert-7112-revert-7110-feat/developer/ldml-constants-ts-module
feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀
2022-08-25 14:13:41 -05:00
Steven R. Loomis
4c5f60ae22
Merge pull request #7125 from keymanapp/feat/core/7043-repadding 2022-08-25 13:54:59 -05:00
Steven R. Loomis
190776ef9c
Update core/tests/unit/ldml/ldml.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-25 13:53:35 -05:00
Steven R. Loomis
a809fc86be
Update core/tests/unit/ldml/ldml.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-25 13:53:21 -05:00
Steven R. Loomis
af3bcd2a15 feat(core): ldml: padding for vkey and loca 🙀
- vkey and loca needed padding before the variable part
- add static asserts to validate this padding
- update sample files (null and tiny)

#7043
2022-08-25 12:54:52 -05:00
Steven R. Loomis
f7aeba5f01 feat(core): ldml: validator for kmxplus 🙀
- also update test case, 002_null_invalid is expected to not validate

#7097
2022-08-25 12:54:08 -05:00
Steven R. Loomis
43b3e3243e fix(core): ldml: emit nothing (no err) on unassigned keys 🙀
- also update test case

#7097
2022-08-25 11:22:19 -05:00
Marc Durdin
23d9aad0a0 chore(developer): cleanup 2022-08-25 15:12:00 +10:00
Marc Durdin
4f05e009d3
Merge branch 'revert-7112-revert-7110-feat/developer/ldml-constants-ts-module' into feat/developer/ldml-compiler-kmxplus-builder 2022-08-24 21:40:45 -05:00
Marc Durdin
aed8f0d11a feat(developer): Write kmxplus section
Adds support for writing kmxplus section of a kmx file. Extends KMXFile
to a KMXPlusFile subclass. Adds KMXPlusBuilder which is called from
KMXBuilder if KMXBuilder finds that the file is a KMXPlusFile.

This means that KMXPlusBuilder generates a buffer that could be stored
independently of its parent .kmx file transport, if that is later
considered desirable.

Adds new constants for sizes of various table components, which makes
building the file much simpler.
2022-08-25 12:26:21 +10:00
Marc Durdin
e84bd590b0 chore: Merge branch 'feat/developer/ldml-compiler-kmx-builder' in 2022-08-25 08:42:23 +10:00
Steven R. Loomis
7aac6a1747 chore(core): update 000_null_keyboard to be a proper null keyboard 🙀
- rewrite 000_null_keyboard.kmx to be valid, but with no keys
- remove outdated comment in kmx_plus.h
- update kmx_plus.cpp to not reverse the order of section ids (#7098)
- add a new test option for expected-error
2022-08-24 17:04:55 -05:00
Steven R. Loomis
b180ca844d fix(core): fixes for kmx_file rearrangement 🙀
fix breaks in #7111 and prior

#7098
2022-08-24 13:49:55 -05:00
Marc Durdin
6aeb20e77c
Revert "Revert "feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀"" 2022-08-24 14:32:22 +10:00
Marc Durdin
8ebae9515e
Revert "feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀" 2022-08-24 14:26:44 +10:00
Marc Durdin
fa28040643
Merge pull request #7100 from keymanapp/fix/core/7098-cxxfun
fix(core): ldml: remove 0-length arrays 🙀
2022-08-23 23:20:14 -05:00
Marc Durdin
eb48fb76f9
Update core/src/kmx/kmx_plus.cpp 2022-08-24 14:20:08 +10:00
Marc Durdin
5e296c98af feat(core): turn @keymanapp/ldml-keyboard-constants into a module
Splits the building of keyboardprocessor_ldml.h into a separate
tsconfig.build.json, and designates keyboardprocessor_ldml.ts as a new
Typescript/Node module @keymanapp/ldml-keyboard-constants.

Includes an example usage in the (currently unused) kmx-plus.ts.
2022-08-24 14:10:20 +10:00
Steven R. Loomis
16b8d725fd fix(core): ldml: include path fix 🙀
#7098
2022-08-23 19:15:20 -05:00
Steven R. Loomis
f8e0abd810 fix(core): ldml: turn off problematic debug 🙀
#7098
2022-08-23 16:37:02 -05:00
Steven R. Loomis
b08f101f37 fix(core): ldml: goodbye multichar strings 🙀
- update builder to print out string as well as hex
- change to symbol name
2022-08-23 16:34:08 -05:00
Marc Durdin
e6ff47f2b3 chore: Merge branch 'master' into feature-ldml 2022-08-24 04:21:21 +10:00
Steven R. Loomis
a31cbc91bf fix(core): ldml: remove 0-length arrays 🙀
- need a pragma to suppress this on MSVC

#7098
2022-08-23 12:40:28 -05:00
Steven R. Loomis
59fc1521dd fix(core): ldml: use nullptr and some other cleanups 🙀
- per review comments

Fixes: #7098
2022-08-23 12:04:57 -05:00
Steven R. Loomis
da3bd0a619 fix(core): ldml: fix a bad unsigned assert 🙀
Fixes: #7098
2022-08-23 11:56:36 -05:00
Steven R. Loomis
fd4945c5ee fix(core): ldml: goodbye multichar strings 🙀
- use 32 bit hex constants instead

#7098
2022-08-23 11:27:59 -05:00
Steven R. Loomis
45cac0da6c fix(core): ldml: remove 0-length arrays 🙀
- use C99 flexible array member instead

#7098
2022-08-23 10:57:36 -05:00
Steven R. Loomis
f5db1749b4 style(core): ldml: review comments
#5015

- add copyright notices
2022-08-22 17:13:45 -05:00
Eberhard Beilharz
b03e7ca32f
chore(core): Remove obsolete python keyboardprocessor
This seems to be not used anywhere and has diverged from the API.
2022-08-22 17:46:27 +02:00
Steven R. Loomis
a4080d2b7a feat(core): ldml: cleanup of header files
- quell debug messages unless KMXPLUS_DEBUG=1
- don't read the file twice

#5015
2022-08-19 17:42:09 -05:00
Steven R. Loomis
747dfe8c0d feat(core): ldml: header cleanup 2022-08-18 14:33:53 -05:00
Steven R. Loomis
a333eade22 feat(core): ldml: tiny passes tests
- removed null from build, invalid
2022-08-18 14:27:17 -05:00