Commit graph

1143 commits

Author SHA1 Message Date
Steven R. Loomis
3de77cda92 feat(core): ldml: improve reading 🙀
- read keys into a std::map instead of on the fly
- no longer retain copy of entire KMX file in read
2022-08-26 16:43:27 -05:00
Steven R. Loomis
892492a634 feat(core): ldml: improve validation 🙀
- clearer asserts
- validate length of data

#5015
2022-08-26 15:45:48 -05:00
Marc Durdin
44d7d5ca88 chore(core): always build deps 2022-08-27 06:23:33 +10:00
Steven R. Loomis
aeeae1c4f4 feat(core): ldml: improve read side 🙀
- don't reparse structs each time
- move validation into constructor
2022-08-26 13:15:05 -05:00
Marc Durdin
3ff5c4c46e chore(core): fix typo 2022-08-26 15:58:25 +10:00
Marc Durdin
a3b1a96960 chore(core): disable failing tests for now 2022-08-26 15:48:19 +10:00
Marc Durdin
4820f99503 chore(core): also build keyman-version as a prereq 2022-08-26 15:41:44 +10:00
Marc Durdin
daf6a7970e feat(core): use kmldmlc to build test keyboards
Removes .kmx from repo and builds them from the source .xml files using
kmldmlc (building that if necessary also with a poor fella's dependency
check for now), and generally a lot of cleanup of the meson.build files
for the ldml tests.

Moves keyboards to a subfolder of the ldml unit test folder to keep them
neat and tidy and avoid embedding huge files into ldml.js for test of
WASM.

Note: 000_null_keyboard and 002_null_invalid do build successfully, but
the core ldml keyboardprocessor currently fails on them, because they
are missing keys and vkey sections.
2022-08-26 15:12:38 +10:00
Marc Durdin
9756c28102 chore(developer): Merge branch 'feature-ldml' into feat/developer/ldml-compiler-kmxplus-builder 2022-08-26 05:17:05 +10:00
Marc Durdin
144025e29f
Merge pull request #7113 from keymanapp/revert-7112-revert-7110-feat/developer/ldml-constants-ts-module
feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀
2022-08-25 14:13:41 -05:00
Steven R. Loomis
4c5f60ae22
Merge pull request #7125 from keymanapp/feat/core/7043-repadding 2022-08-25 13:54:59 -05:00
Steven R. Loomis
190776ef9c
Update core/tests/unit/ldml/ldml.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-25 13:53:35 -05:00
Steven R. Loomis
a809fc86be
Update core/tests/unit/ldml/ldml.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-25 13:53:21 -05:00
Steven R. Loomis
af3bcd2a15 feat(core): ldml: padding for vkey and loca 🙀
- vkey and loca needed padding before the variable part
- add static asserts to validate this padding
- update sample files (null and tiny)

#7043
2022-08-25 12:54:52 -05:00
Steven R. Loomis
f7aeba5f01 feat(core): ldml: validator for kmxplus 🙀
- also update test case, 002_null_invalid is expected to not validate

#7097
2022-08-25 12:54:08 -05:00
Steven R. Loomis
43b3e3243e fix(core): ldml: emit nothing (no err) on unassigned keys 🙀
- also update test case

#7097
2022-08-25 11:22:19 -05:00
Marc Durdin
23d9aad0a0 chore(developer): cleanup 2022-08-25 15:12:00 +10:00
Marc Durdin
4f05e009d3
Merge branch 'revert-7112-revert-7110-feat/developer/ldml-constants-ts-module' into feat/developer/ldml-compiler-kmxplus-builder 2022-08-24 21:40:45 -05:00
Marc Durdin
aed8f0d11a feat(developer): Write kmxplus section
Adds support for writing kmxplus section of a kmx file. Extends KMXFile
to a KMXPlusFile subclass. Adds KMXPlusBuilder which is called from
KMXBuilder if KMXBuilder finds that the file is a KMXPlusFile.

This means that KMXPlusBuilder generates a buffer that could be stored
independently of its parent .kmx file transport, if that is later
considered desirable.

Adds new constants for sizes of various table components, which makes
building the file much simpler.
2022-08-25 12:26:21 +10:00
Marc Durdin
e84bd590b0 chore: Merge branch 'feat/developer/ldml-compiler-kmx-builder' in 2022-08-25 08:42:23 +10:00
Steven R. Loomis
7aac6a1747 chore(core): update 000_null_keyboard to be a proper null keyboard 🙀
- rewrite 000_null_keyboard.kmx to be valid, but with no keys
- remove outdated comment in kmx_plus.h
- update kmx_plus.cpp to not reverse the order of section ids (#7098)
- add a new test option for expected-error
2022-08-24 17:04:55 -05:00
Steven R. Loomis
b180ca844d fix(core): fixes for kmx_file rearrangement 🙀
fix breaks in #7111 and prior

#7098
2022-08-24 13:49:55 -05:00
Marc Durdin
6aeb20e77c
Revert "Revert "feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀"" 2022-08-24 14:32:22 +10:00
Marc Durdin
8ebae9515e
Revert "feat(core): turn @keymanapp/ldml-keyboard-constants into a module 🙀" 2022-08-24 14:26:44 +10:00
Marc Durdin
fa28040643
Merge pull request #7100 from keymanapp/fix/core/7098-cxxfun
fix(core): ldml: remove 0-length arrays 🙀
2022-08-23 23:20:14 -05:00
Marc Durdin
eb48fb76f9
Update core/src/kmx/kmx_plus.cpp 2022-08-24 14:20:08 +10:00
Marc Durdin
5e296c98af feat(core): turn @keymanapp/ldml-keyboard-constants into a module
Splits the building of keyboardprocessor_ldml.h into a separate
tsconfig.build.json, and designates keyboardprocessor_ldml.ts as a new
Typescript/Node module @keymanapp/ldml-keyboard-constants.

Includes an example usage in the (currently unused) kmx-plus.ts.
2022-08-24 14:10:20 +10:00
Steven R. Loomis
16b8d725fd fix(core): ldml: include path fix 🙀
#7098
2022-08-23 19:15:20 -05:00
Steven R. Loomis
f8e0abd810 fix(core): ldml: turn off problematic debug 🙀
#7098
2022-08-23 16:37:02 -05:00
Steven R. Loomis
b08f101f37 fix(core): ldml: goodbye multichar strings 🙀
- update builder to print out string as well as hex
- change to symbol name
2022-08-23 16:34:08 -05:00
Marc Durdin
e6ff47f2b3 chore: Merge branch 'master' into feature-ldml 2022-08-24 04:21:21 +10:00
Steven R. Loomis
a31cbc91bf fix(core): ldml: remove 0-length arrays 🙀
- need a pragma to suppress this on MSVC

#7098
2022-08-23 12:40:28 -05:00
Steven R. Loomis
59fc1521dd fix(core): ldml: use nullptr and some other cleanups 🙀
- per review comments

Fixes: #7098
2022-08-23 12:04:57 -05:00
Steven R. Loomis
da3bd0a619 fix(core): ldml: fix a bad unsigned assert 🙀
Fixes: #7098
2022-08-23 11:56:36 -05:00
Steven R. Loomis
fd4945c5ee fix(core): ldml: goodbye multichar strings 🙀
- use 32 bit hex constants instead

#7098
2022-08-23 11:27:59 -05:00
Steven R. Loomis
45cac0da6c fix(core): ldml: remove 0-length arrays 🙀
- use C99 flexible array member instead

#7098
2022-08-23 10:57:36 -05:00
Steven R. Loomis
f5db1749b4 style(core): ldml: review comments
#5015

- add copyright notices
2022-08-22 17:13:45 -05:00
Eberhard Beilharz
b03e7ca32f
chore(core): Remove obsolete python keyboardprocessor
This seems to be not used anywhere and has diverged from the API.
2022-08-22 17:46:27 +02:00
Steven R. Loomis
a4080d2b7a feat(core): ldml: cleanup of header files
- quell debug messages unless KMXPLUS_DEBUG=1
- don't read the file twice

#5015
2022-08-19 17:42:09 -05:00
Steven R. Loomis
747dfe8c0d feat(core): ldml: header cleanup 2022-08-18 14:33:53 -05:00
Steven R. Loomis
a333eade22 feat(core): ldml: tiny passes tests
- removed null from build, invalid
2022-08-18 14:27:17 -05:00
Steven R. Loomis
70639c585b feat(core): more kmx_plus_dump functions
- also update sample data again
2022-08-17 18:13:17 -05:00
Steven R. Loomis
5926dfe5fe feat(core): more kmx_plus_dump functions
- also update sample data
2022-08-17 17:43:14 -05:00
Steven R. Loomis
590ac8f8f6 feat(core): build out ldml structures
- beginning of str handling
2022-08-17 16:25:41 -05:00
Steven R. Loomis
95b4469ea8 feat(core): build out ldml structures
- checkin spec to repo
- new header: kmx_plus.h with data access structs and functions
- static asserts to validate header sizes
- early validation of data
- internal dump of kmx+ data
2022-08-17 15:53:53 -05:00
Steven R. Loomis
615eec89ca feat(core): augment unit tests for ldml
- add 001_tiny to the build, annotate with expected run
2022-08-17 13:31:09 -05:00
Steven R. Loomis
5e203f8eb5 Update core/src/kmx/kmx_file.h
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-16 10:24:53 -05:00
Steven R. Loomis
9e44a7f9bc feat(core): Sample LDML data updates, kmx_file #5015
Review comments:
- moved KMXPLUSINFO into separate struct
- created resource/standards-data/ldml-keyboards for CLDR data
- corrected Khmer err
2022-08-16 10:24:53 -05:00
Steven R. Loomis
1ccc43df04 Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-16 10:24:53 -05:00
Steven R. Loomis
189124ba69 feat(core): update COMP_KEYBOARD_EX #5015
- uberstruct
2022-08-16 10:24:53 -05:00