Commit graph

16226 commits

Author SHA1 Message Date
Marc Durdin
423debfef7
Merge pull request #7187 from keymanapp/feat/core/ldml-add-meta-version
feat(core): add version field to meta 🙀
2022-09-07 11:27:20 +10:00
Marc Durdin
c171a79915
Merge pull request #7186 from keymanapp/feat/common/hextobin-calculate-size-and-offset
feat(common): add size and offset calculators to hextobin 🙀
2022-09-07 11:27:12 +10:00
Marc Durdin
3307dc63a7
Merge pull request #7184 from keymanapp/feat/developer/7181-use-json-schema-to-verify-source-file
feat(developer): use json schema to verify source file 🙀
2022-09-07 11:27:04 +10:00
Marc Durdin
716b19b810
Merge pull request #7183 from keymanapp/feat/developer/7180-add-kmldmlc-test-to-developer-test-build
feat(developer): add kmlmc and kmldmlc to `make test` 🙀
2022-09-07 11:26:54 +10:00
Marc Durdin
690290bdfe
chore(core): add comment for KMXPLUS_STR
Co-authored-by: Steven R. Loomis <srl295@gmail.com>
2022-09-07 08:14:23 +10:00
Marc Durdin
bc2bcef96f
chore: Update common/tools/hextobin/package.json 2022-09-02 15:06:02 +10:00
Marc Durdin
702db90a71 feat(core): add version field to meta
Also adds semver package for validating the version field.
2022-09-02 14:13:10 +10:00
Marc Durdin
d141274aed chore(core): update hextobin files with new format 2022-09-02 14:06:26 +10:00
Marc Durdin
17371c3801 feat(common): add size and offset calculators to hextobin
This makes it easier to write a binary file which contains offsets and
size values. Using this for unit tests for the LDML keyboard compiler.

Example kmldmlc basic.txt is updated and included.

Future enhancement suggestion: include count() for repeated blocks and
index() for block indices within these repeated blocks. Would be very
useful for string tables in the example ref.
2022-09-02 11:37:59 +10:00
Marc Durdin
918d261f99 feat(developer): add conformsTo validity unit test
This is reasonably random, but wanted to add another test for content
rather than structure, just to verify that the DTD-XSD-jsonschema
translation is doing the right thing according to what we expect.

Looking good.
2022-09-02 08:01:48 +10:00
Marc Durdin
25b2d255aa feat(developer): use json schema to verify source file
As we do not appear to have access to a good DTD validation library in
Node.js, we will instead convert the DTD to a JSON schema file and use
that for validation.

I have not verified every aspect of the JSON schema at this point, but
will incrementally do so as I implement further aspects of the compiler.

This updates all the testing to use the schema. As the schema validation
is slow, I have increased the 'normal' time for tests.
2022-09-01 16:56:50 +10:00
Marc Durdin
14094c8979 feat(developer): add kmlmc and kmldmlc to make test
Also removes `-test` from normal `make build` for kmlmc, moving that
responsibility to `make test` (but uses `-tdd` to avoid rebuild).
2022-09-01 15:28:41 +10:00
Marc Durdin
2b57396f96
Merge pull request #7177 from keymanapp/feat/developer/kmldmlc-loca-minimize-plus-error-cleanup
feat(developer): loca minimization + cleanup kmldmlc error list 🙀
2022-08-31 23:53:25 -05:00
Marc Durdin
8d00d78cef
Merge pull request #7175 from keymanapp/feat/developer/kmldmlc-loca-tests
feat(developer): add loca tests 🙀
2022-08-31 23:53:15 -05:00
Steven R. Loomis
b330a62d43
Merge pull request #7179 from keymanapp/feat/core/7136-splitvalidate
chore(core): split out validators 🙀
2022-08-31 21:46:54 -05:00
Marc Durdin
3a4bbf9d0d
Merge pull request #7154 from keymanapp/feat/developer/kmldmlc-meta-tests-and-cleanup
feat(developer): add unit tests for meta compiler 🙀
2022-08-31 19:05:45 -05:00
Marc Durdin
08870288bd
Merge branch 'feature-ldml' into feat/developer/kmldmlc-meta-tests-and-cleanup 2022-08-31 19:05:34 -05:00
Steven R. Loomis
f3f498e923 fix(core): fix validators 🙀
- copy over asserts from 'old' code

#7136
2022-08-31 17:37:55 -05:00
Steven R. Loomis
3684cc43d0 fix(core): fix validators 🙀
yes, a kmxplus file does not start with a 'sect' !

#7136
2022-08-31 17:37:55 -05:00
Steven R. Loomis
90be01385b chore(core): split out validators 🙀
#7136
2022-08-31 17:37:55 -05:00
Marc Durdin
36b7dcc46a
Merge pull request #7174 from keymanapp/chore/merge-master-into-feature-ldml
chore: merge master into feature-ldml 🙀
2022-08-31 17:04:25 -05:00
Marc Durdin
7d38e26b09
chore: Apply suggestions from code review 2022-08-31 14:46:27 +10:00
Marc Durdin
1c0f68bb75 feat(developer): loca minimization + cleanup kmldmlc error list
Yeah two for the price of one:

- minimize locales at compile time
- cleans up the list of errors to make it easier to maintain and use
2022-08-31 14:30:03 +10:00
Marc Durdin
eca37e8aa2
Merge branch 'feature-ldml' into feat/developer/kmldmlc-meta-tests-and-cleanup 2022-08-30 22:29:30 -05:00
Marc Durdin
17c221f4ce chore: remove comment 2022-08-31 10:21:03 +10:00
Marc Durdin
b58bb36d80 feat(developer): add loca tests
Note: bumps target to es2020 for better Intl.* support. Verified as
supported by Node 16+ and modern browsers.

Adds support for multiple locales and basic canonicalization. Does not
support normalization of locales.
2022-08-31 10:18:57 +10:00
Marc Durdin
5d44d2ecc7 chore(developer): Merge branch 'chore/merge-master-into-feature-ldml' into feat/developer/kmldmlc-meta-tests-and-cleanup 2022-08-31 10:10:03 +10:00
Marc Durdin
696ae9964d
Merge pull request #7153 from keymanapp/refactor/developer/kmldmlc-pure-functional-section-compilers
refactor(developer): each section compiler is a pure function 🙀
2022-08-30 19:04:05 -05:00
Steven R. Loomis
0e29be0c80
Merge pull request #7159 from keymanapp/feat/core/7136-debug
feat(core): set -DDEBUG for debug 🙀
2022-08-30 13:20:27 -05:00
Keyman Server
90b7e38c81
Merge pull request #7160 from keymanapp/auto/version-master-16.0.55
auto: increment master version to 16.0.55
2022-08-31 04:03:16 +10:00
Keyman Build Agent
957c2b04c3 auto: increment master version to 16.0.55 2022-08-30 14:02:40 -04:00
Steven R. Loomis
bef0013bf8 feat(core): set -DDEBUG for debug 🙀
- set -DDEBUG for 'debug' meson builds
- default KXPLUS_DEBUG  to 1 for -DDEBUG

#7136
2022-08-30 11:49:22 -05:00
Marc Durdin
8fc7921ed4 chore(developer): cleanup todo comments 2022-08-30 13:57:16 +10:00
Marc Durdin
995d731ccb feat(developer): add unit tests for meta compiler
First, a little bit of refactoring to make it cleaner to do unit test
for each section.

Then add some global error values.

Throw in a helper function or two.

Dust off the edges of the CompilerCallbacks object.

Duct tape together validation of the normalization attribute.

And ... eventually we get a nice clean-like unit test for the meta
compiler, with three pretty little fixtures and as-neat-as-you-like-it
assertions.
2022-08-30 13:52:36 +10:00
Marc Durdin
d27f288c13
Merge pull request #7149 from keymanapp/chore/wasm-on-linux
chore(core): emcc off path for linux
2022-08-29 20:53:15 -05:00
Marc Durdin
920352b360
Merge pull request #7151 from keymanapp/refactor/developer/kmldmlc-compiler-separate-files-for-sections
refactor(developer): separate files for kmldmlc sections 🙀
2022-08-29 20:53:01 -05:00
Marc Durdin
91283464ca
Merge pull request #7150 from keymanapp/chore/merge-master-feature-ldml
chore: merge master into feature-ldml 🙀
2022-08-29 20:52:44 -05:00
Marc Durdin
db11553c9a refactor(developer): each section compiler is a pure function
Well, almost. There are still callback side-effects for messages. These
may in future be returned as an array as well, eliminating the need for
a message callback.

But restructures each section compiler so that it is self-contained and
responsible only for writing its own section. Also tightens up the types
for the sections constants.

Last, but not least, moves responsibility for writing the output .kmx
out of compiler.ts and into its callers -- removing one dependency from
the compiler. Reading input files still needs to be done in the
compiler, as there may be multiple input files with `<import>` that we
cannot know about until we start parsing xml.
2022-08-30 11:13:11 +10:00
Eberhard Beilharz
aa55fc2571
Merge pull request #7147 from keymanapp/fix/linux/jenkins.sh
fix(linux): Allow downgrades for installing build deps
2022-08-30 00:26:17 +02:00
Steven R. Loomis
9c79a3a109
Merge pull request #7152 from keymanapp/fix/core/5015-fixtiny
fix(core): ldml: fix validator 🙀
2022-08-29 17:26:10 -05:00
Eberhard Beilharz
54a314df0e
Merge pull request #7145 from keymanapp/chore/linux/cherry-pick/changelog
chore(linux): Update debian changelog 🍒
2022-08-30 00:23:23 +02:00
Steven R. Loomis
f03abf2ea5
Update core/src/ldml/ldml_processor.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-29 17:23:17 -05:00
Steven R. Loomis
46d7c96823 fix(core): ldml: fix validator 🙀
- allow keys section to exactly fit in data size, as in #7140
2022-08-29 17:16:45 -05:00
Steven R. Loomis
8c65dea9b3
Merge pull request #7132 from keymanapp/feat/core/5015-fileread
feat(core): ldml: improve file reading/validation 🙀
2022-08-29 16:45:56 -05:00
Steven R. Loomis
c03ab40d5d feat(core): ldml: update spec per code review
#7043
2022-08-29 16:41:39 -05:00
Steven R. Loomis
1c7188f9fd
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-08-29 16:41:12 -05:00
Marc Durdin
c0147dc6ea refactor(developer): separate files for kmldmlc sections
@keymanapp-test-bot skip
2022-08-30 07:16:53 +10:00
Marc Durdin
655c872dc8 chore: Merge branch 'master' into chore/merge-master-feature-ldml 2022-08-30 06:31:04 +10:00
Marc Durdin
9edabec297 chore(core): emcc off path for linux 2022-08-30 06:20:48 +10:00
Marc Durdin
8dc68edc6c
Merge pull request #7140 from keymanapp/feat/developer/unit-tests-for-kmldmlc
feat(developer): unit test framework for kmldmlc 🙀
2022-08-29 15:16:41 -05:00