Commit graph

205 commits

Author SHA1 Message Date
Steven R. Loomis
0c290bd37a feat(developer): update for line numbers
- update to metadata branch of fast-xml-parser
- restructure metadata handling
- use type safety for fast-xml-parser options

Fixes: #10622
2025-03-31 16:11:19 -05:00
Steven R. Loomis
8aa9219410 Merge remote-tracking branch 'upstream/master' into feat/developer/10622-xml-parse-line-numbers-epic-ldml 2025-03-27 16:02:45 -05:00
Steven R. Loomis
692ea6ee9e feat(core): improve kmxplus validation tests
- update basic.txt to include the kvk section #12402
- use run() instead of compile()
- move compile()+serialize to a separate subtest
2025-03-21 17:11:03 -05:00
Steven R. Loomis
4e3379c698 feat(core): improve kmxplus validation tests
- ratchet required coverage level to 90%!
- comment out failing run() test though - run() adds an embedded .kvk

Fixes: #9446
2025-03-18 11:50:28 -05:00
Steven R. Loomis
7487a6bbbe chore(developer): improve code coverage
- add a test for LdmlKeyboardCompiler.run()
- exclude generated abnf files from coverage
2025-03-14 17:20:23 -05:00
Marc Durdin
1a0b830c0b
Merge branch 'master' into chore/b18s3-merge-beta-to-master 2025-03-14 08:54:50 +07:00
Marc Durdin
7645627f21 fix(developer): strip markers when generating KVK from LDML
Markers are permitted in key outputs, but when these are used to
generate a .kvk, they should be stripped out.

Also added a test for disp to verify that markers are not transformed in
`<display display=` attribute values.

Fixes: #13469
2025-03-12 08:06:34 +07:00
Steven R. Loomis
b015221814 feat(developer): line number symbols
- change special symbol to XML_START_INDEX_SYMBOL
- use an array of line lengths rather than an array of strings for the file info
- add an EventResolver interface to the callbacks, such that it has an opportunity to resolve events

Fixes: #10622
2025-03-03 17:08:40 -06:00
Steven R. Loomis
4d9ea2db2d feat(developer): work around line number symbols
- add KeymanXMLReader.removeSymbols() and call it from tests

Fixes: #10622
2025-02-28 14:24:12 -06:00
Steven R. Loomis
819d9bf562 feat(developer): API for resolving line numbers for messages
- change to use a different fast-xml-parser PR, https://github.com/NaturalIntelligence/fast-xml-parser/pull/729

Fixes: #10622
2025-02-28 12:08:31 -06:00
Steven R. Loomis
61b9234daf feat(developer): document test cases
Fixes: #13175
2025-02-20 08:20:50 -06:00
Steven R. Loomis
92463b9a89 feat(developer): ldml: update tests for ABNF
- reinstate errors

Fixes: #13175
2025-02-17 16:17:03 -06:00
Steven R. Loomis
24f629a9f1 feat(developer): ldml: update tests for ABNF
- some error messages are less specific now than they were before ABNF, because we're not using custom code to test them.

Fixes: #13175
2025-02-14 16:29:16 -06:00
Steven R. Loomis
b59bb6d4de feat(developer): serialize KMXPlus into XML 🍒
Cherry pick of #12969
> feat(developer): serialize KMXPlus (back) into XML 🗼

(cherry picked from commit f93c45c9e8 #12969)

Fixes: #12874
2025-02-10 12:12:12 -06:00
Steven R. Loomis
b35ba06a2b chore(developer): remove workaround for base=""
Fixes: #12749
2025-01-30 15:48:02 -06:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Steven R. Loomis
f4f0355109 feat(developer): local imports directory fixes
- use path.join instead of path.resolve
- rename importsPath to cldrImportsPath
- remove k_015_importlocal from core, as it duplicated tests in common and developer

Fixes: #10649
2024-12-02 15:40:54 -06:00
Steven R. Loomis
75d172a6cb
Merge branch 'feat/developer/7394-verify-normalization-epic-ldml' into feat/developer/10649-local-imports 2024-12-02 11:37:29 -06:00
Steven R. Loomis
6f2a6b2ebf feat(common,developer): tests and warning on denormalized content per review comments
- report a warning on each separate instance of a denormalized string (once per string).

Fixes: #7394
2024-12-02 11:35:06 -06:00
Steven R. Loomis
01fed4b2ab feat(developer): test local imports in kmc-ldml
- also improve callback visibility when XML fails to load

Fixes: #10649
2024-11-29 14:11:29 -06:00
Steven R. Loomis
add502e1a9 feat(common,developer): tests and warning on denormalized content
- common: shift the 'strs' processing slightly, because otherwise we normalize to NFD before even tracking the strings
- if any string is neither NFC nor NFD, give a warning
- add tests for the same

Fixes: #7394
2024-11-29 12:16:11 -06:00
Eberhard Beilharz
66404f9503
chore(developer): rename test files
This change rename the test files for Developer according to the
discussion at the Keyman conference in November 2024.
2024-11-28 14:32:09 +01:00
Marc Durdin
b54b005830 refactor(common): move remaining LDML keyboard types into LdmlKeyboardTypes
Follows: #12712
2024-11-27 11:52:53 +07:00
Steven R. Loomis
774eba727e chore(developer): fix typo in test comment
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-11-07 09:37:54 -06:00
Steven R. Loomis
3eac4f12c9 fix(common): check for invalid markers
- pattern parser update
- correct spelling of Error_InvalidVariableIdentifier
- track and report bad marker strings (look like markers but not)
- fix: may have missed markers mentioned in variables
- tests for above

Fixes: 12467
2024-11-04 10:15:56 -06:00
Steven R. Loomis
c057112c9f fix(developer): add a test for empty varable id, fix message
- stray \u in Error_InvalidVariableIdentifer
- add a test for an empty variable id
2024-11-01 13:47:39 -05:00
Steven R. Loomis
ac99a4c54d fix(developer): ldml don't allow a uset as right-hand-side variable
- to="$[triisap]" is a mistake and not allowed.
- it's either meant to be a regular variable, or a map. Both of those are different syntax.
- add a test for the failing case, and also for the escaped case (to="\$[…)
2024-11-01 13:03:31 -05:00
Marc Durdin
71f3d912a4 fix(developer): ignore excess whitespace in <row keys> attribute
Fixes: #12449
2024-09-25 07:33:42 -07:00
Marc Durdin
2c3ae9eba7 fix(developer): prevent invalid string ids
Adds checks for invalid string IDs and corresponding unit tests.

Fixes: #12451
2024-09-24 19:29:10 -07:00
Marc Durdin
201f48a1f0
Merge pull request #12404 from keymanapp/fix/developer/12403-validate-string-use-before-definition
fix(developer): check vars string usage before definition
2024-09-17 00:40:24 +02:00
Marc Durdin
f4321b38d7 fix(developer): LDML compiler, add TSS_VISUALKEYBOARD store when compiling visual keyboard
Fixes: #12395
2024-09-16 14:00:39 +07:00
Marc Durdin
b7810bdd2e fix(developer): check vars string usage before definition
Validation of vars was not properly checking for forward references to
string variables. This, coupled with a null vs undefined bug in
subsequent use, meant that forward reference variables were ending up
with a literal string value of 'undefined'.

This also fixes the test for visual-keyboard-compiler, where the fixture
was actually buggy and was the trigger for investigating this problem.

Fixes: #12403
Relates-to: #12395
2024-09-12 09:18:06 +07:00
Marc Durdin
e4017a8c28 fix(developer): rewrite ldml visual keyboard compiler
The visual keyboard compiler was never finished in 17.0. This rewrites
it to:

1. Use the kmxplus data rather than reading from xml directly
2. Fill in `visualkeyboard.header.kbdname`
3. Support modifiers
4. Handle encoded characters like `\u{1234}`
5. Handle string variables like `${one}`*

Additional unit tests have been added to verify the behavior of the
visual keyboard compiler in more detail.

TODO-LDML: string variables appear to have a secondary bug -- they seem
to be returning the string 'undefined'. I have disabled the related
tests and will examine this separately, and enable those tests once
fixed.

TODO-LDML: we should probably add a compiler warning + unit test for
`<layers formId="us"><layer id="base">`, because this pattern does not
make sense: when using non-touch forms, the `<layer>` element should use
`modifiers` attribute, and correspondingly, `modifiers` attribute should
_not_ be used when `formId` is `touch`.

Other fixes:

1. The LDML XML reader was relying on its input being a Node.js `Buffer`
   even though it was declared `Uint8Array`, as it implicitly used
   `Buffer.toString()` to do text conversion. (`Buffer` subclasses from
   `Uint8Array`). This breaks when using `Uint8Array` directly and means
   we had an implicit dependency on Node.js. See also #12331.
2. XML errors were not captured in the LDML XML reader. See also #12331.
3. The unused and unfinished touch-layout-compiler.ts and
   keymanweb-compiler.ts have been removed along with corresponding unit
   tests and fixtures. These are replaced by Core implementations; see
   #12291.

Fixes: #12395
2024-09-12 06:06:00 +07:00
Marc Durdin
32b4711659
Merge branch 'master' into refactor/common/9665-move-compiler-interfaces-to-developer 2024-08-07 15:15:45 +10:00
Marc Durdin
ed143fb333
Merge branch 'master' into refactor/common/9665-move-ldml-keyboard-source-file-to-developer 2024-08-07 15:15:04 +10:00
Marc Durdin
e67aed1606 refactor(common): move compiler-interfaces to @keymanapp/developer-utils
This should be the last step in the refactoring.

Fixes: #9665
2024-08-02 13:49:59 +07:00
Marc Durdin
d3d3ac1eec refactor(common): move LDML keyboard .xml reader/writer and kmx-plus builder to @keymanapp/developer-utils
Relates-to: #9665
2024-08-02 10:23:11 +07:00
Marc Durdin
d8986be12f refactor(common): merge master into branch 2024-08-01 07:07:07 +07:00
Marc Durdin
d1e351a1ac
Revert "chore(developer): remove redundant references from tsconfig.json" 2024-08-01 08:57:27 +10:00
Marc Durdin
1cff1a5084 chore(developer): remove redundant references from tsconfig.json
With the full move to ES Modules, we no longer need to include
`references` in tsconfig.json, as we can rely on package.json and
build.sh dependency management. Note however that `tsc -b` may not work
to build dependencies -- they need to be built using `build.sh`, which
calculates which dependencies need building.

kmc-keyboard-info was missing a dependency link to kmc-package in
build.sh, correcting this at the same time.

Relates-to: #12027
2024-07-27 05:44:22 +07:00
Marc Durdin
1afa4beb01 fix(developer): remove paths from tsconfig.json
Fixes: #12027
2024-07-26 06:59:45 +07:00
Marc Durdin
909b71b6fc chore(developer): rename kmc-ldml CompilerMessages to LdmlCompilerMessages
It is easier to differentiate between all the compiler message classes
if each one has its own name.
2024-07-22 15:52:37 +07:00
Marc Durdin
632f98d029 chore(developer): update unit test CompilerMessages references 2024-07-22 15:26:09 +07:00
Steven R. Loomis
f390de30da
Merge pull request #11843 from keymanapp/fix/developer/10553-reduce-bogus-unclear-errors
fix(developer): layr: fix modifier err message on layer w/o id 🙀
2024-06-21 08:25:08 -05:00
Steven R. Loomis
4dc02c4952 fix(developer): layr: fix modifier err message on layer w/o id
- the error message was broken when there wasn't a layer id (which is allowed)
- add a test case

Fixes: #10553
2024-06-20 17:19:44 -05:00
Steven R. Loomis
570766ca7c feat(developer): linter on blank keycaps
add a linter and hints to detect these cases:
- 'switch' key with no output and no matching display
- any other key whose only output is markers (so cannot otherwise display) and no matching display

Fixes: #11054
2024-06-20 16:02:43 -05:00
Steven R. Loomis
8deae5137c test(developer): shuffle tests
- split out test-strs from test-compiler-e2e
- split out test-linter from test-compiler-e2e
2024-06-18 10:15:28 -05:00
Steven R. Loomis
612064f1b2 feat(developer): warn on empty keycaps
- add a hint and test for cases where no display matches a marker-only key

Fixes: #11054
2024-06-17 17:49:59 -05:00
Marc Durdin
75d39069ec chore(developer): increase timeout for kmc-ldml compiler test
Fixes: #11632
2024-05-31 15:44:40 +07:00
Steven R. Loomis
7fb97efa43 fix(developer): escape plus and asterisk sign
- additional tests for asterisk and plus

Fixes: #11233
2024-04-19 16:37:16 -05:00