Commit graph

329 commits

Author SHA1 Message Date
Steven R. Loomis
29f387bebe Merge remote-tracking branch 'upstream/master' into feat/developer/10622-xml-parse-line-numbers-epic-ldml 2025-03-31 16:16:10 -05:00
Steven R. Loomis
0c290bd37a feat(developer): update for line numbers
- update to metadata branch of fast-xml-parser
- restructure metadata handling
- use type safety for fast-xml-parser options

Fixes: #10622
2025-03-31 16:11:19 -05:00
Marc Durdin
ab59fa29b4
Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:41:03 +07:00
Marc Durdin
a52437c407 chore: Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:38:08 +07:00
Marc Durdin
de5a02f8ce chore(developer): rewrap detail on semver LDML message 2025-03-29 04:14:38 +07:00
Marc Durdin
dbfe03aa8e chore(developer): improve kmc-ldml 'invalid version' message
Suggested by @Nnyny. Give more detail on the semantic version format for
keyboard version, as well as details on the limitations to semver
imposed by the Keyman toolchain.
2025-03-28 15:01:11 +07:00
Steven R. Loomis
8aa9219410 Merge remote-tracking branch 'upstream/master' into feat/developer/10622-xml-parse-line-numbers-epic-ldml 2025-03-27 16:02:45 -05:00
Steven R. Loomis
18e7686b3e chore(developer): fix build message per review comment
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-24 12:29:32 -05:00
Steven R. Loomis
692ea6ee9e feat(core): improve kmxplus validation tests
- update basic.txt to include the kvk section #12402
- use run() instead of compile()
- move compile()+serialize to a separate subtest
2025-03-21 17:11:03 -05:00
Steven R. Loomis
4e3379c698 feat(core): improve kmxplus validation tests
- ratchet required coverage level to 90%!
- comment out failing run() test though - run() adds an embedded .kvk

Fixes: #9446
2025-03-18 11:50:28 -05:00
Steven R. Loomis
7487a6bbbe chore(developer): improve code coverage
- add a test for LdmlKeyboardCompiler.run()
- exclude generated abnf files from coverage
2025-03-14 17:20:23 -05:00
Marc Durdin
780ca714a1 chore(developer): reduce kmc-ldml coverage threshold to 75%
kmc-ldml no longer meets the 90% coverage threshold with recent changes
in 19.0-alpha. With the fix from #13418, this now causes the build to
fail. Coverage should be increased with additional tests in kmc-ldml,
but this is a stopgap to allow builds to continue to pass.
2025-03-14 09:48:45 +07:00
Marc Durdin
1a0b830c0b
Merge branch 'master' into chore/b18s3-merge-beta-to-master 2025-03-14 08:54:50 +07:00
Marc Durdin
7645627f21 fix(developer): strip markers when generating KVK from LDML
Markers are permitted in key outputs, but when these are used to
generate a .kvk, they should be stripped out.

Also added a test for disp to verify that markers are not transformed in
`<display display=` attribute values.

Fixes: #13469
2025-03-12 08:06:34 +07:00
Steven R. Loomis
92e16b90a0 feat(developer): line number symbols
- move file read bottleneck out of compiler.ts into the LDML XML reader
- move the eventresolver into the LDML XML reader
- add XML_FILENAME_SYMBOL - for tagging when an object is from a different file than the default
(such as with an import)

Fixes: #10622
2025-03-03 19:12:34 -06:00
Steven R. Loomis
b015221814 feat(developer): line number symbols
- change special symbol to XML_START_INDEX_SYMBOL
- use an array of line lengths rather than an array of strings for the file info
- add an EventResolver interface to the callbacks, such that it has an opportunity to resolve events

Fixes: #10622
2025-03-03 17:08:40 -06:00
Steven R. Loomis
d7cb7b63a3 feat(developer): add an EventResolver
- it's a hook to the hook to allow the LDML compiler to resolve error message line numbers
- TODO still not extra performant, but maybe the right separation of concerns

Fixes: #10622
2025-03-03 13:14:41 -06:00
Steven R. Loomis
3e73a3f081 feat(developer): add CompilerEvent.offset
- per review, keep this as a separate field.

Fixes: #10622
2025-03-03 12:36:42 -06:00
Steven R. Loomis
4d9ea2db2d feat(developer): work around line number symbols
- add KeymanXMLReader.removeSymbols() and call it from tests

Fixes: #10622
2025-02-28 14:24:12 -06:00
Steven R. Loomis
819d9bf562 feat(developer): API for resolving line numbers for messages
- change to use a different fast-xml-parser PR, https://github.com/NaturalIntelligence/fast-xml-parser/pull/729

Fixes: #10622
2025-02-28 12:08:31 -06:00
Steven R. Loomis
650042fb6e feat(developer): API for resolving line numbers for messages
- simplify the error call site again

Fixes: #10622
2025-02-28 10:53:21 -06:00
Steven R. Loomis
c4ca359d18 feat(developer): API for resolving line numbers for messages
- add 'column' to CompilerEvent
- Add a LdmlCompilerMessages.col() function to set the column number
- add a LdmlCompilerMessages.resolveLineNumber() to map offsets to line number

Fixes: #10622
2025-02-28 09:27:25 -06:00
Steven R. Loomis
61b9234daf feat(developer): document test cases
Fixes: #13175
2025-02-20 08:20:50 -06:00
Steven R. Loomis
7e28758892
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-02-20 08:18:55 -06:00
Steven R. Loomis
3c3d2586e6 chore(developer): ldml: build updates for ABNF
- fix build output

Fixes: #13175
2025-02-18 11:05:55 -06:00
Steven R. Loomis
92463b9a89 feat(developer): ldml: update tests for ABNF
- reinstate errors

Fixes: #13175
2025-02-17 16:17:03 -06:00
Steven R. Loomis
24f629a9f1 feat(developer): ldml: update tests for ABNF
- some error messages are less specific now than they were before ABNF, because we're not using custom code to test them.

Fixes: #13175
2025-02-14 16:29:16 -06:00
Steven R. Loomis
2e842bc19c feat(developer): update builder for abnf
- just convert over all abnf (as we do with copying imports)
- hard coded CLDR version number for now

Fixes: #13175
2025-02-14 13:12:11 -06:00
Steven R. Loomis
53b20e6adc feat(developer): use peggy grammar in from/to transform
Fixes: #13175
2025-02-14 12:27:15 -06:00
Steven R. Loomis
a2a0e79675 feat(core): convert abnf to peggy grammar
- convert each .abnf to a .pegjs file

Fixes: #13175
2025-02-14 12:27:15 -06:00
Steven R. Loomis
9a33a26d5c feat(developer): preserve CLDR version on serialize
Fixes: #12874
2025-02-10 12:35:25 -06:00
Steven R. Loomis
b59bb6d4de feat(developer): serialize KMXPlus into XML 🍒
Cherry pick of #12969
> feat(developer): serialize KMXPlus (back) into XML 🗼

(cherry picked from commit f93c45c9e8 #12969)

Fixes: #12874
2025-02-10 12:12:12 -06:00
Marc Durdin
bd4633333b feat(developer): improve compiler messages and user interface
* Add unit test to verify the format of message detail strings
  (max line length of 80 chars)
* Add unit test to verify non-null, non-empty message strings
* Tidy up a number of messages (to pass unit tests)
* Add message details for all kmw compiler messages
* Add a clickable link for the TIKE message window for each message
  so that explanations and further message details are more accessible
  to authors (`kmc message` is already available for command-line users)
* Fixes a number of minor issues with the messages window in TIKE,
  mostly relating to when popup menu commands are enabled and how
  messages are selected.

Fixes: #13134
2025-02-07 08:46:46 +07:00
Steven R. Loomis
b35ba06a2b chore(developer): remove workaround for base=""
Fixes: #12749
2025-01-30 15:48:02 -06:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Steven R. Loomis
f4f0355109 feat(developer): local imports directory fixes
- use path.join instead of path.resolve
- rename importsPath to cldrImportsPath
- remove k_015_importlocal from core, as it duplicated tests in common and developer

Fixes: #10649
2024-12-02 15:40:54 -06:00
Steven R. Loomis
75d172a6cb
Merge branch 'feat/developer/7394-verify-normalization-epic-ldml' into feat/developer/10649-local-imports 2024-12-02 11:37:29 -06:00
Steven R. Loomis
17185d79a9
Merge branch 'master' into feat/developer/7394-verify-normalization-epic-ldml 2024-12-02 11:36:21 -06:00
Steven R. Loomis
6f2a6b2ebf feat(common,developer): tests and warning on denormalized content per review comments
- report a warning on each separate instance of a denormalized string (once per string).

Fixes: #7394
2024-12-02 11:35:06 -06:00
Marc Durdin
cd8825d504
Merge pull request #12736 from keymanapp/refactor/developer/unify_test_action
refactor(developer): unify test action
2024-12-01 00:14:11 +01:00
Steven R. Loomis
01fed4b2ab feat(developer): test local imports in kmc-ldml
- also improve callback visibility when XML fails to load

Fixes: #10649
2024-11-29 14:11:29 -06:00
Steven R. Loomis
add502e1a9 feat(common,developer): tests and warning on denormalized content
- common: shift the 'strs' processing slightly, because otherwise we normalize to NFD before even tracking the strings
- if any string is neither NFC nor NFD, give a warning
- add tests for the same

Fixes: #7394
2024-11-29 12:16:11 -06:00
Eberhard Beilharz
66404f9503
chore(developer): rename test files
This change rename the test files for Developer according to the
discussion at the Keyman conference in November 2024.
2024-11-28 14:32:09 +01:00
Eberhard Beilharz
badb23a6ab
Merge pull request #12710 from keymanapp/refactor/developer/tc_tests
refactor(developer): output number of tests when running on TC
2024-11-28 19:40:52 +07:00
Marc Durdin
70a32f00d7 refactor(developer): unify test action 2024-11-28 15:16:25 +07:00
Eberhard Beilharz
21736d8817
refactor(developer): output number of tests when running on TC
This change adds the mocha-teamcity-reporter which outputs the running
tests in a special format that TeamCity can interpret. This allows TC
to show which tests run as well as the number of tests run.

Also refactor some of the build scripts to use functions.

Note that we had to explicitly add `--check-coverage=false` (which is the
default) to get successful builds when we run on TC.
2024-11-27 11:41:06 +01:00
Marc Durdin
b54b005830 refactor(common): move remaining LDML keyboard types into LdmlKeyboardTypes
Follows: #12712
2024-11-27 11:52:53 +07:00
Steven R. Loomis
774eba727e chore(developer): fix typo in test comment
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-11-07 09:37:54 -06:00
Steven R. Loomis
3eac4f12c9 fix(common): check for invalid markers
- pattern parser update
- correct spelling of Error_InvalidVariableIdentifier
- track and report bad marker strings (look like markers but not)
- fix: may have missed markers mentioned in variables
- tests for above

Fixes: 12467
2024-11-04 10:15:56 -06:00
Steven R. Loomis
c057112c9f fix(developer): add a test for empty varable id, fix message
- stray \u in Error_InvalidVariableIdentifer
- add a test for an empty variable id
2024-11-01 13:47:39 -05:00