Commit graph

4688 commits

Author SHA1 Message Date
Marc Durdin
0f05da957f fix(developer): remove redundant Name and RTL fields from .kps LexicalModel
Follows on from the similar changes to `Keyboard` fields in #13600. This
deprecates the `RTL` and `Name` fields. `RTL` was never used. `Name` was
written but never presented to end users, so effectively irrelevant.
`Name` also was present in `File.Description` (already deprecated) for
the corresponding model.js _and_ `Info.Name`, so doubly redundant.

Also updates the schema documentation to mark the deprecated fields.

Relates-to: #13600
2025-04-02 05:39:09 +07:00
Sabine
c39eeff7c6 feat(developer): use fs and path from callbacks 2025-04-01 12:30:14 +02:00
Steven R. Loomis
29f387bebe Merge remote-tracking branch 'upstream/master' into feat/developer/10622-xml-parse-line-numbers-epic-ldml 2025-03-31 16:16:10 -05:00
Steven R. Loomis
0c290bd37a feat(developer): update for line numbers
- update to metadata branch of fast-xml-parser
- restructure metadata handling
- use type safety for fast-xml-parser options

Fixes: #10622
2025-03-31 16:11:19 -05:00
SabineSIL
b89d55e60a
Merge branch 'epic/kmc-convert' into feat/developer/kmc-convert 2025-03-31 18:51:35 +02:00
Sabine
e3e8874091 feat(developer): rename test files according to new standard 2025-03-31 17:50:10 +02:00
Sabine
0fe56a4d88 feat(developer): change use of Backslash as Path Separator; remove all markers 2025-03-31 10:07:05 +02:00
Marc Durdin
9c1c417457 fix(developer): add special handling for 'und' in kmc-keyboard-info
This is a workaround for missing 'und' language tag.

`(new Intl.Locale('und')).language` returns `undefined` in V8, as of 31
March 2025. This means we cannot rely on `Intl.Locale` to parse the
bcp47 string for us. The implemented workaround is to replace `und` with
a known-good language subtag, and then swap it back out in later
processing.

Also adds unit test and new warning messages for invalid language tags
that were exposed during testing of this.

Fixes: #13610
2025-03-31 12:30:05 +07:00
Marc Durdin
4eb03c862e
Merge branch 'epic/kmc-convert' into chore/merge-master-into-kmc-convert 2025-03-31 12:13:38 +07:00
Marc Durdin
ab59fa29b4
Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:41:03 +07:00
Marc Durdin
a52437c407 chore: Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:38:08 +07:00
Marc Durdin
f9739e5e9b chore(developer): remove unreferenced kps fields from test 2025-03-29 04:21:41 +07:00
Marc Durdin
de5a02f8ce chore(developer): rewrap detail on semver LDML message 2025-03-29 04:14:38 +07:00
Sabine
91153a3eaa feat(developer): switch to write/not write rules with associated warnings 2025-03-28 20:12:00 +01:00
Sabine
f24ee130c9 feat(developer): add warning for superior rules 2025-03-28 17:47:50 +01:00
Marc Durdin
dbfe03aa8e chore(developer): improve kmc-ldml 'invalid version' message
Suggested by @Nnyny. Give more detail on the semantic version format for
keyboard version, as well as details on the limitations to semver
imposed by the Keyman toolchain.
2025-03-28 15:01:11 +07:00
Marc Durdin
86d457837f chore(developer): remove unreferenced fields from kmc-generate test fixtures
Relates-to: #13576
2025-03-28 14:59:07 +07:00
Marc Durdin
52d016fe7c chore(developer): remove redundant fields from kps reader unit test
Note: the round-trip test still passes because the XML data passes
basically unmodified through, even though we make no reference to the
fields in the code.

Fixes: #13576
2025-03-28 14:27:08 +07:00
Marc Durdin
9dabcc0306 fix(developer): remove redundant keyboard fields from .kps
The fields 'name', 'version', and 'rtl' were defined in the .kps file,
but were not actually used by the compiler, as kmc-package reads the
metadata from the keyboard file rather than relying on this data, which
often is out of date. This led to confusion for keyboard authors, so
this change removes that metadata from the package.

The one visible side-effect (apart from the data disappearing from
future .kps files) is that the package editor no longer shows this
metadata in the Keyboards view (but as it may be out-of-date, this is
probably a good thing).

Fixes: #13576
Fixes: #13593
2025-03-28 14:07:23 +07:00
Sabine
c25e3d2ffb Merge branch 'feat/developer/kmc-convert' of https://github.com/keymanapp/keyman into feat/developer/kmc-convert 2025-03-28 07:40:04 +01:00
Sabine
8cd457dbce feat(developer): add/remove input files 2025-03-28 07:37:55 +01:00
Steven R. Loomis
8aa9219410 Merge remote-tracking branch 'upstream/master' into feat/developer/10622-xml-parse-line-numbers-epic-ldml 2025-03-27 16:02:45 -05:00
SabineSIL
734c50b18f
Delete developer/src/kmc-convert/data/MySample.keylayout
delete unused input file
2025-03-27 18:38:13 +01:00
SabineSIL
85047d6f77
Delete developer/src/kmc-convert/data/My_dk_Keyboard.keylayout
delete unused input files
2025-03-27 18:37:14 +01:00
SabineSIL
caf11a00e0
Merge branch 'epic/kmc-convert' into feat/developer/kmc-convert 2025-03-27 18:27:44 +01:00
Sabine
c9570b5011 feat(developer): order of modifiers, final check prevent unwritten rule in rare cases 2025-03-27 18:25:14 +01:00
Sabine
7315f6cb35 feat(developer): enable use without specified output file name 2025-03-26 15:53:35 +01:00
Sabine
56c21bd897 feat(developer): add several .keylayout files to git; rearrange methods 2025-03-26 14:21:37 +01:00
Steven R. Loomis
4f22ab0d9e
Merge pull request #13516 from keymanapp/chore/developer/ldmlcov
chore(developer): improve ldml code coverage 🙀
2025-03-24 17:16:34 -05:00
Steven R. Loomis
18e7686b3e chore(developer): fix build message per review comment
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-24 12:29:32 -05:00
Sabine
40e3beca29 feat(developer): split createRuleData to createRuleData and reviewRuleInputData | function comments 2025-03-24 17:24:02 +01:00
Sabine
647fb0afcf feat(developer): remove markers, checkIfCapsIsUsed for upper/lower/mixed case case 2025-03-24 15:36:08 +01:00
Sabine
6505f095e6 Merge branch 'feat/developer/kmc-convert' of https://github.com/keymanapp/keyman into feat/developer/kmc-convert 2025-03-24 10:57:58 +01:00
Marc Durdin
86102c405e
Merge pull request #13191 from keymanapp/dependabot/npm_and_yarn/esbuild-0.25.0
chore(deps-dev): bump esbuild from 0.18.20 to 0.25.0
2025-03-24 11:22:00 +07:00
Marc Durdin
9080d156dc chore(developer): use 'statement' instead of 'command' for messages about kmn statements
In compiler messages, for a long time we have inconsistently used
'command' instead of 'statement' when referring to kmn statements. The
kmn language documentation uses 'statement' fairly consistently.
2025-03-22 05:58:06 +07:00
Steven R. Loomis
692ea6ee9e feat(core): improve kmxplus validation tests
- update basic.txt to include the kvk section #12402
- use run() instead of compile()
- move compile()+serialize to a separate subtest
2025-03-21 17:11:03 -05:00
Sabine
b36b0f6dc6 feat(developer): add test for converter 2025-03-21 16:42:35 +01:00
Sabine
cdada430ee feat(developer): review, tidy up 2025-03-20 18:01:25 +01:00
Marc Durdin
bc4462a312
Merge pull request #13532 from keymanapp/fix/developer/13455-ERROR_TextBeforeOrAfterNulInOutput
fix(developer): add validation of `nul` usage in output part of rule
2025-03-19 06:12:54 +07:00
Steven R. Loomis
4e3379c698 feat(core): improve kmxplus validation tests
- ratchet required coverage level to 90%!
- comment out failing run() test though - run() adds an embedded .kvk

Fixes: #9446
2025-03-18 11:50:28 -05:00
Sabine
169cad76b1 feat(developer): print codepoints instead of numeric character ref. and check for control characters 2025-03-18 14:17:36 +01:00
Marc Durdin
fdd219a481 chore(developer): remove mozhi_malayalam.kmx from baseline fixtures
This is required because it includes rules with nul and other output.
2025-03-18 12:16:40 +07:00
Marc Durdin
5683b370fe fix(developer): add validation of nul usage in output part of rule
If `nul` is used in the output part of a rule, then text or
text-emitting statements cannot also be included in the output of that
rule. This was not enforced in earlier versions of Keyman.

Adds a unit test for the enhanced validation.

Fixes: #13455
Relates-to: keymanapp/keyboards#3379
2025-03-18 09:54:06 +07:00
Sabine
454b0359cb feat(developer): add semicolon where needed; function names 2025-03-17 16:25:32 +01:00
Sabine
794db98409 feat(developer): comment out first occurance+write out later occurance of a rule
if a rule is ambiguous i.e. if it is a C0/C1 rule and has an abiguous C2 or C3 rule we comment out the C0/C1 rule and only use the C2 or C3 rule
c WARNING  + [CAPS K_EQUAL]  > 'ì'  (C0/C1)
+ [CAPS K_EQUAL]  >  dk(A3)				  (C2 or C3)
2025-03-17 11:04:13 +01:00
Marc Durdin
d4540e3c68 chore(developer): use caps for constants in xml-utils 2025-03-17 12:19:10 +07:00
Marc Durdin
7d5c98ddba fix(developer): bump fast-xml-parser to 5.0.9
This is a major dependency version bump (4.5.0 to 5.0.9) which we would
normally avoid during beta. However, we need the fix
NaturalIntelligence/fast-xml-parser#725 for hex escapes in XML, which is
incorporated in 5.0.9.

I have assessed the other changes to fast-xml-parser and found no
breaking changes for us, but I tightened the types in xml-utils
declarations and found some minor inconsistencies which appear to have
no impact, and which I have corrected:
* wrong type in unused parameter to `tagValueProcessor`
* reference to unused property `options.emptyTag`

While doing this, I consolidated the common options for the parser in
order to verify consistency, but made no changes to the resolved
parsing/building options.

Added a test to ldml keyboard reading, to verify that numeric strings
are treated as strings, given the divergence in the `numberParseOptions`
option, and it shows that numeric strings are treated as strings.

Fixes: #13348
2025-03-17 06:22:29 +07:00
Marc Durdin
96a96bef15 fix(developer): add units tests to verify non-BMP numeric entities in XML reader
fast-xml-reader has a bug with numeric entities. See:
  https://github.com/NaturalIntelligence/fast-xml-parser/issues/725

This commit adds a unit test to verify that non-BMP numeric entities
will be parsed correctly. It will fail until we update the
fast-xml-parser dependency.

Relates-to: #13348
2025-03-17 05:50:47 +07:00
Marc Durdin
7b33aed94d chore(developer): reduce kmc coverage to 45%
This was due to platform-specific checks in NodeCompilerCallbacks,
around filename case and drive letters in paths (search terms:
HINT_FilenameHasDifferingCase, verifyFilenameConsistency):

- Linux: 59.01%
- macOS: 60.46%
- Windows: 63.08%

Mitigation: reduce threshold to 45%. More tests will be needed for
kmc-convert before epic merge.
2025-03-15 15:09:18 +07:00
Steven R. Loomis
7487a6bbbe chore(developer): improve code coverage
- add a test for LdmlKeyboardCompiler.run()
- exclude generated abnf files from coverage
2025-03-14 17:20:23 -05:00