Commit graph

196 commits

Author SHA1 Message Date
Steven R. Loomis
0c290bd37a feat(developer): update for line numbers
- update to metadata branch of fast-xml-parser
- restructure metadata handling
- use type safety for fast-xml-parser options

Fixes: #10622
2025-03-31 16:11:19 -05:00
Marc Durdin
a52437c407 chore: Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:38:08 +07:00
Marc Durdin
52d016fe7c chore(developer): remove redundant fields from kps reader unit test
Note: the round-trip test still passes because the XML data passes
basically unmodified through, even though we make no reference to the
fields in the code.

Fixes: #13576
2025-03-28 14:27:08 +07:00
Marc Durdin
9dabcc0306 fix(developer): remove redundant keyboard fields from .kps
The fields 'name', 'version', and 'rtl' were defined in the .kps file,
but were not actually used by the compiler, as kmc-package reads the
metadata from the keyboard file rather than relying on this data, which
often is out of date. This led to confusion for keyboard authors, so
this change removes that metadata from the package.

The one visible side-effect (apart from the data disappearing from
future .kps files) is that the package editor no longer shows this
metadata in the Keyboards view (but as it may be out-of-date, this is
probably a good thing).

Fixes: #13576
Fixes: #13593
2025-03-28 14:07:23 +07:00
Steven R. Loomis
8aa9219410 Merge remote-tracking branch 'upstream/master' into feat/developer/10622-xml-parse-line-numbers-epic-ldml 2025-03-27 16:02:45 -05:00
Marc Durdin
d4540e3c68 chore(developer): use caps for constants in xml-utils 2025-03-17 12:19:10 +07:00
Marc Durdin
7d5c98ddba fix(developer): bump fast-xml-parser to 5.0.9
This is a major dependency version bump (4.5.0 to 5.0.9) which we would
normally avoid during beta. However, we need the fix
NaturalIntelligence/fast-xml-parser#725 for hex escapes in XML, which is
incorporated in 5.0.9.

I have assessed the other changes to fast-xml-parser and found no
breaking changes for us, but I tightened the types in xml-utils
declarations and found some minor inconsistencies which appear to have
no impact, and which I have corrected:
* wrong type in unused parameter to `tagValueProcessor`
* reference to unused property `options.emptyTag`

While doing this, I consolidated the common options for the parser in
order to verify consistency, but made no changes to the resolved
parsing/building options.

Added a test to ldml keyboard reading, to verify that numeric strings
are treated as strings, given the divergence in the `numberParseOptions`
option, and it shows that numeric strings are treated as strings.

Fixes: #13348
2025-03-17 06:22:29 +07:00
Marc Durdin
96a96bef15 fix(developer): add units tests to verify non-BMP numeric entities in XML reader
fast-xml-reader has a bug with numeric entities. See:
  https://github.com/NaturalIntelligence/fast-xml-parser/issues/725

This commit adds a unit test to verify that non-BMP numeric entities
will be parsed correctly. It will fail until we update the
fast-xml-parser dependency.

Relates-to: #13348
2025-03-17 05:50:47 +07:00
Marc Durdin
1a0b830c0b
Merge branch 'master' into chore/b18s3-merge-beta-to-master 2025-03-14 08:54:50 +07:00
Marc Durdin
d2b457b2e4 fix(developer): load Keyman Developer options in Server before use
Fixes: #13420
2025-03-10 06:28:39 +07:00
Marc Durdin
060189a1ab chore(developer): drop coverage threshold for developer-utils to 45% 2025-03-05 00:24:41 +03:00
Steven R. Loomis
0ba005a55b feat(developer): line number symbols
- improve and simplify lookup algorithm
- add test for override filename on imports

Fixes: #10622
2025-03-04 09:34:47 -06:00
Steven R. Loomis
92e16b90a0 feat(developer): line number symbols
- move file read bottleneck out of compiler.ts into the LDML XML reader
- move the eventresolver into the LDML XML reader
- add XML_FILENAME_SYMBOL - for tagging when an object is from a different file than the default
(such as with an import)

Fixes: #10622
2025-03-03 19:12:34 -06:00
Steven R. Loomis
b015221814 feat(developer): line number symbols
- change special symbol to XML_START_INDEX_SYMBOL
- use an array of line lengths rather than an array of strings for the file info
- add an EventResolver interface to the callbacks, such that it has an opportunity to resolve events

Fixes: #10622
2025-03-03 17:08:40 -06:00
Steven R. Loomis
1febf9a0de
Update developer/src/common/web/utils/src/xml-utils.ts
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-03 13:18:02 -06:00
Steven R. Loomis
d7cb7b63a3 feat(developer): add an EventResolver
- it's a hook to the hook to allow the LDML compiler to resolve error message line numbers
- TODO still not extra performant, but maybe the right separation of concerns

Fixes: #10622
2025-03-03 13:14:41 -06:00
Steven R. Loomis
3e73a3f081 feat(developer): add CompilerEvent.offset
- per review, keep this as a separate field.

Fixes: #10622
2025-03-03 12:36:42 -06:00
Steven R. Loomis
4d9ea2db2d feat(developer): work around line number symbols
- add KeymanXMLReader.removeSymbols() and call it from tests

Fixes: #10622
2025-02-28 14:24:12 -06:00
Steven R. Loomis
819d9bf562 feat(developer): API for resolving line numbers for messages
- change to use a different fast-xml-parser PR, https://github.com/NaturalIntelligence/fast-xml-parser/pull/729

Fixes: #10622
2025-02-28 12:08:31 -06:00
Steven R. Loomis
c4ca359d18 feat(developer): API for resolving line numbers for messages
- add 'column' to CompilerEvent
- Add a LdmlCompilerMessages.col() function to set the column number
- add a LdmlCompilerMessages.resolveLineNumber() to map offsets to line number

Fixes: #10622
2025-02-28 09:27:25 -06:00
Steven R. Loomis
b60c64c3d3 feat(developer): test for line number extension to xml-utils
Fixes: #10622
2025-02-27 16:42:41 -06:00
Steven R. Loomis
ec68e74bde feat(developer): add experimental char-offset data
Depends on PR https://github.com/NaturalIntelligence/fast-xml-parser/pull/728 so uses srl repository "temporarily"

Fixes: #10622
2025-02-27 16:24:21 -06:00
Steven R. Loomis
b59bb6d4de feat(developer): serialize KMXPlus into XML 🍒
Cherry pick of #12969
> feat(developer): serialize KMXPlus (back) into XML 🗼

(cherry picked from commit f93c45c9e8 #12969)

Fixes: #12874
2025-02-10 12:12:12 -06:00
Marc Durdin
bd4633333b feat(developer): improve compiler messages and user interface
* Add unit test to verify the format of message detail strings
  (max line length of 80 chars)
* Add unit test to verify non-null, non-empty message strings
* Tidy up a number of messages (to pass unit tests)
* Add message details for all kmw compiler messages
* Add a clickable link for the TIKE message window for each message
  so that explanations and further message details are more accessible
  to authors (`kmc message` is already available for command-line users)
* Fixes a number of minor issues with the messages window in TIKE,
  mostly relating to when popup menu commands are enabled and how
  messages are selected.

Fixes: #13134
2025-02-07 08:46:46 +07:00
Marc Durdin
2df63dddae
Merge pull request #13075 from keymanapp/fix/developer/13072-make-kmc-log-options-consistent
fix(developer): make kmc log options consistent across all commands
2025-02-05 11:30:10 +07:00
Marc Durdin
9db0d086dc
Merge pull request #13113 from keymanapp/feat/developer/kmc-include-command-line-in-sentry-reports
feat(developer): include command line in kmc sentry reports
2025-02-04 11:43:12 +07:00
Marc Durdin
05c17e3439 feat(developer): include command line in kmc sentry reports
In order to reproduce kmc errors, it's very helpful to know how it was
instantiated. This change includes the command line call for kmc. This
includes things such as a keyboard filename and may include file paths,
but does not include private personal information or secrets.
2025-02-03 09:53:45 +07:00
Steven R. Loomis
622900a5d8 chore(developer): LDML update issue
- test needed to be updated

Fixes: #11307
2025-01-30 23:42:10 -06:00
Steven R. Loomis
b54ea40770 chore(developer): update comments per review
- note that 45,46,47 are the same data and so aliased

Fixes: #11307
2025-01-30 23:13:36 -06:00
Steven R. Loomis
b35ba06a2b chore(developer): remove workaround for base=""
Fixes: #12749
2025-01-30 15:48:02 -06:00
Steven R. Loomis
b741a88ca0 chore(core,developer): treat CLDR v45, 46, 46.1, 47 as 46
Fixes: #11307
2025-01-30 15:42:30 -06:00
Steven R. Loomis
2c32fb7903 chore(resources, developer,core): CLDR 46 and workaround
Fixes: #11307
- import from CLDR 46, mostly unchanged

Fixes: #12749
- workaround LDML DTD bug CLDR-18138 where base= does not imply base=cldr
2025-01-30 15:41:34 -06:00
Marc Durdin
d2aa22ff75 fix(developer): make kmc log options consistent across all commands
Makes the `--log-format`, `--log-level`, and `--color`/`--no-color`
options available for all commands and DRYs out some of the options
processing for consistency. This has positive impact in particular on
the Generator classes which removes the need for several unit tests as
the interface can now be checked at compile time.

Fixes: #13072
Unblocks: #13073
2025-01-30 09:20:29 +07:00
Marc Durdin
a08f1aca6e feat(developer): add 'verbose' and 'debug' log levels to kmc
Adds 'verbose' and 'debug' log levels and corresponding error severity
levels. This means the internal error severity values have changed to
match - 'info' is now 2 instead of 0.

Fixes: #12975
2025-01-22 08:58:13 +07:00
Marc Durdin
3ad0dec493 chore(developer): validate keyboard and model IDs in kmc-generate
Fixes: #12957
2025-01-21 12:41:52 +07:00
Marc Durdin
45a6291a3e refactor(developer): move ttfInfo to developer-utils
We will need ttfInfo in kmc-package, so move it to the shared module.
2025-01-20 09:19:47 +07:00
Marc Durdin
dad1df3194 fix(developer): ensure license parameter is required and add tests
Some additional items arose from this:

* Adds a command-line-tests unit test which does some basic sanity tests
  on `kmc`.
* Cleans up some of the error reporting infrastructure in `kmc` to
  ensure that thrown errors are handled better.
* Removes redundant subshell from `builder_run_action`.

Fixes: #12846
2025-01-13 15:10:31 +07:00
Marc Durdin
7be5c62e3b
Merge pull request #12773 from keymanapp/chore/developer/12746-kmc-copy-full-gh-url-to-master
change(developer): use full github url in kmc copy parameters 🍒
2024-12-06 08:01:34 +01:00
Marc Durdin
72d0bac0ee
Merge pull request #12775 from keymanapp/chore/rename-test-callbacks
chore: rename TestCompilerCallbacks.ts
2024-12-05 23:51:40 +01:00
Marc Durdin
389ef7f001 chore: rename TestCompilerCallbacks.ts
There was confusion with two implementations of TestCompilerCallbacks.
Turns out one of them wasn't used.
2024-12-05 13:04:27 +07:00
Marc Durdin
170282bba7 chore: address review comments 2024-12-05 09:23:27 +07:00
Marc Durdin
33b4fa3e89 change(developer): use full github url in kmc copy parameters
Fixes: #12746
Cherry-pick-of: #12754
2024-12-05 09:23:08 +07:00
Steven R. Loomis
9a338abf91 chore(developer): update per code review
- support empty string as import base
2024-12-04 08:34:36 -06:00
Steven R. Loomis
f4f0355109 feat(developer): local imports directory fixes
- use path.join instead of path.resolve
- rename importsPath to cldrImportsPath
- remove k_015_importlocal from core, as it duplicated tests in common and developer

Fixes: #10649
2024-12-02 15:40:54 -06:00
Steven R. Loomis
48da8fd613
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-12-02 11:55:13 -06:00
Steven R. Loomis
75d172a6cb
Merge branch 'feat/developer/7394-verify-normalization-epic-ldml' into feat/developer/10649-local-imports 2024-12-02 11:37:29 -06:00
Marc Durdin
cd8825d504
Merge pull request #12736 from keymanapp/refactor/developer/unify_test_action
refactor(developer): unify test action
2024-12-01 00:14:11 +01:00
Steven R. Loomis
01a47c3ea9 feat(developer): support local imports
- added a new reader callback option, localImportsPaths
- due to the CLDR issue #12749 use base=""
- add tests
- some bugfixes in import messages
- add an ImportStatus section to determine if something is a local import

Fixes: #10649
2024-11-29 13:48:21 -06:00
Eberhard Beilharz
66404f9503
chore(developer): rename test files
This change rename the test files for Developer according to the
discussion at the Keyman conference in November 2024.
2024-11-28 14:32:09 +01:00
Eberhard Beilharz
badb23a6ab
Merge pull request #12710 from keymanapp/refactor/developer/tc_tests
refactor(developer): output number of tests when running on TC
2024-11-28 19:40:52 +07:00