Commit graph

299 commits

Author SHA1 Message Date
Steven R. Loomis
48da8fd613
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-12-02 11:55:13 -06:00
Steven R. Loomis
75d172a6cb
Merge branch 'feat/developer/7394-verify-normalization-epic-ldml' into feat/developer/10649-local-imports 2024-12-02 11:37:29 -06:00
Marc Durdin
cd8825d504
Merge pull request #12736 from keymanapp/refactor/developer/unify_test_action
refactor(developer): unify test action
2024-12-01 00:14:11 +01:00
Steven R. Loomis
01a47c3ea9 feat(developer): support local imports
- added a new reader callback option, localImportsPaths
- due to the CLDR issue #12749 use base=""
- add tests
- some bugfixes in import messages
- add an ImportStatus section to determine if something is a local import

Fixes: #10649
2024-11-29 13:48:21 -06:00
Eberhard Beilharz
66404f9503
chore(developer): rename test files
This change rename the test files for Developer according to the
discussion at the Keyman conference in November 2024.
2024-11-28 14:32:09 +01:00
Eberhard Beilharz
badb23a6ab
Merge pull request #12710 from keymanapp/refactor/developer/tc_tests
refactor(developer): output number of tests when running on TC
2024-11-28 19:40:52 +07:00
Marc Durdin
70a32f00d7 refactor(developer): unify test action 2024-11-28 15:16:25 +07:00
Eberhard Beilharz
21736d8817
refactor(developer): output number of tests when running on TC
This change adds the mocha-teamcity-reporter which outputs the running
tests in a special format that TeamCity can interpret. This allows TC
to show which tests run as well as the number of tests run.

Also refactor some of the build scripts to use functions.

Note that we had to explicitly add `--check-coverage=false` (which is the
default) to get successful builds when we run on TC.
2024-11-27 11:41:06 +01:00
Marc Durdin
e388361e84
Merge pull request #12713 from keymanapp/refactor/common/LdmlKeyboardTypes
refactor(common): move remaining LDML keyboard types into `LdmlKeyboardTypes`
2024-11-27 07:21:06 +01:00
Marc Durdin
b54b005830 refactor(common): move remaining LDML keyboard types into LdmlKeyboardTypes
Follows: #12712
2024-11-27 11:52:53 +07:00
Dr Mark C. Sinclair
f51bb9602b chore(developer): removed ERROR_XXXInVirtualKeySection from Compiler.cpp and kmn_compiler_errors.h 2024-11-26 03:46:58 +00:00
Marc Durdin
ce7593bd2f fix(developer): box package compiler info fields
If a .kps file had an `<Info>` element without a URL attribute, it
would not map correctly in all circumstances. This change ensures that
the kps-file-reader is responsible for normalizing the layout of these
elements on read.
2024-11-13 10:11:44 +07:00
Marc Durdin
5a93bdf6c2 chore(developer): make package subfile description fully optional
The package subfile description field is never really used in the
end-user apps. As it is meaningless metadata, we'll make it fully
optional, and start to remove all references in the designers and
compilers.

TODO: For packages targeting Keyman 18.0+, we should consider removing
the description field altogether from the subfiles.
2024-11-13 10:03:26 +07:00
Marc Durdin
d68a64252e feat(developer): add GitHub and Cloud support to kmc-copy
This requires a significant refactor to the compiler interfaces to allow
for asynchronous filesystem and network access. For now, only kmc-copy
uses the asynchronous interfaces, but over time we may try and
consolidate them where other compiler modules find them useful. This
adds a big set of unit test fixtures to avoid network instability in
tests (the fixtures can be regenerated by setting environment variable
`TEST_SAVE_FIXTURES`).
2024-10-28 10:33:42 +07:00
Marc Durdin
40db1c2051 feat(common): add isDirectory, helper functions for kmc-copy 2024-10-22 10:28:03 +07:00
Marc Durdin
1e084e99f0 refactor(developer): add kps-file-reader and kps-file-writer
Relates to kmc-copy project.

Moves the .kps file reader out of kmc-package and adds corresponding
kps file writer unit in developer-utils. Also moves related test and
error message definition.
2024-10-22 10:28:01 +07:00
Marc Durdin
521a9b24c8 feat(developer): add .kpj file writer 2024-10-22 10:28:00 +07:00
Marc Durdin
7d17913e18 feat(developer): seed kmc-copy 2024-10-22 10:28:00 +07:00
Marc Durdin
89e58fbe00
Merge branch 'master' into feat/developer/kmc-generate 2024-10-12 11:48:31 +02:00
Marc Durdin
7dc787b96b fix(developer): use TextDecoder to convert Uint8Array to string
`toString()` only works when the Uint8Array is actually a Node Buffer.
2024-10-11 14:11:45 +07:00
Marc Durdin
0d9688451d
Merge branch 'master' into feat/developer/kmc-generate 2024-10-11 03:26:54 +02:00
Marc Durdin
473fea9be0
Merge pull request #12464 from keymanapp/fix/developer/12454-use-richedit-for-debugger-memo
fix(developer): use richedit in debug memo to support Egyptian cartouches
2024-10-09 09:49:21 +02:00
Marc Durdin
e208d56419 fix(developer): use RichEdit 4.1 for debugger memos
Adds support for using RichEdit 4.1, and sets the richedit control into
the correct mode for rendering arbitrary Unicode correctly. Removes
some unnecessary per-character refresh for improved performance.

Fixes: #12454
2024-10-08 14:21:22 +07:00
Steven R. Loomis
9a4c7e5820 feat(developer): fast-xml-parser: typo fix in xml-utils.ts
Fixes: #12208
2024-10-07 13:05:50 -05:00
Steven R. Loomis
03a24a1506 feat(developer): fast-xml-parser: fix for ldml empty attributes
- because of the form to="" in ldml, we need to distinguish attributes and sub-elements in the ldml xml parsing
- use an attributePrefix, and fixup the object tree afterwards

Fixes: #12208
2024-10-03 12:41:08 -05:00
Steven R. Loomis
3e168aafb2 chore(developer,common): deps: xml2js / fast-xml-parser
- remove vendored xml2js
- add fast-xml-parser@4.5.0
- Rewire the KeymanXMLReader / KeymanXMLWriter to use fast-xml-parser

Four small changes to the test data on the generated .kvks:
- allow &quot; for XML generation (" is legal)
- use the same XML prologue as the actual .kvks files.
- treat <key></key> as <key/> since we can't distinguish
  (removed a "_": "" entry)
- remove whitespace because we now can ("_": "\n  \n \n") on non-leaf nodes.
  This is ignored by the kvks code, but was difficult to maintain roundtrip.

Fixes: #12208
2024-10-03 08:34:24 -05:00
Steven R. Loomis
158c4a7637 feat(developer): update the unified xml parser per review
- declarative syntax for options
- workaround an issue where xml2js is mutating our options objects
- improve tests

Fixes: #12208
2024-09-30 13:25:38 -05:00
Steven R. Loomis
eba7079694 chore(developer): update per style guide
Fixes: #12208
2024-09-30 10:00:46 -05:00
Steven R. Loomis
1780b1ebf3 feat(developer,common): use unified xml writer
Fixes: #12208
2024-09-27 18:53:54 -05:00
Steven R. Loomis
d540e4cda0 feat(common): unified xml writer test
Fixes: #12208
2024-09-27 18:37:40 -05:00
Steven R. Loomis
3832ca1c12 feat(developer,common): use unified xml parser
Subsystems changed:
- ldml keyboard reader (main and test)
- kpj
- kvks
- kmp compiler

test: made the test-xml-utils less verbose about the pathnames

Fixes: #12208
2024-09-27 17:49:32 -05:00
Steven R. Loomis
7da7375ee5 feat(common): xml: map \r\n to \n
- help make the parsed form identical across platforms

Fixes: #12208
2024-09-27 17:31:00 -05:00
Steven R. Loomis
609faf8047 feat(common): unified xml parser reader test
Fixes: #12208
2024-09-27 16:52:21 -05:00
Steven R. Loomis
7e631c1a73 feat(common): unified xml parser scaffolding
Fixes: #12208
2024-09-27 14:09:52 -05:00
Marc Durdin
10e4bba100
Merge branch 'master' into feat/developer/kmc-generate 2024-09-26 18:55:20 +02:00
Marc Durdin
25ea6a74dc fix(developer): use richedit in debug memo to support Egyptian cartouches
RichEdit allows text selection one character past end-of-string, so we
need to cater for that as well in passing text ranges to the character
grid.

Fixes: #12454
2024-09-24 08:24:47 -07:00
Marc Durdin
e4017a8c28 fix(developer): rewrite ldml visual keyboard compiler
The visual keyboard compiler was never finished in 17.0. This rewrites
it to:

1. Use the kmxplus data rather than reading from xml directly
2. Fill in `visualkeyboard.header.kbdname`
3. Support modifiers
4. Handle encoded characters like `\u{1234}`
5. Handle string variables like `${one}`*

Additional unit tests have been added to verify the behavior of the
visual keyboard compiler in more detail.

TODO-LDML: string variables appear to have a secondary bug -- they seem
to be returning the string 'undefined'. I have disabled the related
tests and will examine this separately, and enable those tests once
fixed.

TODO-LDML: we should probably add a compiler warning + unit test for
`<layers formId="us"><layer id="base">`, because this pattern does not
make sense: when using non-touch forms, the `<layer>` element should use
`modifiers` attribute, and correspondingly, `modifiers` attribute should
_not_ be used when `formId` is `touch`.

Other fixes:

1. The LDML XML reader was relying on its input being a Node.js `Buffer`
   even though it was declared `Uint8Array`, as it implicitly used
   `Buffer.toString()` to do text conversion. (`Buffer` subclasses from
   `Uint8Array`). This breaks when using `Uint8Array` directly and means
   we had an implicit dependency on Node.js. See also #12331.
2. XML errors were not captured in the LDML XML reader. See also #12331.
3. The unused and unfinished touch-layout-compiler.ts and
   keymanweb-compiler.ts have been removed along with corresponding unit
   tests and fixtures. These are replaced by Core implementations; see
   #12291.

Fixes: #12395
2024-09-12 06:06:00 +07:00
Marc Durdin
113b4f7ac8 feat(developer): comments for kmc-generate 2024-09-03 06:20:34 +07:00
Marc Durdin
8274d30788 feat(developer): kmc generate cleanup
Cleanup various todo items for kmc generate. Outstanding TODO item is
icon generation for .kmn.
2024-09-03 06:20:28 +07:00
Marc Durdin
2994fe2aea feat(developer): kmc-generate docs and tests started 2024-09-03 06:17:33 +07:00
Marc Durdin
939134a9a3 feat(developer): kmc-generate module
Plenty of todo items left but this is a mostly-functional module for the
kmc generate command for automatically creating keyboard and model
projects.
2024-09-03 06:15:33 +07:00
Marc Durdin
f5066174cd fix(developer): make LDML import path consistent for all bundlings of kmc
The default import path is relative to ldml-keyboard-xml-reader.js. The
bundling of kmc with Keyman Developer reduces the directory tree depth,
so we move the import path to be a subdirectory of the immediate parent
directory, rather than two levels up, so that
`C:\Program Files\Keyman\Keyman Developer\kmc` can refer to
`C:\Program Files\Keyman\Keyman Developer\import` when kmc is bundled.

Fixes: #12279
2024-08-24 10:28:48 +08:00
dependabot[bot]
6fad148466
chore(deps): bump semver from 7.5.4 to 7.6.0
Bumps [semver](https://github.com/npm/node-semver) from 7.5.4 to 7.6.0.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v7.5.4...v7.6.0)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-14 11:22:58 +00:00
Marc Durdin
062ae33579 refactor(developer): add eventemitter3 and ldml-keyboard-constants deps to Server
Fixes: #12077
2024-08-12 17:23:48 +07:00
Marc Durdin
2814ac995c chore(developer): include common-types dep in developer-utils 2024-08-12 06:08:56 +07:00
Marc Durdin
945da3d8d4
Merge pull request #12113 from keymanapp/chore/common/add-support-for-build-target-platform-exclusions
chore(common): add support for build target platform exclusions
2024-08-09 16:57:56 +10:00
Marc Durdin
d55558fb31
Merge pull request #12101 from keymanapp/refactor/common/9665-move-xml2js-and-related-deps-to-developer
refactor(common): move xml2js and related deps to @keymanapp/developer-utils 🐉
2024-08-07 15:16:07 +10:00
Marc Durdin
32b4711659
Merge branch 'master' into refactor/common/9665-move-compiler-interfaces-to-developer 2024-08-07 15:15:45 +10:00
Marc Durdin
ed143fb333
Merge branch 'master' into refactor/common/9665-move-ldml-keyboard-source-file-to-developer 2024-08-07 15:15:04 +10:00
Marc Durdin
ec72245575
Merge pull request #11765 from keymanapp/refactor/common/9665-move-keyman-touch-layout-file-to-developer
refactor(common): move .keyman-touch-layout reader/writer to @keymanapp/developer-utils 🐉
2024-08-07 15:14:06 +10:00