Commit graph

196 commits

Author SHA1 Message Date
Marc Durdin
70a32f00d7 refactor(developer): unify test action 2024-11-28 15:16:25 +07:00
Eberhard Beilharz
21736d8817
refactor(developer): output number of tests when running on TC
This change adds the mocha-teamcity-reporter which outputs the running
tests in a special format that TeamCity can interpret. This allows TC
to show which tests run as well as the number of tests run.

Also refactor some of the build scripts to use functions.

Note that we had to explicitly add `--check-coverage=false` (which is the
default) to get successful builds when we run on TC.
2024-11-27 11:41:06 +01:00
Marc Durdin
b54b005830 refactor(common): move remaining LDML keyboard types into LdmlKeyboardTypes
Follows: #12712
2024-11-27 11:52:53 +07:00
Marc Durdin
ce7593bd2f fix(developer): box package compiler info fields
If a .kps file had an `<Info>` element without a URL attribute, it
would not map correctly in all circumstances. This change ensures that
the kps-file-reader is responsible for normalizing the layout of these
elements on read.
2024-11-13 10:11:44 +07:00
Marc Durdin
5a93bdf6c2 chore(developer): make package subfile description fully optional
The package subfile description field is never really used in the
end-user apps. As it is meaningless metadata, we'll make it fully
optional, and start to remove all references in the designers and
compilers.

TODO: For packages targeting Keyman 18.0+, we should consider removing
the description field altogether from the subfiles.
2024-11-13 10:03:26 +07:00
Marc Durdin
d68a64252e feat(developer): add GitHub and Cloud support to kmc-copy
This requires a significant refactor to the compiler interfaces to allow
for asynchronous filesystem and network access. For now, only kmc-copy
uses the asynchronous interfaces, but over time we may try and
consolidate them where other compiler modules find them useful. This
adds a big set of unit test fixtures to avoid network instability in
tests (the fixtures can be regenerated by setting environment variable
`TEST_SAVE_FIXTURES`).
2024-10-28 10:33:42 +07:00
Marc Durdin
40db1c2051 feat(common): add isDirectory, helper functions for kmc-copy 2024-10-22 10:28:03 +07:00
Marc Durdin
1e084e99f0 refactor(developer): add kps-file-reader and kps-file-writer
Relates to kmc-copy project.

Moves the .kps file reader out of kmc-package and adds corresponding
kps file writer unit in developer-utils. Also moves related test and
error message definition.
2024-10-22 10:28:01 +07:00
Marc Durdin
521a9b24c8 feat(developer): add .kpj file writer 2024-10-22 10:28:00 +07:00
Marc Durdin
7d17913e18 feat(developer): seed kmc-copy 2024-10-22 10:28:00 +07:00
Marc Durdin
89e58fbe00
Merge branch 'master' into feat/developer/kmc-generate 2024-10-12 11:48:31 +02:00
Marc Durdin
7dc787b96b fix(developer): use TextDecoder to convert Uint8Array to string
`toString()` only works when the Uint8Array is actually a Node Buffer.
2024-10-11 14:11:45 +07:00
Marc Durdin
0d9688451d
Merge branch 'master' into feat/developer/kmc-generate 2024-10-11 03:26:54 +02:00
Steven R. Loomis
9a4c7e5820 feat(developer): fast-xml-parser: typo fix in xml-utils.ts
Fixes: #12208
2024-10-07 13:05:50 -05:00
Steven R. Loomis
03a24a1506 feat(developer): fast-xml-parser: fix for ldml empty attributes
- because of the form to="" in ldml, we need to distinguish attributes and sub-elements in the ldml xml parsing
- use an attributePrefix, and fixup the object tree afterwards

Fixes: #12208
2024-10-03 12:41:08 -05:00
Steven R. Loomis
3e168aafb2 chore(developer,common): deps: xml2js / fast-xml-parser
- remove vendored xml2js
- add fast-xml-parser@4.5.0
- Rewire the KeymanXMLReader / KeymanXMLWriter to use fast-xml-parser

Four small changes to the test data on the generated .kvks:
- allow &quot; for XML generation (" is legal)
- use the same XML prologue as the actual .kvks files.
- treat <key></key> as <key/> since we can't distinguish
  (removed a "_": "" entry)
- remove whitespace because we now can ("_": "\n  \n \n") on non-leaf nodes.
  This is ignored by the kvks code, but was difficult to maintain roundtrip.

Fixes: #12208
2024-10-03 08:34:24 -05:00
Steven R. Loomis
158c4a7637 feat(developer): update the unified xml parser per review
- declarative syntax for options
- workaround an issue where xml2js is mutating our options objects
- improve tests

Fixes: #12208
2024-09-30 13:25:38 -05:00
Steven R. Loomis
eba7079694 chore(developer): update per style guide
Fixes: #12208
2024-09-30 10:00:46 -05:00
Steven R. Loomis
1780b1ebf3 feat(developer,common): use unified xml writer
Fixes: #12208
2024-09-27 18:53:54 -05:00
Steven R. Loomis
d540e4cda0 feat(common): unified xml writer test
Fixes: #12208
2024-09-27 18:37:40 -05:00
Steven R. Loomis
3832ca1c12 feat(developer,common): use unified xml parser
Subsystems changed:
- ldml keyboard reader (main and test)
- kpj
- kvks
- kmp compiler

test: made the test-xml-utils less verbose about the pathnames

Fixes: #12208
2024-09-27 17:49:32 -05:00
Steven R. Loomis
7da7375ee5 feat(common): xml: map \r\n to \n
- help make the parsed form identical across platforms

Fixes: #12208
2024-09-27 17:31:00 -05:00
Steven R. Loomis
609faf8047 feat(common): unified xml parser reader test
Fixes: #12208
2024-09-27 16:52:21 -05:00
Steven R. Loomis
7e631c1a73 feat(common): unified xml parser scaffolding
Fixes: #12208
2024-09-27 14:09:52 -05:00
Marc Durdin
10e4bba100
Merge branch 'master' into feat/developer/kmc-generate 2024-09-26 18:55:20 +02:00
Marc Durdin
e4017a8c28 fix(developer): rewrite ldml visual keyboard compiler
The visual keyboard compiler was never finished in 17.0. This rewrites
it to:

1. Use the kmxplus data rather than reading from xml directly
2. Fill in `visualkeyboard.header.kbdname`
3. Support modifiers
4. Handle encoded characters like `\u{1234}`
5. Handle string variables like `${one}`*

Additional unit tests have been added to verify the behavior of the
visual keyboard compiler in more detail.

TODO-LDML: string variables appear to have a secondary bug -- they seem
to be returning the string 'undefined'. I have disabled the related
tests and will examine this separately, and enable those tests once
fixed.

TODO-LDML: we should probably add a compiler warning + unit test for
`<layers formId="us"><layer id="base">`, because this pattern does not
make sense: when using non-touch forms, the `<layer>` element should use
`modifiers` attribute, and correspondingly, `modifiers` attribute should
_not_ be used when `formId` is `touch`.

Other fixes:

1. The LDML XML reader was relying on its input being a Node.js `Buffer`
   even though it was declared `Uint8Array`, as it implicitly used
   `Buffer.toString()` to do text conversion. (`Buffer` subclasses from
   `Uint8Array`). This breaks when using `Uint8Array` directly and means
   we had an implicit dependency on Node.js. See also #12331.
2. XML errors were not captured in the LDML XML reader. See also #12331.
3. The unused and unfinished touch-layout-compiler.ts and
   keymanweb-compiler.ts have been removed along with corresponding unit
   tests and fixtures. These are replaced by Core implementations; see
   #12291.

Fixes: #12395
2024-09-12 06:06:00 +07:00
Marc Durdin
113b4f7ac8 feat(developer): comments for kmc-generate 2024-09-03 06:20:34 +07:00
Marc Durdin
8274d30788 feat(developer): kmc generate cleanup
Cleanup various todo items for kmc generate. Outstanding TODO item is
icon generation for .kmn.
2024-09-03 06:20:28 +07:00
Marc Durdin
2994fe2aea feat(developer): kmc-generate docs and tests started 2024-09-03 06:17:33 +07:00
Marc Durdin
939134a9a3 feat(developer): kmc-generate module
Plenty of todo items left but this is a mostly-functional module for the
kmc generate command for automatically creating keyboard and model
projects.
2024-09-03 06:15:33 +07:00
Marc Durdin
f5066174cd fix(developer): make LDML import path consistent for all bundlings of kmc
The default import path is relative to ldml-keyboard-xml-reader.js. The
bundling of kmc with Keyman Developer reduces the directory tree depth,
so we move the import path to be a subdirectory of the immediate parent
directory, rather than two levels up, so that
`C:\Program Files\Keyman\Keyman Developer\kmc` can refer to
`C:\Program Files\Keyman\Keyman Developer\import` when kmc is bundled.

Fixes: #12279
2024-08-24 10:28:48 +08:00
dependabot[bot]
6fad148466
chore(deps): bump semver from 7.5.4 to 7.6.0
Bumps [semver](https://github.com/npm/node-semver) from 7.5.4 to 7.6.0.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v7.5.4...v7.6.0)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-14 11:22:58 +00:00
Marc Durdin
062ae33579 refactor(developer): add eventemitter3 and ldml-keyboard-constants deps to Server
Fixes: #12077
2024-08-12 17:23:48 +07:00
Marc Durdin
2814ac995c chore(developer): include common-types dep in developer-utils 2024-08-12 06:08:56 +07:00
Marc Durdin
d55558fb31
Merge pull request #12101 from keymanapp/refactor/common/9665-move-xml2js-and-related-deps-to-developer
refactor(common): move xml2js and related deps to @keymanapp/developer-utils 🐉
2024-08-07 15:16:07 +10:00
Marc Durdin
32b4711659
Merge branch 'master' into refactor/common/9665-move-compiler-interfaces-to-developer 2024-08-07 15:15:45 +10:00
Marc Durdin
ed143fb333
Merge branch 'master' into refactor/common/9665-move-ldml-keyboard-source-file-to-developer 2024-08-07 15:15:04 +10:00
Marc Durdin
ec72245575
Merge pull request #11765 from keymanapp/refactor/common/9665-move-keyman-touch-layout-file-to-developer
refactor(common): move .keyman-touch-layout reader/writer to @keymanapp/developer-utils 🐉
2024-08-07 15:14:06 +10:00
Marc Durdin
e9d054ab2c
Merge branch 'master' into refactor/common/9665-move-kvk-file-to-developer 2024-08-07 15:13:41 +10:00
Marc Durdin
88506cea6b
Merge branch 'master' into refactor/common/9665-move-kps-file-type-to-developer 2024-08-07 15:13:10 +10:00
Marc Durdin
7849b7734b
Merge pull request #11531 from keymanapp/refactor/common/9665-move-project-file-type-to-developer
refactor(common): move kpj-related files into developer-utils 🐉
2024-08-07 15:12:31 +10:00
Marc Durdin
edf47d7f43 refactor(common): move xml2js and related deps to @keymanapp/developer-utils
Fixes: #9665
2024-08-06 05:19:30 +07:00
Marc Durdin
e67aed1606 refactor(common): move compiler-interfaces to @keymanapp/developer-utils
This should be the last step in the refactoring.

Fixes: #9665
2024-08-02 13:49:59 +07:00
Marc Durdin
d3d3ac1eec refactor(common): move LDML keyboard .xml reader/writer and kmx-plus builder to @keymanapp/developer-utils
Relates-to: #9665
2024-08-02 10:23:11 +07:00
Marc Durdin
08803111e5
Merge branch 'refactor/common/9665-move-kvk-file-to-developer' into refactor/common/9665-move-keyman-touch-layout-file-to-developer 2024-08-01 10:35:48 +10:00
Marc Durdin
53551f5662
Merge branch 'refactor/common/9665-move-kps-file-type-to-developer' into refactor/common/9665-move-kvk-file-to-developer 2024-08-01 10:35:33 +10:00
Marc Durdin
ad9ba1f188
Merge branch 'refactor/common/9665-move-project-file-type-to-developer' into refactor/common/9665-move-kps-file-type-to-developer 2024-08-01 10:34:05 +10:00
Marc Durdin
d8986be12f refactor(common): merge master into branch 2024-08-01 07:07:07 +07:00
Marc Durdin
d1e351a1ac
Revert "chore(developer): remove redundant references from tsconfig.json" 2024-08-01 08:57:27 +10:00
Marc Durdin
1cff1a5084 chore(developer): remove redundant references from tsconfig.json
With the full move to ES Modules, we no longer need to include
`references` in tsconfig.json, as we can rely on package.json and
build.sh dependency management. Note however that `tsc -b` may not work
to build dependencies -- they need to be built using `build.sh`, which
calculates which dependencies need building.

kmc-keyboard-info was missing a dependency link to kmc-package in
build.sh, correcting this at the same time.

Relates-to: #12027
2024-07-27 05:44:22 +07:00