Commit graph

184 commits

Author SHA1 Message Date
Marc Durdin
1e084e99f0 refactor(developer): add kps-file-reader and kps-file-writer
Relates to kmc-copy project.

Moves the .kps file reader out of kmc-package and adds corresponding
kps file writer unit in developer-utils. Also moves related test and
error message definition.
2024-10-22 10:28:01 +07:00
Marc Durdin
521a9b24c8 feat(developer): add .kpj file writer 2024-10-22 10:28:00 +07:00
Marc Durdin
7d17913e18 feat(developer): seed kmc-copy 2024-10-22 10:28:00 +07:00
Marc Durdin
89e58fbe00
Merge branch 'master' into feat/developer/kmc-generate 2024-10-12 11:48:31 +02:00
Marc Durdin
7dc787b96b fix(developer): use TextDecoder to convert Uint8Array to string
`toString()` only works when the Uint8Array is actually a Node Buffer.
2024-10-11 14:11:45 +07:00
Marc Durdin
0d9688451d
Merge branch 'master' into feat/developer/kmc-generate 2024-10-11 03:26:54 +02:00
Marc Durdin
473fea9be0
Merge pull request #12464 from keymanapp/fix/developer/12454-use-richedit-for-debugger-memo
fix(developer): use richedit in debug memo to support Egyptian cartouches
2024-10-09 09:49:21 +02:00
Marc Durdin
e208d56419 fix(developer): use RichEdit 4.1 for debugger memos
Adds support for using RichEdit 4.1, and sets the richedit control into
the correct mode for rendering arbitrary Unicode correctly. Removes
some unnecessary per-character refresh for improved performance.

Fixes: #12454
2024-10-08 14:21:22 +07:00
Steven R. Loomis
9a4c7e5820 feat(developer): fast-xml-parser: typo fix in xml-utils.ts
Fixes: #12208
2024-10-07 13:05:50 -05:00
Steven R. Loomis
03a24a1506 feat(developer): fast-xml-parser: fix for ldml empty attributes
- because of the form to="" in ldml, we need to distinguish attributes and sub-elements in the ldml xml parsing
- use an attributePrefix, and fixup the object tree afterwards

Fixes: #12208
2024-10-03 12:41:08 -05:00
Steven R. Loomis
3e168aafb2 chore(developer,common): deps: xml2js / fast-xml-parser
- remove vendored xml2js
- add fast-xml-parser@4.5.0
- Rewire the KeymanXMLReader / KeymanXMLWriter to use fast-xml-parser

Four small changes to the test data on the generated .kvks:
- allow " for XML generation (" is legal)
- use the same XML prologue as the actual .kvks files.
- treat <key></key> as <key/> since we can't distinguish
  (removed a "_": "" entry)
- remove whitespace because we now can ("_": "\n  \n \n") on non-leaf nodes.
  This is ignored by the kvks code, but was difficult to maintain roundtrip.

Fixes: #12208
2024-10-03 08:34:24 -05:00
Steven R. Loomis
158c4a7637 feat(developer): update the unified xml parser per review
- declarative syntax for options
- workaround an issue where xml2js is mutating our options objects
- improve tests

Fixes: #12208
2024-09-30 13:25:38 -05:00
Steven R. Loomis
eba7079694 chore(developer): update per style guide
Fixes: #12208
2024-09-30 10:00:46 -05:00
Steven R. Loomis
1780b1ebf3 feat(developer,common): use unified xml writer
Fixes: #12208
2024-09-27 18:53:54 -05:00
Steven R. Loomis
d540e4cda0 feat(common): unified xml writer test
Fixes: #12208
2024-09-27 18:37:40 -05:00
Steven R. Loomis
3832ca1c12 feat(developer,common): use unified xml parser
Subsystems changed:
- ldml keyboard reader (main and test)
- kpj
- kvks
- kmp compiler

test: made the test-xml-utils less verbose about the pathnames

Fixes: #12208
2024-09-27 17:49:32 -05:00
Steven R. Loomis
7da7375ee5 feat(common): xml: map \r\n to \n
- help make the parsed form identical across platforms

Fixes: #12208
2024-09-27 17:31:00 -05:00
Steven R. Loomis
609faf8047 feat(common): unified xml parser reader test
Fixes: #12208
2024-09-27 16:52:21 -05:00
Steven R. Loomis
7e631c1a73 feat(common): unified xml parser scaffolding
Fixes: #12208
2024-09-27 14:09:52 -05:00
Marc Durdin
10e4bba100
Merge branch 'master' into feat/developer/kmc-generate 2024-09-26 18:55:20 +02:00
Marc Durdin
25ea6a74dc fix(developer): use richedit in debug memo to support Egyptian cartouches
RichEdit allows text selection one character past end-of-string, so we
need to cater for that as well in passing text ranges to the character
grid.

Fixes: #12454
2024-09-24 08:24:47 -07:00
Marc Durdin
e4017a8c28 fix(developer): rewrite ldml visual keyboard compiler
The visual keyboard compiler was never finished in 17.0. This rewrites
it to:

1. Use the kmxplus data rather than reading from xml directly
2. Fill in `visualkeyboard.header.kbdname`
3. Support modifiers
4. Handle encoded characters like `\u{1234}`
5. Handle string variables like `${one}`*

Additional unit tests have been added to verify the behavior of the
visual keyboard compiler in more detail.

TODO-LDML: string variables appear to have a secondary bug -- they seem
to be returning the string 'undefined'. I have disabled the related
tests and will examine this separately, and enable those tests once
fixed.

TODO-LDML: we should probably add a compiler warning + unit test for
`<layers formId="us"><layer id="base">`, because this pattern does not
make sense: when using non-touch forms, the `<layer>` element should use
`modifiers` attribute, and correspondingly, `modifiers` attribute should
_not_ be used when `formId` is `touch`.

Other fixes:

1. The LDML XML reader was relying on its input being a Node.js `Buffer`
   even though it was declared `Uint8Array`, as it implicitly used
   `Buffer.toString()` to do text conversion. (`Buffer` subclasses from
   `Uint8Array`). This breaks when using `Uint8Array` directly and means
   we had an implicit dependency on Node.js. See also #12331.
2. XML errors were not captured in the LDML XML reader. See also #12331.
3. The unused and unfinished touch-layout-compiler.ts and
   keymanweb-compiler.ts have been removed along with corresponding unit
   tests and fixtures. These are replaced by Core implementations; see
   #12291.

Fixes: #12395
2024-09-12 06:06:00 +07:00
Marc Durdin
113b4f7ac8 feat(developer): comments for kmc-generate 2024-09-03 06:20:34 +07:00
Marc Durdin
8274d30788 feat(developer): kmc generate cleanup
Cleanup various todo items for kmc generate. Outstanding TODO item is
icon generation for .kmn.
2024-09-03 06:20:28 +07:00
Marc Durdin
2994fe2aea feat(developer): kmc-generate docs and tests started 2024-09-03 06:17:33 +07:00
Marc Durdin
939134a9a3 feat(developer): kmc-generate module
Plenty of todo items left but this is a mostly-functional module for the
kmc generate command for automatically creating keyboard and model
projects.
2024-09-03 06:15:33 +07:00
Marc Durdin
f5066174cd fix(developer): make LDML import path consistent for all bundlings of kmc
The default import path is relative to ldml-keyboard-xml-reader.js. The
bundling of kmc with Keyman Developer reduces the directory tree depth,
so we move the import path to be a subdirectory of the immediate parent
directory, rather than two levels up, so that
`C:\Program Files\Keyman\Keyman Developer\kmc` can refer to
`C:\Program Files\Keyman\Keyman Developer\import` when kmc is bundled.

Fixes: #12279
2024-08-24 10:28:48 +08:00
dependabot[bot]
6fad148466
chore(deps): bump semver from 7.5.4 to 7.6.0
Bumps [semver](https://github.com/npm/node-semver) from 7.5.4 to 7.6.0.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v7.5.4...v7.6.0)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-08-14 11:22:58 +00:00
Marc Durdin
062ae33579 refactor(developer): add eventemitter3 and ldml-keyboard-constants deps to Server
Fixes: #12077
2024-08-12 17:23:48 +07:00
Marc Durdin
2814ac995c chore(developer): include common-types dep in developer-utils 2024-08-12 06:08:56 +07:00
Marc Durdin
945da3d8d4
Merge pull request #12113 from keymanapp/chore/common/add-support-for-build-target-platform-exclusions
chore(common): add support for build target platform exclusions
2024-08-09 16:57:56 +10:00
Marc Durdin
d55558fb31
Merge pull request #12101 from keymanapp/refactor/common/9665-move-xml2js-and-related-deps-to-developer
refactor(common): move xml2js and related deps to @keymanapp/developer-utils 🐉
2024-08-07 15:16:07 +10:00
Marc Durdin
32b4711659
Merge branch 'master' into refactor/common/9665-move-compiler-interfaces-to-developer 2024-08-07 15:15:45 +10:00
Marc Durdin
ed143fb333
Merge branch 'master' into refactor/common/9665-move-ldml-keyboard-source-file-to-developer 2024-08-07 15:15:04 +10:00
Marc Durdin
ec72245575
Merge pull request #11765 from keymanapp/refactor/common/9665-move-keyman-touch-layout-file-to-developer
refactor(common): move .keyman-touch-layout reader/writer to @keymanapp/developer-utils 🐉
2024-08-07 15:14:06 +10:00
Marc Durdin
e9d054ab2c
Merge branch 'master' into refactor/common/9665-move-kvk-file-to-developer 2024-08-07 15:13:41 +10:00
Marc Durdin
88506cea6b
Merge branch 'master' into refactor/common/9665-move-kps-file-type-to-developer 2024-08-07 15:13:10 +10:00
Marc Durdin
7849b7734b
Merge pull request #11531 from keymanapp/refactor/common/9665-move-project-file-type-to-developer
refactor(common): move kpj-related files into developer-utils 🐉
2024-08-07 15:12:31 +10:00
Marc Durdin
118a947882 chore(common): add support for build target platform exclusions
Add `builder_describe_platform` function to builder.inc.sh, associated
documentation and unit test, and use it in Developer and Common build
scripts.

Relates-to: #11755
2024-08-07 09:25:56 +07:00
Marc Durdin
edf47d7f43 refactor(common): move xml2js and related deps to @keymanapp/developer-utils
Fixes: #9665
2024-08-06 05:19:30 +07:00
Marc Durdin
e67aed1606 refactor(common): move compiler-interfaces to @keymanapp/developer-utils
This should be the last step in the refactoring.

Fixes: #9665
2024-08-02 13:49:59 +07:00
Marc Durdin
d3d3ac1eec refactor(common): move LDML keyboard .xml reader/writer and kmx-plus builder to @keymanapp/developer-utils
Relates-to: #9665
2024-08-02 10:23:11 +07:00
Marc Durdin
08803111e5
Merge branch 'refactor/common/9665-move-kvk-file-to-developer' into refactor/common/9665-move-keyman-touch-layout-file-to-developer 2024-08-01 10:35:48 +10:00
Marc Durdin
53551f5662
Merge branch 'refactor/common/9665-move-kps-file-type-to-developer' into refactor/common/9665-move-kvk-file-to-developer 2024-08-01 10:35:33 +10:00
Marc Durdin
ad9ba1f188
Merge branch 'refactor/common/9665-move-project-file-type-to-developer' into refactor/common/9665-move-kps-file-type-to-developer 2024-08-01 10:34:05 +10:00
Marc Durdin
d8986be12f refactor(common): merge master into branch 2024-08-01 07:07:07 +07:00
Marc Durdin
8cd97e5f5d refactor(developer): move kmcmplib message construction to kmc-kmn
* Remove compiler message definitions from kmcmplib
* Add parameterization to compiler message structures
* Translate parameters for existing parameterized messages (except for
  `ERROR_InvalidToken`, which requires a bigger refactor of
  `GetXStringImpl()` and many friends)
* Add columnNumber to message structures (not yet used in kmc-kmn)
* Add filename to message structures (not yet used in kmcmplib)
* Rename `INFO_Info` to `INFO_MinimumCoreEngineVersion` and
  `INFO_MinimumEngineVersion` to `INFO_MinimumWebEngineVersion`

Relates-to: #10866
2024-07-30 10:01:19 +07:00
Marc Durdin
d53b4c8322 test(developer): validate that messages in kmcmplib and kmc-kmn match
Add a unit test to verify that kmn_compiler_errors.h and
kmn-compiler-messages.ts have identical definitions for all errors in
kmcmplib. Then remove mismatched (and unused) messages identified by
this test.
2024-07-28 12:37:58 +07:00
Marc Durdin
6a48f5e550 refactor(developer): align kmcmplib error codes with kmc
Renames kmcmplib error codes to match those in kmc-kmn, and updates the
bitmasks accordingly. Adds a KmnCompilerMessages namespace so a trivial
enum can be declared without risk of clashes with other ERROR_ or
similar messages.
2024-07-28 11:45:03 +07:00
Marc Durdin
c8b055101f refactor(developer): rename CWARN_Info to CINFO_Info 2024-07-28 08:42:57 +07:00