Commit graph

2034 commits

Author SHA1 Message Date
Marc Durdin
192dafa6ea feat(developer): support file lists in kmc
Fixes #9162.

Adds support for `@files.txt` parameters in kmc. Files listed in this
file, separated by line breaks, will be added to the input file list,
relative to the path of files.txt.

The file can include comments in lines starting with '#' and ignores
whitespace before and after each filename.
2023-07-17 08:15:45 +07:00
Marc Durdin
0fa7d1af5d chore(developer): remove old copy_schema call kmc build.sh 2023-07-17 05:25:16 +07:00
Marc Durdin
3fa8bc5d7a
Merge branch 'feature-kmc-kmw' into chore/common/bundle-schemas 2023-07-14 15:14:09 +10:00
Marc Durdin
3491463d3c
Merge pull request #9251 from keymanapp/fix/developer/9250-kmc-message-relative-paths
fix(developer): use relative paths in kmc messages 🗜
2023-07-14 15:12:54 +10:00
Marc Durdin
ce9c9ec9fc chore: try removing -t from sentry-cli 2023-07-13 14:48:05 +07:00
Marc Durdin
b0652c2afc chore(developer): wasm debug symbols and upload to sentry 2023-07-13 13:42:47 +07:00
Marc Durdin
68c5ba0a18 chore(developer): avoid double-map for bundled sourcemap 2023-07-13 11:11:57 +07:00
Marc Durdin
cbc857fad9 chore(developer): sourcemap def in wrong file 2023-07-13 09:04:19 +07:00
Marc Durdin
4c8d173eab chore(developer): enable source maps for bundle 2023-07-13 07:58:16 +07:00
Marc Durdin
5cc373274c chore(developer): remove --dist from sentry-cli sourcemaps inject 2023-07-13 07:44:52 +07:00
Marc Durdin
31ee7bff63 feat(developer): use sentry inject for sourcemap resolution 2023-07-13 06:07:44 +07:00
Marc Durdin
568a8141dd chore(common): additional cleanup on schema loading 2023-07-12 13:11:06 +07:00
Marc Durdin
1a52e61d4e fix(developer): use relative paths in kmc messages
Fixes #9250.
2023-07-12 12:09:16 +07:00
Marc Durdin
111de31fd6 chore(developer): fix tests and linting 2023-07-12 12:08:48 +07:00
Marc Durdin
fa3980f598 feat(developer): deploy kmc as ES module
Deploy kmc as an ES module, forced now by top-level await that we use in
the sentry load.

Moved NodeCompilerCallbacks to same util folder as schemas are found in,
to avoid rewriting schema loading with imports in this PR. It belongs
better there than messages/ anyway.

Distributed files are now .mjs. Removed --enable-source-maps from
launchers as logging will be managed with sentry anyway.
2023-07-12 10:20:35 +07:00
Marc Durdin
8082f5f3bd chore(developer): workaround TS2367 2023-07-11 17:44:56 +07:00
Marc Durdin
b435471b24 feat(developer): add sentry reporting for kmc
Fixes #9113.

Adds sentry reporting, plus helper functions, for kmc. Includes the
now-standard `-sentry-client-test-exception` command line parameter,
with `kmcmplib` and `event` sub-options available.

Uploads kmc* sourcemaps to sentry on a release build.

On Windows, honours the registry setting for error reporting. Also
includes a command line option `--[no-]error-reporting` (must be used
prior to initial command, e.g. `kmc --no-error-reporting build ...`).
2023-07-11 15:45:41 +07:00
Marc Durdin
d5aeecd7d3 chore(developer): suppress excessive messages in kmc
Fixes #9147.
2023-07-11 12:55:19 +07:00
Marc Durdin
53e2ae0ac2 fix(developer): touch layout font test should be lower case
Fixes #9110.

Also, `!=` has higher precedence than `??`.

This returns a boolean result:

```
(platform.font?.toLowerCase() ?? '') != FTouchLayoutFont
```

But `!=` has higher precedence than `??` so this returns the font name:

```
platform.font?.toLowerCase() ?? '' != FTouchLayoutFont
```
2023-07-11 12:34:11 +07:00
Marc Durdin
31083c2ed3 chore(developer): unit test for compilerWarningsAsErrors
Also adds inverse option -W, --no-compiler-warnings-as-errors
command line parameter.
2023-07-11 12:14:51 +07:00
Marc Durdin
804215145f chore(developer): handle compilerWarningsAsErrors option
Fixes #9100.

The `compilerWarningsAsErrors` option can be passed in as a command-line
option or in the project options. Command-line option, if present, takes
precedence over the project option. If neither is set, then the value is
`false`.

Handling of this is within kmc, rather than individual modules.
2023-07-11 10:16:41 +07:00
Marc Durdin
02a1c48d18 chore(developer): add --color switch to kmc
Fixes #9233.

Adds --color and --no-color switches to kmc command line, with default
being determined by console mode.

Also ensures color initialization happens just once in
NodeCompilerCallbacks and moves constants to top of file.
2023-07-11 06:25:21 +07:00
Marc Durdin
acb2e2e1ff chore(developer): split file and project build messages 2023-07-10 15:36:39 +07:00
Marc Durdin
de1c171f46 chore(developer): cleanup compiler messages in kmc
Fixes #8795.
Relates to #9090.

Cleans up and makes consistent compiler messages in kmc, and related
unit tests.

Adds support for colorized messages.

After installing chalk for ansi coloring in kmc, @types/node needed to
be brought in sync with the Typescript version, per the suggestion of
checking against `npm dist-tags @types/node`,found in a discussion at
https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/64262#discussioncomment-4905069

Additional tidying up includes:
* Deprecating a set of functions in compiler-interfaces, bringing them
  together under a new CompilerError class.
  Separating CompilerErrorSeverity and CompilerErrorMask enums.
* Making CompilerError.formatEvent (formerly compilerEventFormat) print
  messages in the same format as NodeCompilerCallbacks (sans coloring).
* Adding a wrapper class for CompilerCallbacks that manages the filename
  reporting, used currently exclusively by kmc itself.
2023-07-10 15:13:48 +07:00
Marc Durdin
7e8ba8bad3 chore(developer): fixup test parameters 2023-07-10 08:52:59 +07:00
Marc Durdin
5c0f0e87a8 chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw-A17S16 2023-07-07 09:53:34 +07:00
Marc Durdin
306934eb62 fix(developer): kmc-package support for keyboard metadata
Fixes #9208.
Fixes #9209.
Fixes #9203.
Fixes #9198.

The changes here overlap in kmp-compiler.ts. Addresses several issues
with the package compiler:

* refactors the reading of the metadata from keyboards to make use of it
  when refreshing the keyboard metadata in kmp.json (#9208). This is the
  bulk of the changes.
* Fixes case on some fields in the kps file format (#9209)
* Removes code which emitted strings table to kmp.json, as these were
  only ever used for package installer executables (#9203)
* Cleans up code which emitted Start Menu items to make it match spec
  (#9198)
2023-07-06 13:49:04 +07:00
Marc Durdin
3773522e06 feat(developer): kmc-package support for writing kmp.inf
Fixes #9205.
2023-07-06 13:49:03 +07:00
Marc Durdin
2085c26ab0 fix(developer): trim fields when emitting kmp.json
Fixes #9207.
2023-07-06 13:49:03 +07:00
Marc Durdin
84c04d5cbd fix(developer): strip paths from readmeFile and graphicFile
Fixes #9204.
2023-07-06 13:49:02 +07:00
Marc Durdin
8f5ffcc99d chore(developer): fixup ldml test for kmw emit 2023-07-06 13:48:26 +07:00
Steven R. Loomis
1bea614fd5 chore(common): improve coverage, fix todos 🙀
- remove SectionCompiler.required as unused
- minor fixes to common setSplitter
- add edge case tests

for: #8069
2023-07-05 15:48:12 -05:00
Marc Durdin
0a9c95feb4 fix(developer): off-by-one index calculations in js output
Fixes #9185.
2023-07-05 10:08:18 +07:00
Marc Durdin
aa999180c0 fix(developer): kmc: cleanup handling of whitespace in import files
Fixes #9183.

* CSS: blank line required at EOF if not present, but only if the CSS
  file was not empty!
* Embedded HTML needs a blank line at EOF if not present
* `\r` is converted to `\\n` and `\n` is converted to ` `, to match
  kmcomp
2023-07-05 10:03:31 +07:00
Marc Durdin
2d31055f74 fix(developer): escape font name correctly in vk
Fixes #9182.
2023-07-05 09:54:25 +07:00
Marc Durdin
a2c54599a9 fix(developer): call statements were misreferenced in kmc
Fixes #9181.

Two issues:

* First use of a `call` statement had an incorrect index (misuse of
  `push` return value)
* The .call_js files were incorrectly prepended with the basename of the
  keyboard
2023-07-05 09:52:15 +07:00
Steven R. Loomis
2370f97cbe
Merge branch 'chore/developer-7377-refactor-kmn-epic-ldml' into fix/common-7377-multi-escape-reorder-epic-ldml 2023-07-04 15:20:09 -05:00
Steven R. Loomis
5588e3fc34
Merge branch 'fix/developer-8069-coverage-todo-epic-ldml' into chore/developer-7377-refactor-kmn-epic-ldml 2023-07-04 15:19:44 -05:00
Steven R. Loomis
b6992d378b fix(developer): improve coverage, add todos 🙀
- fix to the fix

for: #8069
2023-07-04 15:05:10 -05:00
Steven R. Loomis
8cd846f298 fix(common): support multichar escapes in element sets 🙀
- \u{22 0127} acts like \u{22}\u{0127}

for: #7377
2023-07-04 12:47:58 -05:00
Steven R. Loomis
a5af734160 chore(developer): refactor so KmnCompiler only created once within kmc 🙀
- remove async SectionCompiler.init()  and all callers
- the usetparser lives on the DependencySections, initialized by kmc-ldml (and by the test harnesses)

for: #7377
2023-07-04 11:39:57 -05:00
Steven R. Loomis
69c9ddc99e fix(developer): improve coverage, add todos 🙀
- skip coverage on some actual internal errors
- add todos where coverage is needed

for: #8069
2023-07-04 11:10:05 -05:00
Steven R. Loomis
1126cfb9a2 chore(developer): robust reorder tests 🙀
For: #7377
2023-07-04 10:50:03 -05:00
Steven R. Loomis
52b8dfb205 feat(common): preflighting on uset 🙀
- update docs on UnicodeSet
- uset parser: fix/clarify that the wasm interface takes a bufferSize, but the ts interface takes a range count. This was muddled before.

For: #7377
2023-07-04 10:26:50 -05:00
Marc Durdin
8ebc849345 fix(developer): kmc/kmw was not stripping codes correctly
Fixes #9156.

The compiler was only skipping the first codepoint in a CODE sequence.
2023-07-04 14:48:36 +07:00
Marc Durdin
4d64cb2d38 fix(developer): touch layout was not converting all special keycaps
Fixes #9161.

kmw-compiler was converting only the first key cap with `*special*`
caption, rather than all key caps.
2023-07-04 14:44:42 +07:00
Marc Durdin
d2acfdec5c fix(developer): improve kmcmplib/kmw compiler performance
Fixes #9154.

Improves performance in both kmcmplib and in kmc-kmn/kmw-compiler. After
these changes are applied, vietnamese_telex keyboard builds in around 6
seconds on my machine (down from over 2 minutes). This is certainly only
the surface of performance improvements we could make.

Addresses excessive memory reallocations in kmcmplib, by reallocating
store and key arrays in large chunks of 100 items rather than
reallocating on each new item added. This resulted in a 250x speed-up on
functions such as `AddStore` in WASM build.

Some careful modulus arithmetic was needed for resizing the key array,
which can be grown in larger increments.

Resolved excessive zeroing out of destination buffer in u16ncpy, which
was being called with an 8kb destination buffer on every line of the
file.

In the kmw side, vast majority of the performance cost was in copying of
buffers while loading a .kmx into memory, rather than creating views
into the memory. Essentially replaced this pattern:

```
return this.rString.fromBuffer(source.slice(offset));
```

with:

```
const data = new Uint8Array(source.buffer, source.byteOffset + offset);
return this.rString.fromBuffer(data);
```
2023-07-04 14:36:25 +07:00
Marc Durdin
3a54d9b7f4 fix(developer): kmc-kmn/kmw should strip \r from inline html
Fixes #9157.

Matching kmcomp behaviour once more.
2023-07-04 14:31:01 +07:00
Steven R. Loomis
0f0c1e5e61 feat(common): preflighting on uset 🙀
- allow uset to preflight (calculate # of ranges)

For: #7377
2023-07-03 18:48:48 -05:00
Steven R. Loomis
ebcccbe0ef feat(core): rearrange elem flags 🙀
- per review comments, make the elem flags clearer

For: #7377
2023-07-03 18:20:59 -05:00