Commit graph

310 commits

Author SHA1 Message Date
Steven R. Loomis
b6855e121f feat(core): ldml: support <display id=...> in the 'disp' section 🙀
- "recent" addition to CLDR DTD

Fixes: #8717
2023-08-18 17:02:35 -05:00
Steven R. Loomis
e20123cbf8 fix(common): don't crash on illegal unicodesets 🙀
- UnicodeSetItem was ignoring the error value from sizeUnicodeSet() and calling parseUnicodeSet with an invalid size
- it *is* an error, but should be handled properly.
- an error is already recorded in the callback

The symptom was an internal error where there should have been an error callback.

  2) vars
       should fail to compile: sections/vars/fail-uset-props1.xml:
     Error: Internal Error: uset returned error -1 for [:Adlm:]
      at new UnicodeSetItem
      at VarsCompiler.addUnicodeSet

For: #7377
2023-08-18 17:02:35 -05:00
Steven R. Loomis
0ea1211b72 fix(common): marker test 🙀
- common test lacked update from #9448

#9119
2023-08-17 14:48:50 -05:00
Marc Durdin
add9a43f9e
Merge pull request #9441 from keymanapp/chore/developer/report-fatal-compiler-errors-to-sentry
chore(developer): improve kmc sentry reporting on fatal build errors
2023-08-09 18:35:53 +10:00
Marc Durdin
fa19eea873 chore(developer): improve kmc sentry reporting on fatal build errors
kmc already reported unhandled exceptions, but any handled fatal errors
were captured and only reported to the user. It is better to report
these to Sentry as these are still unexpected.

I have refactored all the fatal exception messages in various kmc
modules to use a common mechanism, keeping all the Sentry integration in
kmc, now passing exception data up in the `CompilerEvent.exceptionVar`
property.

* I took the opportunity to rename messages.ts to
  infrastructureMessages.ts

* @types/chai was missing which gave intellisense errors in vscode

* normal exit of kmc now provides an opportunity for error reports to
  Sentry to be finalized

* Added a unit test for fatal errors in kmc

* Added a manual test pathway with `SENTRY_CLIENT_TEST_BUILD_EXCEPTION`
  env var to trip the build fatal error mechanism and verify that it
  looks ok; the following shows test runs demonstrate how fatal build
  errors are reported:

```
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION

Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
    at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
    at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
    at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
    at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
    at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
    at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
    at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
    at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
    at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
    at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)

    This error has been automatically reported to the Keyman team.
      Identifier:  6f0fca1a26694c22b03f02b2463d39c5
      Application: Keyman Developer
      Reported at: https://sentry.io/organizations/keyman/projects/keyman-developer/events/6f0fca1a26694c22b03f02b2463d39c5/

mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --no-error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION

Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
    at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
    at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
    at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
    at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
    at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
    at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
    at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
    at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
    at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
    at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
```
2023-08-09 09:34:44 +07:00
Marc Durdin
c02f5ab419 feat(developer): add 'default' property for longpress keys
Fixes #9430.

Adds a 'default' property to subkeys, only used by longpress-type
subkeys:

* Added property to schema and readers/writers
* Added property to touch layout validator
* Added editor for property to layout builder
2023-08-08 14:20:52 +07:00
Marc Durdin
49d471ba17
Merge pull request #8954 from keymanapp/feature-kmc-kmw
epic: kmc-kmw KeymanWeb compiler in Typescript 🗜
2023-08-07 11:37:39 +10:00
Marc Durdin
cdf026c2ba chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw-a17s18 2023-08-04 12:54:10 +07:00
Steven R. Loomis
6632aafeec spec(core): update per code review 🙀
- keep 'deadkey' out of the ldml code
- other renames

For: #9119
2023-08-03 19:00:21 -05:00
Steven R. Loomis
d53ae69299
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-08-03 18:56:05 -05:00
Steven R. Loomis
4aacfbf737 chore(developer): improve code coverage 30% 🙀
- … with this one weird trick

For: #9119
2023-08-02 18:09:58 -05:00
Steven R. Loomis
2a818b1ede spec(core,developer): update marker spec 🙀
- use UC_SENTINEL CODE_DEADKEY …
- use U+FFFE for any marker
- add asserts between .ts and C++
- update generation

For: #9119
2023-08-02 18:04:48 -05:00
Marc Durdin
c20a670f91 chore(developer): handle unknown file extensions in file types
Projects can contain any type of file, but the fromFilename file type
utility function would only handle known source and binary file types.
2023-08-02 12:32:07 +07:00
Marc Durdin
d8e03e0695 chore(developer): refactor KeymanDeveloperProjectFile
Turns KeymanDeveloperProjectFile into an interface, and removes
redundant data in filename and fileType fields, calculating these from
filePath instead. Maintains separate KeymanDeveloperProjectFile10 and
KeymanDeveloperProjectFile20 classes implementing the base interface for
now, although that may be a target for future consolidation (lowpri).
2023-08-02 12:05:48 +07:00
Marc Durdin
48c0f09872 chore(developer): use getFileType() instead of fileType
Ensures that we calculate the file type instead of using stored data,
which is a lot cleaner. Means we ignore the filetype field in the .kpj
xml, which is fine.

We should do the same with filename vs filepath.
2023-08-02 11:50:35 +07:00
Steven R. Loomis
7c3424b081 chore(developer): update markers per review 🙀
- remove SENTINEL_ALL_MARKERS

For: #9119
2023-07-31 16:10:59 -05:00
Steven R. Loomis
8c79f14156 feat(developer): marker - use marker in the sections 🙀
- bunch of rework to make 'vars' a dependent of lots of sections
- should be emitting marker ids in the output stream now.

#9119
2023-07-29 17:26:36 -05:00
Steven R. Loomis
a6bce2d6c0 feat(common): marker - utilities to emit sentinel values 🙀
#9119
2023-07-29 16:59:54 -05:00
Steven R. Loomis
97c8380f7c feat(common): marker - emit sentinel values 🙀
#9119
2023-07-29 16:48:33 -05:00
Steven R. Loomis
2211900e5f feat(common): marker - move values into constants 🙀
- constants need to be shared with C++

#9119
2023-07-29 16:15:55 -05:00
Steven R. Loomis
d514dc9915 feat(common): marker processing support 🙀
- common: emit the sentinel value, with range checking, from MarkerParser
2023-07-29 16:06:59 -05:00
Steven R. Loomis
581d65a4c7 fix(common): list fix 🙀
- a falsy list should show up as list #0

#9119
2023-07-29 14:55:21 -05:00
Steven R. Loomis
52e54395f9 feat(developer): marker accounting 🙀
- split out MarkerTracker, could give us more precise messages about marker use
- for now, we parse all markers twice.
- update builder for the markers list

#9119
2023-07-29 14:18:37 -05:00
Steven R. Loomis
ab4a0f36ab feat(developer): Marker tests 🙀
- wip , still not working

For: #9119
2023-07-26 18:06:22 -05:00
Marc Durdin
c9e1e82a76
Merge branch 'feature-kmc-kmw' into feat/developer/introduce-kmc-keyboard-info 2023-07-26 19:24:34 +10:00
Marc Durdin
52869e0d15
Merge pull request #9323 from keymanapp/chore/developer/move-keyman-targets-to-common
refactor(developer): move keyman-targets.ts to common/web/types 🗜
2023-07-26 17:37:41 +10:00
Marc Durdin
393951ce8e
Merge branch 'chore/developer/move-keyman-targets-to-common' into feat/developer/introduce-kmc-keyboard-info 2023-07-26 14:34:40 +10:00
Marc Durdin
a2d45c00c7 chore(common): address review comments 2023-07-26 11:32:38 +07:00
Marc Durdin
7f5e0756d2
Merge pull request #9306 from keymanapp/chore/developer/9305-use-kmc-messages-for-errors-in-kmc-model-info
chore(developer): use kmc messages for kmc-model-info 🗜
2023-07-26 09:48:17 +10:00
Marc Durdin
b369fbb89b feat(developer): implement kmc keyboard_info compiler
Fixes missing components of the kmc-keyboard-info compiler and
integration into kmc. Lots of cleanup making sure that the compiler
actually generates expected output against the keyboards repository.
2023-07-25 14:21:58 +07:00
Marc Durdin
1bfd9e64ae refactor(developer): move keyman-targets.ts to common/web/types
Also adds unit test. Prep for use in kmc-keyboard-info.
2023-07-24 08:59:04 +07:00
Marc Durdin
1810434b34 chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-kmc-kmw-a17s17 2023-07-21 10:09:33 +07:00
Marc Durdin
f89507aedf
Merge pull request #9303 from keymanapp/fix/developer/9302-double-add-of-keyboard_info
fix(developer): don't add .keyboard_info twice 🗜
2023-07-21 12:39:27 +10:00
Marc Durdin
7c1d2e04e2
Merge pull request #9301 from keymanapp/fix/developer/9296-9297-9298-9299-9300-compiler-warning-tweaks
fix(developer): kmc warnings tidyup 🗜
2023-07-21 12:39:20 +10:00
Marc Durdin
1034d6944c
Merge pull request #9287 from keymanapp/refactor/developer/9286-remove-node-deps-from-kmc-model-info
refactor(developer): remove node dependencies from kmc-model-info 🗜
2023-07-19 18:25:18 +10:00
Marc Durdin
23b9f7bf93
Merge pull request #9283 from keymanapp/chore/developer/9272-work-around-experimental-json-modules
chore(developer): work around experimental JSON modules in node 🗜
2023-07-19 18:25:08 +10:00
Marc Durdin
2a1bbad5f8
chore: Update common/web/types/src/util/compiler-interfaces.ts
Co-authored-by: Joshua Horton <joshua_horton@sil.org>
2023-07-19 18:23:42 +10:00
Marc Durdin
f639d1e6d2
Update common/web/types/src/util/file-types.ts
Co-authored-by: Joshua Horton <joshua_horton@sil.org>
2023-07-19 18:22:15 +10:00
Marc Durdin
45dc9761d2
chore: Update common/web/types/build.sh 2023-07-19 18:16:59 +10:00
Marc Durdin
794bf8a3be chore(developer): use kmc messages for kmc-model-info
Fixes #9305.
2023-07-19 15:15:54 +07:00
Marc Durdin
e731e1acb5 fix(developer): don't add .keyboard_info twice
Fixes #9302.
2023-07-19 14:13:41 +07:00
Marc Durdin
fd8e17e05c fix(developer): respect checkFilenameConventions in kmc
Fixes #9300.
2023-07-19 13:56:21 +07:00
Marc Durdin
e2dc338ef1 refactor(developer): remove node dependencies from kmc-model-info
Fixes #9286.

Refactors the use of path and fs into the standard kmc callbacks pattern
and adds `callbacks.fileSize()`, which is needed for populating the
metadata file.

Turns on eslint test for node dependencies for kmc-model-info.
2023-07-18 14:02:04 +07:00
Marc Durdin
b6ba229790 chore(developer): ensure schemas are built 2023-07-18 13:43:25 +07:00
Marc Durdin
88e216d017 chore(developer): work around experimental JSON modules in node
Fixes #9272.

Credit to @srl295.
2023-07-18 12:51:15 +07:00
Marc Durdin
908f5c2470 chore(developer): account for .keyboard_info in .kpj test 2023-07-18 12:34:42 +07:00
Marc Durdin
0e65d7703c feat(developer): add model_info and keyboard_info build activities to kmc
Adds ability to build .model_info and .keyboard_info files to kmc. Note
that .keyboard_info building is currently a stub as the .keyboard_info
compiler has not yet been translated to TypeScript, which will come in a
subsequent PR.

Note that .model_info and .keyboard_info builds depend on the project
loader, as they reference both keyboard and package source files when
constructing the metadata files.

Also:
* Fixed a deep reference to KeymanDeveloperProjectFile.
* Split the project loader into a utility unit.
* Marked the .kpj fileType field as deprecated, as it is easily
  extrapolated from the filename, with the aim to remove it entirely
  from version 2.0 projects.

@keymanapp-test-bot skip
2023-07-18 12:14:46 +07:00
Steven R. Loomis
3268be0c34 chore(common): add BUILDER_U32CHAR 🙀
- fix some other minor nits and documentation todos (no semantics)
- BUILDER_KEYS accidentally repeated 'ident' and 'size' from the parent interface

Fixes #9079
2023-07-14 12:42:35 -05:00
Steven R. Loomis
2e80d984c8 Merge remote-tracking branch 'upstream/master' into chore/common-9079-builder-str-ref-epic-ldml 2023-07-14 12:17:33 -05:00
Marc Durdin
a9cae27e78
chore: apply review suggestion, delete comment
Co-authored-by: Joshua Horton <joshua_horton@sil.org>
2023-07-13 17:02:41 +10:00