Fixes#9324.
As .js keyboards in packages are only used by the iOS and Android apps,
it's kinda helpful if the keyboard itself includes a touch layout. While
Keyman will fall back to a desktop vk layout if a touch layout is not
present, this is less than ideal -- but that's why this is setup as a
hint and not a warning.
Fixes#9325.
Also tweaks test for error_package_must_contain_a_model_or_a_keyboard as
the fixture had a .kmn, triggering the new hint, which was unhelpful in
this case.
Replaces kmcomp's package installer build infrastructure. While this is
mostly legacy, package installers are still used in adhoc distribution
scenarios, so we need to continue to support with the new kmc.
The package installer is a Windows self-extracting zip archive. The user
is expected to find setup-redist.exe and keymandesktop.msi themselves,
from the Keyman Developer release files. They are not included with
Keyman Developer, but are available as standalone downloads from
https://downloads.keyman.com/windows/.
`kmc build` has been updated to include two subcommands:
* `kmc build ldml-test-data`
* `kmc build windows-package-installer`
Both of these subcommands are a little long, but the use cases for them
are fairly narrow, so I believe this is okay. `kmc build-test-data` has
been removed, as `kmc build ldml-test-data` replaces it.
Also fixes a file reference issue in kmpJson.options, where paths were
stripped too early in the process.
Removes a now-irrelevant unit test for keyboard_info.
TODO: markdown to html during the keyboard_info build.
Adds license file to .kps, kmp.json, kmc-package, package editor, and
new project templates. The intent is for LICENSE.md to be the file used,
and for any keyboards on keymanapp/keyboards, it should always be a
standard MIT license.
While a package supports a .ttf or .otf font for the OSK and Display
fonts for use within the Keyman apps, on websites it also supports .woff
and .woff2. This change makes it possible to specify a set of fonts for
a package which will be provided through api.keyman.com to websites.
This means that .woff and .woff2 fonts may also be included in the
package and may not be used on target devices at this time, it further
opens the pathway to deploying .kmp packages to web in the future.
The deployment stage of keymanapp/keyboards will require a new phase to
copy web fonts from packages to s.keyman.com.
Adds relatedPackages support to:
* .kps schema
* .kps file format for Typescript
* kmp.json file format for Typescript
* kmc-package compiler
* Package Editor
kmc already reported unhandled exceptions, but any handled fatal errors
were captured and only reported to the user. It is better to report
these to Sentry as these are still unexpected.
I have refactored all the fatal exception messages in various kmc
modules to use a common mechanism, keeping all the Sentry integration in
kmc, now passing exception data up in the `CompilerEvent.exceptionVar`
property.
* I took the opportunity to rename messages.ts to
infrastructureMessages.ts
* @types/chai was missing which gave intellisense errors in vscode
* normal exit of kmc now provides an opportunity for error reports to
Sentry to be finalized
* Added a unit test for fatal errors in kmc
* Added a manual test pathway with `SENTRY_CLIENT_TEST_BUILD_EXCEPTION`
env var to trip the build fatal error mechanism and verify that it
looks ok; the following shows test runs demonstrate how fatal build
errors are reported:
```
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
This error has been automatically reported to the Keyman team.
Identifier: 6f0fca1a26694c22b03f02b2463d39c5
Application: Keyman Developer
Reported at: https://sentry.io/organizations/keyman/projects/keyman-developer/events/6f0fca1a26694c22b03f02b2463d39c5/
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --no-error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
```
Fixes#8795.
Relates to #9090.
Cleans up and makes consistent compiler messages in kmc, and related
unit tests.
Adds support for colorized messages.
After installing chalk for ansi coloring in kmc, @types/node needed to
be brought in sync with the Typescript version, per the suggestion of
checking against `npm dist-tags @types/node`,found in a discussion at
https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/64262#discussioncomment-4905069
Additional tidying up includes:
* Deprecating a set of functions in compiler-interfaces, bringing them
together under a new CompilerError class.
Separating CompilerErrorSeverity and CompilerErrorMask enums.
* Making CompilerError.formatEvent (formerly compilerEventFormat) print
messages in the same format as NodeCompilerCallbacks (sans coloring).
* Adding a wrapper class for CompilerCallbacks that manages the filename
reporting, used currently exclusively by kmc itself.
Fixes#9208.
Fixes#9209.
Fixes#9203.
Fixes#9198.
The changes here overlap in kmp-compiler.ts. Addresses several issues
with the package compiler:
* refactors the reading of the metadata from keyboards to make use of it
when refreshing the keyboard metadata in kmp.json (#9208). This is the
bulk of the changes.
* Fixes case on some fields in the kps file format (#9209)
* Removes code which emitted strings table to kmp.json, as these were
only ever used for package installer executables (#9203)
* Cleans up code which emitted Start Menu items to make it match spec
(#9198)
Fixes#9145.
* `WARN_KeyboardVersionsDoNotMatch` is now only raised when
`FollowKeyboardVersion` is set.
* `WARN_KeyboardVersionsDoNotMatchPackageVersion` has been removed,
because it did not really make sense; if 'FollowKeyboardVersion' is
set, it could not be raised, and otherwise, the author may wish to
have separate keyboard + package versions anyway.
* Note that the 0x0013 compiler message allocation for
`WARN_KeyboardVersionsDoNotMatchPackageVersion` has been left alone;
as this was only used in pre-release, we can probably re-task it in
the future.
Fixes#9146.
For model packages, 304016 (ERROR_ModelMustHaveAtLeastOneLanguage)
remains as an error (this is a new error for kmc, kmcomp did not
validate model metadata). For keyboard packages, a new warning is
introduced to match the existing kmcomp behaviour, 20401B
(WARN_KeyboardShouldHaveAtLeastOneLanguage).
Fixes#9111.
We have existing packages which have only .js in them, for touch-only
keyboards (mostly legacy but still...), so we need to support the freaky
Javascript regex search which we did in the past for extracting
metadata. While this is not necessarily going to work with hand-crafted
Javascript keyboards, it should work with all kmc- and kmcomp-generated
keyboards, so it will suffice to support these legacy packages.
In the future, we will be giving a hint when a package includes a .js
but not a .kmx, gradually upgrading this to a warning and finally an
error as we attempt to phase out .js-based keyboards in preference for
.kmx keyboards.
Fixes#8998.
kmc-kmn messages were broken down into sub-namespaces, but the range
overlapped the reserved namespace mask range. Fixed the message values,
added a new test to the verifyCompilerMessagesObject function to verify
that messages don't creep outside their namespace, and defined new
masks (in CompilerErrorSeverity enum... hmm) to help.
I opted to remove the leading `0` in the messages for kmc-kmn as I had
to correct a number of them anyway, but not the messages in remaining
units at this time (it's not an error, but just slightly misleading as
we only have 12 bits, not 16 to play with).
It is safe to reassign these messages as they were only assigned in 17.0
alpha.
Now that we have the file type detection in place, we can replace
existing references to the file extensions, where possible, with the
constant declarations. While we could rely on TypeScript's typing to use
string-based values, this helps us to avoid using file extensions
directly in string transforms, regex, etc, and use the `KeymanFileTypes`
alias instead.
Note that only references in common/web, and kmc-* projects have been
touched. References in other Developer projects have not been touched,
as some of those are not yet ESM, so that needs to be addressed first.
kmcmplib no longer has any filesystem access, so it cannot verify if a
referenced filename in a source file has the same case as the actual
filename on disk (a risk when moving projects between platforms). So
I opted to move this to the `loadFile` callback in kmc, which is the
only place where filesystem is actually accessed, and added
corresponding unit test.
Small additional fixes here:
1. Move from `Buffer` to `Uint8Array` in all kmc-* modules, so that we
remove that barrier to running on web.
2. Use `callbacks.loadFile` instead of `callbacks.fs.readFileSync`, so
that we can be sure to run the filename consistency check.
3. Fixed kps parser silently swallowing xml errors on load.
4. Added silent mode to NodeCompilerCallbacks so we could cleanly test
the new filename consistency hint.
5. Noted a location where we still have NodeJS deps in kmc-ldml.
