Works around npm/cli#3466 when bundling internal dependencies using the
bundleDependencies package.json property.
This change works in tandem with the npm pack/publish process -- when we
run `developer/src/kmc/build.sh publish` (or `pack`), we end up with
`npm version` stomping on all our package.json files, so the repo is
dirty after this. We need a copy of the top-level package.json before
this stomping happens, in order to get a simple map of the location of
each of our internal dependencies, from the `dependencies` property (it
would be possible to figure this out with a lot more parsing of
our package.json files, but this is simpler).
This means, in future, we should avoid publishing our internal
dependencies such as those under common/ to npm, as they serve no
practical purpose there.
kmc already reported unhandled exceptions, but any handled fatal errors
were captured and only reported to the user. It is better to report
these to Sentry as these are still unexpected.
I have refactored all the fatal exception messages in various kmc
modules to use a common mechanism, keeping all the Sentry integration in
kmc, now passing exception data up in the `CompilerEvent.exceptionVar`
property.
* I took the opportunity to rename messages.ts to
infrastructureMessages.ts
* @types/chai was missing which gave intellisense errors in vscode
* normal exit of kmc now provides an opportunity for error reports to
Sentry to be finalized
* Added a unit test for fatal errors in kmc
* Added a manual test pathway with `SENTRY_CLIENT_TEST_BUILD_EXCEPTION`
env var to trip the build fatal error mechanism and verify that it
looks ok; the following shows test runs demonstrate how fatal build
errors are reported:
```
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
This error has been automatically reported to the Keyman team.
Identifier: 6f0fca1a26694c22b03f02b2463d39c5
Application: Keyman Developer
Reported at: https://sentry.io/organizations/keyman/projects/keyman-developer/events/6f0fca1a26694c22b03f02b2463d39c5/
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --no-error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
```
Turns KeymanDeveloperProjectFile into an interface, and removes
redundant data in filename and fileType fields, calculating these from
filePath instead. Maintains separate KeymanDeveloperProjectFile10 and
KeymanDeveloperProjectFile20 classes implementing the base interface for
now, although that may be a target for future consolidation (lowpri).
Ensures that we calculate the file type instead of using stored data,
which is a lot cleaner. Means we ignore the filetype field in the .kpj
xml, which is fine.
We should do the same with filename vs filepath.
Fixes#9377.
1. ldml_transforms.cpp used emplace_back with char instead of dword
values (#9377)
2. import was updated to src/ instead of ../ by vscode in build.ts
(no related issue)
Fixes missing components of the kmc-keyboard-info compiler and
integration into kmc. Lots of cleanup making sure that the compiler
actually generates expected output against the keyboards repository.
Fixes#9286.
Refactors the use of path and fs into the standard kmc callbacks pattern
and adds `callbacks.fileSize()`, which is needed for populating the
metadata file.
Turns on eslint test for node dependencies for kmc-model-info.
Adds ability to build .model_info and .keyboard_info files to kmc. Note
that .keyboard_info building is currently a stub as the .keyboard_info
compiler has not yet been translated to TypeScript, which will come in a
subsequent PR.
Note that .model_info and .keyboard_info builds depend on the project
loader, as they reference both keyboard and package source files when
constructing the metadata files.
Also:
* Fixed a deep reference to KeymanDeveloperProjectFile.
* Split the project loader into a utility unit.
* Marked the .kpj fileType field as deprecated, as it is easily
extrapolated from the filename, with the aim to remove it entirely
from version 2.0 projects.
@keymanapp-test-bot skip
Fixes#9162.
Adds support for `@files.txt` parameters in kmc. Files listed in this
file, separated by line breaks, will be added to the input file list,
relative to the path of files.txt.
The file can include comments in lines starting with '#' and ignores
whitespace before and after each filename.
Deploy kmc as an ES module, forced now by top-level await that we use in
the sentry load.
Moved NodeCompilerCallbacks to same util folder as schemas are found in,
to avoid rewriting schema loading with imports in this PR. It belongs
better there than messages/ anyway.
Distributed files are now .mjs. Removed --enable-source-maps from
launchers as logging will be managed with sentry anyway.
Fixes#9113.
Adds sentry reporting, plus helper functions, for kmc. Includes the
now-standard `-sentry-client-test-exception` command line parameter,
with `kmcmplib` and `event` sub-options available.
Uploads kmc* sourcemaps to sentry on a release build.
On Windows, honours the registry setting for error reporting. Also
includes a command line option `--[no-]error-reporting` (must be used
prior to initial command, e.g. `kmc --no-error-reporting build ...`).
Fixes#9100.
The `compilerWarningsAsErrors` option can be passed in as a command-line
option or in the project options. Command-line option, if present, takes
precedence over the project option. If neither is set, then the value is
`false`.
Handling of this is within kmc, rather than individual modules.
Fixes#9233.
Adds --color and --no-color switches to kmc command line, with default
being determined by console mode.
Also ensures color initialization happens just once in
NodeCompilerCallbacks and moves constants to top of file.
Fixes#8795.
Relates to #9090.
Cleans up and makes consistent compiler messages in kmc, and related
unit tests.
Adds support for colorized messages.
After installing chalk for ansi coloring in kmc, @types/node needed to
be brought in sync with the Typescript version, per the suggestion of
checking against `npm dist-tags @types/node`,found in a discussion at
https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/64262#discussioncomment-4905069
Additional tidying up includes:
* Deprecating a set of functions in compiler-interfaces, bringing them
together under a new CompilerError class.
Separating CompilerErrorSeverity and CompilerErrorMask enums.
* Making CompilerError.formatEvent (formerly compilerEventFormat) print
messages in the same format as NodeCompilerCallbacks (sans coloring).
* Adding a wrapper class for CompilerCallbacks that manages the filename
reporting, used currently exclusively by kmc itself.
Fixes#9140.
Fixes#9148.
Keyman Developer 9.0 .kpj files included a lot of additional state
metadata. We need a schema which validates these files, as they are
otherwise valid to load (we'll never save them any more). Rather than
add all the extra metadata to what is otherwise a fairly clean schema,
we'll provide a legacy .schema.json.
In the future, we may be able to merge these schemas, as we move towards
the .kpj 2.0 format which doesn't list files. Ideally, all three formats
(Keyman Developer 9.0 schema, call it legacy, 1.0 schema for Keyman
Developer 10.0+ which has Options and Files listed, 2.0 schema for
capturing just project settings for a folder) will be supported by a
single schema file.
Moves kmc-kmw to being a sub-component of kmc-kmn, as it can never be
independently instantiated anyway. Reorganized tests accordingly.
Note that c8 is currently disabled for kmw-compiler, until we add unit
tests for it.
Moves common remapping code for the OSK rewriter into common/web/types
so that it can be consumed by the kmn and kmw compilers soon. Also
cleans up the command line instantiation of the analyze tools, although
the rewrite tool is less likely to be used once we have the compiler
integration in place.
