Moved the ERROR_Message line above the Error_Message line for every
message in the compiler message files, as it makes the grouping clearer
once we start adding message details. For example, see the start of the
KmnCompilerMessages class.
Also wraps potentially-undefined parameters to the messages with a new
function `def`, shorthand for `CompilerMessageDef`, which converts the
parameter to '<param>' if it is undefined, which is helpful for
documentation.
Relates to #10207.
In order to include message detail in messages, it is helpful to split
out the exception messages which have a separate parameter, so that
parameter at the same position can be the message detail.
Introduces the CompilerMessageSpecWithException function which takes an
exceptionVar parameter to achieve this differentiation.
Relates to #10207.
Adds the `api` action to build.sh for each affected module. The `api`
action for each module will generate the api metadata files, and then
the markdown files are generated by /developer/build.sh api.
Note that developer-utils, kmc, and kmcmplib have no api documentation
at this time (and kmcmplib is not Typescript so would need separate
tooling anyway).
Relates to #10207.
Adds @microsoft/api-extractor and @microsoft/api-documenter NPM
packages for validating and generating API documentation from Typescript
modules.
Adds basic configuration for each module that we will be publishing at
this time.
api-extractor's output will be emitted to /developer/docs/api/etc/, for
future release comparisons. A baseline output will be committed
separately.
The build outputs will be in /developer/build/docs and the intermediate
files will be in /developer/build/api.
Note that at this time, we will not enable the checks and validations.
That is a task for alpha rather than beta. The primary intent at this
time is to select a solid, well-supported toolchain for generating the
API documentation for kmc.
Relates to #10150. Makes the various Messages classes in kmc modules
accessible to other modules. Some file renames involved to reduce
confusion.
No functional changes.
Fixes kmc-analyze metadata.
See #10254 for a related future fix.
Relates to #9473.
Refactors the public API of LexicalModelCompiler to meet KeymanCompiler,
including asyncing a bunch of functions, and moving file write
responsibilities into the class itself. Removes top-level
`compileModel()` and `loadFromFilename()` functions, as you should now
use `LexicalModelCompiler` API.
Most of the test cases needed only minor patching, but introducing the
`async init()` function has caused them to all be async, despite the
model compiler itself having no async requirements. I think this is
fine, because it makes any future additional async initialization tasks
much less painful to implement.
Given we've just run a build, this (a) seems unnecessary, and (b) seems
to go wrong in some circumstances anyway, e.g.
https://build.palaso.org/buildConfiguration/Keyman_Developer_Release/421326?buildTab=log&focusLine=19936&linesState=18839&logView=flowAware:
```
07:01:29 > @keymanapp/kmc@17.0.205-alpha prepublishOnly
07:01:29 > npm run build
07:01:29
07:01:31
07:01:31 > @keymanapp/kmc@17.0.205-alpha build
07:01:31 > tsc -b
07:01:31
07:01:35 ../../../common/web/types/build/src/kmx/kmx.d.ts(1,1): error TS1036: Statements are not allowed in ambient contexts.
07:01:35 ../../../common/web/types/build/src/kmx/kmx.d.ts(1,2): error TS1345: An expression of type 'void' cannot be tested for truthiness.
...
```
I missed adding this as a dependency for @keymanapp/kmc-package. because
node has a very lenient dependency search, it found xml2js from other
@keymanapp/common-types, so never caused errors locally.
kmc already reported unhandled exceptions, but any handled fatal errors
were captured and only reported to the user. It is better to report
these to Sentry as these are still unexpected.
I have refactored all the fatal exception messages in various kmc
modules to use a common mechanism, keeping all the Sentry integration in
kmc, now passing exception data up in the `CompilerEvent.exceptionVar`
property.
* I took the opportunity to rename messages.ts to
infrastructureMessages.ts
* @types/chai was missing which gave intellisense errors in vscode
* normal exit of kmc now provides an opportunity for error reports to
Sentry to be finalized
* Added a unit test for fatal errors in kmc
* Added a manual test pathway with `SENTRY_CLIENT_TEST_BUILD_EXCEPTION`
env var to trip the build fatal error mechanism and verify that it
looks ok; the following shows test runs demonstrate how fatal build
errors are reported:
```
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
This error has been automatically reported to the Keyman team.
Identifier: 6f0fca1a26694c22b03f02b2463d39c5
Application: Keyman Developer
Reported at: https://sentry.io/organizations/keyman/projects/keyman-developer/events/6f0fca1a26694c22b03f02b2463d39c5/
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --no-error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
```
Fixes#9288.
Tweaks a couple of unit tests to ensure callbacks are available, and
stops masking exceptions in one test with `doesNotThrow`... if it's
gonna throw then the test is gonna go bang anyway.
Deploy kmc as an ES module, forced now by top-level await that we use in
the sentry load.
Moved NodeCompilerCallbacks to same util folder as schemas are found in,
to avoid rewriting schema loading with imports in this PR. It belongs
better there than messages/ anyway.
Distributed files are now .mjs. Removed --enable-source-maps from
launchers as logging will be managed with sentry anyway.
Fixes#8795.
Relates to #9090.
Cleans up and makes consistent compiler messages in kmc, and related
unit tests.
Adds support for colorized messages.
After installing chalk for ansi coloring in kmc, @types/node needed to
be brought in sync with the Typescript version, per the suggestion of
checking against `npm dist-tags @types/node`,found in a discussion at
https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/64262#discussioncomment-4905069
Additional tidying up includes:
* Deprecating a set of functions in compiler-interfaces, bringing them
together under a new CompilerError class.
Separating CompilerErrorSeverity and CompilerErrorMask enums.
* Making CompilerError.formatEvent (formerly compilerEventFormat) print
messages in the same format as NodeCompilerCallbacks (sans coloring).
* Adding a wrapper class for CompilerCallbacks that manages the filename
reporting, used currently exclusively by kmc itself.
Fixes#8998.
kmc-kmn messages were broken down into sub-namespaces, but the range
overlapped the reserved namespace mask range. Fixed the message values,
added a new test to the verifyCompilerMessagesObject function to verify
that messages don't creep outside their namespace, and defined new
masks (in CompilerErrorSeverity enum... hmm) to help.
I opted to remove the leading `0` in the messages for kmc-kmn as I had
to correct a number of them anyway, but not the messages in remaining
units at this time (it's not an error, but just slightly misleading as
we only have 12 bits, not 16 to play with).
It is safe to reassign these messages as they were only assigned in 17.0
alpha.
Now that we have the file type detection in place, we can replace
existing references to the file extensions, where possible, with the
constant declarations. While we could rely on TypeScript's typing to use
string-based values, this helps us to avoid using file extensions
directly in string transforms, regex, etc, and use the `KeymanFileTypes`
alias instead.
Note that only references in common/web, and kmc-* projects have been
touched. References in other Developer projects have not been touched,
as some of those are not yet ESM, so that needs to be addressed first.
Adds eslint, effectively disabled, to infrastructure for all of kmc.
Note that eslintNoNodeImports.js will be used in the next commit, for
verifying #8644.
Fixes#7340.
Removes the unused <Version> element from <LexicalModel> element in the
.kps schema. (This needs to be copied to api.keyman.com also.)
Updates compiler to check for <FollowKeyboardVersion> and read the
version data from the .kmx in that case.
Unit tests updated. Some of the fixtures were incorrect as they had
<FollowKeyboardVersion> set for model-type packages or packages that
contained no keyboards. Fixed.
Replaces the existing error reporting in kmc-model with CompilerEvent
style messages. Some of the existing error reporting was kinda nice, but
consistency trumps pretty here.
One downside at present is the global filename and line variables in the
model-compiler-errors.ts unit. This is fragile, and should probably be
refactored further in the future.
Where it made sense, I converted various throws into ModelCompilerError
instances. If it appeared to be an internal error, I left it as-is, but
the external API interfaces wrap all unknown exceptions and convert them
into a compiler error message, so these APIs (compileModel and
loadFromFilename) now have a guarantee that they will never throw.
At this point, all of the error messages in the model compiler are
unrecoverable -- they will halt the compile entirely. However, the
pathway is there to support recoverable errors through the normal
callbacks.reportMessage mechanisms.
The message-counting mechanism is currently missing from this, as it
belongs in the API consumer, rather than in the low-level reporting
infrastructure. This means that models with large numbers of warnings
are going to generate scads of messages for now.
Note that the _users_ of CompilerCallbacks are subject to change in the
future, once we have all the compilers using the callbacks.
Consolidates the two copies of comperr.h (they were identical) and
moves to a common folder. Updates all references to comperr.h, except
for one github commit ref.
The build/ folder for most packages was missing due to being listed in
.gitignore.
Also, @keymanapp/ldml-keyboard-constants package was not being published
despite being a dependency of @keymanapp/kmc-keyboard.
Instead of relying on npm's use of .gitignore / .npmignore (which has
seriously wonky behaviours), we list the files that should be included
in package.json.
NPM Wonky behaviours: it seems that .npmignore causes .gitignore in the
same folder to be ignored. But, higher level .gitignore files still
affect the files included in the package, which makes
specification-by-exclusion painful (negating exclusions, etc). We do not
recommend use of .npmignore anywhere for this reason.
Fixes#8586.
All npm publish actions in the repository are now run as part of the
Developer release build, hosted at present by kmc, until we setup a
higher-level build script to do it.
Adds a 'pack' action which mirrors 'publish' for all npm packages.
Removes unnecessary dev dependencies of models-templates and
models-wordbreakers from kmc-model.
Web Release build CI step has been updated to check for presence of
.build-builder.
