Realtes to #10207.
Adds a new `kmc message` command which prints detailed information about
any or all compiler messages, in JSON, text, or markdown formats.
Markdown format is emitted to file in a structured way to be deployed to
the help site automatically (coming in a subsequent commit).
Relates to #10207.
In order to include message detail in messages, it is helpful to split
out the exception messages which have a separate parameter, so that
parameter at the same position can be the message detail.
Introduces the CompilerMessageSpecWithException function which takes an
exceptionVar parameter to achieve this differentiation.
This module is used only by Keyman Developer compiler. It was originally
in common-types because we didn't have a shared module for Developer.
Moving it to @keymanapp/developer-utils in order to improve its
documentation.
Relates to #10207.
Adds the `api` action to build.sh for each affected module. The `api`
action for each module will generate the api metadata files, and then
the markdown files are generated by /developer/build.sh api.
Note that developer-utils, kmc, and kmcmplib have no api documentation
at this time (and kmcmplib is not Typescript so would need separate
tooling anyway).
Fixes#10678.
Some messages have been reduced to ERROR instead of FATAL, as they are
not internal compiler errors.
Where fatal messages are raised, the message will now be reported
through correctly to Sentry.
Fixes#10488:
1. Use a consistent `exitProcess()` function so that we always wait for Sentry to do its thing.
2. Make `KeymanSentry.isEnabled()` non-async because it is synchronous.
3. Remove duplicate initialization of `KeymanSentry` in kmc.ts.
- when a required section is missing, only throw an internal error IF we haven't already flagged a problem. for example, if 'vars' fails to compile because of a problem, and we already have an error on file, then don't throw an internal error that 'bksp' depends on uninitialized 'vars'.
- mimic this flow in the test helpers.
- add a test case for this in the test helpers, in test-tran
- add a strictError flag to say that no additional errs are allowed.
Fixes: #10291
Fixes#10150.
This feature adds a command line parameter to kmc to control the
severity of INFO, HINT and WARN level messages. The severity of any of
these messages can be reduced all the way to INFO, or even disabled, or
increased to ERROR. The compiler and the reporter will honour the new
code and fail builds when failure-level codes are encountered.
Inputs are validated against all the available compiler messages in all
modules. This also provides a good pathway for automated future
documentation generation of compiler messages.
In order to process the input parameters cleanly, have refactored some
of kmc's build command logic.
This commit increases code coverage for kmc to roughly 50%.
Fixes#10059.
Use of the unsupported and undocumented virtual key output, that doesn't
work in recent Keyman versions, at all, now results in a build warning.
Only a warning, because it did kinda work in old versions of Keyman.
Fixes#9923.
These are no longer used because we now use kmc for builds. This also
means that the kmcomp-x64-structures test is no longer required, because
that was used to keep the Delphi and C++ structures in sync.
There are some minor details around error message identifiers that are
still present in a handful of places, but they are work for another
time.
Fixes#7880.
Virtual keys have never been allowed in context. This should be an error
in a keyboard. Note: virtual keys in output are officially unsupported
but still kinda sorta a little bit work in Windows -- but I doubt they
will ever be officially supported.
Also includes small side journey to tidy up names and constants for two
other tests (error_duplicate_group and error_duplicate_store).
Fixes#9620.
.keyboard_info keys emitted from kmc-keyboard-info had the wrong format
for the example.keys object, which I discovered when I applied the
validation to the kmc-keyboard-info output.
Adds support for calculating isRTL and license fields in .model_info
compiler.
Also:
* establishes @keymanapp/developer-utils shared module
* moves license validation into @keymanapp/developer-utils
* refactors kmc-model-info to a class and general cleanup
Fixes#9296.
208A in kmcmplib is the awkwardly-named CWARN_Info. We now have an
'info' severity in kmc, so this tidies that up (we can keep the same
flag in kmcmplib for now).
Fixes#9286.
Refactors the use of path and fs into the standard kmc callbacks pattern
and adds `callbacks.fileSize()`, which is needed for populating the
metadata file.
Turns on eslint test for node dependencies for kmc-model-info.
Fixes#8795.
Relates to #9090.
Cleans up and makes consistent compiler messages in kmc, and related
unit tests.
Adds support for colorized messages.
After installing chalk for ansi coloring in kmc, @types/node needed to
be brought in sync with the Typescript version, per the suggestion of
checking against `npm dist-tags @types/node`,found in a discussion at
https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/64262#discussioncomment-4905069
Additional tidying up includes:
* Deprecating a set of functions in compiler-interfaces, bringing them
together under a new CompilerError class.
Separating CompilerErrorSeverity and CompilerErrorMask enums.
* Making CompilerError.formatEvent (formerly compilerEventFormat) print
messages in the same format as NodeCompilerCallbacks (sans coloring).
* Adding a wrapper class for CompilerCallbacks that manages the filename
reporting, used currently exclusively by kmc itself.
Fixes#8998.
kmc-kmn messages were broken down into sub-namespaces, but the range
overlapped the reserved namespace mask range. Fixed the message values,
added a new test to the verifyCompilerMessagesObject function to verify
that messages don't creep outside their namespace, and defined new
masks (in CompilerErrorSeverity enum... hmm) to help.
I opted to remove the leading `0` in the messages for kmc-kmn as I had
to correct a number of them anyway, but not the messages in remaining
units at this time (it's not an error, but just slightly misleading as
we only have 12 bits, not 16 to play with).
It is safe to reassign these messages as they were only assigned in 17.0
alpha.
The url module is a node module. We need to move responsibility for
resolving the path of the LDML XML <import> statements out of
common/web/types, and into the ultimate consumer, so it's now surfaced
as an option, along with a helper constant that reports the
import.meta.url-relative base path of the standard imports that are
compiled into common/web/types.
This hopefully means we can use this module in both browser and node
contexts without trouble.
kmcmplib no longer has any filesystem access, so it cannot verify if a
referenced filename in a source file has the same case as the actual
filename on disk (a risk when moving projects between platforms). So
I opted to move this to the `loadFile` callback in kmc, which is the
only place where filesystem is actually accessed, and added
corresponding unit test.
Small additional fixes here:
1. Move from `Buffer` to `Uint8Array` in all kmc-* modules, so that we
remove that barrier to running on web.
2. Use `callbacks.loadFile` instead of `callbacks.fs.readFileSync`, so
that we can be sure to run the filename consistency check.
3. Fixed kps parser silently swallowing xml errors on load.
4. Added silent mode to NodeCompilerCallbacks so we could cleanly test
the new filename consistency hint.
5. Noted a location where we still have NodeJS deps in kmc-ldml.
Relates to #8493.
* Removes kmcmplib calls from kmcmpdll (now that we have kmc)
* Removes old kmcmp_CompileKeyboardFile and
kmcmp_CompileKeyboardFileToBuffer functions in preference for a much
cleaner kmcmp_CompileKeyboard function
* Removes json validation helper from kmcmplib (we'll use js-native
json schema validation instead)
This change means that we no longer need to keep compfile.h consistent
between kmcmplib and kmcmpdll. This will simplify upcoming refactoring
of kmcmplib.
kmcmplib no longer writes files, but it does still read them. The next
refactor will move file load responsibility into the caller.
FILE_KEYBOARD is a structure used in multiple libraries and languages.
It is passed around between kmcmplib, kmcmpdll, and kmcomp, with three
separate definitions (kmcmplib/compfile.h, kmcmpdll/compfile.h, and
kmcomp/compile.pas). These duplicate definitions must be maintained for
now because of type issues between the legacy kmcmpdll and kmcmplib,
which means that the definitions must be kept in sync.
Because FILE_KEYBOARD_EXTRA has a C++ class in it, it will be
initialized in CompileKeyboardHandle rather than in the caller, so
FILE_KEYBOARD.extra is now a pointer rather than a struct.
When original kmcomp and kmcmpdll are eventually removed, these
duplications will disappear and some of the memory management and
lifecycle will become easier to manage.
Refactor the `extractKeyboardVersionFromKmx` function into a new class
and add extra version checking. This caused a bit of a cascade of test
failures due to some of the 'invalid' fixtures actually being invalid
in multiple ways, so the fixtures have been corrected to only be wrong
in a single way -- the way they are supposed to be broken. This means
additional fixture files in the 'invalid' folder.
Now updates keyboard version metadata for all keyboards in the package.
Upgrades `WARN_FollowKeyboardVersionButNoKeyboards` to
`ERROR_FollowKeyboardVersionButNoKeyboards`, as this leads to invalid
package metadata on build.
Renames `ERROR_KeyboardFileNotFound` to
`ERROR_KeyboardContentFileNotFound` to better reflect that no `<File>`
in the package is found to match a given `<Keyboard>` entry.
Adds `ERROR_KeyboardFileNotFound` when a referenced .kmx does not exist.
Adds and updates corresponding unit tests.
developer:
- update wasm machinery in kmc-kmn to be more self contained
- improve exception situation in wasm functions
common:
- compilerErrorFormatCode() for formatting the raw code such as for tests
- compilerExceptionToString() for formatting exceptions in messages
for: #7234
