This starts the refactor of the .kmn compiler interfaces for WASM,
moving more file writes into typescript and out of the compiler. Adds a
cleaner binding and structures for passing compile metadata back to
typescript for additional build steps (e.g. kvk, js builds).
Will rebase the Typescript-based KMW compiler onto this so it can start
to benefit from the refactoring.
* Fixes a minor issue with loading a .kvks file with an empty flags set.
* Adds a writeFileSync method to the compiler callback interfaces.
* Adds an e2e compiler test to verify that .kmx and .kvk are both
correctly compiled.
developer:
- update wasm machinery in kmc-kmn to be more self contained
- improve exception situation in wasm functions
common:
- compilerErrorFormatCode() for formatting the raw code such as for tests
- compilerExceptionToString() for formatting exceptions in messages
for: #7234
This sets us up to run all common/ and resources/ tests from a single
script, common/build.sh. This test will be run on each platform for all
changes made in common/ or resources/.
Also renamed files and paths for consistency in resources/ and updated
tests accordingly.
Fixes#8644.
This is a signficant cleanup and refactor of code that was accessing the
fs or path modules in Node directly. Given we want to be able to run
these modules on web as well in the future, it is important that we
avoid that.
This also redesigns the CompilerCallback interfaces to provide Node-like
interfaces for path and fs, with a minimal set of functions. The aim
here is to provide a surface that will hopefully match a future
web-based component such as path-browserify without significant
retooling of the kmc-* modules.
Enables linting for common/web/types and turns on the
eslintNoNodeImports checks for that module and kmc-package.
Note: Buffer is still in use in a few places also, so we'll need to
refactor that in a future commit.
Adds eslint, effectively disabled, to infrastructure for all of kmc.
Note that eslintNoNodeImports.js will be used in the next commit, for
verifying #8644.
Cleans up majority of remaining kmc-* modules to use CompilerEvent
messages instead of console.log and friends, adding
InfrastructureMessages class for kmc CLI messages.
Consolidates schema loading functions into a single `loadSchema`
function in the CompilerCallbacks interface and implementations.
Moves responsibility for instantiating NodeCompilerCallbacks out of
each individual BuildActivity and into higher level modules.
Fixes return value for failing builds.
Fixes#7340.
Removes the unused <Version> element from <LexicalModel> element in the
.kps schema. (This needs to be copied to api.keyman.com also.)
Updates compiler to check for <FollowKeyboardVersion> and read the
version data from the .kmx in that case.
Unit tests updated. Some of the fixtures were incorrect as they had
<FollowKeyboardVersion> set for model-type packages or packages that
contained no keyboards. Fixed.
Read a .kmx file into memory in Typescript. Currently only reads stores
from the file, as these are the most common things needed for other
processes. Will add support for reading other metadata and rules in
future when required.
Replaces the existing error reporting in kmc-model with CompilerEvent
style messages. Some of the existing error reporting was kinda nice, but
consistency trumps pretty here.
One downside at present is the global filename and line variables in the
model-compiler-errors.ts unit. This is fragile, and should probably be
refactored further in the future.
Where it made sense, I converted various throws into ModelCompilerError
instances. If it appeared to be an internal error, I left it as-is, but
the external API interfaces wrap all unknown exceptions and convert them
into a compiler error message, so these APIs (compileModel and
loadFromFilename) now have a guarantee that they will never throw.
At this point, all of the error messages in the model compiler are
unrecoverable -- they will halt the compile entirely. However, the
pathway is there to support recoverable errors through the normal
callbacks.reportMessage mechanisms.
The message-counting mechanism is currently missing from this, as it
belongs in the API consumer, rather than in the low-level reporting
infrastructure. This means that models with large numbers of warnings
are going to generate scads of messages for now.
Note that the _users_ of CompilerCallbacks are subject to change in the
future, once we have all the compilers using the callbacks.
Consolidates the two copies of comperr.h (they were identical) and
moves to a common folder. Updates all references to comperr.h, except
for one github commit ref.
Relates to #8150.
Legacy .kmp compiler would transform xml-format .kvk files into a binary
.kvk file; now we want that to remain the responsibility of the keyboard
compiler, so we'll warn the few users who are still doing this.
Moves to using the `reportMessage` callback for reporting all errors in
kmc-kmn.
This required some patching to common/web/types in order to get the new
constants in place and add support for line numbers to `CompilerEvent`,
so at the same time I refactored `compilerErrorSeverityName` into the
module.
After the refactor, coverage tests were failing because they included
the test/ folder, so cleaned that up at the same time.
The build/ folder for most packages was missing due to being listed in
.gitignore.
Also, @keymanapp/ldml-keyboard-constants package was not being published
despite being a dependency of @keymanapp/kmc-keyboard.
Instead of relying on npm's use of .gitignore / .npmignore (which has
seriously wonky behaviours), we list the files that should be included
in package.json.
NPM Wonky behaviours: it seems that .npmignore causes .gitignore in the
same folder to be ignored. But, higher level .gitignore files still
affect the files included in the package, which makes
specification-by-exclusion painful (negating exclusions, etc). We do not
recommend use of .npmignore anywhere for this reason.
