The url module is a node module. We need to move responsibility for
resolving the path of the LDML XML <import> statements out of
common/web/types, and into the ultimate consumer, so it's now surfaced
as an option, along with a helper constant that reports the
import.meta.url-relative base path of the standard imports that are
compiled into common/web/types.
This hopefully means we can use this module in both browser and node
contexts without trouble.
kmcmplib no longer has any filesystem access, so it cannot verify if a
referenced filename in a source file has the same case as the actual
filename on disk (a risk when moving projects between platforms). So
I opted to move this to the `loadFile` callback in kmc, which is the
only place where filesystem is actually accessed, and added
corresponding unit test.
Small additional fixes here:
1. Move from `Buffer` to `Uint8Array` in all kmc-* modules, so that we
remove that barrier to running on web.
2. Use `callbacks.loadFile` instead of `callbacks.fs.readFileSync`, so
that we can be sure to run the filename consistency check.
3. Fixed kps parser silently swallowing xml errors on load.
4. Added silent mode to NodeCompilerCallbacks so we could cleanly test
the new filename consistency hint.
5. Noted a location where we still have NodeJS deps in kmc-ldml.
Fixes#8886.
.kvks compiler now returns helpful errors for xml parse failures and
schema validation errors, and for invalid virtual key codes. Introduces
extra infrastructure for reporting and unit testing messages to kmc-kmn.
This also fixes unhandled xml load exceptions and simplifies the error
reporting coming out of the kvks loader, on the basis that only one type
of error was being reported anyway.
Relates to #8493.
* Removes kmcmplib calls from kmcmpdll (now that we have kmc)
* Removes old kmcmp_CompileKeyboardFile and
kmcmp_CompileKeyboardFileToBuffer functions in preference for a much
cleaner kmcmp_CompileKeyboard function
* Removes json validation helper from kmcmplib (we'll use js-native
json schema validation instead)
This change means that we no longer need to keep compfile.h consistent
between kmcmplib and kmcmpdll. This will simplify upcoming refactoring
of kmcmplib.
kmcmplib no longer writes files, but it does still read them. The next
refactor will move file load responsibility into the caller.
This starts the refactor of the .kmn compiler interfaces for WASM,
moving more file writes into typescript and out of the compiler. Adds a
cleaner binding and structures for passing compile metadata back to
typescript for additional build steps (e.g. kvk, js builds).
Will rebase the Typescript-based KMW compiler onto this so it can start
to benefit from the refactoring.
* Fixes a minor issue with loading a .kvks file with an empty flags set.
* Adds a writeFileSync method to the compiler callback interfaces.
* Adds an e2e compiler test to verify that .kmx and .kvk are both
correctly compiled.
developer:
- update wasm machinery in kmc-kmn to be more self contained
- improve exception situation in wasm functions
common:
- compilerErrorFormatCode() for formatting the raw code such as for tests
- compilerExceptionToString() for formatting exceptions in messages
for: #7234
This sets us up to run all common/ and resources/ tests from a single
script, common/build.sh. This test will be run on each platform for all
changes made in common/ or resources/.
Also renamed files and paths for consistency in resources/ and updated
tests accordingly.
Fixes#8644.
This is a signficant cleanup and refactor of code that was accessing the
fs or path modules in Node directly. Given we want to be able to run
these modules on web as well in the future, it is important that we
avoid that.
This also redesigns the CompilerCallback interfaces to provide Node-like
interfaces for path and fs, with a minimal set of functions. The aim
here is to provide a surface that will hopefully match a future
web-based component such as path-browserify without significant
retooling of the kmc-* modules.
Enables linting for common/web/types and turns on the
eslintNoNodeImports checks for that module and kmc-package.
Note: Buffer is still in use in a few places also, so we'll need to
refactor that in a future commit.
Adds eslint, effectively disabled, to infrastructure for all of kmc.
Note that eslintNoNodeImports.js will be used in the next commit, for
verifying #8644.
Cleans up majority of remaining kmc-* modules to use CompilerEvent
messages instead of console.log and friends, adding
InfrastructureMessages class for kmc CLI messages.
Consolidates schema loading functions into a single `loadSchema`
function in the CompilerCallbacks interface and implementations.
Moves responsibility for instantiating NodeCompilerCallbacks out of
each individual BuildActivity and into higher level modules.
Fixes return value for failing builds.
Fixes#7340.
Removes the unused <Version> element from <LexicalModel> element in the
.kps schema. (This needs to be copied to api.keyman.com also.)
Updates compiler to check for <FollowKeyboardVersion> and read the
version data from the .kmx in that case.
Unit tests updated. Some of the fixtures were incorrect as they had
<FollowKeyboardVersion> set for model-type packages or packages that
contained no keyboards. Fixed.
Read a .kmx file into memory in Typescript. Currently only reads stores
from the file, as these are the most common things needed for other
processes. Will add support for reading other metadata and rules in
future when required.
Replaces the existing error reporting in kmc-model with CompilerEvent
style messages. Some of the existing error reporting was kinda nice, but
consistency trumps pretty here.
One downside at present is the global filename and line variables in the
model-compiler-errors.ts unit. This is fragile, and should probably be
refactored further in the future.
Where it made sense, I converted various throws into ModelCompilerError
instances. If it appeared to be an internal error, I left it as-is, but
the external API interfaces wrap all unknown exceptions and convert them
into a compiler error message, so these APIs (compileModel and
loadFromFilename) now have a guarantee that they will never throw.
At this point, all of the error messages in the model compiler are
unrecoverable -- they will halt the compile entirely. However, the
pathway is there to support recoverable errors through the normal
callbacks.reportMessage mechanisms.
The message-counting mechanism is currently missing from this, as it
belongs in the API consumer, rather than in the low-level reporting
infrastructure. This means that models with large numbers of warnings
are going to generate scads of messages for now.
Note that the _users_ of CompilerCallbacks are subject to change in the
future, once we have all the compilers using the callbacks.
Consolidates the two copies of comperr.h (they were identical) and
moves to a common folder. Updates all references to comperr.h, except
for one github commit ref.
