Commit graph

1690 commits

Author SHA1 Message Date
Steven R. Loomis
9fdffdb7e3 fix(developer): try to unbreak windows kmcmplib 🙀
feat(developer): compiler updates for ldml transforms 🙀 #7377
2023-05-01 16:54:09 -05:00
Steven R. Loomis
6f568bbadf feat(developer): uset api in kmcmplib 🙀
- another windows fix

For: #7377
2023-04-28 19:10:22 -05:00
Steven R. Loomis
3996bd0046 feat(developer): fix for kmcmplib 🙀
- fix build issue on windows

For: #7377
2023-04-28 15:58:15 -05:00
Steven R. Loomis
ea4da7fc92 feat(developer): fix for kmcmplib 🙀
- include icu static lib

For: #7377
2023-04-28 14:49:48 -05:00
Steven R. Loomis
13d34065c7 feat(developer): uset api in kmcmplib 🙀
- use a wrapped ICU

For: #7377
2023-04-28 14:25:26 -05:00
Steven R. Loomis
2efd503a03 fix(developer): updates to warning fixes in kmcmplib
- improvements per code review

For: #8688

Co-authored-by: Marc Durdin <marc@durdin.net>
2023-04-28 12:31:41 -05:00
Steven R. Loomis
55405e5cf1 fix(developer): warning fixes in kmcmplib
- failing build on XCode
- fix sprintf, tmpnam, others

For: #8688
2023-04-27 15:31:23 -05:00
Steven R. Loomis
473d4e31d4 feat(developer): fix basic.xml… for now 🙀
- comment out much of the transform related content
- fix broken build-fixtures

for #7377
2023-04-27 12:21:04 -05:00
Steven R. Loomis
07392ed460 feat(developer): update tests and reader for transforms 🙀
- not working yet, but 'update' some XML

for #7377
2023-04-27 12:21:03 -05:00
Marc Durdin
6fb9b72437 fix(developer): project relative path calculation and add test 2023-04-26 15:56:00 +07:00
Marc Durdin
5e0471f0b7 refactor(developer): use callbacks to avoid direct fs access in kmc-package
Fixes #8644.

This is a signficant cleanup and refactor of code that was accessing the
fs or path modules in Node directly. Given we want to be able to run
these modules on web as well in the future, it is important that we
avoid that.

This also redesigns the CompilerCallback interfaces to provide Node-like
interfaces for path and fs, with a minimal set of functions. The aim
here is to provide a surface that will hopefully match a future
web-based component such as path-browserify without significant
retooling of the kmc-* modules.

Enables linting for common/web/types and turns on the
eslintNoNodeImports checks for that module and kmc-package.

Note: Buffer is still in use in a few places also, so we'll need to
refactor that in a future commit.
2023-04-26 12:31:50 +07:00
Marc Durdin
5f9ccab4ce refactor(common): move basename func to common/web/types
This is the best place for it for now. Later we may move elsewhere.
2023-04-25 14:49:45 +07:00
Marc Durdin
ccb84b0c20 chore(developer): kmc-keyboard should not use any node-specific modules
One of a set of updates for kmc-*, as all of them currently have either
wasm-fs or node core module references. This is the easiest patch.
2023-04-25 14:18:26 +07:00
Marc Durdin
ac39427f60 chore(developer): add eslint devDependency for kmc
Adds eslint, effectively disabled, to infrastructure for all of kmc.

Note that eslintNoNodeImports.js will be used in the next commit, for
verifying #8644.
2023-04-25 13:44:56 +07:00
Marc Durdin
9d75e0265a
Merge branch 'master' into chore/developer/8656-uniqueness-test-for-messages 2023-04-25 13:29:10 +10:00
Marc Durdin
f475176f8f chore(developer): update dependency refs 2023-04-25 09:47:30 +07:00
Marc Durdin
c9bca0bff2 chore(developer): test message correctness in compiler modules 2023-04-25 07:51:07 +07:00
Marc Durdin
82a28f56f2 refactor(developer): move message correctness test into shared unit 2023-04-25 06:01:36 +07:00
Marc Durdin
71e5cb5b1a
Merge pull request #8652 from keymanapp/chore/developer/build-dependencies
chore(developer): fixup deps for kmc modules
2023-04-24 22:51:31 +10:00
Marc Durdin
ec3adcf9d6 chore(developer): additional messages 2023-04-24 12:43:23 +07:00
Marc Durdin
f11ff9d208 chore(developer): fixup deps for kmc modules 2023-04-24 09:01:29 +07:00
Marc Durdin
f9c818f103 chore(common): update dependency paths 2023-04-24 08:51:39 +07:00
Marc Durdin
cf4022ac8a feat(developer): use CompilerEvent for messages in all of kmc
Cleans up majority of remaining kmc-* modules to use CompilerEvent
messages instead of console.log and friends, adding
InfrastructureMessages class for kmc CLI messages.

Consolidates schema loading functions into a single `loadSchema`
function in the CompilerCallbacks interface and implementations.

Moves responsibility for instantiating NodeCompilerCallbacks out of
each individual BuildActivity and into higher level modules.

Fixes return value for failing builds.
2023-04-24 07:26:43 +07:00
Marc Durdin
34ae4a1397 chore(developer): additional unit tests 2023-04-23 06:48:31 +07:00
Marc Durdin
66a7a6fc4e feat(developer): support FollowKeyboardVersion in kmc-package
Fixes #7340.

Removes the unused <Version> element from <LexicalModel> element in the
.kps schema. (This needs to be copied to api.keyman.com also.)

Updates compiler to check for <FollowKeyboardVersion> and read the
version data from the .kmx in that case.

Unit tests updated. Some of the fixtures were incorrect as they had
<FollowKeyboardVersion> set for model-type packages or packages that
contained no keyboards. Fixed.
2023-04-21 14:04:06 +07:00
Marc Durdin
e30c513fe5 chore(developer): add missing dependency 2023-04-21 08:18:07 +07:00
Marc Durdin
e0ed058e89 refactor(developer): kmc-model to use CompilerEvent
Replaces the existing error reporting in kmc-model with CompilerEvent
style messages. Some of the existing error reporting was kinda nice, but
consistency trumps pretty here.

One downside at present is the global filename and line variables in the
model-compiler-errors.ts unit. This is fragile, and should probably be
refactored further in the future.

Where it made sense, I converted various throws into ModelCompilerError
instances. If it appeared to be an internal error, I left it as-is, but
the external API interfaces wrap all unknown exceptions and convert them
into a compiler error message, so these APIs (compileModel and
loadFromFilename) now have a guarantee that they will never throw.

At this point, all of the error messages in the model compiler are
unrecoverable -- they will halt the compile entirely. However, the
pathway is there to support recoverable errors through the normal
callbacks.reportMessage mechanisms.

The message-counting mechanism is currently missing from this, as it
belongs in the API consumer, rather than in the low-level reporting
infrastructure. This means that models with large numbers of warnings
are going to generate scads of messages for now.

Note that the _users_ of CompilerCallbacks are subject to change in the
future, once we have all the compilers using the callbacks.
2023-04-20 16:40:04 +07:00
Marc Durdin
39d473fb2d refactor(developer): move comperr.h to kmn_compiler_errors.h
Consolidates the two copies of comperr.h (they were identical) and
moves to a common folder. Updates all references to comperr.h, except
for one github commit ref.
2023-04-20 13:20:29 +07:00
Marc Durdin
d9713a2e32 chore(developer): only run keyboards-repo tests for a full-test
If kmcmplib build includes the --full-test flag, then configure and run
tests that include pulling in the keyboards repo; otherwise skip all of
that.

This will require CI changes to run full tests, relating to #8361.
2023-04-20 12:59:00 +07:00
Marc Durdin
0e8861d407 fix(developer): kmc-package path reference 2023-04-20 12:05:26 +07:00
Marc Durdin
0ce42db399 chore: address review comments 2023-04-20 10:42:42 +07:00
Marc Durdin
2f7407b4b8 feat(developer): warn if .kps file includes a non-binary .kvk file
Relates to #8150.

Legacy .kmp compiler would transform xml-format .kvk files into a binary
.kvk file; now we want that to remain the responsibility of the keyboard
compiler, so we'll warn the few users who are still doing this.
2023-04-19 15:32:29 +07:00
Marc Durdin
e89568c2d3 fix(developer): KmpCompiler call in kmlmp, kmlmi 2023-04-19 15:31:28 +07:00
Marc Durdin
88601f2409 feat(developer): add file-existence check to kmc-package 2023-04-19 12:44:52 +07:00
Marc Durdin
7644979dee chore(developer): unit test for absolute path warning 2023-04-19 12:17:29 +07:00
Marc Durdin
161fb924b9 feat(developer): add CompilerMessages support to kmc-package
Adds basic infrastructure for CompilerMessages events. Additional warnings,
errors, and hints to come.
2023-04-19 11:57:02 +07:00
Marc Durdin
5e1fbfa0b7 refactor(developer): move shared test helpers into common
Sadly, a lot of plumbing required to share code. But it should work now;
these test helpers are used only in dev so should not require
publication.
2023-04-19 11:43:27 +07:00
Marc Durdin
57bc98c096 chore(developer): ignore interface-only files for c8 2023-04-19 11:42:49 +07:00
Marc Durdin
1d496b7f6c chore(developer): ignore lexical-model.ts interface-only file for c8 2023-04-19 11:32:42 +07:00
Marc Durdin
c3b06ace63 chore(developer): add c8 exclusions to remaining modules 2023-04-19 10:18:18 +07:00
Marc Durdin
66542f6323 feat(developer): use new message infrastructure in kmc-kmn
Moves to using the `reportMessage` callback for reporting all errors in
kmc-kmn.

This required some patching to common/web/types in order to get the new
constants in place and add support for line numbers to `CompilerEvent`,
so at the same time I refactored `compilerErrorSeverityName` into the
module.

After the refactor, coverage tests were failing because they included
the test/ folder, so cleaned that up at the same time.
2023-04-19 10:13:14 +07:00
Marc Durdin
2c8dadb327 feat(developer): add warning options to kmc
Adds --compiler-warnings-as-errors and --no-warn-deprecated-code options
to kmc command line and interfaces.

Note that most compilers do not yet honour these options, only kmc-kmn.

Both of these should be handled in error filtering and reporting rather
than at the compiler level, so management of these may be factored out
of kmcmplib and into kmc later (although this goal may be blocked by
kmcmpdll's need to also use them).
2023-04-19 05:35:39 +07:00
Marc Durdin
2f7fbd9068 refactor(developer): Add BuildTestDataOptions interface 2023-04-19 05:22:48 +07:00
Marc Durdin
bfdd4a5fd9 refactor(developer): move build activities
Moves build activities under commands/build/, and build-test-data
action under commands/buildTestData. Clarifies which activites are
used where.
2023-04-19 05:18:34 +07:00
Marc Durdin
e926580c12 fix(developer): kmc no longer emits double file extension for models
Fixes #8602.
2023-04-18 13:40:37 +07:00
Marc Durdin
5f88bbf50c fix(developer): kmc-kmn would abort build on warnings
Fixes #8615.

The msgproc was never being located as we were missing the UTF8ToString
wrapper for the context.

This also ensures error message codes are emitted in hexadecimal, and
that the return value from the msgproc callback is 1, meaning 'continue
build'.
2023-04-18 13:04:59 +07:00
Marc Durdin
3eb29c3427 fix(developer): add missing dependencies 2023-04-18 05:30:12 +07:00
Marc Durdin
8057b39066 chore(developer): add comma 2023-04-17 12:12:14 +07:00
Marc Durdin
75a4bac5c6 chore(developer): tweak build of extension list in kmc
Co-authored-by: Joshua Horton <jahorton@sil.org>
2023-04-17 12:07:35 +07:00
Marc Durdin
501cf03dc2 chore(common): update description for 'build.sh pack' 2023-04-12 14:18:31 +07:00