Commit graph

1667 commits

Author SHA1 Message Date
Marc Durdin
34ae4a1397 chore(developer): additional unit tests 2023-04-23 06:48:31 +07:00
Marc Durdin
66a7a6fc4e feat(developer): support FollowKeyboardVersion in kmc-package
Fixes #7340.

Removes the unused <Version> element from <LexicalModel> element in the
.kps schema. (This needs to be copied to api.keyman.com also.)

Updates compiler to check for <FollowKeyboardVersion> and read the
version data from the .kmx in that case.

Unit tests updated. Some of the fixtures were incorrect as they had
<FollowKeyboardVersion> set for model-type packages or packages that
contained no keyboards. Fixed.
2023-04-21 14:04:06 +07:00
Marc Durdin
e30c513fe5 chore(developer): add missing dependency 2023-04-21 08:18:07 +07:00
Marc Durdin
e0ed058e89 refactor(developer): kmc-model to use CompilerEvent
Replaces the existing error reporting in kmc-model with CompilerEvent
style messages. Some of the existing error reporting was kinda nice, but
consistency trumps pretty here.

One downside at present is the global filename and line variables in the
model-compiler-errors.ts unit. This is fragile, and should probably be
refactored further in the future.

Where it made sense, I converted various throws into ModelCompilerError
instances. If it appeared to be an internal error, I left it as-is, but
the external API interfaces wrap all unknown exceptions and convert them
into a compiler error message, so these APIs (compileModel and
loadFromFilename) now have a guarantee that they will never throw.

At this point, all of the error messages in the model compiler are
unrecoverable -- they will halt the compile entirely. However, the
pathway is there to support recoverable errors through the normal
callbacks.reportMessage mechanisms.

The message-counting mechanism is currently missing from this, as it
belongs in the API consumer, rather than in the low-level reporting
infrastructure. This means that models with large numbers of warnings
are going to generate scads of messages for now.

Note that the _users_ of CompilerCallbacks are subject to change in the
future, once we have all the compilers using the callbacks.
2023-04-20 16:40:04 +07:00
Marc Durdin
39d473fb2d refactor(developer): move comperr.h to kmn_compiler_errors.h
Consolidates the two copies of comperr.h (they were identical) and
moves to a common folder. Updates all references to comperr.h, except
for one github commit ref.
2023-04-20 13:20:29 +07:00
Marc Durdin
d9713a2e32 chore(developer): only run keyboards-repo tests for a full-test
If kmcmplib build includes the --full-test flag, then configure and run
tests that include pulling in the keyboards repo; otherwise skip all of
that.

This will require CI changes to run full tests, relating to #8361.
2023-04-20 12:59:00 +07:00
Marc Durdin
0e8861d407 fix(developer): kmc-package path reference 2023-04-20 12:05:26 +07:00
Marc Durdin
0ce42db399 chore: address review comments 2023-04-20 10:42:42 +07:00
Marc Durdin
2f7407b4b8 feat(developer): warn if .kps file includes a non-binary .kvk file
Relates to #8150.

Legacy .kmp compiler would transform xml-format .kvk files into a binary
.kvk file; now we want that to remain the responsibility of the keyboard
compiler, so we'll warn the few users who are still doing this.
2023-04-19 15:32:29 +07:00
Marc Durdin
e89568c2d3 fix(developer): KmpCompiler call in kmlmp, kmlmi 2023-04-19 15:31:28 +07:00
Marc Durdin
88601f2409 feat(developer): add file-existence check to kmc-package 2023-04-19 12:44:52 +07:00
Marc Durdin
7644979dee chore(developer): unit test for absolute path warning 2023-04-19 12:17:29 +07:00
Marc Durdin
161fb924b9 feat(developer): add CompilerMessages support to kmc-package
Adds basic infrastructure for CompilerMessages events. Additional warnings,
errors, and hints to come.
2023-04-19 11:57:02 +07:00
Marc Durdin
5e1fbfa0b7 refactor(developer): move shared test helpers into common
Sadly, a lot of plumbing required to share code. But it should work now;
these test helpers are used only in dev so should not require
publication.
2023-04-19 11:43:27 +07:00
Marc Durdin
57bc98c096 chore(developer): ignore interface-only files for c8 2023-04-19 11:42:49 +07:00
Marc Durdin
1d496b7f6c chore(developer): ignore lexical-model.ts interface-only file for c8 2023-04-19 11:32:42 +07:00
Marc Durdin
c3b06ace63 chore(developer): add c8 exclusions to remaining modules 2023-04-19 10:18:18 +07:00
Marc Durdin
66542f6323 feat(developer): use new message infrastructure in kmc-kmn
Moves to using the `reportMessage` callback for reporting all errors in
kmc-kmn.

This required some patching to common/web/types in order to get the new
constants in place and add support for line numbers to `CompilerEvent`,
so at the same time I refactored `compilerErrorSeverityName` into the
module.

After the refactor, coverage tests were failing because they included
the test/ folder, so cleaned that up at the same time.
2023-04-19 10:13:14 +07:00
Marc Durdin
2c8dadb327 feat(developer): add warning options to kmc
Adds --compiler-warnings-as-errors and --no-warn-deprecated-code options
to kmc command line and interfaces.

Note that most compilers do not yet honour these options, only kmc-kmn.

Both of these should be handled in error filtering and reporting rather
than at the compiler level, so management of these may be factored out
of kmcmplib and into kmc later (although this goal may be blocked by
kmcmpdll's need to also use them).
2023-04-19 05:35:39 +07:00
Marc Durdin
2f7fbd9068 refactor(developer): Add BuildTestDataOptions interface 2023-04-19 05:22:48 +07:00
Marc Durdin
bfdd4a5fd9 refactor(developer): move build activities
Moves build activities under commands/build/, and build-test-data
action under commands/buildTestData. Clarifies which activites are
used where.
2023-04-19 05:18:34 +07:00
Marc Durdin
e926580c12 fix(developer): kmc no longer emits double file extension for models
Fixes #8602.
2023-04-18 13:40:37 +07:00
Marc Durdin
5f88bbf50c fix(developer): kmc-kmn would abort build on warnings
Fixes #8615.

The msgproc was never being located as we were missing the UTF8ToString
wrapper for the context.

This also ensures error message codes are emitted in hexadecimal, and
that the return value from the msgproc callback is 1, meaning 'continue
build'.
2023-04-18 13:04:59 +07:00
Marc Durdin
3eb29c3427 fix(developer): add missing dependencies 2023-04-18 05:30:12 +07:00
Marc Durdin
8057b39066 chore(developer): add comma 2023-04-17 12:12:14 +07:00
Marc Durdin
75a4bac5c6 chore(developer): tweak build of extension list in kmc
Co-authored-by: Joshua Horton <jahorton@sil.org>
2023-04-17 12:07:35 +07:00
Marc Durdin
501cf03dc2 chore(common): update description for 'build.sh pack' 2023-04-12 14:18:31 +07:00
Marc Durdin
3265a714f2 chore(common): add missing deps and files for npm publish
The build/ folder for most packages was missing due to being listed in
.gitignore.

Also, @keymanapp/ldml-keyboard-constants package was not being published
despite being a dependency of @keymanapp/kmc-keyboard.

Instead of relying on npm's use of .gitignore / .npmignore (which has
seriously wonky behaviours), we list the files that should be included
in package.json.

NPM Wonky behaviours: it seems that .npmignore causes .gitignore in the
same folder to be ignored. But, higher level .gitignore files still
affect the files included in the package, which makes
specification-by-exclusion painful (negating exclusions, etc). We do not
recommend use of .npmignore anywhere for this reason.
2023-04-12 14:05:05 +07:00
Marc Durdin
10a59f789d fix(common): support npm pack and consolidate npm publishing
Fixes #8586.

All npm publish actions in the repository are now run as part of the
Developer release build, hosted at present by kmc, until we setup a
higher-level build script to do it.

Adds a 'pack' action which mirrors 'publish' for all npm packages.

Removes unnecessary dev dependencies of models-templates and
models-wordbreakers from kmc-model.

Web Release build CI step has been updated to check for presence of
.build-builder.
2023-04-12 11:55:31 +07:00
Marc Durdin
02638023bf fix(developer): improve 'does not exist' error message
Fixes #8508.
2023-04-11 20:17:18 +07:00
Marc Durdin
354291c390 fix(developer): kmc should not validate as well as compile keyboard
Fixes #8512.

Note: the compile phase is identical to the validation phase at this
point, so no need to run validate prior to compile.
2023-04-11 20:16:16 +07:00
Marc Durdin
cdf9bbf9f8 chore(developer): kmc-kmn should build all baseline fixtures
Fixes #8540.
2023-04-11 20:15:53 +07:00
Marc Durdin
e22ae91eae chore(developer): filename case 2023-04-11 20:15:31 +07:00
Marc Durdin
43cffa2ec3 chore(developer): filename case change 2023-04-11 20:13:44 +07:00
Marc Durdin
a151ac476d refactor(developer): DRY kmc module references 2023-04-11 14:07:55 +07:00
Marc Durdin
ab1514eb93
Merge pull request #8576 from keymanapp/fix/developer/8574-dry-run-var
fix(developer): add DRY_RUN variable to build.sh
2023-04-11 13:11:55 +10:00
Marc Durdin
135060edfc fix(developer): add DRY_RUN variable to build.sh
Fixes #8574.

Also ensures that kmc is published after its dependencies.
2023-04-11 09:59:11 +07:00
Marc Durdin
e31ad60d3c chore(developer): Further AddWarning and SetError cleanup
* Splits compiler interface functions that return bool into
  CompilerInterfaces.cpp, and moves `SetError` define to that unit so
  it can't be used accidentally elsewhere
* Audit of `AddWarning` calls and only those that are used in bool
  functions now `return FALSE` (with new `AddWarningBool`) whereas all
  others `return CERR_Break`
2023-04-10 16:04:46 +07:00
Marc Durdin
5e761f56eb chore(developer): add basic unit test for kmcmplib apis 2023-04-10 15:24:03 +07:00
Marc Durdin
732c0e4786 chore(developer): -d flag should not be overridden in build.sh 2023-04-10 15:23:43 +07:00
Marc Durdin
59b981880f chore(developer): cleanup SetError and return types for public apis in kmcmplib 2023-04-10 15:23:17 +07:00
Marc Durdin
c5d79a9fb4
Merge pull request #7330 from keymanapp/feature-kmcompx
epic: kmcompx - kmcmpdll cross-platform compiler 🚑
2023-04-10 07:34:04 +10:00
Marc Durdin
787b32ba34
Revert "chore(web): merge master into feature-esmodule-web-engine (A17S9 end) 🧩" 2023-03-31 14:17:37 +11:00
Joshua Horton
6df63eddc5
Merge pull request #8549 from keymanapp/chore/master-to-esmodule-merge-a17s9 2023-03-31 08:51:27 +07:00
Marc Durdin
1a5592d43f
Merge pull request #8531 from keymanapp/chore/developer/8530-move-filesystem-h-cassert
chore(developer): move cassert ref to filesystem.cpp
2023-03-29 14:47:41 +11:00
Marc Durdin
ef61cc3815
Merge pull request #8529 from keymanapp/refactor/common/shared-kmx-tests
refactor(common): move core baseline tests into common
2023-03-29 14:47:33 +11:00
Marc Durdin
c8347b3c1e
Merge pull request #8523 from keymanapp/chore/developer/publish-all-required-npm-modules
chore(developer): publish all required npm modules
2023-03-29 14:47:23 +11:00
Marc Durdin
82a8ef0f63
Merge pull request #8503 from keymanapp/feat/core/use-kmc-for-kmx-builds
feat(core): use kmc to build kmx tests
2023-03-29 14:47:14 +11:00
Marc Durdin
0f5589f37f chore: fix typo 2023-03-28 13:38:43 +07:00
Marc Durdin
f31cc92848 chore: address review comments 2023-03-28 13:17:56 +07:00