Commit graph

216 commits

Author SHA1 Message Date
Marc Durdin
663e4195ff
chore: Apply suggestions from code review 2023-05-15 23:16:36 +10:00
Joshua A. Horton
1308e74833 chore(web): Merge branch 'feature-esmodule-web-engine' into fix/web/webview-android-launch 2023-05-15 08:50:28 +07:00
Marc Durdin
135d6f9c0f
chore(developer): rename misnamed error message 2023-05-14 17:23:21 +10:00
Marc Durdin
7f1c05cba4
chore: rename Error_PackageMustContainAModelOrAKeyboard
Co-authored-by: Darcy Wong <darcy_wong@sil.org>
2023-05-14 17:18:59 +10:00
Marc Durdin
abd174557c chore(developer): verify that package has at least a model or keyboard
Adds `ERROR_PackageMustContainAPackageOrAKeyboard` and corresponding
unit tests. A couple of unit tests tweaked as their fixtures were no
longer valid!
2023-05-14 09:53:36 +07:00
Marc Durdin
3c9492dd56 chore(developer): verify file types of content files in package
This is a transfer of the functionality in the legacy package compiler;
we could go much further in verifying file types and excluding certain
files, but that's a big design session. For now, just refreshed to
include the set of Keyman for Windows and Keyman Engine for Windows
files which are most likely to be accidentally included.

Adds WARN_RedistFileShouldNotBeInPackage and WARN_DocFileDangerous and
corresponding unit tests and constant declarations.
2023-05-14 09:31:32 +07:00
Marc Durdin
8446775dd5 feat(developer): verify at least one language in package
Adds `ERROR_MustHaveAtLeastOneLanguage` and corresponding unit test.
2023-05-12 14:42:07 +07:00
Marc Durdin
e4ad95550e feat(developer): verify bcp47 tags are valid and minimal in kmc-package
Adds checks for bcp47 tag metadata for keyboards and lexical models --
both validity and minimality. Adds `ERROR_LanguageTagIsNotValid` and
`WARN_LanguageTagIsNotMinimal` messages and corresponding unit tests.

This was implemented by extending the existing duplicate id check, so
renamed that function accordingly.
2023-05-12 13:21:13 +07:00
Joshua A. Horton
7b67b2f095 fix(developer): keyman-version package reference 2023-05-12 11:53:34 +07:00
Marc Durdin
3ee1132200
Merge pull request #8757 from keymanapp/feat/developer/validate-package-name
feat(developer): validate package name in compiler
2023-05-12 13:22:58 +10:00
Marc Durdin
34ba77ff2b
Merge pull request #8755 from keymanapp/feat/developer/validate-content-file-names-in-packages
feat(developer): validate content file names in packages
2023-05-12 13:22:50 +10:00
Marc Durdin
25f3a42bd5
Merge pull request #8751 from keymanapp/feat/developer/validate-package-filename
feat(developer): add validation of package filenames
2023-05-12 12:25:33 +10:00
Marc Durdin
090170ce8d
Merge pull request #8740 from keymanapp/feat/developer/package-validation-in-kmc-package
feat(developer): add package validation
2023-05-12 12:25:26 +10:00
Marc Durdin
7982bd955c
Merge pull request #8729 from keymanapp/chore/developer/move-package-formats-to-common-types
chore(developer): move package formats to common/web/types
2023-05-12 12:25:19 +10:00
Marc Durdin
d3e4762b52 chore(developer): Info_KeyboardFileHasNoKeyboardVersion
Downgrades `Warn_KeyboardFileHasNoKeyboardVersion` to
`Info_KeyboardFileHasNoKeyboardVersion`, because this is not an error,
or even something wrong necessarily; it's just something it's good to be
aware of. Updates unit test for kmc to cater for the extra message.
2023-05-12 09:07:46 +07:00
Marc Durdin
53c51b79e6 feat(developer): add tests for keyboard and package version matching
Adds `WARN_KeyboardVersionsDoNotMatch` and
`WARN_KeyboardVersionsDoNotMatchPackageVersion`.

Adds unit tests. Several other 'invalid' packages needed corrections
after adding this validation. Note that one unit test was deleted
because it could never be satisfied after adding this validation step.
2023-05-12 08:26:54 +07:00
Marc Durdin
3c3a363478 feat(developer): verify keyboard versions in kmc-package
Refactor the `extractKeyboardVersionFromKmx` function into a new class
and add extra version checking. This caused a bit of a cascade of test
failures due to some of the 'invalid' fixtures actually being invalid
in multiple ways, so the fixtures have been corrected to only be wrong
in a single way -- the way they are supposed to be broken. This means
additional fixture files in the 'invalid' folder.

Now updates keyboard version metadata for all keyboards in the package.

Upgrades `WARN_FollowKeyboardVersionButNoKeyboards` to
`ERROR_FollowKeyboardVersionButNoKeyboards`, as this leads to invalid
package metadata on build.

Renames `ERROR_KeyboardFileNotFound` to
`ERROR_KeyboardContentFileNotFound` to better reflect that no `<File>`
in the package is found to match a given `<Keyboard>` entry.

Adds `ERROR_KeyboardFileNotFound` when a referenced .kmx does not exist.

Adds and updates corresponding unit tests.
2023-05-12 07:19:33 +07:00
Marc Durdin
70279914a8 feat(developer): validate package name in compiler 2023-05-11 15:03:53 +07:00
Marc Durdin
b2be12c258 feat(developer): validate content file names in packages 2023-05-11 14:42:26 +07:00
Marc Durdin
88b31ba49e feat(developer): add validation of package filenames
Adds `WARN_PackageNameDoesNotFollowKeyboardConventions` and
`WARN_PackageNameDoesNotFollowLexicalModelConventions`, refactoring as
required to test filenames, and adds unit tests for same.
2023-05-11 13:23:33 +07:00
Marc Durdin
a77aa8b54c chore(developer): refactor checkForModelsAndKeyboardsInSamePackage into separate func 2023-05-11 12:44:45 +07:00
Marc Durdin
6c17f4851a feat(developer): add validation for keyboards in package
Adds validation for WARN_PackageShouldNotRepeatLanguages for keyboards
plus unit test.
2023-05-10 11:57:40 +07:00
Marc Durdin
dbc64bcbb0 feat(developer): add package validation
Adds a package-validation.ts module, plus adds a call to the validation
step in the compilers.

Rearranges the kmc-package files to match other compilers better.

Removes `default` from exports.

And finally, adds ERROR_PackageCannotContainBothModelsAndKeyboards and
WARN_PackageShouldNotRepeatLanguages messages, plus tests.
2023-05-10 11:50:54 +07:00
Marc Durdin
53ccdbe241 chore(developer): move package formats to common/web/types
Package file interfaces were stored in developer/kmc-package but belong
better now in common/web/types.
2023-05-09 15:48:50 +07:00
Marc Durdin
4838ba1ffe chore: rename kmc-keyboard to kmc-ldml
Part of #8719.
2023-05-09 14:08:45 +07:00
Joshua A. Horton
fd21d267f3 fix(developer): right, gotta update keyman-version references from master 2023-05-03 08:45:48 +07:00
Marc Durdin
5e0471f0b7 refactor(developer): use callbacks to avoid direct fs access in kmc-package
Fixes #8644.

This is a signficant cleanup and refactor of code that was accessing the
fs or path modules in Node directly. Given we want to be able to run
these modules on web as well in the future, it is important that we
avoid that.

This also redesigns the CompilerCallback interfaces to provide Node-like
interfaces for path and fs, with a minimal set of functions. The aim
here is to provide a surface that will hopefully match a future
web-based component such as path-browserify without significant
retooling of the kmc-* modules.

Enables linting for common/web/types and turns on the
eslintNoNodeImports checks for that module and kmc-package.

Note: Buffer is still in use in a few places also, so we'll need to
refactor that in a future commit.
2023-04-26 12:31:50 +07:00
Marc Durdin
ac39427f60 chore(developer): add eslint devDependency for kmc
Adds eslint, effectively disabled, to infrastructure for all of kmc.

Note that eslintNoNodeImports.js will be used in the next commit, for
verifying #8644.
2023-04-25 13:44:56 +07:00
Marc Durdin
f475176f8f chore(developer): update dependency refs 2023-04-25 09:47:30 +07:00
Marc Durdin
c9bca0bff2 chore(developer): test message correctness in compiler modules 2023-04-25 07:51:07 +07:00
Marc Durdin
f11ff9d208 chore(developer): fixup deps for kmc modules 2023-04-24 09:01:29 +07:00
Marc Durdin
34ae4a1397 chore(developer): additional unit tests 2023-04-23 06:48:31 +07:00
Marc Durdin
66a7a6fc4e feat(developer): support FollowKeyboardVersion in kmc-package
Fixes #7340.

Removes the unused <Version> element from <LexicalModel> element in the
.kps schema. (This needs to be copied to api.keyman.com also.)

Updates compiler to check for <FollowKeyboardVersion> and read the
version data from the .kmx in that case.

Unit tests updated. Some of the fixtures were incorrect as they had
<FollowKeyboardVersion> set for model-type packages or packages that
contained no keyboards. Fixed.
2023-04-21 14:04:06 +07:00
Marc Durdin
0ce42db399 chore: address review comments 2023-04-20 10:42:42 +07:00
Marc Durdin
2f7407b4b8 feat(developer): warn if .kps file includes a non-binary .kvk file
Relates to #8150.

Legacy .kmp compiler would transform xml-format .kvk files into a binary
.kvk file; now we want that to remain the responsibility of the keyboard
compiler, so we'll warn the few users who are still doing this.
2023-04-19 15:32:29 +07:00
Marc Durdin
88601f2409 feat(developer): add file-existence check to kmc-package 2023-04-19 12:44:52 +07:00
Marc Durdin
7644979dee chore(developer): unit test for absolute path warning 2023-04-19 12:17:29 +07:00
Marc Durdin
161fb924b9 feat(developer): add CompilerMessages support to kmc-package
Adds basic infrastructure for CompilerMessages events. Additional warnings,
errors, and hints to come.
2023-04-19 11:57:02 +07:00
Marc Durdin
57bc98c096 chore(developer): ignore interface-only files for c8 2023-04-19 11:42:49 +07:00
Marc Durdin
c3b06ace63 chore(developer): add c8 exclusions to remaining modules 2023-04-19 10:18:18 +07:00
Marc Durdin
501cf03dc2 chore(common): update description for 'build.sh pack' 2023-04-12 14:18:31 +07:00
Marc Durdin
3265a714f2 chore(common): add missing deps and files for npm publish
The build/ folder for most packages was missing due to being listed in
.gitignore.

Also, @keymanapp/ldml-keyboard-constants package was not being published
despite being a dependency of @keymanapp/kmc-keyboard.

Instead of relying on npm's use of .gitignore / .npmignore (which has
seriously wonky behaviours), we list the files that should be included
in package.json.

NPM Wonky behaviours: it seems that .npmignore causes .gitignore in the
same folder to be ignored. But, higher level .gitignore files still
affect the files included in the package, which makes
specification-by-exclusion painful (negating exclusions, etc). We do not
recommend use of .npmignore anywhere for this reason.
2023-04-12 14:05:05 +07:00
Marc Durdin
10a59f789d fix(common): support npm pack and consolidate npm publishing
Fixes #8586.

All npm publish actions in the repository are now run as part of the
Developer release build, hosted at present by kmc, until we setup a
higher-level build script to do it.

Adds a 'pack' action which mirrors 'publish' for all npm packages.

Removes unnecessary dev dependencies of models-templates and
models-wordbreakers from kmc-model.

Web Release build CI step has been updated to check for presence of
.build-builder.
2023-04-12 11:55:31 +07:00
Marc Durdin
2a871c8165 chore(developer): publish all required npm modules 2023-03-27 11:36:21 +07:00
Joshua Horton
10518dd62e
Merge pull request #8429 from keymanapp/chore/common/ldml-ts-and-node-updates
chore(common): TS updates to non-sync'd packages, feature-esmodule merge conflict prevention
2023-03-14 12:06:15 +07:00
Joshua A. Horton
2439ba4e97 fix(developer): updates kmc, web/types module resolution, adds missing type import/exports 2023-03-14 08:41:29 +07:00
Joshua A. Horton
0da7ac24be chore(common): updates TS, sets export map for keyman-version 2023-03-14 08:22:26 +07:00
Marc Durdin
92ed99c4d6 chore(common): build script performance improvements
Improves build script performance by:
* using built-ins wherever possible (e.g. string splitting)
* eliminating redundant code
* using absolute (to $KEYMAN_ROOT) rather than relative paths to avoid realpath
* removing unnecessary `npm run` calls

BEFORE                         | AFTER
-------------------------------|----------------------
time ./core/build.sh --help    |
real    0m2.116s               | real    0m0.874s
user    0m0.578s               | user    0m0.198s
sys     0m0.984s               | sys     0m0.289s
-------------------------------|----------------------
time ./web/build.sh --help     |
real    0m3.523s               | real    0m0.757s
user    0m1.166s               | user    0m0.320s
sys     0m2.273s               | sys     0m0.455s
-------------------------------|----------------------
time ./web/build.sh -d         |
real    1m34.750s              | real    0m59.974s
user    0m9.721s               | user    0m6.284s
sys     0m19.652s              | sys     0m13.202s

@keymanapp-test-bot skip
2023-03-13 07:18:09 +07:00
Marc Durdin
adfe3fac63 feat(developer): kmc compile .kps files
Adds basic support for compiling .kps files using kmc-package to kmc.
2023-02-24 14:29:48 +07:00
Marc Durdin
a2ceb98948 chore: rewrite extra standard build script prolog 2023-02-23 08:09:41 +07:00