Commit graph

41 commits

Author SHA1 Message Date
Marc Durdin
53ccdbe241 chore(developer): move package formats to common/web/types
Package file interfaces were stored in developer/kmc-package but belong
better now in common/web/types.
2023-05-09 15:48:50 +07:00
Marc Durdin
5e0471f0b7 refactor(developer): use callbacks to avoid direct fs access in kmc-package
Fixes #8644.

This is a signficant cleanup and refactor of code that was accessing the
fs or path modules in Node directly. Given we want to be able to run
these modules on web as well in the future, it is important that we
avoid that.

This also redesigns the CompilerCallback interfaces to provide Node-like
interfaces for path and fs, with a minimal set of functions. The aim
here is to provide a surface that will hopefully match a future
web-based component such as path-browserify without significant
retooling of the kmc-* modules.

Enables linting for common/web/types and turns on the
eslintNoNodeImports checks for that module and kmc-package.

Note: Buffer is still in use in a few places also, so we'll need to
refactor that in a future commit.
2023-04-26 12:31:50 +07:00
Marc Durdin
ac39427f60 chore(developer): add eslint devDependency for kmc
Adds eslint, effectively disabled, to infrastructure for all of kmc.

Note that eslintNoNodeImports.js will be used in the next commit, for
verifying #8644.
2023-04-25 13:44:56 +07:00
Marc Durdin
f475176f8f chore(developer): update dependency refs 2023-04-25 09:47:30 +07:00
Marc Durdin
c9bca0bff2 chore(developer): test message correctness in compiler modules 2023-04-25 07:51:07 +07:00
Marc Durdin
f11ff9d208 chore(developer): fixup deps for kmc modules 2023-04-24 09:01:29 +07:00
Marc Durdin
34ae4a1397 chore(developer): additional unit tests 2023-04-23 06:48:31 +07:00
Marc Durdin
66a7a6fc4e feat(developer): support FollowKeyboardVersion in kmc-package
Fixes #7340.

Removes the unused <Version> element from <LexicalModel> element in the
.kps schema. (This needs to be copied to api.keyman.com also.)

Updates compiler to check for <FollowKeyboardVersion> and read the
version data from the .kmx in that case.

Unit tests updated. Some of the fixtures were incorrect as they had
<FollowKeyboardVersion> set for model-type packages or packages that
contained no keyboards. Fixed.
2023-04-21 14:04:06 +07:00
Marc Durdin
0ce42db399 chore: address review comments 2023-04-20 10:42:42 +07:00
Marc Durdin
2f7407b4b8 feat(developer): warn if .kps file includes a non-binary .kvk file
Relates to #8150.

Legacy .kmp compiler would transform xml-format .kvk files into a binary
.kvk file; now we want that to remain the responsibility of the keyboard
compiler, so we'll warn the few users who are still doing this.
2023-04-19 15:32:29 +07:00
Marc Durdin
88601f2409 feat(developer): add file-existence check to kmc-package 2023-04-19 12:44:52 +07:00
Marc Durdin
7644979dee chore(developer): unit test for absolute path warning 2023-04-19 12:17:29 +07:00
Marc Durdin
161fb924b9 feat(developer): add CompilerMessages support to kmc-package
Adds basic infrastructure for CompilerMessages events. Additional warnings,
errors, and hints to come.
2023-04-19 11:57:02 +07:00
Marc Durdin
57bc98c096 chore(developer): ignore interface-only files for c8 2023-04-19 11:42:49 +07:00
Marc Durdin
c3b06ace63 chore(developer): add c8 exclusions to remaining modules 2023-04-19 10:18:18 +07:00
Marc Durdin
501cf03dc2 chore(common): update description for 'build.sh pack' 2023-04-12 14:18:31 +07:00
Marc Durdin
3265a714f2 chore(common): add missing deps and files for npm publish
The build/ folder for most packages was missing due to being listed in
.gitignore.

Also, @keymanapp/ldml-keyboard-constants package was not being published
despite being a dependency of @keymanapp/kmc-keyboard.

Instead of relying on npm's use of .gitignore / .npmignore (which has
seriously wonky behaviours), we list the files that should be included
in package.json.

NPM Wonky behaviours: it seems that .npmignore causes .gitignore in the
same folder to be ignored. But, higher level .gitignore files still
affect the files included in the package, which makes
specification-by-exclusion painful (negating exclusions, etc). We do not
recommend use of .npmignore anywhere for this reason.
2023-04-12 14:05:05 +07:00
Marc Durdin
10a59f789d fix(common): support npm pack and consolidate npm publishing
Fixes #8586.

All npm publish actions in the repository are now run as part of the
Developer release build, hosted at present by kmc, until we setup a
higher-level build script to do it.

Adds a 'pack' action which mirrors 'publish' for all npm packages.

Removes unnecessary dev dependencies of models-templates and
models-wordbreakers from kmc-model.

Web Release build CI step has been updated to check for presence of
.build-builder.
2023-04-12 11:55:31 +07:00
Marc Durdin
2a871c8165 chore(developer): publish all required npm modules 2023-03-27 11:36:21 +07:00
Joshua Horton
10518dd62e
Merge pull request #8429 from keymanapp/chore/common/ldml-ts-and-node-updates
chore(common): TS updates to non-sync'd packages, feature-esmodule merge conflict prevention
2023-03-14 12:06:15 +07:00
Joshua A. Horton
2439ba4e97 fix(developer): updates kmc, web/types module resolution, adds missing type import/exports 2023-03-14 08:41:29 +07:00
Joshua A. Horton
0da7ac24be chore(common): updates TS, sets export map for keyman-version 2023-03-14 08:22:26 +07:00
Marc Durdin
92ed99c4d6 chore(common): build script performance improvements
Improves build script performance by:
* using built-ins wherever possible (e.g. string splitting)
* eliminating redundant code
* using absolute (to $KEYMAN_ROOT) rather than relative paths to avoid realpath
* removing unnecessary `npm run` calls

BEFORE                         | AFTER
-------------------------------|----------------------
time ./core/build.sh --help    |
real    0m2.116s               | real    0m0.874s
user    0m0.578s               | user    0m0.198s
sys     0m0.984s               | sys     0m0.289s
-------------------------------|----------------------
time ./web/build.sh --help     |
real    0m3.523s               | real    0m0.757s
user    0m1.166s               | user    0m0.320s
sys     0m2.273s               | sys     0m0.455s
-------------------------------|----------------------
time ./web/build.sh -d         |
real    1m34.750s              | real    0m59.974s
user    0m9.721s               | user    0m6.284s
sys     0m19.652s              | sys     0m13.202s

@keymanapp-test-bot skip
2023-03-13 07:18:09 +07:00
Marc Durdin
adfe3fac63 feat(developer): kmc compile .kps files
Adds basic support for compiling .kps files using kmc-package to kmc.
2023-02-24 14:29:48 +07:00
Marc Durdin
a2ceb98948 chore: rewrite extra standard build script prolog 2023-02-23 08:09:41 +07:00
Steven R. Loomis
e63745baff chore(coverage): ldml setup for visual code coverage 🙀
- generate coverage/lcov.info files in several places that were already using c8
- change kmc-package's 'npm test' to include coverage, leave 'coverage' as alias
- gitignore the lcov files

Fixes #8060
2023-01-20 11:22:44 -06:00
Steven R. Loomis
ac461e97b8 fix(developer): ldml: merge fixes: fix scope of JSZip in the test 🙀
- JSZip() needs to be in an inner function otherwise the single zip grows and grows
- also don't read the kps version when building the kmp

For #7042
2022-12-22 14:31:49 -06:00
Steven R. Loomis
ec9a5952fc fix(developer): ldml: merge fixes: fix scope of JSZip 🙀
- JSZip() needs to be in an inner function otherwise the single zip grows and grows

For #7042
2022-12-22 14:20:39 -06:00
Steven R. Loomis
80b764174a
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-12-22 13:56:08 -06:00
Steven R. Loomis
47a9d16be9 chore(core): updates to kmc-package after merge 🙀
Try to get this to pass tests again

test data:
- put withfolders.qaa.sencoten.model in the right spot
- add kmp.json as it seems to be included
- remove copyLocation: 0 as kmp-compiler drops it

test-package-compiler:
- improve the jszip-contents test
- assume kmp.json is included in the zip
- bug: withfolders.qaa.sencoten.model.js and example.qaa.sencoten.model.js show up in an unrelated KPS. But, include for now.

KmpCompiler
- attempt to pull version from kps.info.version: comments to the contrary, some kps files do not have kmxes.
- map backslash to forward slash so that relative path resolution works

For #7042
2022-12-19 19:50:19 -06:00
Steven R. Loomis
bc14f18460 Merge remote-tracking branch 'origin/master' into chore/core/mergemaster4-epic-ldml 2022-12-15 17:55:13 -06:00
Marc Durdin
2cf6bf3f60 chore(common): update build.sh dependencies for epic-ldml 2022-10-14 10:45:00 +11:00
Marc Durdin
828cbdf584
Merge pull request #7341 from keymanapp/feat/developer/epic-ldml/7277-kmc-makes-keyboard-packages
feat(developer): fixes to kmc-package for keyboard support 🙀
2022-09-24 10:12:38 +10:00
Marc Durdin
cae408edc5 chore: merge feature-ldml into branch 2022-09-21 14:56:45 +10:00
Marc Durdin
f3fbf01d30 feat(developer): fixes to kmc-package for keyboard support
kmc-package had a number of gaps and errors. This fixes a bunch of
problems:

* Wrong case on XML elements
* Wrong metadata in system element
* Missing kmp.json in Files array (needs test in consumer apps later)
* Missing displayFont, oskFont in keyboard elements
* Including copyLocation when == 0
* Including an empty startMenu element
* Incorrect assumptions about location of source files in source/ folder

Note that support for followKeyboardVersion will come later; for now, it
uses the possibly-stale value from the Keyboard element in the .kps
file.
2022-09-21 14:37:08 +10:00
Marc Durdin
bd5f6877bb feat(developer): fixup fixture json 2022-09-21 14:33:04 +10:00
Marc Durdin
2b9a2949a6 feat(developer): kmc-package khmer_angkor fixture 2022-09-21 14:32:42 +10:00
Marc Durdin
e840f04ccf chore(common): make scripts executable and add pre-commit test 2022-09-19 06:39:13 +10:00
Marc Durdin
5a9495c548 chore(developer): fixup tests for kmlmc, kmlmp 2022-09-19 06:37:30 +10:00
Marc Durdin
003f23d7b4 feat(developer): fixup and prepare build scripts and docs for kmc 2022-09-19 06:36:06 +10:00
Marc Durdin
9c623ed872 refactor(developer): split kmlmc into kmc-model, kmc-model-info, kmc-package 2022-09-19 06:30:50 +10:00