Commit graph

129 commits

Author SHA1 Message Date
Marc Durdin
2f7407b4b8 feat(developer): warn if .kps file includes a non-binary .kvk file
Relates to #8150.

Legacy .kmp compiler would transform xml-format .kvk files into a binary
.kvk file; now we want that to remain the responsibility of the keyboard
compiler, so we'll warn the few users who are still doing this.
2023-04-19 15:32:29 +07:00
Marc Durdin
66542f6323 feat(developer): use new message infrastructure in kmc-kmn
Moves to using the `reportMessage` callback for reporting all errors in
kmc-kmn.

This required some patching to common/web/types in order to get the new
constants in place and add support for line numbers to `CompilerEvent`,
so at the same time I refactored `compilerErrorSeverityName` into the
module.

After the refactor, coverage tests were failing because they included
the test/ folder, so cleaned that up at the same time.
2023-04-19 10:13:14 +07:00
Marc Durdin
501cf03dc2 chore(common): update description for 'build.sh pack' 2023-04-12 14:18:31 +07:00
Marc Durdin
3265a714f2 chore(common): add missing deps and files for npm publish
The build/ folder for most packages was missing due to being listed in
.gitignore.

Also, @keymanapp/ldml-keyboard-constants package was not being published
despite being a dependency of @keymanapp/kmc-keyboard.

Instead of relying on npm's use of .gitignore / .npmignore (which has
seriously wonky behaviours), we list the files that should be included
in package.json.

NPM Wonky behaviours: it seems that .npmignore causes .gitignore in the
same folder to be ignored. But, higher level .gitignore files still
affect the files included in the package, which makes
specification-by-exclusion painful (negating exclusions, etc). We do not
recommend use of .npmignore anywhere for this reason.
2023-04-12 14:05:05 +07:00
Marc Durdin
10a59f789d fix(common): support npm pack and consolidate npm publishing
Fixes #8586.

All npm publish actions in the repository are now run as part of the
Developer release build, hosted at present by kmc, until we setup a
higher-level build script to do it.

Adds a 'pack' action which mirrors 'publish' for all npm packages.

Removes unnecessary dev dependencies of models-templates and
models-wordbreakers from kmc-model.

Web Release build CI step has been updated to check for presence of
.build-builder.
2023-04-12 11:55:31 +07:00
Marc Durdin
c5d79a9fb4
Merge pull request #7330 from keymanapp/feature-kmcompx
epic: kmcompx - kmcmpdll cross-platform compiler 🚑
2023-04-10 07:34:04 +10:00
Marc Durdin
787b32ba34
Revert "chore(web): merge master into feature-esmodule-web-engine (A17S9 end) 🧩" 2023-03-31 14:17:37 +11:00
Joshua Horton
6df63eddc5
Merge pull request #8549 from keymanapp/chore/master-to-esmodule-merge-a17s9 2023-03-31 08:51:27 +07:00
Marc Durdin
2a871c8165 chore(developer): publish all required npm modules 2023-03-27 11:36:21 +07:00
Steven R. Loomis
93ca0473e5 fix(developer): ldml add ABNT2 🙀
- compiles, at least

For: #8161
2023-03-24 12:49:00 -05:00
Steven R. Loomis
a474d8cbc6 fix(developer): ldml add JIS 🙀
- compiles, at least

For: #8161
2023-03-24 12:13:36 -05:00
Joshua A. Horton
b85087f392 chore(web): Merge branch 'master' into chore/web/update-feature-esmodule-for-configure-fix 2023-03-23 15:15:07 +07:00
Steven R. Loomis
f0cc2b56e6
Merge pull request #8436 from keymanapp/fix/developer-8069/todo2-epic-ldml 2023-03-14 21:51:08 -05:00
Steven R. Loomis
f7e5b114cf
Merge pull request #8412 from keymanapp/chore/resources-7042-cldr-update-202303-epic-ldml 2023-03-14 21:50:17 -05:00
Marc Durdin
409b722eb6
Merge pull request #8426 from keymanapp/fix/common-8419
fix(common): fix broken common/web/types cases 🙀
2023-03-15 11:33:44 +11:00
Steven R. Loomis
f7499238a2 fix(core): ldml more TODO-LDML fixes 🙀
- drop some TODO-LDMLs, document others.

For: #8069
2023-03-14 18:54:28 -05:00
Steven R. Loomis
f92cd36125 chore(developer): ldml update compiler for updated spec 🙀
#7042
2023-03-14 17:38:30 -05:00
Steven R. Loomis
5c25bf429f fix(common): fix broken common/web/types cases
Fixes: #8419
2023-03-14 17:34:37 -05:00
Joshua Horton
10518dd62e
Merge pull request #8429 from keymanapp/chore/common/ldml-ts-and-node-updates
chore(common): TS updates to non-sync'd packages, feature-esmodule merge conflict prevention
2023-03-14 12:06:15 +07:00
Joshua A. Horton
f7d082981c chore: Merge branch 'master' into chore/feature-esmodule-master-update 2023-03-14 11:51:57 +07:00
Joshua A. Horton
f5e64a10a5 chore: Merge branch 'chore/common/ldml-ts-and-node-updates' into chore/feature-esmodule-master-update 2023-03-14 09:23:39 +07:00
Joshua A. Horton
2439ba4e97 fix(developer): updates kmc, web/types module resolution, adds missing type import/exports 2023-03-14 08:41:29 +07:00
Joshua A. Horton
0da7ac24be chore(common): updates TS, sets export map for keyman-version 2023-03-14 08:22:26 +07:00
Marc Durdin
92ed99c4d6 chore(common): build script performance improvements
Improves build script performance by:
* using built-ins wherever possible (e.g. string splitting)
* eliminating redundant code
* using absolute (to $KEYMAN_ROOT) rather than relative paths to avoid realpath
* removing unnecessary `npm run` calls

BEFORE                         | AFTER
-------------------------------|----------------------
time ./core/build.sh --help    |
real    0m2.116s               | real    0m0.874s
user    0m0.578s               | user    0m0.198s
sys     0m0.984s               | sys     0m0.289s
-------------------------------|----------------------
time ./web/build.sh --help     |
real    0m3.523s               | real    0m0.757s
user    0m1.166s               | user    0m0.320s
sys     0m2.273s               | sys     0m0.455s
-------------------------------|----------------------
time ./web/build.sh -d         |
real    1m34.750s              | real    0m59.974s
user    0m9.721s               | user    0m6.284s
sys     0m19.652s              | sys     0m13.202s

@keymanapp-test-bot skip
2023-03-13 07:18:09 +07:00
Marc Durdin
be4f036a3c feat(developer): Add support for building a folder to kmc
Relates to #2761.

kmc now supports passing in a folder and building all Keyman source
files found in the folder, even if a .kpj is not present in the folder.

If a .kpj file exists, `KeymanDeveloperProject/Options/Version` will be
checked to determine behaviour. If this field is missing or "1.0", then
the classical behaviour persists.

If the Version field is "2.0", the `Files` key is ignored, and .kmn,
.xml, .model.ts, and .kps files under `sourcePath` are enumerated and
added to the in-memory project for build.

If a .kpj is not present in the folder, a virtual project is constructed
in memory with the following defaults:

* `version`: `2.0`
* `buildPath`: `$PROJECTPATH/build`
* `sourcePath`: `$PROJECTPATH/source`
* `compilerWarningsAsErrors`: `false`
* `warnDeprecatedCode`: `true`
* `checkFilenameConventions`: `true`

It is still considered an error to include both a keyboard and a model
in the same file, but a version 2.0 project will not include the field
determining its type in `Options/ProjectType`; this will instead be
checked during build.

The call to `kmc` can reference a folder or a .kpj file. The .kpj file
must have the same basename as the folder name (e.g.
`khmer_angkor/khmer_angkor.kpj`).

WARNING: this does not add support for 2.0 projects to Keyman Developer;
it is likely that opening a version 2.0 project in Developer will
currently corrupt it.

WARNING: it is not anticipated that kmcomp.exe will be updated to
support version 2.0 projects -- rather kmcomp.exe will be removed before
release of v17, and replaced with kmc.
2023-02-27 13:11:22 +07:00
Marc Durdin
b04489a47b feat(developer): kmc .kpj support
Adds support for building .kpj projects. Required minor tweak to kmcomp
to allow it to build both .kmx and .js from .kmn command line.
Currently, calls kmcomp.exe to build .kmn files, until we support a
cross-platform kmc-kmn C++ project.
2023-02-27 06:14:06 +07:00
Marc Durdin
35c706576b chore(common): export kpj from module 2023-02-26 06:02:43 +07:00
Marc Durdin
647803ef28 feat(common): add .kpj typescript reader and schema 2023-02-25 11:17:39 +07:00
Marc Durdin
a2ceb98948 chore: rewrite extra standard build script prolog 2023-02-23 08:09:41 +07:00
Marc Durdin
a0ecb30a9c
Merge pull request #8265 from keymanapp/feat/developer/epic-ldml/7238-touch-layout-compiler
feat(developer): touch layout compiler for LDML 🙀
2023-02-23 08:47:46 +11:00
Steven R. Loomis
e1e5eea469 feat(core): ldml rename key2 to keys 🙀
- So in summary, since before #7532 basically keys was restructured, and the old 'keys' was moved into 'keys.kmap'.

#8069
2023-02-20 17:40:34 -06:00
Marc Durdin
88df443ba1 feat(developer): touch layout compiler [unfinished] 2023-02-20 10:18:01 +07:00
Steven R. Loomis
83adfdd1c6
Update common/web/types/src/kmx/kmx-plus-builder/build-key2.ts
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2023-02-16 21:43:45 -06:00
Steven R. Loomis
8ea87aa828 feat(common): ldml: merge keys into key2 🙀 2023-02-16 12:29:19 -06:00
Steven R. Loomis
dd76a1fc36 feat(developer): add -T option to kmc 🙀
- -T converts testdata.xml into testdata.json

#7535
2023-02-15 17:58:52 -06:00
Steven R. Loomis
228cdd6281 feat(common): ldml read keyboard testdata 🙀
- full data test

#7535
2023-02-15 17:58:52 -06:00
Steven R. Loomis
c2556acc4a feat(common): ldml read keyboard testdata 🙀
- full data read

#7535
2023-02-15 17:58:52 -06:00
Steven R. Loomis
9a31e866df feat(common): ldml read keyboard testdata 🙀
- early scaffolding
- with some basic tests

#7535
2023-02-15 17:58:52 -06:00
Steven R. Loomis
ac2366f460 feat(core): ldml C++ updates to modifiers 🙀
- spec: change caps value to 0x100 #7533
- add a 'mod_all'  for validating valid modifier keys, plus tests
- add asserts on C++ side mapping modifiers to keyman constants
- use 'mod_all' to validate modifiers
- further fix modifier -> mod in binary

For: #7985 and related to #7533
2023-02-14 09:23:03 -06:00
Steven R. Loomis
7983fd9609 fix(developer): ldml spec: layr…mod needs to be a bitfield 🙀
- layr.layers.layer.mod needs to be a bit field just as keys.key.mod

Related to #7533
2023-02-14 09:23:03 -06:00
Steven R. Loomis
7c02ab85af feat(core): ldml C++ feature support 🙀
- make ldml.cpp print FAILED on failure especially because of https://github.com/microsoft/vscode-cpptools/issues/487
- add vkey validation (#7135)
- add disp validation
- fix: a latent bug in disp! disp.count was === vkey.count, oops. Fix and test cases.

For: #7985
2023-02-14 09:23:03 -06:00
Steven R. Loomis
a8832ea893
Merge pull request #8172 from keymanapp/feat/developer/7965-morehardware-epic-ldml
feat(common): ldml different hardware support 🙀
2023-02-07 17:06:35 -06:00
Steven R. Loomis
9d6289d420
Merge pull request #8171 from keymanapp/feat/core/7533-modifier-epic-ldml
feat(common): ldml modifier support 🙀
2023-02-07 17:05:55 -06:00
Steven R. Loomis
75fb36b9e0
Merge pull request #8143 from keymanapp/fix/common/7534-more-import-errs-epic-ldml
feat(common): ldml: import error messages from the compiler   🙀
2023-02-06 11:25:20 -06:00
Steven R. Loomis
c4f7f0eb12
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-02-06 11:24:24 -06:00
Steven R. Loomis
f74a419bc6 chore: Review comment 2023-02-03 19:03:26 -06:00
Steven R. Loomis
41f77a874c feat(common): ldml different hardware support 🙀
- add HardwareToKeymap in common/web/types
- add a mt-iso keyboard, very close to stock mt.xml
- add machinery for parsing modifier key sequences
- more test machinery
- add assertions against invalid modifier sequences

Fixes: #7965
2023-02-03 19:00:57 -06:00
Steven R. Loomis
af10312f84 feat(common): ldml modifier support 🙀
- spec: update spec for layr.mod field
- core/include/ldm: update constants
- bitfield modelled after s_modifier_names[]  / LCTRLFLAG etc.
- new test case k_004_tinyshift, also update k_010_mt
- update const builder to ignore all *_map properties
- split utility out to utils, add test

For: #7533
(cherry picked from commit 0346bea488)
2023-02-03 18:54:22 -06:00
Steven R. Loomis
6f51be0684 feat(common): ldml: unit tests for import error messages from the compiler 🙀
- test fixup

For: #7534
2023-02-03 18:54:07 -06:00
Steven R. Loomis
7ffd2b94f5 feat(common): ldml: updates to xml loading 🙀
per review comments

- exit-early on import errs

For: #7534
2023-02-03 18:43:06 -06:00