Commit graph

532 commits

Author SHA1 Message Date
Eberhard Beilharz
1389d40965
Merge branch 'master' into change/web/11878_remove-es5 2024-07-31 15:28:38 +02:00
Eberhard Beilharz
e42a895977
fix(web): export consts 2024-07-30 09:47:02 +02:00
Eberhard Beilharz
fd8ffb5efc
chore(web): don't deep import and remove sub-export of util
Addresses code review comments and reverts eb9205f46e and
partially reverts 5552fdd5ad.
2024-07-29 17:43:34 +02:00
Eberhard Beilharz
02170dd062
chore(web): update common/web/types/src/main.ts
Co-authored-by: Joshua Horton <joshua_horton@sil.org>
2024-07-25 12:28:11 +02:00
Eberhard Beilharz
5552fdd5ad
chore(web): expose util.ts as separate sub-export
The exports of the `util.ts` can now be imported as
`@keymanapp/common-types/utils`. Those methods are used by developer.
We still export the `Uni_IsSurrogate*` methods in `main.js` because they
are needed by the keyboard-processor, but exporting everything would
increase the size of the resulting `keymanweb.js`. Doing it this way
keeps the size the same.
2024-07-24 17:33:48 +02:00
Marc Durdin
3497e97b00 feat(developer): automatic version detection for U_xxxx_yyyy ids
Currently, automatic Keyman minimum version determination works for all
features in Keyman keyboards _except_ `U_xxxx_yyyy` touch key
identifiers.

This change adds the automatic keyboard versioning functionality to the
keymanweb compiler as well, following the pattern that's in kmcmplib.

This makes the `store(&version)` line completely optional, and it may
be possible to deprecate it in future versions of Keyman Developer.
2024-07-13 07:28:18 +10:00
Joshua A. Horton
877cd9767d
fix(common): add workaround for divergence of EventEmitter3 from node's version
Node's version has special behavior for events named 'error' - if no handler, it auto-throws.  EventEmitter3 does not include this.
2024-07-04 20:49:31 +02:00
Eberhard Beilharz
afe25eda24
change(web): remove support for es5
Fixes: #11878
2024-07-04 20:49:30 +02:00
Marc Durdin
e081e0c72e fix(developer): check HISTORY.md to get last modified date for keyboard_info and model_info
This updates the calculation of the lastModifiedDate property of the
.keyboard_info and .model_info file. It now looks for the most recent
commit to HISTORY.md, and then falls back to the project folder for
.keyboard_info and .model_info.

getLastGitCommitDate now also returns UTC date (it was giving TZ=0
which is affected by daylight saving), and works for a file rather than
just for a folder, and a corresponding unit test has been added.

The date strings returned will now always include '.000' msec suffix. I
have manually verified that the import into the api.keyman.com database
copes with this, and that this change still conforms to the expected
schema for .keyboard_info and .model_info files.

Last, but important, it turned out that the unit tests for kmc were not
running. This has been resolved in kmc/build.sh.

Fixes: #11793
2024-06-18 07:03:08 +10:00
Joshua A. Horton
d07fa7e953 chore(common): eliminate ts-node ref from most subpackages 2024-06-10 10:07:58 +07:00
Joshua A. Horton
e527317d3a fix(common): remove subpackge entries for older TS version 2024-06-10 09:14:03 +07:00
Marc Durdin
32e92846a3
Merge pull request #11690 from keymanapp/chore/common/move-cldr-import-copy-to-build-step
chore(common): move CLDR import copy into build step for common/web/types
2024-06-06 10:08:39 +07:00
Marc Durdin
93ba57dcd8 chore(common): move CLDR import copy into build step for common/web/types
The CLDR imports are copied into the build/ folder, so it makes more
sense for them to be done during the build phase, particularly as it is
a very quick operation. (Before this, removing the build/ folder did not
trigger a build.sh configure, so subsequent tests and usage will fail
with confusing LDML import errors. An alternative solution would have
been to declare the configure output to be one of these files.)
2024-06-05 10:53:44 +07:00
Marc Durdin
9c204ea071 fix(common): import EventEmitter only instead of all events in parser.js
When importing all of events with:

    import * as events from 'events';

tsc for kmc-ldml (but not for common/web/types) would error with:

    ../../../common/web/types/build/src/deps/xml2js/parser.d.ts:17:25 - error TS2306: File '.../node_modules/@types/node/events.d.ts' is not a module.

    17 import * as events from "node/events.js";

I am not clear why. Changing to the following side-stepped this:

    import { EventEmitter } from 'events';
2024-06-05 10:07:37 +07:00
Marc Durdin
e6d59fc173 chore: move xml2js into the repo to eliminate npm git dependency
Small rewrites to the xml2js code as ES6 modules. Based on
https://github.com/keymanapp/dependency-node-xml2js

Fixes: #8616
2024-06-03 16:07:29 +07:00
Marc Durdin
b874de25e8 chore: replace git dep on restructure with 3.0.1 in npm
3.0.1 includes our fix. However, 3.0.2 does not yet appear to have been
published, which has a patch for our fix?

Fixes: #8616
2024-06-03 14:58:39 +07:00
Marc Durdin
ee0fc0ce14 chore(developer): cleanup npm publish
Use a new flag `--npm-publish` in conjunction with `publish` action,
so that the default will always be to `npm pack` if the new flag is not
specified. This flag is also guarded in the actual npm publish code to
ensure that it can only run in the appropriate CI alpha/beta/stable
environment, and not in local or test.

This then also removes the separate `pack` action.

Also removes legacy boilerplate from a number of build scripts.
2024-05-20 06:34:57 +07:00
Joshua A. Horton
079e68393a fix(common): Chai 5.0 update strictness broke some common/web/types tests 2024-05-08 08:50:18 +07:00
Joshua A. Horton
398b9e0ae2 chore(common): drops package-specific 'chai', '@types/chai' versions
chore(web): missed leaving these out of the rebase
2024-05-07 14:35:09 +07:00
Eberhard Beilharz
1b066de292
chore(web): Improve dependencies 2024-05-06 16:41:06 +02:00
Marc Durdin
cd8121ef65 chore(common): builder scripts now use /resources/build/builder.inc.sh
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).

This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
2024-05-02 15:54:31 +07:00
Marc Durdin
0f390d47cc chore(common): maintenance on build scripts - cd
Fixes #11324.

* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
2024-05-02 15:54:29 +07:00
Steven R. Loomis
f0ff05a9a9 fix(developer): escape plus sign
Fixes: #11233
2024-04-19 13:58:31 -05:00
Marc Durdin
41e16b3c5f chore(developer): use keyboard3 tag rather than DTD to identify LDML keyboard xml files
Relates to #10469. Apply same test in kmc that we are using in TIKE.
2024-04-12 06:49:43 +10:00
Steven R. Loomis
5b1c853557 fix(common,core): support other lookup for modifiers
- the 'other' keyword was incorrectly called 'default'

Fixes: #11072
2024-04-01 23:21:33 -05:00
Steven R. Loomis
5615a5eb4c chore(common): add more references to 0x10000
- at least add comments pointing back to keyman_core_ldml.ts

Fixes: #11072
2024-04-01 12:26:00 -05:00
Steven R. Loomis
5f836d72f0
Merge pull request #11059 from keymanapp/fix/core/11045-assert-marker
fix(core,developer): variable/marker substitution in sets and strings🙀
2024-03-28 00:11:28 -05:00
Steven R. Loomis
baeb3c80fc fix(developer): fix an additional issue on illegal UnicodeSets
- update comments per review comments
- improve logic of checks around malloc
2024-03-26 17:24:56 -05:00
Steven R. Loomis
f2ad7d4e68 fix(common): further improvement to string/set substitution
- tran: check for \uXXXX earlier in the chain, because we emit this format further down
- add test cases for syntax chars being used in variables

Fixes: bug(core): assert when matching markers #11045
Fixes: bug(developer): escape on string substitution if syntax char #11037
2024-03-25 14:52:32 -05:00
Steven R. Loomis
f773066fa0 fix(developer): store set variables in literal form, unescape in tran
- update the tran compiler to call escapeStringForRegex() on set items being substituted into an expression
2024-03-25 12:46:55 -05:00
Steven R. Loomis
67838e60a2 fix(developer,core): fix for marker substitution in sets
- for bug(core): assert when matching markers #11045
- track raw and marker-substituted sets
2024-03-25 12:00:55 -05:00
Steven R. Loomis
18ba46d874 fix(common): fix an additional issue on illegal UnicodeSets
- similar fix to https://github.com/keymanapp/keyman/pull/9492
- again may be due to a wasm version variant
- also additional rangechecking on the input side
2024-03-25 10:26:33 -05:00
Marc Durdin
812e914e87 fix(developer): min Keyman version for LDML is 17.0
Fixes #10959.
2024-03-11 15:15:47 -05:00
Marc Durdin
ffe43d29ef
Merge branch 'beta' into fix/developer/10942-normalize-paths-in-kmc-package 2024-03-07 18:13:43 +07:00
Marc Durdin
7d03010479 fix(developer): normalize all input paths in .kps compiler
Fixes #10942.
2024-03-05 17:34:06 -06:00
Steven R. Loomis
8975a38109 chore(common): ldml update out of techpreview 🙀
- updates to support techpreview -> 45

Fixes: #10900
2024-03-02 22:40:38 -06:00
Marc Durdin
abd5a86196
Merge branch 'beta' into refactor/developer/10207-message-detail 2024-03-01 07:54:31 +07:00
Marc Durdin
4bb9b39ba2 refactor(developer): reorganize messages for adding details
Moved the ERROR_Message line above the Error_Message line for every
message in the compiler message files, as it makes the grouping clearer
once we start adding message details. For example, see the start of the
KmnCompilerMessages class.

Also wraps potentially-undefined parameters to the messages with a new
function `def`, shorthand for `CompilerMessageDef`, which converts the
parameter to '<param>' if it is undefined, which is helpful for
documentation.
2024-02-28 16:00:02 +07:00
Marc Durdin
c7afed7201 refactor(developer): split messages with callstacks into separate generator
Relates to #10207.

In order to include message detail in messages, it is helpful to split
out the exception messages which have a separate parameter, so that
parameter at the same position can be the message detail.

Introduces the CompilerMessageSpecWithException function which takes an
exceptionVar parameter to achieve this differentiation.
2024-02-28 11:34:49 +07:00
Steven R. Loomis
9c2a73a290 chore(common,developer): rename unicodeSet to uset in code
Fixes: rename unicodeSet to uset 🙀  #10657
2024-02-27 18:15:57 -06:00
Marc Durdin
e1a8ce47e3
Merge pull request #10821 from keymanapp/fix/developer/10396-repeated-options-in-kmc
fix(developer): repeated options in kmc must now be fully specified
2024-02-27 16:04:50 +07:00
Marc Durdin
70a50f0f67 refactor(developer): move osk module from common-types to developer-utils
This module is used only by Keyman Developer compiler. It was originally
in common-types because we didn't have a shared module for Developer.

Moving it to @keymanapp/developer-utils in order to improve its
documentation.
2024-02-26 14:57:27 +07:00
Marc Durdin
7b198eb4a4 fix(developer): repeated options in kmc must now be fully specified
Fixes #10396.

Instead of `--message 20ae 5006`, which is ambiguous, because 5006 could
be a filename, you must now specify the parameter option flag each time:
`--message 20ae --message 5006` (or `-m 20ae -m 5006`).
2024-02-23 13:17:11 +07:00
Steven R. Loomis
7cd2e2d4e0 fix(developer): fix for incorrectly-stateful regex
- take out the /g flag for regex used with .exec

For #10734
2024-02-15 15:47:41 -06:00
Steven R. Loomis
8787073c40 feat(developer): errs on \uXXXX escapes 🙀
- note, some older test cases had to be fixed also.

For: #10389
2024-02-13 12:26:46 -06:00
Steven R. Loomis
da4b544471 feat(developer): errs on unparseable regex 🙀
- also, make sure NFD-incompliant ranges get mapped to escaped chars
- also, propagate UnicodeSet errs out

For: #10389
2024-02-12 18:16:34 -06:00
Marc Durdin
44abf2c41e
Merge pull request #10651 from markcsinclair/fix/developer/ensure-project-populate-files-actually-adds-files
fix(developer): ensure project populate files actually adds files
2024-02-07 16:01:56 +11:00
Steven R. Loomis
5360cff8e5 Merge branch 'feat/developer/10554-sometimes-no-norm' into feat/developer/10316-range-warning 2024-02-06 16:14:54 -06:00
Steven R. Loomis
9085a4fcda feat(developer): support normalization=disabled 🙀
- assume normalization is enabled

#10554
2024-02-06 16:11:20 -06:00
Steven R. Loomis
50e0b31e9e feat(developer): support normalization=disabled 🙀
- fix check normalization state

#10554
2024-02-06 14:54:26 -06:00