Commit graph

521 commits

Author SHA1 Message Date
Marc Durdin
32e92846a3
Merge pull request #11690 from keymanapp/chore/common/move-cldr-import-copy-to-build-step
chore(common): move CLDR import copy into build step for common/web/types
2024-06-06 10:08:39 +07:00
Marc Durdin
93ba57dcd8 chore(common): move CLDR import copy into build step for common/web/types
The CLDR imports are copied into the build/ folder, so it makes more
sense for them to be done during the build phase, particularly as it is
a very quick operation. (Before this, removing the build/ folder did not
trigger a build.sh configure, so subsequent tests and usage will fail
with confusing LDML import errors. An alternative solution would have
been to declare the configure output to be one of these files.)
2024-06-05 10:53:44 +07:00
Marc Durdin
9c204ea071 fix(common): import EventEmitter only instead of all events in parser.js
When importing all of events with:

    import * as events from 'events';

tsc for kmc-ldml (but not for common/web/types) would error with:

    ../../../common/web/types/build/src/deps/xml2js/parser.d.ts:17:25 - error TS2306: File '.../node_modules/@types/node/events.d.ts' is not a module.

    17 import * as events from "node/events.js";

I am not clear why. Changing to the following side-stepped this:

    import { EventEmitter } from 'events';
2024-06-05 10:07:37 +07:00
Marc Durdin
e6d59fc173 chore: move xml2js into the repo to eliminate npm git dependency
Small rewrites to the xml2js code as ES6 modules. Based on
https://github.com/keymanapp/dependency-node-xml2js

Fixes: #8616
2024-06-03 16:07:29 +07:00
Marc Durdin
b874de25e8 chore: replace git dep on restructure with 3.0.1 in npm
3.0.1 includes our fix. However, 3.0.2 does not yet appear to have been
published, which has a patch for our fix?

Fixes: #8616
2024-06-03 14:58:39 +07:00
Marc Durdin
ee0fc0ce14 chore(developer): cleanup npm publish
Use a new flag `--npm-publish` in conjunction with `publish` action,
so that the default will always be to `npm pack` if the new flag is not
specified. This flag is also guarded in the actual npm publish code to
ensure that it can only run in the appropriate CI alpha/beta/stable
environment, and not in local or test.

This then also removes the separate `pack` action.

Also removes legacy boilerplate from a number of build scripts.
2024-05-20 06:34:57 +07:00
Joshua A. Horton
079e68393a fix(common): Chai 5.0 update strictness broke some common/web/types tests 2024-05-08 08:50:18 +07:00
Joshua A. Horton
398b9e0ae2 chore(common): drops package-specific 'chai', '@types/chai' versions
chore(web): missed leaving these out of the rebase
2024-05-07 14:35:09 +07:00
Eberhard Beilharz
1b066de292
chore(web): Improve dependencies 2024-05-06 16:41:06 +02:00
Marc Durdin
cd8121ef65 chore(common): builder scripts now use /resources/build/builder.inc.sh
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).

This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
2024-05-02 15:54:31 +07:00
Marc Durdin
0f390d47cc chore(common): maintenance on build scripts - cd
Fixes #11324.

* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
2024-05-02 15:54:29 +07:00
Steven R. Loomis
f0ff05a9a9 fix(developer): escape plus sign
Fixes: #11233
2024-04-19 13:58:31 -05:00
Marc Durdin
41e16b3c5f chore(developer): use keyboard3 tag rather than DTD to identify LDML keyboard xml files
Relates to #10469. Apply same test in kmc that we are using in TIKE.
2024-04-12 06:49:43 +10:00
Steven R. Loomis
5b1c853557 fix(common,core): support other lookup for modifiers
- the 'other' keyword was incorrectly called 'default'

Fixes: #11072
2024-04-01 23:21:33 -05:00
Steven R. Loomis
5615a5eb4c chore(common): add more references to 0x10000
- at least add comments pointing back to keyman_core_ldml.ts

Fixes: #11072
2024-04-01 12:26:00 -05:00
Steven R. Loomis
5f836d72f0
Merge pull request #11059 from keymanapp/fix/core/11045-assert-marker
fix(core,developer): variable/marker substitution in sets and strings🙀
2024-03-28 00:11:28 -05:00
Steven R. Loomis
baeb3c80fc fix(developer): fix an additional issue on illegal UnicodeSets
- update comments per review comments
- improve logic of checks around malloc
2024-03-26 17:24:56 -05:00
Steven R. Loomis
f2ad7d4e68 fix(common): further improvement to string/set substitution
- tran: check for \uXXXX earlier in the chain, because we emit this format further down
- add test cases for syntax chars being used in variables

Fixes: bug(core): assert when matching markers #11045
Fixes: bug(developer): escape on string substitution if syntax char #11037
2024-03-25 14:52:32 -05:00
Steven R. Loomis
f773066fa0 fix(developer): store set variables in literal form, unescape in tran
- update the tran compiler to call escapeStringForRegex() on set items being substituted into an expression
2024-03-25 12:46:55 -05:00
Steven R. Loomis
67838e60a2 fix(developer,core): fix for marker substitution in sets
- for bug(core): assert when matching markers #11045
- track raw and marker-substituted sets
2024-03-25 12:00:55 -05:00
Steven R. Loomis
18ba46d874 fix(common): fix an additional issue on illegal UnicodeSets
- similar fix to https://github.com/keymanapp/keyman/pull/9492
- again may be due to a wasm version variant
- also additional rangechecking on the input side
2024-03-25 10:26:33 -05:00
Marc Durdin
812e914e87 fix(developer): min Keyman version for LDML is 17.0
Fixes #10959.
2024-03-11 15:15:47 -05:00
Marc Durdin
ffe43d29ef
Merge branch 'beta' into fix/developer/10942-normalize-paths-in-kmc-package 2024-03-07 18:13:43 +07:00
Marc Durdin
7d03010479 fix(developer): normalize all input paths in .kps compiler
Fixes #10942.
2024-03-05 17:34:06 -06:00
Steven R. Loomis
8975a38109 chore(common): ldml update out of techpreview 🙀
- updates to support techpreview -> 45

Fixes: #10900
2024-03-02 22:40:38 -06:00
Marc Durdin
abd5a86196
Merge branch 'beta' into refactor/developer/10207-message-detail 2024-03-01 07:54:31 +07:00
Marc Durdin
4bb9b39ba2 refactor(developer): reorganize messages for adding details
Moved the ERROR_Message line above the Error_Message line for every
message in the compiler message files, as it makes the grouping clearer
once we start adding message details. For example, see the start of the
KmnCompilerMessages class.

Also wraps potentially-undefined parameters to the messages with a new
function `def`, shorthand for `CompilerMessageDef`, which converts the
parameter to '<param>' if it is undefined, which is helpful for
documentation.
2024-02-28 16:00:02 +07:00
Marc Durdin
c7afed7201 refactor(developer): split messages with callstacks into separate generator
Relates to #10207.

In order to include message detail in messages, it is helpful to split
out the exception messages which have a separate parameter, so that
parameter at the same position can be the message detail.

Introduces the CompilerMessageSpecWithException function which takes an
exceptionVar parameter to achieve this differentiation.
2024-02-28 11:34:49 +07:00
Steven R. Loomis
9c2a73a290 chore(common,developer): rename unicodeSet to uset in code
Fixes: rename unicodeSet to uset 🙀  #10657
2024-02-27 18:15:57 -06:00
Marc Durdin
e1a8ce47e3
Merge pull request #10821 from keymanapp/fix/developer/10396-repeated-options-in-kmc
fix(developer): repeated options in kmc must now be fully specified
2024-02-27 16:04:50 +07:00
Marc Durdin
70a50f0f67 refactor(developer): move osk module from common-types to developer-utils
This module is used only by Keyman Developer compiler. It was originally
in common-types because we didn't have a shared module for Developer.

Moving it to @keymanapp/developer-utils in order to improve its
documentation.
2024-02-26 14:57:27 +07:00
Marc Durdin
7b198eb4a4 fix(developer): repeated options in kmc must now be fully specified
Fixes #10396.

Instead of `--message 20ae 5006`, which is ambiguous, because 5006 could
be a filename, you must now specify the parameter option flag each time:
`--message 20ae --message 5006` (or `-m 20ae -m 5006`).
2024-02-23 13:17:11 +07:00
Steven R. Loomis
7cd2e2d4e0 fix(developer): fix for incorrectly-stateful regex
- take out the /g flag for regex used with .exec

For #10734
2024-02-15 15:47:41 -06:00
Steven R. Loomis
8787073c40 feat(developer): errs on \uXXXX escapes 🙀
- note, some older test cases had to be fixed also.

For: #10389
2024-02-13 12:26:46 -06:00
Steven R. Loomis
da4b544471 feat(developer): errs on unparseable regex 🙀
- also, make sure NFD-incompliant ranges get mapped to escaped chars
- also, propagate UnicodeSet errs out

For: #10389
2024-02-12 18:16:34 -06:00
Marc Durdin
44abf2c41e
Merge pull request #10651 from markcsinclair/fix/developer/ensure-project-populate-files-actually-adds-files
fix(developer): ensure project populate files actually adds files
2024-02-07 16:01:56 +11:00
Steven R. Loomis
5360cff8e5 Merge branch 'feat/developer/10554-sometimes-no-norm' into feat/developer/10316-range-warning 2024-02-06 16:14:54 -06:00
Steven R. Loomis
9085a4fcda feat(developer): support normalization=disabled 🙀
- assume normalization is enabled

#10554
2024-02-06 16:11:20 -06:00
Steven R. Loomis
50e0b31e9e feat(developer): support normalization=disabled 🙀
- fix check normalization state

#10554
2024-02-06 14:54:26 -06:00
Dr Mark C. Sinclair
29b6601018 fix(developer): ensure project populate files actually adds files and test 2024-02-06 10:39:47 +00:00
Steven R. Loomis
4b22d3a527
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-02-05 17:17:29 -06:00
Steven R. Loomis
10940be7e0 feat(developer): hint/warn on range check 🙀
- analyzer

#10316
2024-02-02 15:18:08 -06:00
Steven R. Loomis
107e8e98f9 feat(developer): hint/warn on range check 🙀
- analyzer

#10316
2024-02-02 12:09:23 -06:00
Steven R. Loomis
d77ed9099b feat(developer): support normalization=disabled 🙀
#10554
2024-01-31 23:16:51 -06:00
Steven R. Loomis
6c1447bb4b feat(core,developer): simplify markers 🙀
- clean up removing multi-segment processing
2024-01-31 18:17:17 -06:00
Steven R. Loomis
32d2e6ea2c feat(core,developer): simplify markers 🙀
- maybe don't need NFD table

#10317
2024-01-31 18:15:37 -06:00
Steven R. Loomis
1988c43a05 feat(common): updates to marker parsing 🙀
- clear the marker map at the end of remove_markers if no markers were found.
- add some suggested tests from @jahorton and @mcdurdin

For: #10317
2024-01-31 18:04:29 -06:00
Steven R. Loomis
722131dcf3 feat(developer): double markers once again 🙀
- add another greek double reorder.  with chips.

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
1cfeeed3e4 feat(developer): double markers once again 🙀
- fix off by one in adding denormalized glue chars

Fixes: #10516
2024-01-31 17:47:21 -06:00
Steven R. Loomis
75aa04a4ee feat(developer): double markers once again 🙀
- revert comment
- reorder tests

Fixes: #10516
2024-01-31 17:47:21 -06:00