Commit graph

316 commits

Author SHA1 Message Date
Marc Durdin
df212364a2 maint(resources): add repository record for all published packages
While not documented as such, it appears that the repository record is
required with trusted publishing of npm packages.

Test-bot: skip
Build-bot: skip
2025-10-29 14:35:31 +01:00
Marc Durdin
ba411774f6 maint(resources): move NPM package publishing to GitHub Actions
Due to recent changes in NPM package publishing security requirements,
we have to move from TeamCity build to a GitHub Action to publish our
NPM packages, so we can take advantage of trusted publishing. This
change also consolidates and centralizes the npm publishing into
resources/build/ci/npm-publish.sh, which removes a lot of boilerplate
from each of the build.sh scripts, and ensures consistency.

Packages will be `npm pack`ed on PR and test builds, and published in
release builds.

Ref: https://docs.npmjs.com/trusted-publishers
Ref: https://github.blog/changelog/2025-09-29-strengthening-npm-security-important-changes-to-authentication-and-token-management/
Fixes: #14963
Test-bot: skip
Build-bot: release:developer
2025-10-27 16:31:55 +01:00
Marc Durdin
c6a96fc457 fix(developer): handle missing begin Unicode in KMW compiler
This probably arose with the move to kmcmplib in WASM, and as it was
never unit tested, we missed it. A simple signed vs unsigned issue.

Fixes: #14411
Test-bot: skip
2025-10-23 15:08:12 +02:00
Marc Durdin
bd3d268bcc fix(developer): address review comments, adding comments and renaming variables 2025-09-12 17:01:21 +02:00
Marc Durdin
7fcdfb8f65 fix(developer): restrict invalid characters in identifiers in kmcmplib
The compiler has always been very ambiguous on which characters were
accepted in group and store names, even to the point of accepting
things like comma in a store name, which would then make it impossible
to reference in an `index` statement!

This commit clarifies the allowable characters in an identifier. While
it would have been possible to use UAX#31 for this, that would have
extended the requirements for this change substantially, and may have
caused us more trouble with legacy keyboards. Given kmcmplib is
end-of-life (see epic/ng-compiler), I have chosen a lower friction
approach. There are certainly other characters that could be excluded,
but in general I have chosen to exclude only those that will definitely
be problematic.

The set of allowable characters for deadkeys has actually been expanded
in this release to match the store and group name rules.

It is expected that there may be some impacted keyboards, but addressing
this change will be relatively straightforward, so I consider this to be
an acceptable back-compatibility trade-off, see
https://github.com/keymanapp/keyman/wiki/Principles-of-Keyman-Code-Changes#4-source-backward-compatibility-keyboard-model-and-package-source-file-formats-should-be-backward-compatible

Fixes: #14604
Test-bot: skip
Build-bot: skip build:developer
2025-09-12 10:52:31 +02:00
Marc Durdin
f3fb3ac87b maint(common): move build-utils-ci.inc.sh to ci/ci-publish.inc.sh
Removes a number of unused functions from the file, renames remaining
functions to match our builder script prefix patterns, removes unused
imports from various build scripts (e.g. Android).

Fixes: #14453
2025-08-05 12:54:56 +10:00
Marc Durdin
acefdf281a maint(common): move typescript script functions into typescript.inc.sh
Fixes: #14448
Build-bot: build
Test-bot: skip
2025-08-03 07:01:57 +10:00
Marc Durdin
2726050601 maint(common): rename and move node-related script functions into node.inc.sh
Consolidates the node-related script functions into node.inc.sh, as part
of cleaning up the build scripts and making them easier to maintain into
the future.

Fixes: #14447
2025-08-02 13:34:29 +10:00
Marc Durdin
f394245636 maint(common): consolidate builder scripts
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.

Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh

Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
  android/build.sh.
* Renames functions in builder-basic.inc.sh

More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.

Fixes: #14065
Build-bot: build all
Test-bot: skip
2025-08-02 08:11:24 +10:00
Marc Durdin
65158a0526 maint(common): rename shellHelperFunctions.sh to utils.inc.sh
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.

Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
2025-08-02 06:58:40 +10:00
Steven R. Loomis
0a35181bed chore(developer): rename x: to compileContext:
Fixes: #14117
2025-06-26 15:58:43 -05:00
Steven R. Loomis
28ba904b37 chore(developer): rename ObjectWithMetadata to ObjectWithCompileContext
Fixes: #14117
2025-06-26 15:56:52 -05:00
Steven R. Loomis
fab29aad77 feat(developer): ldml: update errors for unparseable reorder
Fixes: #14091
2025-06-02 08:52:00 -05:00
Joshua Horton
155c1a2b12 maint: update mocha
Some of the npm audit warnings left after #13897 require an updated `mocha` version to remedy.  As we only really use `mocha` to do unit-testing, it should be safe to update if all of our existing tests pass with the new version.

Test-bot: skip
2025-05-13 14:02:02 +07:00
Joshua Horton
da7bfc0982 change(developer): clean up developer/src/kmc-kmn linter issues 2025-04-30 09:18:45 +07:00
Marc Durdin
9080d156dc chore(developer): use 'statement' instead of 'command' for messages about kmn statements
In compiler messages, for a long time we have inconsistently used
'command' instead of 'statement' when referring to kmn statements. The
kmn language documentation uses 'statement' fairly consistently.
2025-03-22 05:58:06 +07:00
Marc Durdin
5683b370fe fix(developer): add validation of nul usage in output part of rule
If `nul` is used in the output part of a rule, then text or
text-emitting statements cannot also be included in the output of that
rule. This was not enforced in earlier versions of Keyman.

Adds a unit test for the enhanced validation.

Fixes: #13455
Relates-to: keymanapp/keyboards#3379
2025-03-18 09:54:06 +07:00
Marc Durdin
a04e977749 fix(developer): include nul in offset calculations for kmw v10 compiler
The cached context comparison for v10 kmw compiler includes the `nul`
statement, unlike the comparison in the earlier compiler. This
difference was missed in earlier iterations including the recent patch
in #13003. This patch fixes the problem for both `context(n)` and
`index(store,n)` offset calculations.

Fixes: #13306
Fixes: #13307
Follows: #13003
2025-02-23 16:25:51 +01:00
Marc Durdin
4a797d0e5a fix(developer): verify context(n) offsets are valid in kmcmplib
Add validity checks to kmcmplib to verify that offsets provided to
`context(n)` are in range for the context, and do not point to
non-character elements (`nul` and `if`) in the context string.

Fixes: #13301
Fixes: #13302
Relates-to: #13299
Relates-to: #13276
2025-02-21 09:58:26 +07:00
Marc Durdin
bd4633333b feat(developer): improve compiler messages and user interface
* Add unit test to verify the format of message detail strings
  (max line length of 80 chars)
* Add unit test to verify non-null, non-empty message strings
* Tidy up a number of messages (to pass unit tests)
* Add message details for all kmw compiler messages
* Add a clickable link for the TIKE message window for each message
  so that explanations and further message details are more accessible
  to authors (`kmc message` is already available for command-line users)
* Fixes a number of minor issues with the messages window in TIKE,
  mostly relating to when popup menu commands are enabled and how
  messages are selected.

Fixes: #13134
2025-02-07 08:46:46 +07:00
Marc Durdin
71ce8d6c91 chore(developer): extend notany tests in kmc-kmn 2025-01-24 08:56:21 +07:00
Marc Durdin
4a99018f1b fix(developer): produce correct index parameter value for context in kmw compiler
The KMW compiler would produce incorrect indices for the corresponding
character in the context after adjusting for the presence of `nul` and
`if` statements at the start of the context string, when using the
`context` and `context(n)` statements in the output.

This fix addresses the offset calculation for those scenarios and adds a
unit test to verify that the new offsets are correct.

Fixes: #12980
2025-01-23 14:20:31 +07:00
Marc Durdin
74f4392d75 fix(developer): detect invalid key ids in touch layout files
Fixes: #12870
2025-01-15 09:59:50 +07:00
Marc Durdin
4202f46550 fix(developer): support hint property in displaymap
Fixes: #12795
2024-12-09 12:20:53 +07:00
Marc Durdin
cd8825d504
Merge pull request #12736 from keymanapp/refactor/developer/unify_test_action
refactor(developer): unify test action
2024-12-01 00:14:11 +01:00
Eberhard Beilharz
66404f9503
chore(developer): rename test files
This change rename the test files for Developer according to the
discussion at the Keyman conference in November 2024.
2024-11-28 14:32:09 +01:00
Eberhard Beilharz
badb23a6ab
Merge pull request #12710 from keymanapp/refactor/developer/tc_tests
refactor(developer): output number of tests when running on TC
2024-11-28 19:40:52 +07:00
Marc Durdin
70a32f00d7 refactor(developer): unify test action 2024-11-28 15:16:25 +07:00
Eberhard Beilharz
21736d8817
refactor(developer): output number of tests when running on TC
This change adds the mocha-teamcity-reporter which outputs the running
tests in a special format that TeamCity can interpret. This allows TC
to show which tests run as well as the number of tests run.

Also refactor some of the build scripts to use functions.

Note that we had to explicitly add `--check-coverage=false` (which is the
default) to get successful builds when we run on TC.
2024-11-27 11:41:06 +01:00
Marc Durdin
e388361e84
Merge pull request #12713 from keymanapp/refactor/common/LdmlKeyboardTypes
refactor(common): move remaining LDML keyboard types into `LdmlKeyboardTypes`
2024-11-27 07:21:06 +01:00
Marc Durdin
b54b005830 refactor(common): move remaining LDML keyboard types into LdmlKeyboardTypes
Follows: #12712
2024-11-27 11:52:53 +07:00
Dr Mark C. Sinclair
c6ef37fdef chore(developer): remove ERROR_XXXInVirtualKeySection from kmn-compiler-messages.ts 2024-11-26 06:57:44 +00:00
Marc Durdin
78999fb7bc
Merge pull request #12588 from keymanapp/feat/developer/12505-report-key-address-in-layout-compiler
feat(developer): Report key 'address' in validation failures in layout compiler
2024-11-07 07:06:51 +01:00
Marc Durdin
6002024525 chore(developer): add address to Error_TouchLayoutInvalidIdentifier message 2024-11-07 07:27:02 +07:00
Marc Durdin
55e3f73a36 fix(developer): handle missing files in kmc-kmn
If a file is not found, the loadfile callback now returns an empty array
rather than null, which kmcmplib interprets as a missing file. This
means a zero-byte file will have the same outcome but as it is also
generally invalid, I think that is acceptable.

Fixes: #12546
Fixes: KEYMAN-DEVELOPER-292
2024-10-31 09:08:44 +07:00
Marc Durdin
91fc7f99bd chore(developer): fix import path 2024-10-30 13:11:42 +07:00
Marc Durdin
a38f46040e feat(developer): Report key 'address' in validation failures in layout compiler
All messages that reference keys in the touch layout will now give the
row+column index of the key with the issue (and longpress/flick/multitap
index as well if relevant).

Fixes: #12505
2024-10-30 12:34:15 +07:00
Eberhard Beilharz
1fe6a0370f
refactor(web): move common/web/eslintcommon/tools/eslint
Fixes: #12386
2024-09-10 19:16:52 +02:00
Marc Durdin
86fd36a713
Merge pull request #12117 from keymanapp/chore/developer/build-ci-linux-mac
chore(developer): start building Developer on linux,mac
2024-08-09 18:57:38 +10:00
Marc Durdin
38bb33d744 chore(developer): ignore line endings in kmw unit test 2024-08-07 12:32:01 +07:00
Marc Durdin
32b4711659
Merge branch 'master' into refactor/common/9665-move-compiler-interfaces-to-developer 2024-08-07 15:15:45 +10:00
Marc Durdin
ed143fb333
Merge branch 'master' into refactor/common/9665-move-ldml-keyboard-source-file-to-developer 2024-08-07 15:15:04 +10:00
Marc Durdin
ec72245575
Merge pull request #11765 from keymanapp/refactor/common/9665-move-keyman-touch-layout-file-to-developer
refactor(common): move .keyman-touch-layout reader/writer to @keymanapp/developer-utils 🐉
2024-08-07 15:14:06 +10:00
Marc Durdin
e9d054ab2c
Merge branch 'master' into refactor/common/9665-move-kvk-file-to-developer 2024-08-07 15:13:41 +10:00
Marc Durdin
e67aed1606 refactor(common): move compiler-interfaces to @keymanapp/developer-utils
This should be the last step in the refactoring.

Fixes: #9665
2024-08-02 13:49:59 +07:00
Marc Durdin
d3d3ac1eec refactor(common): move LDML keyboard .xml reader/writer and kmx-plus builder to @keymanapp/developer-utils
Relates-to: #9665
2024-08-02 10:23:11 +07:00
Marc Durdin
924b1aad78 refactor(common): use developer-utils, not common-types for TouchLayoutFileReader/Writer 2024-08-01 12:38:28 +07:00
Marc Durdin
53551f5662
Merge branch 'refactor/common/9665-move-kps-file-type-to-developer' into refactor/common/9665-move-kvk-file-to-developer 2024-08-01 10:35:33 +10:00
Marc Durdin
d8986be12f refactor(common): merge master into branch 2024-08-01 07:07:07 +07:00
Marc Durdin
8cd97e5f5d refactor(developer): move kmcmplib message construction to kmc-kmn
* Remove compiler message definitions from kmcmplib
* Add parameterization to compiler message structures
* Translate parameters for existing parameterized messages (except for
  `ERROR_InvalidToken`, which requires a bigger refactor of
  `GetXStringImpl()` and many friends)
* Add columnNumber to message structures (not yet used in kmc-kmn)
* Add filename to message structures (not yet used in kmcmplib)
* Rename `INFO_Info` to `INFO_MinimumCoreEngineVersion` and
  `INFO_MinimumEngineVersion` to `INFO_MinimumWebEngineVersion`

Relates-to: #10866
2024-07-30 10:01:19 +07:00