Commit graph

216 commits

Author SHA1 Message Date
Marc Durdin
ad9ba1f188
Merge branch 'refactor/common/9665-move-project-file-type-to-developer' into refactor/common/9665-move-kps-file-type-to-developer 2024-08-01 10:34:05 +10:00
Marc Durdin
d8986be12f refactor(common): merge master into branch 2024-08-01 07:07:07 +07:00
Marc Durdin
d1e351a1ac
Revert "chore(developer): remove redundant references from tsconfig.json" 2024-08-01 08:57:27 +10:00
Marc Durdin
1cff1a5084 chore(developer): remove redundant references from tsconfig.json
With the full move to ES Modules, we no longer need to include
`references` in tsconfig.json, as we can rely on package.json and
build.sh dependency management. Note however that `tsc -b` may not work
to build dependencies -- they need to be built using `build.sh`, which
calculates which dependencies need building.

kmc-keyboard-info was missing a dependency link to kmc-package in
build.sh, correcting this at the same time.

Relates-to: #12027
2024-07-27 05:44:22 +07:00
Marc Durdin
1afa4beb01 fix(developer): remove paths from tsconfig.json
Fixes: #12027
2024-07-26 06:59:45 +07:00
Marc Durdin
1b2ca768b7 chore(developer): rename kmc-package CompilerMessages to PackageCompilerMessages
It is easier to differentiate between all the compiler message classes
if each one has its own name.
2024-07-22 16:02:43 +07:00
Marc Durdin
b0ff102b45
Merge pull request #11735 from keymanapp/fix/developer/11362-verify-email-addresses-in-package
fix(developer): verify email addresses in .kps and .keyboard_info
2024-06-13 10:04:00 +10:00
Marc Durdin
de9ab217e9 refactor(common): move kps-file.ts to @keymanapp/developer-utils
Part of #9665.
2024-06-13 08:57:49 +10:00
Joshua A. Horton
d07fa7e953 chore(common): eliminate ts-node ref from most subpackages 2024-06-10 10:07:58 +07:00
Joshua A. Horton
e527317d3a fix(common): remove subpackge entries for older TS version 2024-06-10 09:14:03 +07:00
Marc Durdin
2506ff7c01 fix(developer): verify email addresses in .kps and .keyboard_info
Add verification of email address formats to kmc-package,
kmc-keyboard-info, and kmc-model-info. Add corresponding unit tests for
kmc-package and kmc-keyboard-info (kmc-model-info unit tests will be a
later addition). Uses AJV's email validation regex, as we rely on AJV
for schema compliance tests.

Fixes: #11362
Fixes: KEYMAN-DEVELOPER-1XE
2024-06-07 13:30:21 +07:00
Marc Durdin
e6d59fc173 chore: move xml2js into the repo to eliminate npm git dependency
Small rewrites to the xml2js code as ES6 modules. Based on
https://github.com/keymanapp/dependency-node-xml2js

Fixes: #8616
2024-06-03 16:07:29 +07:00
Marc Durdin
8efba7186e fix(developer): handle missing data in .kps <Keyboard>
Handles when `<Version/>`, `<Name/>`, or `<ID/>` elements are missing
from `<Keyboard>` elements in the .kps file in kmc-package. Adds a unit
test.

Fixes: #11562
Fixes: KEYMAN-DEVELOPER-1YY
2024-05-27 15:04:53 +07:00
Marc Durdin
ee0fc0ce14 chore(developer): cleanup npm publish
Use a new flag `--npm-publish` in conjunction with `publish` action,
so that the default will always be to `npm pack` if the new flag is not
specified. This flag is also guarded in the actual npm publish code to
ensure that it can only run in the appropriate CI alpha/beta/stable
environment, and not in local or test.

This then also removes the separate `pack` action.

Also removes legacy boilerplate from a number of build scripts.
2024-05-20 06:34:57 +07:00
Marc Durdin
c8a1c6aa39 chore(developer): use builder.inc.sh instead of build-utils.sh in build scripts
also, remove old Makefiles
2024-05-20 06:34:46 +07:00
Joshua Horton
887a47930f
Merge pull request #11318 from keymanapp/chore/web/chai-update
chore(web): updates chai, @types/chai dependencies 🏃
2024-05-15 14:10:27 +07:00
Darcy Wong
cbf1fd5cb1 chore(common): Merge beta to master for sprint A18S1 part 2 2024-05-10 06:27:02 +07:00
Joshua A. Horton
398b9e0ae2 chore(common): drops package-specific 'chai', '@types/chai' versions
chore(web): missed leaving these out of the rebase
2024-05-07 14:35:09 +07:00
Marc Durdin
4fc94a9e70 fix(developer): handle missing Description element for File element in package compiler
Fixes #11353.
2024-05-03 14:31:10 +07:00
Marc Durdin
78902e3891 fix(developer): handle missing Name element for File element in package compiler
Fixes #11351.
2024-05-03 14:25:35 +07:00
Marc Durdin
cd8121ef65 chore(common): builder scripts now use /resources/build/builder.inc.sh
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).

This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
2024-05-02 15:54:31 +07:00
Marc Durdin
0f390d47cc chore(common): maintenance on build scripts - cd
Fixes #11324.

* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
2024-05-02 15:54:29 +07:00
Marc Durdin
ffe43d29ef
Merge branch 'beta' into fix/developer/10942-normalize-paths-in-kmc-package 2024-03-07 18:13:43 +07:00
Marc Durdin
7d03010479 fix(developer): normalize all input paths in .kps compiler
Fixes #10942.
2024-03-05 17:34:06 -06:00
Marc Durdin
80a0fea025 docs(developer): kmc-package api documentation
Note: WindowsPackageInstallerSources fields are not yet documented.
2024-03-04 19:15:37 +07:00
Marc Durdin
4bb9b39ba2 refactor(developer): reorganize messages for adding details
Moved the ERROR_Message line above the Error_Message line for every
message in the compiler message files, as it makes the grouping clearer
once we start adding message details. For example, see the start of the
KmnCompilerMessages class.

Also wraps potentially-undefined parameters to the messages with a new
function `def`, shorthand for `CompilerMessageDef`, which converts the
parameter to '<param>' if it is undefined, which is helpful for
documentation.
2024-02-28 16:00:02 +07:00
Marc Durdin
c7afed7201 refactor(developer): split messages with callstacks into separate generator
Relates to #10207.

In order to include message detail in messages, it is helpful to split
out the exception messages which have a separate parameter, so that
parameter at the same position can be the message detail.

Introduces the CompilerMessageSpecWithException function which takes an
exceptionVar parameter to achieve this differentiation.
2024-02-28 11:34:49 +07:00
Marc Durdin
602b519057 docs(developer): npm package readme files 2024-02-26 14:00:51 +07:00
Marc Durdin
a8d0d8454c feat(developer): add api documentation to build
Relates to #10207.

Adds the `api` action to build.sh for each affected module. The `api`
action for each module will generate the api metadata files, and then
the markdown files are generated by /developer/build.sh api.

Note that developer-utils, kmc, and kmcmplib have no api documentation
at this time (and kmcmplib is not Typescript so would need separate
tooling anyway).
2024-02-26 11:45:36 +07:00
Marc Durdin
661069c0c3 feat(developer): add api-extractor and api-documenter and config
Relates to #10207.

Adds @microsoft/api-extractor and @microsoft/api-documenter NPM
packages for validating and generating API documentation from Typescript
modules.

Adds basic configuration for each module that we will be publishing at
this time.

api-extractor's output will be emitted to /developer/docs/api/etc/, for
future release comparisons. A baseline output will be committed
separately.

The build outputs will be in /developer/build/docs and the intermediate
files will be in /developer/build/api.

Note that at this time, we will not enable the checks and validations.
That is a task for alpha rather than beta. The primary intent at this
time is to select a solid, well-supported toolchain for generating the
API documentation for kmc.
2024-02-26 11:37:32 +07:00
Marc Durdin
bfc35e9272 feat(developer): common Messages exports from kmc modules
Relates to #10150. Makes the various Messages classes in kmc modules
accessible to other modules. Some file renames involved to reduce
confusion.

No functional changes.

Fixes kmc-analyze metadata.

See #10254 for a related future fix.
2023-12-14 09:40:18 +07:00
Marc Durdin
e2ce64d9be fix(web): mocha describe does not do async
mocha describe() does not accept an async function. Any async prep
should be done in a before() function, which does support async.

One helpful ref: https://github.com/mochajs/mocha/issues/2975#issuecomment-1004176440
2023-12-13 05:38:51 +07:00
Marc Durdin
69a2448176 feat(developer): PackageCompiler and WindowPackageInstallerCompiler now implement KeymanCompiler
Relates to #9473.

Refactors the public API of PackageCompiler and
WindowsPackageInstallerCompiler to meet KeymanCompiler, including
asyncing a bunch of functions, and moving file write responsibilities
into the classes themselves.

The classes were updated together in a single commit because
WindowsPackageInstallerCompiler depends on PackageCompiler, and needed
refactoring for the updated API access there anyway.

Most of the test cases needed only minor patching, but introducing the
`async init()` function has caused them to all be async. The
test-messages module has a slight functional change with the compile
process running completely rather than stopping on first message, which
meant we needed to add extra dummy fixtures to avoid other errors that
arose later in the compile process. The keyman.exe fixture is a text
file, not a Windows executable.
2023-12-09 06:48:53 +07:00
Marc Durdin
e5da0aacc2 chore(developer): fixup test fixture 2023-11-25 05:16:03 +10:00
Marc Durdin
b3b2825168 fix(developer): path separator for kmc-package
Fixes #10027.
2023-11-23 16:45:12 +10:00
Marc Durdin
c5c5b35415 chore(common): remove prepublish step from package.json
Given we've just run a build, this (a) seems unnecessary, and (b) seems
to go wrong in some circumstances anyway, e.g.
https://build.palaso.org/buildConfiguration/Keyman_Developer_Release/421326?buildTab=log&focusLine=19936&linesState=18839&logView=flowAware:

```
07:01:29   > @keymanapp/kmc@17.0.205-alpha prepublishOnly
07:01:29   > npm run build
07:01:29
07:01:31
07:01:31   > @keymanapp/kmc@17.0.205-alpha build
07:01:31   > tsc -b
07:01:31
07:01:35   ../../../common/web/types/build/src/kmx/kmx.d.ts(1,1): error TS1036: Statements are not allowed in ambient contexts.
07:01:35   ../../../common/web/types/build/src/kmx/kmx.d.ts(1,2): error TS1345: An expression of type 'void' cannot be tested for truthiness.
...
```
2023-11-04 07:34:18 +07:00
Marc Durdin
45b5db1974
Merge pull request #9844 from keymanapp/chore/developer/verify-kmp-json-output-from-kmc-package
chore(developer): verify kmp.json output from kmc-package
2023-11-02 14:50:50 +11:00
Marc Durdin
d92bd1e70a chore(developer): handle failures to build kmp.json in kmc-package 2023-10-26 09:32:26 +07:00
Marc Durdin
cdac2dd01b chore(developer): verify kmp.json output from kmc-package 2023-10-25 12:07:46 +07:00
Marc Durdin
234add1b38 fix(developer): handle xml errors in package compiler
Fixes #9797.
2023-10-22 07:36:45 +07:00
Marc Durdin
a4652cebb9
Merge pull request #9716 from keymanapp/chore/update-kmp-compiler-xml2js-dependency
chore(developer): Update kmc-package xml2js dependency 🎺
2023-10-11 11:19:24 +11:00
Marc Durdin
98af6052e4
Merge pull request #9714 from keymanapp/chore/common/9713-avoid-xml2js-firstCharLowerCase
chore(common): stop using xml2js.processors.firstCharLowerCase 🎺
2023-10-11 11:19:17 +11:00
Marc Durdin
efe08dcbcd
chore: Update developer/src/kmc-package/src/compiler/kmp-compiler.ts 2023-10-11 11:02:28 +11:00
Marc Durdin
7544b33216 fix(developer): reduce WARN_PackageShouldNotRepeatLanguages to HINT
Part of #9266.

Also cleans up filenames for hint_language_tag_is_not_minimal, and fixes
a bug in transformKpsToKmpObject where a null object could be passed on
to another function in case of failure.
2023-10-09 12:58:19 +07:00
Marc Durdin
0f5e88192c chore(developer): Update kmc-package xml2js dependency
I missed adding this as a dependency for @keymanapp/kmc-package. because
node has a very lenient dependency search, it found xml2js from other
@keymanapp/common-types, so never caused errors locally.
2023-10-09 06:06:34 +07:00
Marc Durdin
f76efc38cc chore(common): stop using xml2js.processors.firstCharLowerCase
Fixes #9713.
2023-10-09 05:43:29 +07:00
Marc Durdin
36a3a27bac fix(developer): non-minimal BCP 47 tags in kps should be a hint
Fixes #9266.

For historical reasons there are many non-minimal BCP 47 tags in package
metadata in the repository (over 2500). Furthermore, earlier versions of
kmcomp did not fail the build on package compilation warnings. This
means it is better to reduce WARN_LanguageTagIsNotMinimal to
HINT_LanguageTagIsNotMinimal, so that we can otherwise respect the
'treat warnings as errors' flag on the keyboard projects.

We may be able to upgrade this to a warning again one day in the future.
2023-10-09 05:19:43 +07:00
Marc Durdin
400444933c feat(developer): introduce WelcomeFile property to packages
Fixes #9478.

This adds a property WelcomeFile to .kps and kmp.json, which allows us
to move away from the hardcoded welcome.htm filename in the future, and
makes transform from Markdown (#9477) a simpler operation, and just
generally starts the cleanup of the messiness of hard-coded filenames.

The package compiler will fallback to injecting welcome.htm into this
property if (a) welcome.htm is present, and (b) the property does not
already have a value. This doesn't buy us much because we still need to
support welcome.htm for existing legacy packages, but does mean that
our .kmp package metadata will be more consistent for packages compiled
with 17.0+ compilers.
2023-10-08 07:22:57 +07:00
Marc Durdin
693aee5c99
Merge branch 'epic/package-metadata' into chore/merge-master-into-package-metadata 2023-10-04 19:34:16 +11:00
Marc Durdin
248297fe5f
Merge branch 'master' into feat/developer/9324-warn-if-kps-includes-non-touch-js 2023-10-04 03:46:40 +02:00