Commit graph

153 commits

Author SHA1 Message Date
Marc Durdin
ee0fc0ce14 chore(developer): cleanup npm publish
Use a new flag `--npm-publish` in conjunction with `publish` action,
so that the default will always be to `npm pack` if the new flag is not
specified. This flag is also guarded in the actual npm publish code to
ensure that it can only run in the appropriate CI alpha/beta/stable
environment, and not in local or test.

This then also removes the separate `pack` action.

Also removes legacy boilerplate from a number of build scripts.
2024-05-20 06:34:57 +07:00
Marc Durdin
c8a1c6aa39 chore(developer): use builder.inc.sh instead of build-utils.sh in build scripts
also, remove old Makefiles
2024-05-20 06:34:46 +07:00
Joshua Horton
887a47930f
Merge pull request #11318 from keymanapp/chore/web/chai-update
chore(web): updates chai, @types/chai dependencies 🏃
2024-05-15 14:10:27 +07:00
Darcy Wong
cbf1fd5cb1 chore(common): Merge beta to master for sprint A18S1 part 2 2024-05-10 06:27:02 +07:00
Joshua A. Horton
398b9e0ae2 chore(common): drops package-specific 'chai', '@types/chai' versions
chore(web): missed leaving these out of the rebase
2024-05-07 14:35:09 +07:00
Marc Durdin
4fc94a9e70 fix(developer): handle missing Description element for File element in package compiler
Fixes #11353.
2024-05-03 14:31:10 +07:00
Marc Durdin
78902e3891 fix(developer): handle missing Name element for File element in package compiler
Fixes #11351.
2024-05-03 14:25:35 +07:00
Marc Durdin
cd8121ef65 chore(common): builder scripts now use /resources/build/builder.inc.sh
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).

This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
2024-05-02 15:54:31 +07:00
Marc Durdin
0f390d47cc chore(common): maintenance on build scripts - cd
Fixes #11324.

* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
2024-05-02 15:54:29 +07:00
Marc Durdin
ffe43d29ef
Merge branch 'beta' into fix/developer/10942-normalize-paths-in-kmc-package 2024-03-07 18:13:43 +07:00
Marc Durdin
7d03010479 fix(developer): normalize all input paths in .kps compiler
Fixes #10942.
2024-03-05 17:34:06 -06:00
Marc Durdin
80a0fea025 docs(developer): kmc-package api documentation
Note: WindowsPackageInstallerSources fields are not yet documented.
2024-03-04 19:15:37 +07:00
Marc Durdin
4bb9b39ba2 refactor(developer): reorganize messages for adding details
Moved the ERROR_Message line above the Error_Message line for every
message in the compiler message files, as it makes the grouping clearer
once we start adding message details. For example, see the start of the
KmnCompilerMessages class.

Also wraps potentially-undefined parameters to the messages with a new
function `def`, shorthand for `CompilerMessageDef`, which converts the
parameter to '<param>' if it is undefined, which is helpful for
documentation.
2024-02-28 16:00:02 +07:00
Marc Durdin
c7afed7201 refactor(developer): split messages with callstacks into separate generator
Relates to #10207.

In order to include message detail in messages, it is helpful to split
out the exception messages which have a separate parameter, so that
parameter at the same position can be the message detail.

Introduces the CompilerMessageSpecWithException function which takes an
exceptionVar parameter to achieve this differentiation.
2024-02-28 11:34:49 +07:00
Marc Durdin
602b519057 docs(developer): npm package readme files 2024-02-26 14:00:51 +07:00
Marc Durdin
a8d0d8454c feat(developer): add api documentation to build
Relates to #10207.

Adds the `api` action to build.sh for each affected module. The `api`
action for each module will generate the api metadata files, and then
the markdown files are generated by /developer/build.sh api.

Note that developer-utils, kmc, and kmcmplib have no api documentation
at this time (and kmcmplib is not Typescript so would need separate
tooling anyway).
2024-02-26 11:45:36 +07:00
Marc Durdin
661069c0c3 feat(developer): add api-extractor and api-documenter and config
Relates to #10207.

Adds @microsoft/api-extractor and @microsoft/api-documenter NPM
packages for validating and generating API documentation from Typescript
modules.

Adds basic configuration for each module that we will be publishing at
this time.

api-extractor's output will be emitted to /developer/docs/api/etc/, for
future release comparisons. A baseline output will be committed
separately.

The build outputs will be in /developer/build/docs and the intermediate
files will be in /developer/build/api.

Note that at this time, we will not enable the checks and validations.
That is a task for alpha rather than beta. The primary intent at this
time is to select a solid, well-supported toolchain for generating the
API documentation for kmc.
2024-02-26 11:37:32 +07:00
Marc Durdin
bfc35e9272 feat(developer): common Messages exports from kmc modules
Relates to #10150. Makes the various Messages classes in kmc modules
accessible to other modules. Some file renames involved to reduce
confusion.

No functional changes.

Fixes kmc-analyze metadata.

See #10254 for a related future fix.
2023-12-14 09:40:18 +07:00
Marc Durdin
e2ce64d9be fix(web): mocha describe does not do async
mocha describe() does not accept an async function. Any async prep
should be done in a before() function, which does support async.

One helpful ref: https://github.com/mochajs/mocha/issues/2975#issuecomment-1004176440
2023-12-13 05:38:51 +07:00
Marc Durdin
69a2448176 feat(developer): PackageCompiler and WindowPackageInstallerCompiler now implement KeymanCompiler
Relates to #9473.

Refactors the public API of PackageCompiler and
WindowsPackageInstallerCompiler to meet KeymanCompiler, including
asyncing a bunch of functions, and moving file write responsibilities
into the classes themselves.

The classes were updated together in a single commit because
WindowsPackageInstallerCompiler depends on PackageCompiler, and needed
refactoring for the updated API access there anyway.

Most of the test cases needed only minor patching, but introducing the
`async init()` function has caused them to all be async. The
test-messages module has a slight functional change with the compile
process running completely rather than stopping on first message, which
meant we needed to add extra dummy fixtures to avoid other errors that
arose later in the compile process. The keyman.exe fixture is a text
file, not a Windows executable.
2023-12-09 06:48:53 +07:00
Marc Durdin
e5da0aacc2 chore(developer): fixup test fixture 2023-11-25 05:16:03 +10:00
Marc Durdin
b3b2825168 fix(developer): path separator for kmc-package
Fixes #10027.
2023-11-23 16:45:12 +10:00
Marc Durdin
c5c5b35415 chore(common): remove prepublish step from package.json
Given we've just run a build, this (a) seems unnecessary, and (b) seems
to go wrong in some circumstances anyway, e.g.
https://build.palaso.org/buildConfiguration/Keyman_Developer_Release/421326?buildTab=log&focusLine=19936&linesState=18839&logView=flowAware:

```
07:01:29   > @keymanapp/kmc@17.0.205-alpha prepublishOnly
07:01:29   > npm run build
07:01:29
07:01:31
07:01:31   > @keymanapp/kmc@17.0.205-alpha build
07:01:31   > tsc -b
07:01:31
07:01:35   ../../../common/web/types/build/src/kmx/kmx.d.ts(1,1): error TS1036: Statements are not allowed in ambient contexts.
07:01:35   ../../../common/web/types/build/src/kmx/kmx.d.ts(1,2): error TS1345: An expression of type 'void' cannot be tested for truthiness.
...
```
2023-11-04 07:34:18 +07:00
Marc Durdin
45b5db1974
Merge pull request #9844 from keymanapp/chore/developer/verify-kmp-json-output-from-kmc-package
chore(developer): verify kmp.json output from kmc-package
2023-11-02 14:50:50 +11:00
Marc Durdin
d92bd1e70a chore(developer): handle failures to build kmp.json in kmc-package 2023-10-26 09:32:26 +07:00
Marc Durdin
cdac2dd01b chore(developer): verify kmp.json output from kmc-package 2023-10-25 12:07:46 +07:00
Marc Durdin
234add1b38 fix(developer): handle xml errors in package compiler
Fixes #9797.
2023-10-22 07:36:45 +07:00
Marc Durdin
a4652cebb9
Merge pull request #9716 from keymanapp/chore/update-kmp-compiler-xml2js-dependency
chore(developer): Update kmc-package xml2js dependency 🎺
2023-10-11 11:19:24 +11:00
Marc Durdin
98af6052e4
Merge pull request #9714 from keymanapp/chore/common/9713-avoid-xml2js-firstCharLowerCase
chore(common): stop using xml2js.processors.firstCharLowerCase 🎺
2023-10-11 11:19:17 +11:00
Marc Durdin
efe08dcbcd
chore: Update developer/src/kmc-package/src/compiler/kmp-compiler.ts 2023-10-11 11:02:28 +11:00
Marc Durdin
7544b33216 fix(developer): reduce WARN_PackageShouldNotRepeatLanguages to HINT
Part of #9266.

Also cleans up filenames for hint_language_tag_is_not_minimal, and fixes
a bug in transformKpsToKmpObject where a null object could be passed on
to another function in case of failure.
2023-10-09 12:58:19 +07:00
Marc Durdin
0f5e88192c chore(developer): Update kmc-package xml2js dependency
I missed adding this as a dependency for @keymanapp/kmc-package. because
node has a very lenient dependency search, it found xml2js from other
@keymanapp/common-types, so never caused errors locally.
2023-10-09 06:06:34 +07:00
Marc Durdin
f76efc38cc chore(common): stop using xml2js.processors.firstCharLowerCase
Fixes #9713.
2023-10-09 05:43:29 +07:00
Marc Durdin
36a3a27bac fix(developer): non-minimal BCP 47 tags in kps should be a hint
Fixes #9266.

For historical reasons there are many non-minimal BCP 47 tags in package
metadata in the repository (over 2500). Furthermore, earlier versions of
kmcomp did not fail the build on package compilation warnings. This
means it is better to reduce WARN_LanguageTagIsNotMinimal to
HINT_LanguageTagIsNotMinimal, so that we can otherwise respect the
'treat warnings as errors' flag on the keyboard projects.

We may be able to upgrade this to a warning again one day in the future.
2023-10-09 05:19:43 +07:00
Marc Durdin
400444933c feat(developer): introduce WelcomeFile property to packages
Fixes #9478.

This adds a property WelcomeFile to .kps and kmp.json, which allows us
to move away from the hardcoded welcome.htm filename in the future, and
makes transform from Markdown (#9477) a simpler operation, and just
generally starts the cleanup of the messiness of hard-coded filenames.

The package compiler will fallback to injecting welcome.htm into this
property if (a) welcome.htm is present, and (b) the property does not
already have a value. This doesn't buy us much because we still need to
support welcome.htm for existing legacy packages, but does mean that
our .kmp package metadata will be more consistent for packages compiled
with 17.0+ compilers.
2023-10-08 07:22:57 +07:00
Marc Durdin
693aee5c99
Merge branch 'epic/package-metadata' into chore/merge-master-into-package-metadata 2023-10-04 19:34:16 +11:00
Marc Durdin
248297fe5f
Merge branch 'master' into feat/developer/9324-warn-if-kps-includes-non-touch-js 2023-10-04 03:46:40 +02:00
Marc Durdin
4367bac8f7
Merge pull request #9658 from keymanapp/feat/developer/9325-hint-if-package-includes-keyboard-source-files
feat(developer): issue hint if package includes keyboard source files
2023-10-04 03:45:26 +02:00
Marc Durdin
1b9c6fba51 feat(developer): warn if .kps includes a .js which is not touch-capable
Fixes #9324.

As .js keyboards in packages are only used by the iOS and Android apps,
it's kinda helpful if the keyboard itself includes a touch layout. While
Keyman will fall back to a desktop vk layout if a touch layout is not
present, this is less than ideal -- but that's why this is setup as a
hint and not a warning.
2023-10-03 15:28:22 +07:00
Marc Durdin
886c64d1c3 chore(developer): add test for checkFilenameConventions == false or unset
Fixes #9317.
2023-10-03 13:29:34 +07:00
Marc Durdin
a2fa1f918e feat(developer): issue hint if package includes keyboard source files
Fixes #9325.

Also tweaks test for error_package_must_contain_a_model_or_a_keyboard as
the fixture had a .kmn, triggering the new hint, which was unhelpful in
this case.
2023-10-02 15:55:00 +07:00
Marc Durdin
5a6758d12e chore(developer): move package version unit tests into Typescript 2023-09-25 08:44:11 +07:00
Marc Durdin
db6a836e31 chore(developer): remove extra parameter 2023-09-21 17:44:08 +07:00
Marc Durdin
a1a7b39481 feat(developer): Windows package installer compiler
Replaces kmcomp's package installer build infrastructure. While this is
mostly legacy, package installers are still used in adhoc distribution
scenarios, so we need to continue to support with the new kmc.

The package installer is a Windows self-extracting zip archive. The user
is expected to find setup-redist.exe and keymandesktop.msi themselves,
from the Keyman Developer release files. They are not included with
Keyman Developer, but are available as standalone downloads from
https://downloads.keyman.com/windows/.

`kmc build` has been updated to include two subcommands:
* `kmc build ldml-test-data`
* `kmc build windows-package-installer`

Both of these subcommands are a little long, but the use cases for them
are fairly narrow, so I believe this is okay. `kmc build-test-data` has
been removed, as `kmc build ldml-test-data` replaces it.
2023-09-21 17:35:16 +07:00
Marc Durdin
72efe0cd5c feat(developer): markdown conversion for description in package editor 2023-08-17 10:17:52 +07:00
Marc Durdin
47e05a7b05 feat(developer): build .keyboard_info without source .keyboard_info
Also fixes a file reference issue in kmpJson.options, where paths were
stripped too early in the process.

Removes a now-irrelevant unit test for keyboard_info.

TODO: markdown to html during the keyboard_info build.
2023-08-16 16:01:14 +07:00
Marc Durdin
158bc28e85 feat(developer): add LicenseFile as property of package
Adds license file to .kps, kmp.json, kmc-package, package editor, and
new project templates. The intent is for LICENSE.md to be the file used,
and for any keyboards on keymanapp/keyboards, it should always be a
standard MIT license.
2023-08-16 09:13:38 +07:00
Marc Durdin
fea10f0eeb feat(developer): add web font references to package
While a package supports a .ttf or .otf font for the OSK and Display
fonts for use within the Keyman apps, on websites it also supports .woff
and .woff2. This change makes it possible to specify a set of fonts for
a package which will be provided through api.keyman.com to websites.

This means that .woff and .woff2 fonts may also be included in the
package and may not be used on target devices at this time, it further
opens the pathway to deploying .kmp packages to web in the future.

The deployment stage of keymanapp/keyboards will require a new phase to
copy web fonts from packages to s.keyman.com.
2023-08-15 15:13:24 +07:00
Marc Durdin
8a9243eba5 chore(developer): tweak Relationship values 2023-08-14 14:31:17 +07:00
Marc Durdin
38c3415eed feat(developer): add relatedPackages metadata
Adds relatedPackages support to:
* .kps schema
* .kps file format for Typescript
* kmp.json file format for Typescript
* kmc-package compiler
* Package Editor
2023-08-14 14:18:01 +07:00