Commit graph

97 commits

Author SHA1 Message Date
Marc Durdin
2b7ceff3e1 chore(developer): add baseline tests for bcp47 codes to kmc-package
Relates-to: #12012
2024-10-07 14:27:15 +07:00
Marc Durdin
ba96d26858 fix(developer): enforce presence of Version field when FollowKeyboardVersion is not set, in package compiler
Fixes: #12193
2024-08-16 08:24:37 +02:00
Marc Durdin
86fd36a713
Merge pull request #12117 from keymanapp/chore/developer/build-ci-linux-mac
chore(developer): start building Developer on linux,mac
2024-08-09 18:57:38 +10:00
Marc Durdin
7bf0d17282 chore(developer): ignore line endings in unit test for kmc-keyboard-info 2024-08-07 13:11:37 +07:00
Marc Durdin
32b4711659
Merge branch 'master' into refactor/common/9665-move-compiler-interfaces-to-developer 2024-08-07 15:15:45 +10:00
Marc Durdin
e67aed1606 refactor(common): move compiler-interfaces to @keymanapp/developer-utils
This should be the last step in the refactoring.

Fixes: #9665
2024-08-02 13:49:59 +07:00
Marc Durdin
d8986be12f refactor(common): merge master into branch 2024-08-01 07:07:07 +07:00
Marc Durdin
d1e351a1ac
Revert "chore(developer): remove redundant references from tsconfig.json" 2024-08-01 08:57:27 +10:00
Marc Durdin
1cff1a5084 chore(developer): remove redundant references from tsconfig.json
With the full move to ES Modules, we no longer need to include
`references` in tsconfig.json, as we can rely on package.json and
build.sh dependency management. Note however that `tsc -b` may not work
to build dependencies -- they need to be built using `build.sh`, which
calculates which dependencies need building.

kmc-keyboard-info was missing a dependency link to kmc-package in
build.sh, correcting this at the same time.

Relates-to: #12027
2024-07-27 05:44:22 +07:00
Marc Durdin
1afa4beb01 fix(developer): remove paths from tsconfig.json
Fixes: #12027
2024-07-26 06:59:45 +07:00
Marc Durdin
1b2ca768b7 chore(developer): rename kmc-package CompilerMessages to PackageCompilerMessages
It is easier to differentiate between all the compiler message classes
if each one has its own name.
2024-07-22 16:02:43 +07:00
Marc Durdin
2506ff7c01 fix(developer): verify email addresses in .kps and .keyboard_info
Add verification of email address formats to kmc-package,
kmc-keyboard-info, and kmc-model-info. Add corresponding unit tests for
kmc-package and kmc-keyboard-info (kmc-model-info unit tests will be a
later addition). Uses AJV's email validation regex, as we rely on AJV
for schema compliance tests.

Fixes: #11362
Fixes: KEYMAN-DEVELOPER-1XE
2024-06-07 13:30:21 +07:00
Marc Durdin
8efba7186e fix(developer): handle missing data in .kps <Keyboard>
Handles when `<Version/>`, `<Name/>`, or `<ID/>` elements are missing
from `<Keyboard>` elements in the .kps file in kmc-package. Adds a unit
test.

Fixes: #11562
Fixes: KEYMAN-DEVELOPER-1YY
2024-05-27 15:04:53 +07:00
Marc Durdin
7d03010479 fix(developer): normalize all input paths in .kps compiler
Fixes #10942.
2024-03-05 17:34:06 -06:00
Marc Durdin
bfc35e9272 feat(developer): common Messages exports from kmc modules
Relates to #10150. Makes the various Messages classes in kmc modules
accessible to other modules. Some file renames involved to reduce
confusion.

No functional changes.

Fixes kmc-analyze metadata.

See #10254 for a related future fix.
2023-12-14 09:40:18 +07:00
Marc Durdin
e2ce64d9be fix(web): mocha describe does not do async
mocha describe() does not accept an async function. Any async prep
should be done in a before() function, which does support async.

One helpful ref: https://github.com/mochajs/mocha/issues/2975#issuecomment-1004176440
2023-12-13 05:38:51 +07:00
Marc Durdin
69a2448176 feat(developer): PackageCompiler and WindowPackageInstallerCompiler now implement KeymanCompiler
Relates to #9473.

Refactors the public API of PackageCompiler and
WindowsPackageInstallerCompiler to meet KeymanCompiler, including
asyncing a bunch of functions, and moving file write responsibilities
into the classes themselves.

The classes were updated together in a single commit because
WindowsPackageInstallerCompiler depends on PackageCompiler, and needed
refactoring for the updated API access there anyway.

Most of the test cases needed only minor patching, but introducing the
`async init()` function has caused them to all be async. The
test-messages module has a slight functional change with the compile
process running completely rather than stopping on first message, which
meant we needed to add extra dummy fixtures to avoid other errors that
arose later in the compile process. The keyman.exe fixture is a text
file, not a Windows executable.
2023-12-09 06:48:53 +07:00
Marc Durdin
e5da0aacc2 chore(developer): fixup test fixture 2023-11-25 05:16:03 +10:00
Marc Durdin
234add1b38 fix(developer): handle xml errors in package compiler
Fixes #9797.
2023-10-22 07:36:45 +07:00
Marc Durdin
7544b33216 fix(developer): reduce WARN_PackageShouldNotRepeatLanguages to HINT
Part of #9266.

Also cleans up filenames for hint_language_tag_is_not_minimal, and fixes
a bug in transformKpsToKmpObject where a null object could be passed on
to another function in case of failure.
2023-10-09 12:58:19 +07:00
Marc Durdin
36a3a27bac fix(developer): non-minimal BCP 47 tags in kps should be a hint
Fixes #9266.

For historical reasons there are many non-minimal BCP 47 tags in package
metadata in the repository (over 2500). Furthermore, earlier versions of
kmcomp did not fail the build on package compilation warnings. This
means it is better to reduce WARN_LanguageTagIsNotMinimal to
HINT_LanguageTagIsNotMinimal, so that we can otherwise respect the
'treat warnings as errors' flag on the keyboard projects.

We may be able to upgrade this to a warning again one day in the future.
2023-10-09 05:19:43 +07:00
Marc Durdin
400444933c feat(developer): introduce WelcomeFile property to packages
Fixes #9478.

This adds a property WelcomeFile to .kps and kmp.json, which allows us
to move away from the hardcoded welcome.htm filename in the future, and
makes transform from Markdown (#9477) a simpler operation, and just
generally starts the cleanup of the messiness of hard-coded filenames.

The package compiler will fallback to injecting welcome.htm into this
property if (a) welcome.htm is present, and (b) the property does not
already have a value. This doesn't buy us much because we still need to
support welcome.htm for existing legacy packages, but does mean that
our .kmp package metadata will be more consistent for packages compiled
with 17.0+ compilers.
2023-10-08 07:22:57 +07:00
Marc Durdin
693aee5c99
Merge branch 'epic/package-metadata' into chore/merge-master-into-package-metadata 2023-10-04 19:34:16 +11:00
Marc Durdin
248297fe5f
Merge branch 'master' into feat/developer/9324-warn-if-kps-includes-non-touch-js 2023-10-04 03:46:40 +02:00
Marc Durdin
4367bac8f7
Merge pull request #9658 from keymanapp/feat/developer/9325-hint-if-package-includes-keyboard-source-files
feat(developer): issue hint if package includes keyboard source files
2023-10-04 03:45:26 +02:00
Marc Durdin
1b9c6fba51 feat(developer): warn if .kps includes a .js which is not touch-capable
Fixes #9324.

As .js keyboards in packages are only used by the iOS and Android apps,
it's kinda helpful if the keyboard itself includes a touch layout. While
Keyman will fall back to a desktop vk layout if a touch layout is not
present, this is less than ideal -- but that's why this is setup as a
hint and not a warning.
2023-10-03 15:28:22 +07:00
Marc Durdin
886c64d1c3 chore(developer): add test for checkFilenameConventions == false or unset
Fixes #9317.
2023-10-03 13:29:34 +07:00
Marc Durdin
a2fa1f918e feat(developer): issue hint if package includes keyboard source files
Fixes #9325.

Also tweaks test for error_package_must_contain_a_model_or_a_keyboard as
the fixture had a .kmn, triggering the new hint, which was unhelpful in
this case.
2023-10-02 15:55:00 +07:00
Marc Durdin
5a6758d12e chore(developer): move package version unit tests into Typescript 2023-09-25 08:44:11 +07:00
Marc Durdin
a1a7b39481 feat(developer): Windows package installer compiler
Replaces kmcomp's package installer build infrastructure. While this is
mostly legacy, package installers are still used in adhoc distribution
scenarios, so we need to continue to support with the new kmc.

The package installer is a Windows self-extracting zip archive. The user
is expected to find setup-redist.exe and keymandesktop.msi themselves,
from the Keyman Developer release files. They are not included with
Keyman Developer, but are available as standalone downloads from
https://downloads.keyman.com/windows/.

`kmc build` has been updated to include two subcommands:
* `kmc build ldml-test-data`
* `kmc build windows-package-installer`

Both of these subcommands are a little long, but the use cases for them
are fairly narrow, so I believe this is okay. `kmc build-test-data` has
been removed, as `kmc build ldml-test-data` replaces it.
2023-09-21 17:35:16 +07:00
Marc Durdin
72efe0cd5c feat(developer): markdown conversion for description in package editor 2023-08-17 10:17:52 +07:00
Marc Durdin
fea10f0eeb feat(developer): add web font references to package
While a package supports a .ttf or .otf font for the OSK and Display
fonts for use within the Keyman apps, on websites it also supports .woff
and .woff2. This change makes it possible to specify a set of fonts for
a package which will be provided through api.keyman.com to websites.

This means that .woff and .woff2 fonts may also be included in the
package and may not be used on target devices at this time, it further
opens the pathway to deploying .kmp packages to web in the future.

The deployment stage of keymanapp/keyboards will require a new phase to
copy web fonts from packages to s.keyman.com.
2023-08-15 15:13:24 +07:00
Marc Durdin
38c3415eed feat(developer): add relatedPackages metadata
Adds relatedPackages support to:
* .kps schema
* .kps file format for Typescript
* kmp.json file format for Typescript
* kmc-package compiler
* Package Editor
2023-08-14 14:18:01 +07:00
Marc Durdin
bf1d5b0f7a chore(developer): fix casing 2023-08-14 10:06:32 +07:00
Marc Durdin
675f6bc91e feat(developer): description, font and example metadata in packages
Adds info.description, options.fonts, keyboard.examples metadata to
.kps and kmp.json files, updates schemas, and Typescript compiler.
2023-08-14 09:23:20 +07:00
Marc Durdin
fd8e17e05c fix(developer): respect checkFilenameConventions in kmc
Fixes #9300.
2023-07-19 13:56:21 +07:00
Marc Durdin
de1c171f46 chore(developer): cleanup compiler messages in kmc
Fixes #8795.
Relates to #9090.

Cleans up and makes consistent compiler messages in kmc, and related
unit tests.

Adds support for colorized messages.

After installing chalk for ansi coloring in kmc, @types/node needed to
be brought in sync with the Typescript version, per the suggestion of
checking against `npm dist-tags @types/node`,found in a discussion at
https://github.com/DefinitelyTyped/DefinitelyTyped/discussions/64262#discussioncomment-4905069

Additional tidying up includes:
* Deprecating a set of functions in compiler-interfaces, bringing them
  together under a new CompilerError class.
  Separating CompilerErrorSeverity and CompilerErrorMask enums.
* Making CompilerError.formatEvent (formerly compilerEventFormat) print
  messages in the same format as NodeCompilerCallbacks (sans coloring).
* Adding a wrapper class for CompilerCallbacks that manages the filename
  reporting, used currently exclusively by kmc itself.
2023-07-10 15:13:48 +07:00
Marc Durdin
306934eb62 fix(developer): kmc-package support for keyboard metadata
Fixes #9208.
Fixes #9209.
Fixes #9203.
Fixes #9198.

The changes here overlap in kmp-compiler.ts. Addresses several issues
with the package compiler:

* refactors the reading of the metadata from keyboards to make use of it
  when refreshing the keyboard metadata in kmp.json (#9208). This is the
  bulk of the changes.
* Fixes case on some fields in the kps file format (#9209)
* Removes code which emitted strings table to kmp.json, as these were
  only ever used for package installer executables (#9203)
* Cleans up code which emitted Start Menu items to make it match spec
  (#9198)
2023-07-06 13:49:04 +07:00
Marc Durdin
3773522e06 feat(developer): kmc-package support for writing kmp.inf
Fixes #9205.
2023-07-06 13:49:03 +07:00
Marc Durdin
92c8c6497e fix(developer): improve warnings around keyboard version matching
Fixes #9145.

* `WARN_KeyboardVersionsDoNotMatch` is now only raised when
  `FollowKeyboardVersion` is set.
* `WARN_KeyboardVersionsDoNotMatchPackageVersion` has been removed,
  because it did not really make sense; if 'FollowKeyboardVersion' is
  set, it could not be raised, and otherwise, the author may wish to
  have separate keyboard + package versions anyway.
* Note that the 0x0013 compiler message allocation for
  `WARN_KeyboardVersionsDoNotMatchPackageVersion` has been left alone;
  as this was only used in pre-release, we can probably re-task it in
  the future.
2023-07-03 09:40:09 +07:00
Marc Durdin
9a1b87824a fix(developer): split keyboard and model language metadata tests
Fixes #9146.

For model packages, 304016 (ERROR_ModelMustHaveAtLeastOneLanguage)
remains as an error (this is a new error for kmc, kmcomp did not
validate model metadata). For keyboard packages, a new warning is
introduced to match the existing kmcomp behaviour, 20401B
(WARN_KeyboardShouldHaveAtLeastOneLanguage).
2023-07-03 09:31:29 +07:00
Marc Durdin
f85478ac6f fix(developer): kmc needs to support .js-only packages
Fixes #9111.

We have existing packages which have only .js in them, for touch-only
keyboards (mostly legacy but still...), so we need to support the freaky
Javascript regex search which we did in the past for extracting
metadata. While this is not necessarily going to work with hand-crafted
Javascript keyboards, it should work with all kmc- and kmcomp-generated
keyboards, so it will suffice to support these legacy packages.

In the future, we will be giving a hint when a package includes a .js
but not a .kmx, gradually upgrading this to a warning and finally an
error as we attempt to phase out .js-based keyboards in preference for
.kmx keyboards.
2023-06-30 08:45:03 +07:00
Joshua Horton
2ac8a4503b
Merge branch 'master' into feature-esmodule-web-engine 2023-06-26 09:09:43 +07:00
Marc Durdin
cbc5b13034 fix(developer): keep kmc-kmn messages within namespace
Fixes #8998.

kmc-kmn messages were broken down into sub-namespaces, but the range
overlapped the reserved namespace mask range. Fixed the message values,
added a new test to the verifyCompilerMessagesObject function to verify
that messages don't creep outside their namespace, and defined new
masks (in CompilerErrorSeverity enum... hmm) to help.

I opted to remove the leading `0` in the messages for kmc-kmn as I had
to correct a number of them anyway, but not the messages in remaining
units at this time (it's not an error, but just slightly misleading as
we only have 12 bits, not 16 to play with).

It is safe to reassign these messages as they were only assigned in 17.0
alpha.
2023-06-13 06:22:15 +07:00
Joshua A. Horton
718ca50b23 chore(web): Merge branch 'master' into chore/web/feature-esmodule-a17s14-mid-update 2023-06-06 12:37:15 +07:00
Marc Durdin
e293d7608c chore(developer): add test for missing .js file 2023-05-23 09:28:26 +07:00
Marc Durdin
3828a5c16b refactor(developer): move message tests into test-messages.ts 2023-05-23 08:31:29 +07:00
Joshua A. Horton
ca894d0991 chore(web): Merge branch 'master' into chore/web/merge-master-esmodule-a17s13-mid 2023-05-22 13:49:45 +07:00
Marc Durdin
4ad975d3a8
Merge pull request #8793 from keymanapp/chore/developer/verify-package-has-a-model-or-keyboard
chore(developer): verify that package has at least a model or keyboard
2023-05-16 17:21:41 +10:00
Marc Durdin
ced54e260a
Merge pull request #8792 from keymanapp/chore/developer/verify-included-files-in-package
chore(developer): verify file types of content files in package
2023-05-16 17:21:24 +10:00