The new --for-publishing flag allows us to enforce additional
requirements for keyboards and lexical models that are to be published
to the keymanapp repositories. This flag will always be switched on in
the repository builds. This flag overrides the 'skipMetadata' option
which is a project-level option.
There is a bit of a delicate balance of requirements here:
1. We want to be able to automatically verify keyboards and package
licenses if they are in the repository.
2. We do not want to force license checks on privately built keyboards
and models.
3. We want to provide pathways for users to check locally before
submitting to the repository.
4. If possible, we want to be able to build the .keyboard_info and
.model_info files locally, but this should be up to the user for
local builds.
For now, the only additional check that this flag provides is to verify
that the license is MIT, which was an unchecked requirement for the
repositories in the past.
Future checks can be added for file layout, additional required
metadata files.
Anticipate adding this as a tool to Keyman Developer IDE -- a
'pre-publish' check -- in the future.
Fixes#9610.
If a build fails when "treat warnings as errors" is on, but there are
only warnings and no error messages, it can be confusing for the
developer. Adds the following `INFO_WarningsHaveFailedBuild` message:
`The build failed because option "treat warnings as errors" is enabled
and there are one or more warnings.`
Replaces kmcomp's package installer build infrastructure. While this is
mostly legacy, package installers are still used in adhoc distribution
scenarios, so we need to continue to support with the new kmc.
The package installer is a Windows self-extracting zip archive. The user
is expected to find setup-redist.exe and keymandesktop.msi themselves,
from the Keyman Developer release files. They are not included with
Keyman Developer, but are available as standalone downloads from
https://downloads.keyman.com/windows/.
`kmc build` has been updated to include two subcommands:
* `kmc build ldml-test-data`
* `kmc build windows-package-installer`
Both of these subcommands are a little long, but the use cases for them
are fairly narrow, so I believe this is okay. `kmc build-test-data` has
been removed, as `kmc build ldml-test-data` replaces it.
Removes kmcomp.dpr, related source, compilekeymanweb.pas, and related
unit tests.
Cleans up a couple of bits and pieces around kmc in order to help tests
to pass, and updates some tests that had legacy code in them
(particularly missing &TARGETS stores).
Works around npm/cli#3466 when bundling internal dependencies using the
bundleDependencies package.json property.
This change works in tandem with the npm pack/publish process -- when we
run `developer/src/kmc/build.sh publish` (or `pack`), we end up with
`npm version` stomping on all our package.json files, so the repo is
dirty after this. We need a copy of the top-level package.json before
this stomping happens, in order to get a simple map of the location of
each of our internal dependencies, from the `dependencies` property (it
would be possible to figure this out with a lot more parsing of
our package.json files, but this is simpler).
This means, in future, we should avoid publishing our internal
dependencies such as those under common/ to npm, as they serve no
practical purpose there.
Works around npm/cli#3466 when bundling internal dependencies using the
bundleDependencies package.json property.
This change works in tandem with the npm pack/publish process -- when we
run `developer/src/kmc/build.sh publish` (or `pack`), we end up with
`npm version` stomping on all our package.json files, so the repo is
dirty after this. We need a copy of the top-level package.json before
this stomping happens, in order to get a simple map of the location of
each of our internal dependencies, from the `dependencies` property (it
would be possible to figure this out with a lot more parsing of
our package.json files, but this is simpler).
This means, in future, we should avoid publishing our internal
dependencies such as those under common/ to npm, as they serve no
practical purpose there.
Adds support for calculating isRTL and license fields in .model_info
compiler.
Also:
* establishes @keymanapp/developer-utils shared module
* moves license validation into @keymanapp/developer-utils
* refactors kmc-model-info to a class and general cleanup
Removes all references to source .keyboard_info and .model_info files in
the kmc compiler and types. Updates the model info compiler to support
building purely from model sources (isRTL, license fields are TODO).
Moves responsibility for loading .kps into kmc-keyboard-info, away from
the caller, and removes other fields with kmc-keyboard-info can
calculate by itself.
Have not moved project parsing into kmc-keyboard-info, because that's a
bigger job, as currently that is mostly happening within kmc itself. A
project for a future version I think.
Relates to #9351.
Adds support for building a .keyboard_info file without having source
.keyboard_info file:
* Constructs a default source .keyboard_info in memory
* Hints if license is missing
* If LICENSE.md is present, verifies it matches the MIT license text
* Adds project option to turn on or off metadata generation. This will
default to False for version 1.0 projects, and to True for version 2.0
projects. This means that the keyboard repository will need a PR to
enable metadata generation for existing projects in the repository,
but this is important to avoid breaking builds for existing projects
that are not in the repository.
* Turns on additional c8 coverage for kmc projects
kmc already reported unhandled exceptions, but any handled fatal errors
were captured and only reported to the user. It is better to report
these to Sentry as these are still unexpected.
I have refactored all the fatal exception messages in various kmc
modules to use a common mechanism, keeping all the Sentry integration in
kmc, now passing exception data up in the `CompilerEvent.exceptionVar`
property.
* I took the opportunity to rename messages.ts to
infrastructureMessages.ts
* @types/chai was missing which gave intellisense errors in vscode
* normal exit of kmc now provides an opportunity for error reports to
Sentry to be finalized
* Added a unit test for fatal errors in kmc
* Added a manual test pathway with `SENTRY_CLIENT_TEST_BUILD_EXCEPTION`
env var to trip the build fatal error mechanism and verify that it
looks ok; the following shows test runs demonstrate how fatal build
errors are reported:
```
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
This error has been automatically reported to the Keyman team.
Identifier: 6f0fca1a26694c22b03f02b2463d39c5
Application: Keyman Developer
Reported at: https://sentry.io/organizations/keyman/projects/keyman-developer/events/6f0fca1a26694c22b03f02b2463d39c5/
mcdurdin@THARK MINGW64 /c/Projects/keyman/app/developer/src/kmc (chore/developer/report-fatal-compiler-errors-to-sentry)
$ SENTRY_CLIENT_TEST_BUILD_EXCEPTION=1 node . --no-error-reporting build
fatal KM05001: Unexpected exception: Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
Call stack:
Error: Test exception from SENTRY_CLIENT_TEST_BUILD_EXCEPTION
at build (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:78:19)
at Command.<anonymous> (file:///C:/Projects/keyman/app/developer/src/kmc/build/src/commands/build.js:66:24)
at Command.listener [as _actionHandler] (C:\Projects\keyman\app\node_modules\commander\lib\command.js:482:17)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:65
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1283:27)
at C:\Projects\keyman\app\node_modules\commander\lib\command.js:1081:27
at Command._chainOrCall (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1177:12)
at Command._dispatchSubcommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1077:23)
at Command._parseCommand (C:\Projects\keyman\app\node_modules\commander\lib\command.js:1248:19)
```
Turns KeymanDeveloperProjectFile into an interface, and removes
redundant data in filename and fileType fields, calculating these from
filePath instead. Maintains separate KeymanDeveloperProjectFile10 and
KeymanDeveloperProjectFile20 classes implementing the base interface for
now, although that may be a target for future consolidation (lowpri).
Ensures that we calculate the file type instead of using stored data,
which is a lot cleaner. Means we ignore the filetype field in the .kpj
xml, which is fine.
We should do the same with filename vs filepath.
Fixes#9377.
1. ldml_transforms.cpp used emplace_back with char instead of dword
values (#9377)
2. import was updated to src/ instead of ../ by vscode in build.ts
(no related issue)
