Commit graph

226 commits

Author SHA1 Message Date
Marc Durdin
33affc660f fix(common): legacy .kpj schema
Fixes #9140.
Fixes #9148.

Keyman Developer 9.0 .kpj files included a lot of additional state
metadata. We need a schema which validates these files, as they are
otherwise valid to load (we'll never save them any more). Rather than
add all the extra metadata to what is otherwise a fairly clean schema,
we'll provide a legacy .schema.json.

In the future, we may be able to merge these schemas, as we move towards
the .kpj 2.0 format which doesn't list files. Ideally, all three formats
(Keyman Developer 9.0 schema, call it legacy, 1.0 schema for Keyman
Developer 10.0+ which has Options and Files listed, 2.0 schema for
capturing just project settings for a folder) will be supported by a
single schema file.
2023-07-03 09:06:03 +07:00
Marc Durdin
d021526e38 fix(common): give friendly error when .kvks is in binary format
Fixes #9124.
2023-06-30 08:43:01 +07:00
Marc Durdin
6027360544 fix(developer): handle empty File Details in package reader
Fixes #9109.
2023-06-28 07:27:42 +07:00
Marc Durdin
d914b6918d chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw-a17s16-start 2023-06-26 14:33:20 +07:00
Joshua Horton
2ac8a4503b
Merge branch 'master' into feature-esmodule-web-engine 2023-06-26 09:09:43 +07:00
dependabot[bot]
7d08b08fb2
chore(deps): bump semver from 7.3.8 to 7.5.2
Bumps [semver](https://github.com/npm/node-semver) from 7.3.8 to 7.5.2.
- [Release notes](https://github.com/npm/node-semver/releases)
- [Changelog](https://github.com/npm/node-semver/blob/main/CHANGELOG.md)
- [Commits](https://github.com/npm/node-semver/compare/v7.3.8...v7.5.2)

---
updated-dependencies:
- dependency-name: semver
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-24 12:37:06 +00:00
Steven R. Loomis
8a3a08dcf7 feat(core): update documentation and builder for uset 🙀
- pre-sort the uset items instead of copying the patterns and re-finding
- update docs on the uset subtable

#7377
2023-06-23 10:52:34 -05:00
Marc Durdin
d6721f68ba chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw 2023-06-23 15:44:06 +07:00
Marc Durdin
164823a643
chore: fix comment in common 2023-06-21 14:31:50 +10:00
Marc Durdin
3f507f974a chore(developer): remove font style/color from kvk
While .kvk includes a font color field, it is essentially unused. There
was also some sort of support provided for font style in the KeymanWeb
keyboard compiler, but this was never streamed into .kvk or .kvks files,
so was always a no-op.

Thus, this PR removes any semblance of support for font style and color
from the .kvk and .kvks readers and writers. When we write the font
color field which is present in .kvk, we always use the default
TColor.clWindowText which is what the legacy Delphi-based writer would
always have written.

Also sorted out the default font size and name in the .kvk transform
from .kvks, and in so doing cleaned up the basic.kvk and basic.js in
LDML keyboard compiler to match what we are doing in the .kmn compiler.
2023-06-21 10:53:45 +07:00
Steven R. Loomis
248d423535 feat(common,developer,core): add uset section 🙀
- not emitted yet, basic scaffolding
- expanded type field in elem:  str/char/set

For: #7377
2023-06-20 18:23:54 -05:00
Marc Durdin
6da441da04 chore(common): fixup tests for common/web/ci 2023-06-20 15:57:50 +07:00
Marc Durdin
c68403fabf chore(developer): validate displaymap schema 2023-06-20 12:57:57 +07:00
Marc Durdin
0bd2ca4821 refactor(developer): kmc-kmw becomes component of kmc-kmn
Moves kmc-kmw to being a sub-component of kmc-kmn, as it can never be
independently instantiated anyway. Reorganized tests accordingly.

Note that c8 is currently disabled for kmw-compiler, until we add unit
tests for it.
2023-06-20 11:52:23 +07:00
Marc Durdin
adfa032e6f feat(developer): add &displayMap system store
The &displayMap system store adds support for a mapping file that remaps
the On Screen Keyboard files -- .kvks and .keyman-touch-layout, at
compile time, to new ranges. This is specifically intended to resolve a
limitation of unattached marks and diacritics which display
inconsistently across platforms; this is described in more detail in
issue #9031.
2023-06-17 16:21:26 +07:00
Marc Durdin
e9391f57ff chore(developer): refactor osk rewriter
Moves common remapping code for the OSK rewriter into common/web/types
so that it can be consumed by the kmn and kmw compilers soon. Also
cleans up the command line instantiation of the analyze tools, although
the rewrite tool is less likely to be used once we have the compiler
integration in place.
2023-06-17 15:03:02 +07:00
Marc Durdin
3d56bc667e chore(common): export KvksFileWriter 2023-06-14 15:55:56 +07:00
Marc Durdin
d19f9027b7
Merge pull request #8994 from keymanapp/chore/developer/complete-error-checking-in-kmc-analyze
chore(developer): complete error checking in kmc-analyze 🗜
2023-06-13 23:10:46 +10:00
Marc Durdin
f7b524b8db
Merge pull request #8984 from keymanapp/chore/common/web/file-types-module-first-steps
feat(common): add file-types.ts module 🗜
2023-06-13 12:56:46 +10:00
Marc Durdin
cbc5b13034 fix(developer): keep kmc-kmn messages within namespace
Fixes #8998.

kmc-kmn messages were broken down into sub-namespaces, but the range
overlapped the reserved namespace mask range. Fixed the message values,
added a new test to the verifyCompilerMessagesObject function to verify
that messages don't creep outside their namespace, and defined new
masks (in CompilerErrorSeverity enum... hmm) to help.

I opted to remove the leading `0` in the messages for kmc-kmn as I had
to correct a number of them anyway, but not the messages in remaining
units at this time (it's not an error, but just slightly misleading as
we only have 12 bits, not 16 to play with).

It is safe to reassign these messages as they were only assigned in 17.0
alpha.
2023-06-13 06:22:15 +07:00
Marc Durdin
d684e58baa
Merge branch 'feature-kmc-kmw' into chore/common/web/file-types-module-first-steps 2023-06-13 08:37:55 +10:00
Marc Durdin
da29d82e01 chore(developer): complete error checking in kmc-analyze 2023-06-12 13:02:04 +07:00
Marc Durdin
f6900d9e85 chore(developer): merge kmc-kmw messages with kmc-kmn messages
These two modules had duplicated messages from kmcmplib. Rewrites the
kmc-kmw messages module to inherit from kmc-kmn and define message
strings where necessary.

While this refactor is not ideal, because it diverges slightly from the
patterns of other message modules, and has a little bit of spaghetti
import cross-modules, I think it is still a step in the right direction
for consolidated message reporting.
2023-06-12 10:29:35 +07:00
Marc Durdin
7ad2df06d9 chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw 2023-06-12 09:31:18 +07:00
Marc Durdin
8a4c005d40 chore(common): define constants for file types
Now that we have the file type detection in place, we can replace
existing references to the file extensions, where possible, with the
constant declarations. While we could rely on TypeScript's typing to use
string-based values, this helps us to avoid using file extensions
directly in string transforms, regex, etc, and use the `KeymanFileTypes`
alias instead.

Note that only references in common/web, and kmc-* projects have been
touched. References in other Developer projects have not been touched,
as some of those are not yet ESM, so that needs to be addressed first.
2023-06-12 09:17:09 +07:00
Marc Durdin
d2029c40cd chore(common): Create file-types.ts module
This module is used for identification of Keyman source and binary file
formats by file extension. It is not yet appropriate to use for
constants; this will come in a future commit.
2023-06-12 08:20:02 +07:00
Marc Durdin
8bfd4c0bae chore(developer): refactor LdmlCompilerOptions
LdmlCompilerOptions is now based on the shared CompilerOptions. Also
fixes command line parameter inheritance for sub-commands.
2023-06-10 14:27:40 +07:00
Marc Durdin
459f42b273 chore(developer): refactor CompilerOptions
Part 1 of the `CompilerOptions` reorg. kmc now uses `CompilerOptions`
throughout, and `CompilerBaseOptions` maps to the set of options
available to all activities.
2023-06-10 14:27:39 +07:00
Marc Durdin
cc956117cf chore: remove commented code 2023-06-10 14:26:26 +07:00
Marc Durdin
db06dfd810 chore(developer): Merge branch 'feature-kmc-kmw' into feat/developer/kmc-analyze 2023-06-09 14:00:14 +07:00
Marc Durdin
fc043c960e chore: tweak CompilerLogLevel 2023-06-09 13:02:17 +07:00
Marc Durdin
3b5c38aa80 feat(developer): use new memory-based kmx compiler in analyze
* Also adds info messages for scanning.
2023-06-09 12:38:20 +07:00
Marc Durdin
fde428fb70 chore(developer): add logLevel to kmc
Tidies up silent vs quiet vs verbose by adding logLevel as an option.

* Notes that we need to consolidate all the various CompilerOptions
  interfaces before they get too much further out of hand.
* Cleans up a couple of other minor TODO items.
2023-06-09 10:23:06 +07:00
Steven R. Loomis
4febfcb451 feat(developer): improve tran/bksp tests, other improvements 🙀
- delete dead code
- share common ID regex for vars
- move bksp/finl/order tests into tran

For: #7377
2023-06-08 18:39:44 -05:00
Steven R. Loomis
26053a5ba3
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-06-08 18:10:11 -05:00
Marc Durdin
c02b271394 chore: cleanup unneeded changes 2023-06-08 15:17:09 +07:00
Marc Durdin
f6e52270ed chore(developer): Merge branch 'feature-kmc-kmw' into feat/developer/kmc-analyze 2023-06-08 14:59:28 +07:00
Marc Durdin
9f71f093cb chore(common): export KmxFileReaderError 2023-06-08 08:51:08 +07:00
Marc Durdin
a1fdc7b133 refactor(developer): CompilerOptions as shared type
Also fixes handling of kmx errors in kmc-package.
2023-06-08 05:42:23 +07:00
Marc Durdin
4c3acaf651 chore(common): update restructure to support typescript 2023-06-08 05:40:48 +07:00
Steven R. Loomis
ad1d4e734c feat(common): fix bug in build-tran 🙀
For: #7377
2023-06-06 18:37:07 -05:00
Steven R. Loomis
8548b0fd33 Merge remote-tracking branch 'upstream/master' into feat/developer-7377-transform-format-epic-ldml3 2023-06-06 09:45:21 -05:00
Marc Durdin
fdd313a6a2 chore(common): cleanup error handling for KmxFileReader 2023-06-06 15:22:00 +07:00
Joshua A. Horton
718ca50b23 chore(web): Merge branch 'master' into chore/web/feature-esmodule-a17s14-mid-update 2023-06-06 12:37:15 +07:00
Steven R. Loomis
70fcea815a feat(developer): tran support 🙀
- tran builder

For: #7377
2023-06-05 19:18:59 -05:00
Marc Durdin
eaf7a6b628 chore(developer): merge changed interfaces in compiler 2023-06-05 16:34:23 +07:00
Marc Durdin
3f993bd651 chore: Merge branch 'master' into feat/typescript-keymanweb-compiler 2023-06-05 15:57:42 +07:00
Steven R. Loomis
2054eeb703 feat(developer): tran support 🙀
- support regex format for sets and unicodeSets
- support mapFrom and mapTo detection

For: #7377
2023-06-02 18:31:25 -05:00
Steven R. Loomis
4b81137a61 feat(developer): section dependencies 🙀
- fix test helper for dependencies
- moved VariableParser and MarkerParser into common

For: #7377
2023-06-02 16:58:39 -05:00
Steven R. Loomis
9b177b2f00 feat(developer): section dependencies 🙀
- make explicit section dependencies, add a test

For: #7377
2023-06-02 15:19:00 -05:00