Commit graph

210 commits

Author SHA1 Message Date
Marc Durdin
164823a643
chore: fix comment in common 2023-06-21 14:31:50 +10:00
Marc Durdin
3f507f974a chore(developer): remove font style/color from kvk
While .kvk includes a font color field, it is essentially unused. There
was also some sort of support provided for font style in the KeymanWeb
keyboard compiler, but this was never streamed into .kvk or .kvks files,
so was always a no-op.

Thus, this PR removes any semblance of support for font style and color
from the .kvk and .kvks readers and writers. When we write the font
color field which is present in .kvk, we always use the default
TColor.clWindowText which is what the legacy Delphi-based writer would
always have written.

Also sorted out the default font size and name in the .kvk transform
from .kvks, and in so doing cleaned up the basic.kvk and basic.js in
LDML keyboard compiler to match what we are doing in the .kmn compiler.
2023-06-21 10:53:45 +07:00
Marc Durdin
6da441da04 chore(common): fixup tests for common/web/ci 2023-06-20 15:57:50 +07:00
Marc Durdin
c68403fabf chore(developer): validate displaymap schema 2023-06-20 12:57:57 +07:00
Marc Durdin
0bd2ca4821 refactor(developer): kmc-kmw becomes component of kmc-kmn
Moves kmc-kmw to being a sub-component of kmc-kmn, as it can never be
independently instantiated anyway. Reorganized tests accordingly.

Note that c8 is currently disabled for kmw-compiler, until we add unit
tests for it.
2023-06-20 11:52:23 +07:00
Marc Durdin
adfa032e6f feat(developer): add &displayMap system store
The &displayMap system store adds support for a mapping file that remaps
the On Screen Keyboard files -- .kvks and .keyman-touch-layout, at
compile time, to new ranges. This is specifically intended to resolve a
limitation of unattached marks and diacritics which display
inconsistently across platforms; this is described in more detail in
issue #9031.
2023-06-17 16:21:26 +07:00
Marc Durdin
e9391f57ff chore(developer): refactor osk rewriter
Moves common remapping code for the OSK rewriter into common/web/types
so that it can be consumed by the kmn and kmw compilers soon. Also
cleans up the command line instantiation of the analyze tools, although
the rewrite tool is less likely to be used once we have the compiler
integration in place.
2023-06-17 15:03:02 +07:00
Marc Durdin
3d56bc667e chore(common): export KvksFileWriter 2023-06-14 15:55:56 +07:00
Marc Durdin
d19f9027b7
Merge pull request #8994 from keymanapp/chore/developer/complete-error-checking-in-kmc-analyze
chore(developer): complete error checking in kmc-analyze 🗜
2023-06-13 23:10:46 +10:00
Marc Durdin
f7b524b8db
Merge pull request #8984 from keymanapp/chore/common/web/file-types-module-first-steps
feat(common): add file-types.ts module 🗜
2023-06-13 12:56:46 +10:00
Marc Durdin
d684e58baa
Merge branch 'feature-kmc-kmw' into chore/common/web/file-types-module-first-steps 2023-06-13 08:37:55 +10:00
Marc Durdin
da29d82e01 chore(developer): complete error checking in kmc-analyze 2023-06-12 13:02:04 +07:00
Marc Durdin
f6900d9e85 chore(developer): merge kmc-kmw messages with kmc-kmn messages
These two modules had duplicated messages from kmcmplib. Rewrites the
kmc-kmw messages module to inherit from kmc-kmn and define message
strings where necessary.

While this refactor is not ideal, because it diverges slightly from the
patterns of other message modules, and has a little bit of spaghetti
import cross-modules, I think it is still a step in the right direction
for consolidated message reporting.
2023-06-12 10:29:35 +07:00
Marc Durdin
7ad2df06d9 chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw 2023-06-12 09:31:18 +07:00
Marc Durdin
8a4c005d40 chore(common): define constants for file types
Now that we have the file type detection in place, we can replace
existing references to the file extensions, where possible, with the
constant declarations. While we could rely on TypeScript's typing to use
string-based values, this helps us to avoid using file extensions
directly in string transforms, regex, etc, and use the `KeymanFileTypes`
alias instead.

Note that only references in common/web, and kmc-* projects have been
touched. References in other Developer projects have not been touched,
as some of those are not yet ESM, so that needs to be addressed first.
2023-06-12 09:17:09 +07:00
Marc Durdin
d2029c40cd chore(common): Create file-types.ts module
This module is used for identification of Keyman source and binary file
formats by file extension. It is not yet appropriate to use for
constants; this will come in a future commit.
2023-06-12 08:20:02 +07:00
Marc Durdin
8bfd4c0bae chore(developer): refactor LdmlCompilerOptions
LdmlCompilerOptions is now based on the shared CompilerOptions. Also
fixes command line parameter inheritance for sub-commands.
2023-06-10 14:27:40 +07:00
Marc Durdin
459f42b273 chore(developer): refactor CompilerOptions
Part 1 of the `CompilerOptions` reorg. kmc now uses `CompilerOptions`
throughout, and `CompilerBaseOptions` maps to the set of options
available to all activities.
2023-06-10 14:27:39 +07:00
Marc Durdin
cc956117cf chore: remove commented code 2023-06-10 14:26:26 +07:00
Marc Durdin
db06dfd810 chore(developer): Merge branch 'feature-kmc-kmw' into feat/developer/kmc-analyze 2023-06-09 14:00:14 +07:00
Marc Durdin
fc043c960e chore: tweak CompilerLogLevel 2023-06-09 13:02:17 +07:00
Marc Durdin
3b5c38aa80 feat(developer): use new memory-based kmx compiler in analyze
* Also adds info messages for scanning.
2023-06-09 12:38:20 +07:00
Marc Durdin
fde428fb70 chore(developer): add logLevel to kmc
Tidies up silent vs quiet vs verbose by adding logLevel as an option.

* Notes that we need to consolidate all the various CompilerOptions
  interfaces before they get too much further out of hand.
* Cleans up a couple of other minor TODO items.
2023-06-09 10:23:06 +07:00
Steven R. Loomis
4febfcb451 feat(developer): improve tran/bksp tests, other improvements 🙀
- delete dead code
- share common ID regex for vars
- move bksp/finl/order tests into tran

For: #7377
2023-06-08 18:39:44 -05:00
Steven R. Loomis
26053a5ba3
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-06-08 18:10:11 -05:00
Marc Durdin
c02b271394 chore: cleanup unneeded changes 2023-06-08 15:17:09 +07:00
Marc Durdin
f6e52270ed chore(developer): Merge branch 'feature-kmc-kmw' into feat/developer/kmc-analyze 2023-06-08 14:59:28 +07:00
Marc Durdin
9f71f093cb chore(common): export KmxFileReaderError 2023-06-08 08:51:08 +07:00
Marc Durdin
a1fdc7b133 refactor(developer): CompilerOptions as shared type
Also fixes handling of kmx errors in kmc-package.
2023-06-08 05:42:23 +07:00
Marc Durdin
4c3acaf651 chore(common): update restructure to support typescript 2023-06-08 05:40:48 +07:00
Steven R. Loomis
ad1d4e734c feat(common): fix bug in build-tran 🙀
For: #7377
2023-06-06 18:37:07 -05:00
Steven R. Loomis
8548b0fd33 Merge remote-tracking branch 'upstream/master' into feat/developer-7377-transform-format-epic-ldml3 2023-06-06 09:45:21 -05:00
Marc Durdin
fdd313a6a2 chore(common): cleanup error handling for KmxFileReader 2023-06-06 15:22:00 +07:00
Steven R. Loomis
70fcea815a feat(developer): tran support 🙀
- tran builder

For: #7377
2023-06-05 19:18:59 -05:00
Marc Durdin
eaf7a6b628 chore(developer): merge changed interfaces in compiler 2023-06-05 16:34:23 +07:00
Marc Durdin
3f993bd651 chore: Merge branch 'master' into feat/typescript-keymanweb-compiler 2023-06-05 15:57:42 +07:00
Steven R. Loomis
2054eeb703 feat(developer): tran support 🙀
- support regex format for sets and unicodeSets
- support mapFrom and mapTo detection

For: #7377
2023-06-02 18:31:25 -05:00
Steven R. Loomis
4b81137a61 feat(developer): section dependencies 🙀
- fix test helper for dependencies
- moved VariableParser and MarkerParser into common

For: #7377
2023-06-02 16:58:39 -05:00
Steven R. Loomis
9b177b2f00 feat(developer): section dependencies 🙀
- make explicit section dependencies, add a test

For: #7377
2023-06-02 15:19:00 -05:00
Marc Durdin
0ec5c08b26 chore(common): move const url to class prop
We still have cjs modules that rely on the ldml-keyboard-xml-reader, and
this meant that an exported const was being calculated with an invalid
meta url, which crashed the cjs module require() call.
2023-06-02 12:28:13 +07:00
Marc Durdin
4e535832d8 chore: cleanup 2023-06-02 11:45:59 +07:00
Marc Durdin
a281e1ac60 chore(common): remove redundant path 2023-06-02 11:43:25 +07:00
Marc Durdin
b313272250 chore(common): remove url module ref from common/web/types
The url module is a node module. We need to move responsibility for
resolving the path of the LDML XML <import> statements out of
common/web/types, and into the ultimate consumer, so it's now surfaced
as an option, along with a helper constant that reports the
import.meta.url-relative base path of the standard imports that are
compiled into common/web/types.

This hopefully means we can use this module in both browser and node
contexts without trouble.
2023-06-02 11:38:17 +07:00
Marc Durdin
c3458d040c refactor(developer): move filename consistency check to kmc
kmcmplib no longer has any filesystem access, so it cannot verify if a
referenced filename in a source file has the same case as the actual
filename on disk (a risk when moving projects between platforms). So
I opted to move this to the `loadFile` callback in kmc, which is the
only place where filesystem is actually accessed, and added
corresponding unit test.

Small additional fixes here:

1. Move from `Buffer` to `Uint8Array` in all kmc-* modules, so that we
   remove that barrier to running on web.
2. Use `callbacks.loadFile` instead of `callbacks.fs.readFileSync`, so
   that we can be sure to run the filename consistency check.
3. Fixed kps parser silently swallowing xml errors on load.
4. Added silent mode to NodeCompilerCallbacks so we could cleanly test
   the new filename consistency hint.
5. Noted a location where we still have NodeJS deps in kmc-ldml.
2023-06-01 15:11:38 +07:00
Marc Durdin
9d351f4798 chore(developer): verify kvks files and report errors
Fixes #8886.

.kvks compiler now returns helpful errors for xml parse failures and
schema validation errors, and for invalid virtual key codes. Introduces
extra infrastructure for reporting and unit testing messages to kmc-kmn.

This also fixes unhandled xml load exceptions and simplifies the error
reporting coming out of the kvks loader, on the basis that only one type
of error was being reported anyway.
2023-06-01 05:02:26 +07:00
Steven R. Loomis
ca602385b3 feat(developer): vars: hooray for more errors 🙀
For: #7377
2023-05-27 17:01:10 -05:00
Marc Durdin
d1d6dc5d99 refactor(developer): compiler interface part 1 for wasm
This starts the refactor of the .kmn compiler interfaces for WASM,
moving more file writes into typescript and out of the compiler. Adds a
cleaner binding and structures for passing compile metadata back to
typescript for additional build steps (e.g. kvk, js builds).

Will rebase the Typescript-based KMW compiler onto this so it can start
to benefit from the refactoring.

* Fixes a minor issue with loading a .kvks file with an empty flags set.
* Adds a writeFileSync method to the compiler callback interfaces.
* Adds an e2e compiler test to verify that .kmx and .kvk are both
  correctly compiled.
2023-05-23 15:26:28 +07:00
Steven R. Loomis
518fcd8fc2 feat(developer): tran: compiler path complete 🙀
- still doesn't actually precompile vars or regex

For: #7377
2023-05-19 19:09:44 -05:00
Steven R. Loomis
3165eae0ea feat(developer): tran: verify steps 🙀
- also add function compilerEventFormat() to format an event or array of events

For: #7377
2023-05-19 13:23:05 -05:00
Steven R. Loomis
4a53bd18de feat(developer): tran: group steps 🙀
For: #7377
2023-05-17 17:29:16 -05:00