Commit graph

2740 commits

Author SHA1 Message Date
Marc Durdin
3730f2db53
Merge pull request #9058 from keymanapp/chore/developer/kmw-support-embed-files
chore(developer): support embedded files in kmw compiler 🗜
2023-06-23 13:25:23 +10:00
Marc Durdin
fd640288fb
Merge pull request #9057 from keymanapp/chore/developer/wasm-load-only-once
chore(developer): load wasm module only once per process 🗜
2023-06-23 13:25:03 +10:00
Marc Durdin
2222e875c9
Merge pull request #9056 from keymanapp/chore/developer/cleanup-display-underlying-flag
chore(developer): cleanup displayUnderlying flag in kmw compiler 🗜
2023-06-23 13:24:52 +10:00
Marc Durdin
6928188d13
Merge pull request #9054 from keymanapp/chore/developer/cleanup-font-style-in-kvk
chore(developer): remove font style/color from kvk 🗜
2023-06-23 13:24:41 +10:00
Marc Durdin
23ef2da1be
Merge pull request #9046 from keymanapp/chore/developer/kmc-web-compiler-cleanup
refactor(developer): move constants and globals for kmw compiler 🗜
2023-06-23 13:24:26 +10:00
Marc Durdin
9216a69636
chore: Update developer/src/kmc-kmn/src/compiler/compiler.ts
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2023-06-23 13:23:29 +10:00
Marc Durdin
557c400d57
chore: Apply suggestions from code review
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2023-06-23 13:21:23 +10:00
Marc Durdin
8cd28bdc97 chore(developer): rename data to extra 2023-06-22 07:47:39 +07:00
Marc Durdin
24910fb25a chore(developer): cleanup data passing from wasm
Reorganizes the data structures passed out of kmcmplib via WASM to make
it easier to work with the extra metadata and reduce the number of
places we have to touch when we add extra metadata fields.

Preparation for supporting the metadata that the kmw compiler requires.
2023-06-21 15:33:09 +07:00
Marc Durdin
165dcbe4aa chore(developer): support embedded files in kmw compiler
Adds support for `&kmw_embedcss`, `&kmw_embedjs`, `&kmw_helpfile`.
2023-06-21 12:29:51 +07:00
Marc Durdin
2c2e6feaa3 chore(developer): load wasm module only once per process
Improves performance, and also resolves warning from node:

(node:21032) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 uncaughtException listeners added to [process]. Use emitter.setMaxListeners() to
increase limit
2023-06-21 11:52:57 +07:00
Marc Durdin
a67abadb2f chore(developer): cleanup displayUnderlying flag in kmw compiler 2023-06-21 11:46:24 +07:00
Marc Durdin
3f507f974a chore(developer): remove font style/color from kvk
While .kvk includes a font color field, it is essentially unused. There
was also some sort of support provided for font style in the KeymanWeb
keyboard compiler, but this was never streamed into .kvk or .kvks files,
so was always a no-op.

Thus, this PR removes any semblance of support for font style and color
from the .kvk and .kvks readers and writers. When we write the font
color field which is present in .kvk, we always use the default
TColor.clWindowText which is what the legacy Delphi-based writer would
always have written.

Also sorted out the default font size and name in the .kvk transform
from .kvks, and in so doing cleaned up the basic.kvk and basic.js in
LDML keyboard compiler to match what we are doing in the .kmn compiler.
2023-06-21 10:53:45 +07:00
Steven R. Loomis
248d423535 feat(common,developer,core): add uset section 🙀
- not emitted yet, basic scaffolding
- expanded type field in elem:  str/char/set

For: #7377
2023-06-20 18:23:54 -05:00
Marc Durdin
78d56822c9
Merge pull request #9043 from keymanapp/chore/developer/use-schema-to-validate-display-map
chore(developer): validate displaymap schema 🗜
2023-06-20 18:16:10 +10:00
Marc Durdin
1890176c11 chore(developer): remove extraneous comment 2023-06-20 15:09:14 +07:00
Marc Durdin
749e42e0a0 chore(developer): move additional consts and globals 2023-06-20 15:08:33 +07:00
Marc Durdin
dae94fd4d5 chore(developer): move constants for kmw-compiler 2023-06-20 14:59:11 +07:00
Marc Durdin
57320435cc
Update developer/src/kmc-kmn/test/kmw/test-compiler-manual.ts 2023-06-20 17:11:47 +10:00
Marc Durdin
c68403fabf chore(developer): validate displaymap schema 2023-06-20 12:57:57 +07:00
Marc Durdin
0bd2ca4821 refactor(developer): kmc-kmw becomes component of kmc-kmn
Moves kmc-kmw to being a sub-component of kmc-kmn, as it can never be
independently instantiated anyway. Reorganized tests accordingly.

Note that c8 is currently disabled for kmw-compiler, until we add unit
tests for it.
2023-06-20 11:52:23 +07:00
Marc Durdin
cf91dd378b chore(developer): remove extraneous comment 2023-06-19 15:32:34 +07:00
Marc Durdin
6462b5fc63 refactor(developer): kmc-kmn now builds .js when needed
kmc-kmn calls into kmc-kmw to build .js when it is needed, rather than
kmc-kmw depending on kmc-kmn. This means examining the `&targets` system store
to determine which files need to be generated.

This also implements the `&displayMap` rewrite for both .kvks and
.keyman-touch-layout as part of the KMW compiler.

I suspect that the next step is to place kmc-kmw into a subfolder of
kmc-kmn, given they are part of the same process, and there is no
practical way to separate out the .js generation from the .kmx
generation.
2023-06-19 15:25:09 +07:00
Marc Durdin
adfa032e6f feat(developer): add &displayMap system store
The &displayMap system store adds support for a mapping file that remaps
the On Screen Keyboard files -- .kvks and .keyman-touch-layout, at
compile time, to new ranges. This is specifically intended to resolve a
limitation of unattached marks and diacritics which display
inconsistently across platforms; this is described in more detail in
issue #9031.
2023-06-17 16:21:26 +07:00
Marc Durdin
e9391f57ff chore(developer): refactor osk rewriter
Moves common remapping code for the OSK rewriter into common/web/types
so that it can be consumed by the kmn and kmw compilers soon. Also
cleans up the command line instantiation of the analyze tools, although
the rewrite tool is less likely to be used once we have the compiler
integration in place.
2023-06-17 15:03:02 +07:00
Marc Durdin
ce90742eb9 feat(developer): support to rewrite .kvks and .keyman-touch-layout with PUA 2023-06-14 16:01:19 +07:00
Marc Durdin
df4ec7d94c feat(developer): analyze osk char use continued
Refactoring, additional options for analyze, moving project scanning
into kmc rather than kmc-analyze.
2023-06-14 14:06:45 +07:00
Joshua A. Horton
f35ba0973e chore(developer): kmc-model no longer needs to bundle a .cjs version for input-processor 2023-06-14 10:15:56 +07:00
Marc Durdin
fac3cb02d0 chore(developer): fixup merge bump 2023-06-13 20:42:10 +07:00
Marc Durdin
d19f9027b7
Merge pull request #8994 from keymanapp/chore/developer/complete-error-checking-in-kmc-analyze
chore(developer): complete error checking in kmc-analyze 🗜
2023-06-13 23:10:46 +10:00
Marc Durdin
d2bf05c94a
Merge pull request #8991 from keymanapp/chore/developer/prevent-project-recursion-in-analyze
chore(developer): prevent recursion when analyzing projects 🗜
2023-06-13 23:10:11 +10:00
Marc Durdin
f7b524b8db
Merge pull request #8984 from keymanapp/chore/common/web/file-types-module-first-steps
feat(common): add file-types.ts module 🗜
2023-06-13 12:56:46 +10:00
Marc Durdin
3f6f11e1da
Merge pull request #8999 from keymanapp/fix/developer/8998-message-namespace-creep
fix(developer): keep kmc-kmn messages within namespace
2023-06-13 12:53:31 +10:00
Marc Durdin
b29c78a83b
chore: remove redundant check 2023-06-13 09:39:45 +10:00
Marc Durdin
cbc5b13034 fix(developer): keep kmc-kmn messages within namespace
Fixes #8998.

kmc-kmn messages were broken down into sub-namespaces, but the range
overlapped the reserved namespace mask range. Fixed the message values,
added a new test to the verifyCompilerMessagesObject function to verify
that messages don't creep outside their namespace, and defined new
masks (in CompilerErrorSeverity enum... hmm) to help.

I opted to remove the leading `0` in the messages for kmc-kmn as I had
to correct a number of them anyway, but not the messages in remaining
units at this time (it's not an error, but just slightly misleading as
we only have 12 bits, not 16 to play with).

It is safe to reassign these messages as they were only assigned in 17.0
alpha.
2023-06-13 06:22:15 +07:00
Marc Durdin
d684e58baa
Merge branch 'feature-kmc-kmw' into chore/common/web/file-types-module-first-steps 2023-06-13 08:37:55 +10:00
Marc Durdin
da29d82e01 chore(developer): complete error checking in kmc-analyze 2023-06-12 13:02:04 +07:00
Marc Durdin
b050d018f2 chore(developer): prevent recursion when analyzing projects 2023-06-12 12:44:09 +07:00
Marc Durdin
6db014224c chore(developer): enable hint message for non-Unicode source files 2023-06-12 12:00:48 +07:00
Marc Durdin
57af86ea9c chore(developer): support cp1252 'ansi' keyboards
Fixes #8955.

Adds a mapping table for CP1252 to USV so that we can compile very old
"ANSI"-encoded Keyman keyboard files with kmcmplib, to match kmcmpdll
support.
2023-06-12 11:43:29 +07:00
Marc Durdin
f6900d9e85 chore(developer): merge kmc-kmw messages with kmc-kmn messages
These two modules had duplicated messages from kmcmplib. Rewrites the
kmc-kmw messages module to inherit from kmc-kmn and define message
strings where necessary.

While this refactor is not ideal, because it diverges slightly from the
patterns of other message modules, and has a little bit of spaghetti
import cross-modules, I think it is still a step in the right direction
for consolidated message reporting.
2023-06-12 10:29:35 +07:00
Marc Durdin
769a08f027 chore(developer): remove old comment 2023-06-12 09:49:12 +07:00
Marc Durdin
f04900a9e9 chore(developer): fixup .kpj reference 2023-06-12 09:36:36 +07:00
Marc Durdin
7ad2df06d9 chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw 2023-06-12 09:31:18 +07:00
Marc Durdin
66f2489573
Merge pull request #8982 from keymanapp/chore/developer/refactor-compiler-options
chore(developer): refactor CompilerOptions 🗜
2023-06-12 12:20:39 +10:00
Marc Durdin
0b5b674d67
Merge pull request #8968 from keymanapp/chore/developer/enable-eslint-kmc-kmw
chore(developer): enable eslint for kmc-kmw 🗜
2023-06-12 12:20:31 +10:00
Marc Durdin
a25f8a723f
Merge pull request #8962 from keymanapp/refactor/developer/kmcmplib-error-code-patterns
refactor(developer): import kmcmplib errors into kmc-kmn
2023-06-12 12:20:10 +10:00
Marc Durdin
8a4c005d40 chore(common): define constants for file types
Now that we have the file type detection in place, we can replace
existing references to the file extensions, where possible, with the
constant declarations. While we could rely on TypeScript's typing to use
string-based values, this helps us to avoid using file extensions
directly in string transforms, regex, etc, and use the `KeymanFileTypes`
alias instead.

Note that only references in common/web, and kmc-* projects have been
touched. References in other Developer projects have not been touched,
as some of those are not yet ESM, so that needs to be addressed first.
2023-06-12 09:17:09 +07:00
Marc Durdin
01b575128f
chore: address review comments 2023-06-12 11:55:03 +10:00
Marc Durdin
d2029c40cd chore(common): Create file-types.ts module
This module is used for identification of Keyman source and binary file
formats by file extension. It is not yet appropriate to use for
constants; this will come in a future commit.
2023-06-12 08:20:02 +07:00