Commit graph

3368 commits

Author SHA1 Message Date
Marc Durdin
adfa032e6f feat(developer): add &displayMap system store
The &displayMap system store adds support for a mapping file that remaps
the On Screen Keyboard files -- .kvks and .keyman-touch-layout, at
compile time, to new ranges. This is specifically intended to resolve a
limitation of unattached marks and diacritics which display
inconsistently across platforms; this is described in more detail in
issue #9031.
2023-06-17 16:21:26 +07:00
Marc Durdin
e9391f57ff chore(developer): refactor osk rewriter
Moves common remapping code for the OSK rewriter into common/web/types
so that it can be consumed by the kmn and kmw compilers soon. Also
cleans up the command line instantiation of the analyze tools, although
the rewrite tool is less likely to be used once we have the compiler
integration in place.
2023-06-17 15:03:02 +07:00
Marc Durdin
ce90742eb9 feat(developer): support to rewrite .kvks and .keyman-touch-layout with PUA 2023-06-14 16:01:19 +07:00
Marc Durdin
df4ec7d94c feat(developer): analyze osk char use continued
Refactoring, additional options for analyze, moving project scanning
into kmc rather than kmc-analyze.
2023-06-14 14:06:45 +07:00
Joshua A. Horton
f35ba0973e chore(developer): kmc-model no longer needs to bundle a .cjs version for input-processor 2023-06-14 10:15:56 +07:00
Marc Durdin
fac3cb02d0 chore(developer): fixup merge bump 2023-06-13 20:42:10 +07:00
Marc Durdin
d19f9027b7
Merge pull request #8994 from keymanapp/chore/developer/complete-error-checking-in-kmc-analyze
chore(developer): complete error checking in kmc-analyze 🗜
2023-06-13 23:10:46 +10:00
Marc Durdin
d2bf05c94a
Merge pull request #8991 from keymanapp/chore/developer/prevent-project-recursion-in-analyze
chore(developer): prevent recursion when analyzing projects 🗜
2023-06-13 23:10:11 +10:00
Marc Durdin
f7b524b8db
Merge pull request #8984 from keymanapp/chore/common/web/file-types-module-first-steps
feat(common): add file-types.ts module 🗜
2023-06-13 12:56:46 +10:00
Marc Durdin
3f6f11e1da
Merge pull request #8999 from keymanapp/fix/developer/8998-message-namespace-creep
fix(developer): keep kmc-kmn messages within namespace
2023-06-13 12:53:31 +10:00
Marc Durdin
b29c78a83b
chore: remove redundant check 2023-06-13 09:39:45 +10:00
Marc Durdin
cbc5b13034 fix(developer): keep kmc-kmn messages within namespace
Fixes #8998.

kmc-kmn messages were broken down into sub-namespaces, but the range
overlapped the reserved namespace mask range. Fixed the message values,
added a new test to the verifyCompilerMessagesObject function to verify
that messages don't creep outside their namespace, and defined new
masks (in CompilerErrorSeverity enum... hmm) to help.

I opted to remove the leading `0` in the messages for kmc-kmn as I had
to correct a number of them anyway, but not the messages in remaining
units at this time (it's not an error, but just slightly misleading as
we only have 12 bits, not 16 to play with).

It is safe to reassign these messages as they were only assigned in 17.0
alpha.
2023-06-13 06:22:15 +07:00
Marc Durdin
d684e58baa
Merge branch 'feature-kmc-kmw' into chore/common/web/file-types-module-first-steps 2023-06-13 08:37:55 +10:00
Marc Durdin
da29d82e01 chore(developer): complete error checking in kmc-analyze 2023-06-12 13:02:04 +07:00
Marc Durdin
b050d018f2 chore(developer): prevent recursion when analyzing projects 2023-06-12 12:44:09 +07:00
Marc Durdin
6db014224c chore(developer): enable hint message for non-Unicode source files 2023-06-12 12:00:48 +07:00
Marc Durdin
57af86ea9c chore(developer): support cp1252 'ansi' keyboards
Fixes #8955.

Adds a mapping table for CP1252 to USV so that we can compile very old
"ANSI"-encoded Keyman keyboard files with kmcmplib, to match kmcmpdll
support.
2023-06-12 11:43:29 +07:00
Marc Durdin
f6900d9e85 chore(developer): merge kmc-kmw messages with kmc-kmn messages
These two modules had duplicated messages from kmcmplib. Rewrites the
kmc-kmw messages module to inherit from kmc-kmn and define message
strings where necessary.

While this refactor is not ideal, because it diverges slightly from the
patterns of other message modules, and has a little bit of spaghetti
import cross-modules, I think it is still a step in the right direction
for consolidated message reporting.
2023-06-12 10:29:35 +07:00
Marc Durdin
769a08f027 chore(developer): remove old comment 2023-06-12 09:49:12 +07:00
Marc Durdin
f04900a9e9 chore(developer): fixup .kpj reference 2023-06-12 09:36:36 +07:00
Marc Durdin
7ad2df06d9 chore: Merge branch 'feature-kmc-kmw' into chore/merge-master-into-feature-kmc-kmw 2023-06-12 09:31:18 +07:00
Marc Durdin
66f2489573
Merge pull request #8982 from keymanapp/chore/developer/refactor-compiler-options
chore(developer): refactor CompilerOptions 🗜
2023-06-12 12:20:39 +10:00
Marc Durdin
0b5b674d67
Merge pull request #8968 from keymanapp/chore/developer/enable-eslint-kmc-kmw
chore(developer): enable eslint for kmc-kmw 🗜
2023-06-12 12:20:31 +10:00
Marc Durdin
a25f8a723f
Merge pull request #8962 from keymanapp/refactor/developer/kmcmplib-error-code-patterns
refactor(developer): import kmcmplib errors into kmc-kmn
2023-06-12 12:20:10 +10:00
Marc Durdin
8a4c005d40 chore(common): define constants for file types
Now that we have the file type detection in place, we can replace
existing references to the file extensions, where possible, with the
constant declarations. While we could rely on TypeScript's typing to use
string-based values, this helps us to avoid using file extensions
directly in string transforms, regex, etc, and use the `KeymanFileTypes`
alias instead.

Note that only references in common/web, and kmc-* projects have been
touched. References in other Developer projects have not been touched,
as some of those are not yet ESM, so that needs to be addressed first.
2023-06-12 09:17:09 +07:00
Marc Durdin
01b575128f
chore: address review comments 2023-06-12 11:55:03 +10:00
Marc Durdin
d2029c40cd chore(common): Create file-types.ts module
This module is used for identification of Keyman source and binary file
formats by file extension. It is not yet appropriate to use for
constants; this will come in a future commit.
2023-06-12 08:20:02 +07:00
Marc Durdin
8bfd4c0bae chore(developer): refactor LdmlCompilerOptions
LdmlCompilerOptions is now based on the shared CompilerOptions. Also
fixes command line parameter inheritance for sub-commands.
2023-06-10 14:27:40 +07:00
Marc Durdin
459f42b273 chore(developer): refactor CompilerOptions
Part 1 of the `CompilerOptions` reorg. kmc now uses `CompilerOptions`
throughout, and `CompilerBaseOptions` maps to the set of options
available to all activities.
2023-06-10 14:27:39 +07:00
Marc Durdin
db06dfd810 chore(developer): Merge branch 'feature-kmc-kmw' into feat/developer/kmc-analyze 2023-06-09 14:00:14 +07:00
Marc Durdin
16c40d0d99
Merge pull request #8951 from keymanapp/feat/KmxFileReader-error-handling-and-more
chore(common): cleanup error handling for KmxFileReader and CompilerOptions as shared type
2023-06-09 16:57:54 +10:00
Marc Durdin
fc043c960e chore: tweak CompilerLogLevel 2023-06-09 13:02:17 +07:00
Marc Durdin
3b5c38aa80 feat(developer): use new memory-based kmx compiler in analyze
* Also adds info messages for scanning.
2023-06-09 12:38:20 +07:00
Marc Durdin
fde428fb70 chore(developer): add logLevel to kmc
Tidies up silent vs quiet vs verbose by adding logLevel as an option.

* Notes that we need to consolidate all the various CompilerOptions
  interfaces before they get too much further out of hand.
* Cleans up a couple of other minor TODO items.
2023-06-09 10:23:06 +07:00
Marc Durdin
1670b3099f chore(developer): cleanup kmc-analyze build.sh 2023-06-09 10:20:54 +07:00
Steven R. Loomis
2d367756a8
Merge pull request #8967 from keymanapp/fix/developer-7377-codecleanup-transform-epic-ldml 2023-06-08 21:42:34 -05:00
Marc Durdin
3476977d16 chore(developer): enable eslint for kmc-kmw
Relates to #8959.

Also makes build.sh responsible for build rather than package.json.
2023-06-09 09:06:37 +07:00
Marc Durdin
38fde28165
Merge pull request #8695 from keymanapp/feat/developer-7377-transform-format-epic-ldml3
spec(core): spec/impl for transform and vars 🙀
2023-06-09 11:44:40 +10:00
Marc Durdin
b06981fff8 chore(developer): fixup kmc-analyze call in kmc 2023-06-09 06:51:10 +07:00
Steven R. Loomis
4febfcb451 feat(developer): improve tran/bksp tests, other improvements 🙀
- delete dead code
- share common ID regex for vars
- move bksp/finl/order tests into tran

For: #7377
2023-06-08 18:39:44 -05:00
Steven R. Loomis
5cf025b62e
Merge pull request #8942 from keymanapp/feat/core-7377-transform-epic-ldml
feat(core): fixups to C++ side 🙀
2023-06-08 18:17:06 -05:00
Steven R. Loomis
4562b90502
Merge branch 'master' into feat/developer-7377-transform-format-epic-ldml3 2023-06-08 18:12:12 -05:00
Steven R. Loomis
26053a5ba3
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2023-06-08 18:10:11 -05:00
Marc Durdin
915311c5fc chore(developer): fixup gosh ref 2023-06-09 06:03:28 +07:00
Marc Durdin
427c326682 chore(developer): cleanup kmc-analyze build and test 2023-06-09 05:20:25 +07:00
Steven R. Loomis
9f9589b095 feat(developer): fix e2e test 🙀
- re-enable e2e
- fix basic.txt to match

tools (hextobin):
- support diff(block1,block2,divisor)
- document the comma rule

For: #7377
2023-06-08 13:29:15 -05:00
Marc Durdin
c02b271394 chore: cleanup unneeded changes 2023-06-08 15:17:09 +07:00
Marc Durdin
57d3a4617a chore: cleanup unneeded changes 2023-06-08 15:13:59 +07:00
Marc Durdin
f6e52270ed chore(developer): Merge branch 'feature-kmc-kmw' into feat/developer/kmc-analyze 2023-06-08 14:59:28 +07:00
Marc Durdin
1b4bce7f65 chore(developer): skip c8 test for kmc-kmw for now 2023-06-08 14:42:57 +07:00