Commit graph

1145 commits

Author SHA1 Message Date
Steven R. Loomis
56dcbc9a88 fix(developer): support updated CLDR structure 🙀
- backspace is now a transform type. Keep the same test.
2022-10-18 12:53:06 -05:00
Steven R. Loomis
85da473a85 chore(developer): update DTD location 🙀
- dtd is in a subdirectory to match CLDR layout
2022-10-18 12:52:27 -05:00
Steven R. Loomis
66a5cdb1f9 chore(developer): ldml: update tools and data for keyboard-preview 🙀
- update tooling in common and kmc
- update spec
- update test files
2022-10-18 11:32:41 -05:00
Steven R. Loomis
85ed41e16c
Merge pull request #7410 from keymanapp/fix/developer/7404-elem0-epic-ldml 2022-10-17 08:53:23 -05:00
Steven R. Loomis
7e5c9ec675 fix(developer): offset of 0th elem str must be 0000 🙀
- update test fixture
- update kmx_plus.cpp to require this

Fix: 7404
2022-10-14 15:00:49 -05:00
Marc Durdin
2cf6bf3f60 chore(common): update build.sh dependencies for epic-ldml 2022-10-14 10:45:00 +11:00
Marc Durdin
0945b92732 chore: Merge branch 'master' into chore/merge-master-into-feature-ldml 2022-10-13 13:38:31 +11:00
Marc Durdin
47da67a742 chore(developer): fixup import 2022-10-10 11:51:55 +11:00
Marc Durdin
faa37f24a9 chore(developer): handle EOL diff in tests 2022-10-10 10:55:35 +11:00
Marc Durdin
b4641efe69 feat(developer): generate .js from ldml .xml
Compiles a LDML keyboard .xml down to a KeymanWeb .js file. Currently
only supports some metadata.

Next step is to generate touch layout data, then to embed the binary kmx
data.
2022-10-10 10:08:37 +11:00
Marc Durdin
8d7ea6dcbb chore(developer): enable sourcemaps for kmc 2022-10-10 05:42:18 +11:00
Marc Durdin
1dd4fe5315 chore(developer): enable source maps for kmc commands
Fixes #7408.

For kmc*.cmd, enables source maps. If you are running kmc from node on
your own system, you may need to enable source maps in other ways.

Note that error reports should be going through to Sentry, and source
maps should already be used for those.
2022-10-06 17:48:22 +11:00
Marc Durdin
61a058fcec
Merge pull request #7371 from keymanapp/fix/developer/7368-handle-invalid-kps-in-project-view
fix(developer): handle invalid kps xml when loading project view
2022-10-03 14:26:59 +11:00
Marc Durdin
e28da7b1dd fix(developer): kmc was overwriting kmx with kvk 2022-09-30 07:54:39 +10:00
Marc Durdin
687c8bae99
Merge pull request #7370 from keymanapp/fix/developer/7090-7091-use-path-backslash-format-for-parameters
fix(developer): ensure backslash in paths passed to kmcomp
2022-09-29 09:22:57 +10:00
Marc Durdin
73e0a2e461 fix(developer): generate valid js when using unquoted digits in stores
Fixes #7005.

If a developer used single digit values in stores, unquoted (which is
not really the usual approach, but should work just fine for non-zero
values), the KMW compiler would emit invalid single-digit hexadecimal
escapes for them, e.g. `"\x1"` instead of `"\x01"`.

Sample code:

```
store(option) 1

if(option = 1) + 'a' > 'one'
if(option = 2) + 'a' > 'two'
+ '1' > set(option = 1)
+ '2' > set(option = 2)
```

It is not immediately obvious from this code, but the value `1` is
actually a character with value `'\x01'` or `U+0001`!

The more usual approach will not encounter this problem:

```
store(option) '1'

if(option = '1') + 'a' > 'one'
if(option = '2') + 'a' > 'two'
+ '1' > set(option = '1')
+ '2' > set(option = '2')
```

Note that the following is a compile error (due to internal use of
null terminated strings).

```
store(option) 0
```
2022-09-28 17:00:55 +10:00
Marc Durdin
f5b8effaae fix(developer): ensure backslash in paths passed to kmcomp
Fixes #7090.
Fixes #7091.

While kmcomp loads most files just fine when paths with forward slashes
are passed in, it fails to parse target path correctly when building one
file out of a project.

For example, if the user runs:

```
kmcomp.exe -t khmer_angkor.kmn release/k/khmer_angkor/khmer_angkor.kpj
```

Then the following error arises:

```
khmer_angkor.kps: Error: 0001 Validation error: The system cannot find the path specified.
khmer_angkor.kps: Failure: Package C:\Projects\keyman\keyboards\source\khmer_angkor.kps had validation errors.
```

This patch ensures all paths passed in as command-line parameters are
converted to backslashes, matching the expected Windows format. While I
only needed to fix the `FParamTarget` parameter in order to address the
reported issues, for consistency I applied the same fix to all input
paths.
2022-09-28 16:50:58 +10:00
Marc Durdin
50614fb088 fix(developer): handle invalid kps xml when loading project view
Fixes #7368.
2022-09-28 16:32:36 +10:00
Marc Durdin
0b701275fd chore(developer): remove unused var 2022-09-28 15:25:35 +10:00
Marc Durdin
0f961bd674 feat(developer): visual keyboard compiler in kmc
Adds support for building .kvk files to kmc. At present, .kvk will
always be generated, even if the .xml does not have a hardware layer.
2022-09-27 15:16:07 +10:00
Marc Durdin
b6abe7432a feat(developer): emit .kvk plus e2e test
Adds visual-keyboard-compiler and corresponding e2e test.
2022-09-27 15:03:31 +10:00
Marc Durdin
03c3c40f29 chore(developer): remove visual-keyboard-compiler.ts 2022-09-27 13:42:56 +10:00
Marc Durdin
26e32a0124 refactor(developer): split file type management into package
Splits file format reading, writing, and in-memory structures of .kmx,
.kvk, .kvks, .xml (ldml keyboard) into separate common module. Lots of
little fixups, sorry...

This will make it much easier for us to reuse these readers and writers
in future code.
2022-09-27 12:33:10 +10:00
Marc Durdin
52dd3e0e85 refactor(developer): move compiler callbacks out of kvks file reader 2022-09-27 05:04:04 +10:00
Marc Durdin
5c65405f65 refactor(developer): move compiler callbacks out of ldml xml reader 2022-09-27 04:52:00 +10:00
Marc Durdin
9480d8931e chore(developer): remove commented code 2022-09-26 20:41:59 +10:00
Marc Durdin
74fc795c1c feat(developer): refactor CompilerCallbacks in tests
Now has a shared object for all tests, reset between tests, and if the
test fails, prints any reported compiler messages.

Also updates kvk and kvks tests with refactored names from previous
commit.
2022-09-26 20:07:00 +10:00
Marc Durdin
50db7a2c9b feat(developer): add schemas for .kvk and .kvks
Also minor tweaks around reading .xml files.
2022-09-26 20:05:52 +10:00
Marc Durdin
c7740a39a2 feat(developer): kvk support
Part of #7238.

Supports reading and writing .kvk files, and reading .kvks files,
to/from an in-memory VisualKeyboard object.

TODO:
 - Full validation of .kvks files
 - Writing .kvks files
 - Some cleanup of API surfaces for consistency
2022-09-26 09:29:38 +10:00
Marc Durdin
828cbdf584
Merge pull request #7341 from keymanapp/feat/developer/epic-ldml/7277-kmc-makes-keyboard-packages
feat(developer): fixes to kmc-package for keyboard support 🙀
2022-09-24 10:12:38 +10:00
Marc Durdin
0502d6ccbd
Merge pull request #7342 from keymanapp/chore/epic-ldml/merge-master-to-feature-ldml
chore: merge master into feature-ldml 🙀
2022-09-24 10:12:00 +10:00
Steven R. Loomis
03012248a2 chore(windows): remove crc32 from tike 2022-09-23 11:48:47 -05:00
Steven R. Loomis
700db810e0 fix(core): fix for incorrect earlier fix 2022-09-22 17:26:39 -05:00
Steven R. Loomis
de3225a4f5 Merge remote-tracking branch 'origin/feature-ldml' into chore/core/7276-checksums-epic-ldml 2022-09-22 17:13:27 -05:00
Marc Durdin
33fd8cffba chore(developer): ts v4.8.3 upgrade breaks mocha 10.0.0 with ts-node
Mocha with ts-register was failing with an error for Keyman Developer
Server.

```
Error: Debug Failure. False expression: Non-string value passed to
`ts.resolveTypeReferenceDirective`, likely by a wrapping package working
with an outdated `resolveTypeReferenceDirectives` signature. This is
probably not a problem in TS itself.
```

The `^4.5.4` version reference for typescript was too lax, and so npm
gave us typescript 4.8.3, which was the latest version in use in the
repo in this branch (in kmc-model). This is also why the build only
failed on this branch -- on master branch, the latest version referenced
was still 4.5.4.

Changing to `~4.5.4` ensures that it stayed on the 4.5.x series, and
mocha then ran version 4.5.5 of tsc.

This is something we should resolve when we move the entire repo to tsc
4.8+. Currently we are still on 4.5.

Note: it may be sensible to review all versioning for typescript,
because the minor version changes are causing functionality breaks,
which kinda violates the spirit of semver ("MINOR version when you add
functionality in a backwards compatible manner"). We can be more
restrictive by updating all package.json references to typescript to
`~4.5.4` instead of the current `^4.5.4` in use throughout. If/when we
do this, we will need to check our ts-node references also.
2022-09-23 05:25:24 +10:00
Marc Durdin
6f949ad63f chore(developer): fixup dependency type changes
* xml2js, `emptyTag` property: https://github.com/DefinitelyTyped/DefinitelyTyped/pull/59259
* semver `parse` `includePrerelease` property: https://github.com/DefinitelyTyped/DefinitelyTyped/pull/61586#discussion_r939877412
* commander use needed to be declared for hextobin
2022-09-22 11:46:44 +10:00
Steven R. Loomis
93a9509302 chore(core): delete crc32.cpp and crc32.pas from some places 2022-09-21 18:13:19 -05:00
Marc Durdin
cae408edc5 chore: merge feature-ldml into branch 2022-09-21 14:56:45 +10:00
Marc Durdin
f3fbf01d30 feat(developer): fixes to kmc-package for keyboard support
kmc-package had a number of gaps and errors. This fixes a bunch of
problems:

* Wrong case on XML elements
* Wrong metadata in system element
* Missing kmp.json in Files array (needs test in consumer apps later)
* Missing displayFont, oskFont in keyboard elements
* Including copyLocation when == 0
* Including an empty startMenu element
* Incorrect assumptions about location of source files in source/ folder

Note that support for followKeyboardVersion will come later; for now, it
uses the possibly-stale value from the Keyboard element in the .kps
file.
2022-09-21 14:37:08 +10:00
Marc Durdin
bd5f6877bb feat(developer): fixup fixture json 2022-09-21 14:33:04 +10:00
Marc Durdin
2b9a2949a6 feat(developer): kmc-package khmer_angkor fixture 2022-09-21 14:32:42 +10:00
Marc Durdin
55f302d929
Merge pull request #7322 from keymanapp/chore/merge-master-to-feature-ldml
chore: merge master into feature-ldml 🙀
2022-09-20 13:42:40 +10:00
Marc Durdin
d1de02d43e chore(developer): Merge branch 'master' into chore/developer/server-node-gyp-dependency 2022-09-20 10:19:11 +10:00
Marc Durdin
9f290cac21 chore(developer): Merge branch 'master' into chore/developer/server-update-multer-dependency 2022-09-20 10:16:46 +10:00
Marc Durdin
aab9d1aad5 chore(developer): Merge branch 'master' into chore/developer/server-node-gyp-dependency 2022-09-20 10:14:16 +10:00
Steven R. Loomis
66281da0e0
Update developer/src/kmcmpdll/Compiler.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-09-19 14:59:46 -05:00
Marc Durdin
93a1fc88b9 chore(developer): remove crc generation from kmc-keyboard 2022-09-19 15:30:38 +10:00
Marc Durdin
a59a6c334a chore: update node-windows-trayicon to update node-gyp to 9.1.0 2022-09-19 15:07:24 +10:00
Marc Durdin
3d61d18371 chore(developer): update multer dependency for server
Removes Keyman Developer Server's transitive dependency on dicer by
updating multer to `1.4.5-lts.1`, which updates its dependency on
busboy.

See
https://github.com/expressjs/multer/pull/1097#issuecomment-1141286771
for reasoning behind use of  `-lts.1` rather than a full release
version.

At some point in the future, multer will publish a full release with
this fix, at which point we can move back to a full release version.
2022-09-19 12:36:51 +10:00
Marc Durdin
ee47e49568 chore(developer): fixup build path for kmc 2022-09-19 08:42:27 +10:00