Commit graph

4256 commits

Author SHA1 Message Date
Sabine
2738052264 chore(developer): check for '_' after Errornumber in Filename 2022-10-06 16:38:49 +02:00
Sabine
d879761de7 chore(developer): change ErrExtra, change CompileMsg, change _finddata in FileExists 2022-10-06 16:33:52 +02:00
Marc Durdin
1dd4fe5315 chore(developer): enable source maps for kmc commands
Fixes #7408.

For kmc*.cmd, enables source maps. If you are running kmc from node on
your own system, you may need to enable source maps in other ways.

Note that error reports should be going through to Sentry, and source
maps should already be used for those.
2022-10-06 17:48:22 +11:00
Sabine
b9acd28703 chore(developer): remove python from meson.build 2022-10-05 17:06:18 +02:00
Sabine
6b11fff8a9 chore(developer): revert changes in AddCompileMessage, tidy up CreateTempFile 2022-10-05 17:04:50 +02:00
Sabine
4536dbfc91 chore(developer): include check for correct Filename + rename var + change whitespace changes 2022-10-04 14:22:08 +02:00
Sabine
fa9a66b3ec chore(developer): change mostly whitespace changes 2022-10-04 14:08:59 +02:00
Sabine
4141b90c33 chore(developer): move CERR-tests into test/keyboards/invalid 2022-10-03 15:50:58 +02:00
Marc Durdin
61a058fcec
Merge pull request #7371 from keymanapp/fix/developer/7368-handle-invalid-kps-in-project-view
fix(developer): handle invalid kps xml when loading project view
2022-10-03 14:26:59 +11:00
Marc Durdin
e28da7b1dd fix(developer): kmc was overwriting kmx with kvk 2022-09-30 07:54:39 +10:00
Joshua A. Horton
fe585b88af chore(web): removal of 'kmw_ip' checks, references 2022-09-29 08:17:47 +07:00
Marc Durdin
687c8bae99
Merge pull request #7370 from keymanapp/fix/developer/7090-7091-use-path-backslash-format-for-parameters
fix(developer): ensure backslash in paths passed to kmcomp
2022-09-29 09:22:57 +10:00
Marc Durdin
7a819598e7 fix(developer): Handle hints and warnings cleanly
Relates to #4324 (but slightly orthogonal.)

We wanted some message classes to be handled in the lexical model editor
more cleanly, including drill-down to line of error. A few changes
required to make this work well:

1. Parse error messages from kmlmc. This is not perfect but works while
   the two projects are kept in sync, which they always are for Keyman
   Developer. Note that at this stage, .ts warnings are not captured in
   this parser, as they are generated by tsc.
2. Drill-down in Wordlist Editor Frame to find line of error
3. Model editor reports ownership of .tsv files so they can be loaded
4. In case of model editor not open (e.g. building from project view),
   the TSV standalone editor was not displaying the frame, so it never
   actually worked.
5. If the text editor had never loaded, then FindError was effectively
   a no-op; adds code to seek to error line after page load finishes.
6. Ensures that if we attempt to seek an error in a sub-file owned by
   an editor (e.g. .tsv owned by .model.ts, .kvks owned by .kmn), that
   the parent editor will be focused first. Does not verify all paths
   here, just the tsv one.
2022-09-28 20:17:44 +10:00
Marc Durdin
73e0a2e461 fix(developer): generate valid js when using unquoted digits in stores
Fixes #7005.

If a developer used single digit values in stores, unquoted (which is
not really the usual approach, but should work just fine for non-zero
values), the KMW compiler would emit invalid single-digit hexadecimal
escapes for them, e.g. `"\x1"` instead of `"\x01"`.

Sample code:

```
store(option) 1

if(option = 1) + 'a' > 'one'
if(option = 2) + 'a' > 'two'
+ '1' > set(option = 1)
+ '2' > set(option = 2)
```

It is not immediately obvious from this code, but the value `1` is
actually a character with value `'\x01'` or `U+0001`!

The more usual approach will not encounter this problem:

```
store(option) '1'

if(option = '1') + 'a' > 'one'
if(option = '2') + 'a' > 'two'
+ '1' > set(option = '1')
+ '2' > set(option = '2')
```

Note that the following is a compile error (due to internal use of
null terminated strings).

```
store(option) 0
```
2022-09-28 17:00:55 +10:00
Marc Durdin
f5b8effaae fix(developer): ensure backslash in paths passed to kmcomp
Fixes #7090.
Fixes #7091.

While kmcomp loads most files just fine when paths with forward slashes
are passed in, it fails to parse target path correctly when building one
file out of a project.

For example, if the user runs:

```
kmcomp.exe -t khmer_angkor.kmn release/k/khmer_angkor/khmer_angkor.kpj
```

Then the following error arises:

```
khmer_angkor.kps: Error: 0001 Validation error: The system cannot find the path specified.
khmer_angkor.kps: Failure: Package C:\Projects\keyman\keyboards\source\khmer_angkor.kps had validation errors.
```

This patch ensures all paths passed in as command-line parameters are
converted to backslashes, matching the expected Windows format. While I
only needed to fix the `FParamTarget` parameter in order to address the
reported issues, for consistency I applied the same fix to all input
paths.
2022-09-28 16:50:58 +10:00
Marc Durdin
50614fb088 fix(developer): handle invalid kps xml when loading project view
Fixes #7368.
2022-09-28 16:32:36 +10:00
Marc Durdin
0b701275fd chore(developer): remove unused var 2022-09-28 15:25:35 +10:00
Marc Durdin
0f961bd674 feat(developer): visual keyboard compiler in kmc
Adds support for building .kvk files to kmc. At present, .kvk will
always be generated, even if the .xml does not have a hardware layer.
2022-09-27 15:16:07 +10:00
Marc Durdin
b6abe7432a feat(developer): emit .kvk plus e2e test
Adds visual-keyboard-compiler and corresponding e2e test.
2022-09-27 15:03:31 +10:00
Marc Durdin
03c3c40f29 chore(developer): remove visual-keyboard-compiler.ts 2022-09-27 13:42:56 +10:00
Marc Durdin
26e32a0124 refactor(developer): split file type management into package
Splits file format reading, writing, and in-memory structures of .kmx,
.kvk, .kvks, .xml (ldml keyboard) into separate common module. Lots of
little fixups, sorry...

This will make it much easier for us to reuse these readers and writers
in future code.
2022-09-27 12:33:10 +10:00
Marc Durdin
52dd3e0e85 refactor(developer): move compiler callbacks out of kvks file reader 2022-09-27 05:04:04 +10:00
Marc Durdin
5c65405f65 refactor(developer): move compiler callbacks out of ldml xml reader 2022-09-27 04:52:00 +10:00
Marc Durdin
9480d8931e chore(developer): remove commented code 2022-09-26 20:41:59 +10:00
Marc Durdin
74fc795c1c feat(developer): refactor CompilerCallbacks in tests
Now has a shared object for all tests, reset between tests, and if the
test fails, prints any reported compiler messages.

Also updates kvk and kvks tests with refactored names from previous
commit.
2022-09-26 20:07:00 +10:00
Marc Durdin
50db7a2c9b feat(developer): add schemas for .kvk and .kvks
Also minor tweaks around reading .xml files.
2022-09-26 20:05:52 +10:00
Marc Durdin
c7740a39a2 feat(developer): kvk support
Part of #7238.

Supports reading and writing .kvk files, and reading .kvks files,
to/from an in-memory VisualKeyboard object.

TODO:
 - Full validation of .kvks files
 - Writing .kvks files
 - Some cleanup of API surfaces for consistency
2022-09-26 09:29:38 +10:00
Marc Durdin
828cbdf584
Merge pull request #7341 from keymanapp/feat/developer/epic-ldml/7277-kmc-makes-keyboard-packages
feat(developer): fixes to kmc-package for keyboard support 🙀
2022-09-24 10:12:38 +10:00
Marc Durdin
0502d6ccbd
Merge pull request #7342 from keymanapp/chore/epic-ldml/merge-master-to-feature-ldml
chore: merge master into feature-ldml 🙀
2022-09-24 10:12:00 +10:00
Steven R. Loomis
03012248a2 chore(windows): remove crc32 from tike 2022-09-23 11:48:47 -05:00
Steven R. Loomis
700db810e0 fix(core): fix for incorrect earlier fix 2022-09-22 17:26:39 -05:00
Steven R. Loomis
de3225a4f5 Merge remote-tracking branch 'origin/feature-ldml' into chore/core/7276-checksums-epic-ldml 2022-09-22 17:13:27 -05:00
Marc Durdin
33fd8cffba chore(developer): ts v4.8.3 upgrade breaks mocha 10.0.0 with ts-node
Mocha with ts-register was failing with an error for Keyman Developer
Server.

```
Error: Debug Failure. False expression: Non-string value passed to
`ts.resolveTypeReferenceDirective`, likely by a wrapping package working
with an outdated `resolveTypeReferenceDirectives` signature. This is
probably not a problem in TS itself.
```

The `^4.5.4` version reference for typescript was too lax, and so npm
gave us typescript 4.8.3, which was the latest version in use in the
repo in this branch (in kmc-model). This is also why the build only
failed on this branch -- on master branch, the latest version referenced
was still 4.5.4.

Changing to `~4.5.4` ensures that it stayed on the 4.5.x series, and
mocha then ran version 4.5.5 of tsc.

This is something we should resolve when we move the entire repo to tsc
4.8+. Currently we are still on 4.5.

Note: it may be sensible to review all versioning for typescript,
because the minor version changes are causing functionality breaks,
which kinda violates the spirit of semver ("MINOR version when you add
functionality in a backwards compatible manner"). We can be more
restrictive by updating all package.json references to typescript to
`~4.5.4` instead of the current `^4.5.4` in use throughout. If/when we
do this, we will need to check our ts-node references also.
2022-09-23 05:25:24 +10:00
Marc Durdin
6f949ad63f chore(developer): fixup dependency type changes
* xml2js, `emptyTag` property: https://github.com/DefinitelyTyped/DefinitelyTyped/pull/59259
* semver `parse` `includePrerelease` property: https://github.com/DefinitelyTyped/DefinitelyTyped/pull/61586#discussion_r939877412
* commander use needed to be declared for hextobin
2022-09-22 11:46:44 +10:00
Steven R. Loomis
93a9509302 chore(core): delete crc32.cpp and crc32.pas from some places 2022-09-21 18:13:19 -05:00
Marc Durdin
cae408edc5 chore: merge feature-ldml into branch 2022-09-21 14:56:45 +10:00
Marc Durdin
f3fbf01d30 feat(developer): fixes to kmc-package for keyboard support
kmc-package had a number of gaps and errors. This fixes a bunch of
problems:

* Wrong case on XML elements
* Wrong metadata in system element
* Missing kmp.json in Files array (needs test in consumer apps later)
* Missing displayFont, oskFont in keyboard elements
* Including copyLocation when == 0
* Including an empty startMenu element
* Incorrect assumptions about location of source files in source/ folder

Note that support for followKeyboardVersion will come later; for now, it
uses the possibly-stale value from the Keyboard element in the .kps
file.
2022-09-21 14:37:08 +10:00
Marc Durdin
bd5f6877bb feat(developer): fixup fixture json 2022-09-21 14:33:04 +10:00
Marc Durdin
2b9a2949a6 feat(developer): kmc-package khmer_angkor fixture 2022-09-21 14:32:42 +10:00
Marc Durdin
55f302d929
Merge pull request #7322 from keymanapp/chore/merge-master-to-feature-ldml
chore: merge master into feature-ldml 🙀
2022-09-20 13:42:40 +10:00
Marc Durdin
f9b4f3d173
Merge branch 'refactor/developer/6026-kmcompx-compiler-cross-platform-part-2-NEW_VERSION' into chore/merge-master-into-kmcompx 2022-09-20 11:06:55 +10:00
Marc Durdin
d1de02d43e chore(developer): Merge branch 'master' into chore/developer/server-node-gyp-dependency 2022-09-20 10:19:11 +10:00
Marc Durdin
9f290cac21 chore(developer): Merge branch 'master' into chore/developer/server-update-multer-dependency 2022-09-20 10:16:46 +10:00
Marc Durdin
aab9d1aad5 chore(developer): Merge branch 'master' into chore/developer/server-node-gyp-dependency 2022-09-20 10:14:16 +10:00
Steven R. Loomis
66281da0e0
Update developer/src/kmcmpdll/Compiler.cpp
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-09-19 14:59:46 -05:00
Marc Durdin
93a1fc88b9 chore(developer): remove crc generation from kmc-keyboard 2022-09-19 15:30:38 +10:00
Marc Durdin
a59a6c334a chore: update node-windows-trayicon to update node-gyp to 9.1.0 2022-09-19 15:07:24 +10:00
Marc Durdin
3d61d18371 chore(developer): update multer dependency for server
Removes Keyman Developer Server's transitive dependency on dicer by
updating multer to `1.4.5-lts.1`, which updates its dependency on
busboy.

See
https://github.com/expressjs/multer/pull/1097#issuecomment-1141286771
for reasoning behind use of  `-lts.1` rather than a full release
version.

At some point in the future, multer will publish a full release with
this fix, at which point we can move back to a full release version.
2022-09-19 12:36:51 +10:00
Marc Durdin
ee47e49568 chore(developer): fixup build path for kmc 2022-09-19 08:42:27 +10:00
Marc Durdin
e840f04ccf chore(common): make scripts executable and add pre-commit test 2022-09-19 06:39:13 +10:00