Commit graph

871 commits

Author SHA1 Message Date
Marc Durdin
d1dd8b022e fix(developer): buffer size for range expansions
Fixes #4830.

There remain some additional buffer size checks we should be doing in
`GetXString` but this will address the current issue by using the
standard maximum buffer size.
2021-03-31 19:10:12 +11:00
Marc Durdin
071bfd9011 fix(developer): requote font names
Fixes #4810.
2021-03-29 20:53:12 +11:00
Marc Durdin
5eb0aec0e8
Merge pull request #4691 from keymanapp/fix/developer/named-code-constants-end-of-line-test
fix(developer): support named character codes at end of line
2021-03-16 23:26:03 +11:00
Marc Durdin
9ae541940a
Merge pull request #4699 from keymanapp/fix/developer/validate-keyboard_info-bcp47-canonicalisation-info
chore(developer): upgrade bcp 47 canonicalisation to warning
2021-03-16 23:25:28 +11:00
Marc Durdin
de74fddbc2 chore(developer): upgrade bcp 47 canonicalisation to warning
This does not trigger a build failure; validation warnings currently always pass.
Info messages are currently suppressed with -s which also suppresses the banner,
which is more plumbing than we should change just now.
2021-03-16 20:07:27 +11:00
Marc Durdin
aa8538a8d9
Merge pull request #4689 from keymanapp/fix/developer/validate-keyboard_info-bcp47-canonicalisation-info
fix(developer): validate keyboard_info should give info on non-canonical bcp47
2021-03-16 19:32:36 +11:00
Marc Durdin
12184f1857 fix(developer): support named character codes at end of line
If a named character code was followed by a `\n`, it would not be
recognised.
2021-03-16 13:49:55 +11:00
Marc Durdin
4f958985a0 fix(developer): &CasedKeys and &MnemonicLayout are not compatible together
The test for `&mnemoniclayout` was incorrectly failing to check the
actual state of `&casedkeys`, which broke all mnemonic layouts.
2021-03-16 13:46:34 +11:00
Marc Durdin
4c82c2b545 fix(developer): validate keyboard_info should give info on non-canonical bcp47 2021-03-16 13:44:23 +11:00
Marc Durdin
523cf39d10 chore(developer): add B11 ISO code for ABNT2 keyboard 2021-03-12 11:11:42 +11:00
Marc Durdin
c10b010d59 fix(developer): Expand filenames before load
Fixes #4583.
2021-03-05 13:44:55 +11:00
Marc Durdin
690a7fd787
Merge pull request #4586 from keymanapp/feat/developer/2241-caps-controlled-keys
feat(developer): &CasedKeys system store
2021-03-05 11:41:16 +11:00
Marc Durdin
103e46c0d5
Merge pull request #4584 from keymanapp/feat/developer/2241-expansions
feat(developer): Range expansions
2021-03-04 15:05:32 +11:00
Marc Durdin
b7765483ec
Merge pull request #4576 from keymanapp/feat/developer/4314-open-containing-folder-options
feat(developer): Add different Open Containing Folder buttons
2021-03-04 15:05:19 +11:00
Marc Durdin
d536345a20
Merge pull request #4574 from keymanapp/fix/developer/2789-support-all-fonts-keyboard-fonts-dialog
fix(developer): Support all fonts in Keyboard Fonts dialog
2021-03-04 15:05:10 +11:00
Marc Durdin
0cbd053cda
Merge pull request #4571 from keymanapp/fix/developer/4278-improve-cef-location-search-stability
fix(developer): improve CEF location search stability
2021-03-04 15:05:02 +11:00
Marc Durdin
e9e4e7bcb6 feat(developer): &CasedKeys system store
Fixes #2241.

The `&CasedKeys` system store is a compiler feature that reduces the
repetitive nature of keyboard rules for `CAPS` and `NCAPS`. The
`&CasedKeys` system store defines a list of virtual keys for which
'normal' Caps Lock rules apply. This store has no default value, for
backward compatibility.

Once this store is defined, then you can define just the unshifted and
shifted versions of a rule, and Keyman Developer will synthesize the
`CAPS` and `NCAPS` versions of the rule. For example, you may have the
following rules:

```
store(&CasedKeys) [K_A]
+ [K_A] > 'α'
+ [SHIFT K_A] > 'Α'
```

These would be replaced by the compiler with:

```
store(&CasedKeys) [K_A]
+ [NCAPS K_A] > 'α'
+ [SHIFT CAPS K_A] > 'α'
+ [CAPS K_A] > 'Α'
+ [SHIFT NCAPS K_A] > 'Α'
```

You can also use this functionality with characters in the key part of
the rule:

```
store(&CasedKeys) 'a'..'c'
+ 'a' > 'α'
+ 'A' > 'Α'
```

and the compiled expansion would be similar:

```
store(&CasedKeys) [K_A] [K_B] [K_C]
+ [NCAPS K_A] > 'α'
+ [SHIFT CAPS K_A] > 'α'
+ [CAPS K_A] > 'Α'
+ [SHIFT NCAPS K_A] > 'Α'
```

This feature is backwardly compatible with Keyman 6.0, as it is entirely
implemented in the compiler.

This feature is not compatible with mnemonic layouts, and the keys
defined in the `&CasedKeys` store must be the unshifted base keys as
found on a US English keyboard, or you can use ISO9995 identifiers if
you prefer.

If you define a rule where you specify either `NCAPS` or `CAPS`, for a
key found in the store, then no change will be made to that rule. You
can also continue to define rules which use `NCAPS` or `CAPS` for keys
not found in the store.

As a side-benefit, this allows the visual designer to be used and
support Caps Lock, although at this stage, the `&CasedKeys` store is not
surfaced in the visual designer.
2021-03-04 13:41:04 +11:00
Marc Durdin
fe5de67c1f feat(developer): Range expansions
Relates to #2241.

Range expansions are a new language feature that reduce verbosity in
Keyman Keyboard Language (.kmn) files by making it possible to collapse
sequential ranges with a new `..` operator. It is envisioned that these
will be used primarily in stores.

Two kinds of expansions are available: character ranges, and virtual key
ranges.

The syntax is:

```
char '..' char
vkey '..' vkey
```

Where the `char` terminator is a single normal character, and the `vkey`
terminator is a single virtual key. `outs` is permissible for
terminators but no other statement is allowed. Whitespace is per normal
Keyman keyboard language syntax.

Character ranges will replace the range with the set of characters
between the terminators, based on the Unicode value of the terminators.

Examples:

```
store(alphabet) 'a' .. 'z'
store(alpha_not_bq) 'ac'..'pr'..'z'
store(capitals) U+0041 .. U+005A
```

Virtual key ranges work in a similar fashion, but with virtual keys. The
starting and terminating virtual key must have the same shift state.
This will be most useful for character and numeric virtual keys, as the
ranges are based on the numeric value of the virtual key as defined by
Windows, not its position on the keyboard.

Examples:

```
store(alphakeys) [K_A] .. [K_Z]
store(numkeys) [K_0] .. [K_9]
```

Ranges must be positive, that is, you cannot use `'z' .. 'a'`.

While this expansion is not expected to be very useful in rules, it is
still permitted. `context()` and other offsets are calculated on the
expanded form, not the collapsed form. Similarly, `any()`/`index()`
offsets are calculated on the expanded form.

'a' .. 'z' > context(3)  c this produces 'c', not 'z'

Because ranges are expanded at compile time, ranges that are overly
long will result in an error, so you cannot use, for example:

```
store(AllUnicode) U+0020 .. U+10FFFF
```

The precise maximum length of a range is dependent on the context where
it is used.

The changes are entirely compiler-based; no change is made to the .kmx
file format or to the Keyman apps.
2021-03-04 09:43:30 +11:00
Marc Durdin
c595767e39 feat(developer): Add different Open Containing Folder buttons
Fixes #4314.

Adds 'Open Source Folder', 'Open Build Folder', and
'Open Project Folder' actions to various parts of Keyman Developer.
2021-03-03 15:36:11 +11:00
Marc Durdin
49f05e879d fix(developer): Support all fonts in Keyboard Fonts dialog
Fixes #2789.

The Keyboard Fonts dialog skipped some fonts which it did not recognise
as valid TrueType fonts. As a bonus, I've rewritten a few parts of it to
make it significantly faster to load.

I also turned off the font formatting for the list, as it was not very
helpful and hurt performance considerably.
2021-03-03 14:24:59 +11:00
Marc Durdin
6180b01a31
Merge pull request #4562 from keymanapp/fix/developer/3337-track-modified-state-wordlist-editor
fix(developer): track modified state in wordlist editor better
2021-03-03 10:48:21 +11:00
Marc Durdin
726e93bcc3
Merge pull request #4559 from keymanapp/feat/developer/2556-isRTL-support-for-lexical-models
feat(developer): isRTL support for lexical model editor
2021-03-03 10:47:34 +11:00
Marc Durdin
290d21f4c6 chore(developer): address review feedback 2021-03-03 10:45:52 +11:00
Marc Durdin
9ffc378fef chore(developer): fixup wxs path 2021-03-03 10:42:35 +11:00
Marc Durdin
2e61cb8e32 fix(developer): improve CEF location search stability
Fixes #4278.

When Keyman Developer is installed without Keyman for Windows, the
program silently crashes, because it is looking for libcef.dll in the
Keyman Desktop install folder rather than using its own copy of it.

This improves the search for libcef.dll and kmbrowserhost.exe, and
includes kmbrowserhost.exe with the Keyman Developer installation.

In the future, we could consider moving kmbrowserhost.exe into a common
folder (perhaps when we split Keyman for Windows and Keyman Developer
projects in 15.0).

I threw in a one-liner addressing a deprecated function warning in the
Developer build.
2021-03-03 10:05:04 +11:00
Marc Durdin
e4d09ece72
Merge pull request #4564 from keymanapp/fix/developer/3011-osk-import-avoid-blank-keys
fix(developer): Avoid blank keys when importing KMX to KVKS
2021-03-03 09:28:29 +11:00
Marc Durdin
a0734f153b
Merge pull request #4552 from keymanapp/fix/developer/2740-touch-editor-importing-osk-right-alt-layer
fix(developer): touch layout osk import handling of multiple modifiers
2021-03-03 09:27:11 +11:00
Marc Durdin
84c0202d85 chore(developer): tweak casing 2021-03-02 13:27:17 +11:00
Marc Durdin
85b9e04345 chore(developer): address review comment 2021-03-02 13:26:40 +11:00
Marc Durdin
36f5788651 fix(developer): Avoid blank keys when importing KMX to KVKS
Fixes #3011.

Keys that have only whitespace will no longer generate separate key data
for the on screen keyboard, as they have no visual presentation anyway.
2021-03-02 12:16:09 +11:00
Marc Durdin
7ea7ba0e11 fix(developer): track modified state in wordlist editor better
Fixes #3337.

In some situations, the wordlist editor was not tracking modified state
100% correctly, which meant that edits could be lost. This resolves that
by simplifying the modification state tracking.
2021-03-02 09:27:18 +11:00
Marc Durdin
eb713ec274 feat(developer): isRTL support for lexical model editor
Fixes #2556.

This adds support for `isRTL`, `insertAfterWord`, and
`quotesForKeepSuggestions` punctuation options for lexical models, both
in the parsing and in the user interface. A number of options are
presented to the user for the quotes and word breaking characters.

The parser continues to be a regex-based typescriptish parser. Expect it
to be gross. It's too big a job to do properly right now, so it is
somewhat limited and will bail if it finds structures it can't
understand.

I would hope to use something a bit smarter in the future to interface
with the editor. Probably when we are actually fully web-based and can
depend on a Typescript compiler directly.

Please don't get mad at me! 😟😟😟
2021-03-01 21:46:51 +11:00
Marc Durdin
1fcfefa3cf fix(developer): support for smp names 2021-03-01 16:05:17 +11:00
Marc Durdin
3f54c81461 fix(developer): touch layout osk import handling of multiple modifiers
Fixes #2740.

The definitions for several of the combined modifiers was incorrect,
which led to a mismatch when attempting to import those layers from a
visual keyboard into the touch layout editor.
2021-03-01 15:44:55 +11:00
Marc Durdin
26b1d777f7 fix(developer): Improve stability of named code constants
Fixes #4423.

Using a named character constant with a store name that included
characters outside ascii could cause kmcmpdll to crash with an assertion
failure.

This fix allows store names defined in the .kmn file to use characters
outside ascii.

Note that more work needs to be done on which characters are acceptable
to use, as this is somewhat implementation-specific according to the C++
specification for `iswalpha`.

Added a test case to verify various named code constant examples.

Also removes dead code for `IsCJKUnifiedIdeograph` as this was never
used and somewhat pointless in any case.

NamedCodeConstants.cpp could stand to be rewritten using `std::map` or
equivalent. Current implementation is pretty icky.
2021-03-01 10:25:06 +11:00
Marc Durdin
c966f297cb
Merge branch 'beta' into feat/developer/improve-bcp47-canonicalization 2021-02-09 07:03:28 +11:00
Marc Durdin
7a59d073ff chore(windows): address review comments, add test 2021-02-09 06:56:43 +11:00
Marc Durdin
9b0daf5652
Merge pull request #4410 from keymanapp/fix/developer/4280-if-nul-start-of-context
fix(developer): Compiler check for if and nul at start of context
2021-02-09 06:47:14 +11:00
Marc Durdin
3a4038a74e
Merge pull request #4408 from keymanapp/fix/developer/4250-debug-information-with-unicode-identifiers
fix(developer): debug information with unicode identifiers
2021-02-09 06:45:58 +11:00
Marc Durdin
70e6aa9f4e chore(windows): rename variable per review 2021-02-08 20:12:24 +11:00
Marc Durdin
558c8013ee feat(developer): improve BCP 47 canonicalization
Ensures we get a canonical tag per langtags.json as far as we possibly
can. This is a breaking change for the compiler as tags which were
formerly regarded as canonical are no longer regarded that way. This
mostly relates to script subtag but a secondary bug meant that some
other tags would have lost data in the canonicalization process
(because we did a lookup based only on the language subtag previously,
which is a no-no).

See keymanapp/keyboards#1452 for related work.
2021-02-05 16:56:21 +11:00
Marc Durdin
0c782e22d3 fix(developer): Compiler check for if and nul at start of context
Fixes #4280.

Adds checks to verify that `if()`, `platform()`, `baselayout()` and
`nul` are at the start of the context in the appropriate order.

Adds unit tests to validate these checks.

These are conditions that have been present in earlier versions of
Keyman but not enforced until now. Keyboards that do not meet these
conditions would not work correctly in all circumstances and should be
updated to meet the requirements.

An alternative would have been to reorder the context string but that is
much more complex as manipulation would also have been required for the
output string. The enforced order is logical and reduces confusion in
any case.

Updated documentation coming along shortly.
2021-02-03 15:28:53 +11:00
Marc Durdin
171b6fac54 fix(developer): debug information with unicode identifiers
Fixes #4250.

This implements @jahorton's suggestion of including the original name
in a comment alongside the munged name, as well as appending the unique
integer identifier of each symbol.

Note: the error reporting code needed a massage to support Unicode error
strings; in the future we should update the C++ compiler to emit
Unicode error strings also.
2021-02-03 10:54:20 +11:00
Marc Durdin
953cb15581 fix(windows): Use forward slashes in wordlist paths
Fixes #3336.
2021-02-03 10:15:30 +11:00
Marc Durdin
8f1fc61cc6 fix(windows): crash deleting wordlist
Fixes #4393.
Fixes KEYMAN-DEVELOPER-46.
Fixes KEYMAN-DEVELOPER-45.

This arose because deleting a tab caused Delphi VCL to switch to another
tab in the page control, which is IMHO a bug in the TPageControl VCL
component. Furthermore, when the active page is changed under these
circumstances, the normal OnChange and OnChanging events are not fired,
which means that the Source tab (which is the lucky tab to be activated)
never gets initialised. This cascades eventually to a crash due to the
Source tab's text editor not being properly loaded.

Two parts to this fix:

1. Restore the active page when we delete a wordlist tab (RSP-32327);
   this does not resolve the crash but does avoid weird side-effect of
   the active tab changing.
2. Initialise the source tab when we load the form so that the
   initialisation issue described above is avoided.

A separate fix in modelTsProjectFile avoids a (handled) null variant
conversion exception.
2021-02-03 09:43:48 +11:00
Marc Durdin
adafdb9c84 fix(developer): Import Keyboard support for Targets
Fixes #2657.

Importing a Windows Keyboard had an option to select default targets for
the imported keyboard, but the option was ignored. This propagates the
option setting through the import process.
2021-02-02 16:12:45 +11:00
Marc Durdin
5208404cd6 fix(developer): Generate platforms correctly from template
Fixes #2679.

If web is specified but not a touch platform, exclude touch layout from
a generated project. Also, fixup list of supported platforms in the
generated README.md.
2021-02-02 15:50:56 +11:00
Marc Durdin
7c3bbeda15 fix(windows): tweaks for strings.xml support
* Process setup strings also
* Remove legacy references to cust files from TIKE
* Add unit test to Makefile
2021-01-25 13:41:56 +11:00
Marc Durdin
11612a193a fix(developer): Debug character grid performance
The character grid in the debugger would become very slow as the number
of characters increased, with substantial flickering. This refactor
removes the unnecessary recalculations of the grid cell count and
thus resolves performance problems.
2021-01-13 14:09:43 +11:00
Marc Durdin
d1513315bf
Merge pull request #4200 from keymanapp/fix/developer/1692-respect-tab-setting
fix(developer): Respect tab editor option
2020-12-23 08:34:51 +11:00