Relates to #5853.
Two things happened here:
1. Construction of Mocks made an assumption that the selection should
always be deleted (outputTarget.ts:363). However, for NewContext and
PostKeystroke processes, we don't want to change anything.
2. Even if nothing is changed, the transcription would emit what is
in theory a no-op ruleTransform (insert="", deleteLeft=0,
deleteRight=0). But apps would treat this as deleting the selection.
This fix goes a little broader than I would have preferred, but adds a
readonly mode to the transcription and mock model, so that we can
control explicitly when changes are applied to the text store.
If the OSK had a non-integer width or height stored in the OSK cookie,
then the width and height would be parsed incorrectly and the OSK would
fail to be sized correctly on first load; this also caused a script
error and made the OSK impossible to interact with.
Fixes#3620.
Implements the Caps Lock layer support and the double-tap gesture on the
shift key to access it.
The double-tap gesture has been implemented with a view to extension to
support other multi-tap gestures in the future. However, for now, it is
limited to supporting the Shift key, if and only if the keyboard
includes a Caps layer.
The reason for this v15 limitation is that multi-tap on regular keys
would involve either rewinding the previous keystroke (the first tap),
or forcing keyboard developers to consider 'rota' style rules in their
keyboards to support the multi-tap gestures, as we need to make sure
that the first tap is accepted and processed for immediate feedback.
This needs more design, to avoid unnecessary complexity in the keyboards
and/or the rewinding of the keystroke (even though that is conceptually
supported in Keyman Engine for Web already). Basically, we don't want to
constrain the way that a keyboard author may use the multi-tap gesture
by hard-coding the rewind, but neither do we want to make all multi-tap
gestures needlessly complex to author.
The shift key (and other modifiers, potentially in future) needs special
support for multi-tap as the key that is being tapped changes with the
layer change. This is currently managed through recognising `K_SHIFT` in
the key id.
I have tried to follow the `PendingGesture` pattern for multi-tap, and
the gesture itself supports a series of taps, not just a double-tap. The
maximum time to complete the tap series is 125msec * number-of-taps, so
for a double-tap is 250msec.
The changes to support a Caps Lock layer itself were minimal; just
adding the `text.KeyboardProcessor.getStateFromLayer` function and
calling it during `KeyEvent` construction. The remaining changes relate
to the multi-tap gesture.
Minor changes:
* I moved `constructNullKeyEvent` to `KeyEvent` in order to make it
more accessible to other classes.
* The multi-tap gesture does not have a promise to complete, so that is
now an optional member of the `PendingGesture` interface.
This makes two corrections:
1. Removed an obsolete test for screen size based on dpi, which is
unreliable, meant that we had been treating many Android tablets as
phones (domManager.ts).
2. Font scale for Android tablets had some incorrect assumptions around
keyboard height and devicePixelRatio, which I was able to simplify
into a fixed ratio, and was much better across all devices I
tested (however, all emulated in Chrome).
Summary of these changes:
Two targeted improvements; more optimization is possible but IMO this is
a significant enough improvement to go through test:
1. In oskView.ts, avoid calling `VisualKeyboard.refreshLayout` twice
from `OSKView.refreshLayout`. This roughly halves the time spent in
`VisualKeyboard.refreshLayout` for a keyboard switch.
2. In modelManager.ts, we avoid a very expensive `unloadModel` /
`loadModel` sequence from `registerModel` by checking to see if the
model spec we are passed is already registered. This avoids an
expensive callback to Keyman for Android which causes the banner to
'bounce'.
Longer notes:
So `VisualKeyboard.refreshLayout` is called 20+ times during a keyboard
switch. It is a very expensive call. It is the bulk of the time spent
when you press the globe key in Keyman for Android. This seems ripe for
optimization; see the following functions in keyboard.html:
* `setKeymanLanguage` -> `setActiveKeyboard` -> 6 calls
* `enableSuggestions` -> `registerModel` -> 8 calls
* `stateChange` -> 6+ calls
`OSKView.refreshLayout` always seems to call
`VisualKeyboard.refreshLayout` twice: first with `setSize`, and then
again itself.
Keyman for Android and KeymanWeb seem to fight over who is responsible
for selecting models (KMW first calls `loadModel`, then Keyman for
Android calls `unloadModel`, `loadModel`, through `enableSuggestions`).
The fix for rotation (`correctOSKTextSize()`), as called from
`stateChange()`, is expensive as well, because it rebuilds the entire
keyboard from scratch.
Along with that, we have the size of the webview changing as the model
is unloaded and reloaded and lots of back-and-forth between the
KeymanWeb and KMEA which seems unnecessary.
Shouldn't KeymanWeb be responsible for loading and unloading models,
once they are all registered? We should only need to register the models
once at page load time (when config changes, do we just reload the
page?).
Fixes#5950.
If a user starts a longpress gesture and then flicks up, that
immediately brings up the longpress menu, and the timeout for display of
the menu should be cancelled -- otherwise the end result is a key that
stays 'stuck' on.
Fixes#5831.
iOS 15 has a significant crashing bug whereby use of a canvas element
completely crashes the browser in some contexts. In particular, we have
observed this when using a WKWebView in a keyboard extension without
'Allow Full Access' switched on.
Keyman uses canvas element to draw a nice looking key preview (key tip)
on iPhones. This is a critical issue for Keyman for iPhone. Keyman for
iPad is not affected because the iPad version does not use key previews.
The resolution here is to remove use of the canvas drawing for key tips
and use a simplified pure HTML/CSS shape. I have not conditioned this
fix on platform; I have currently opted to apply this to all platforms.
This issue will be reported to Apple for resolution. The issue applies
so far to iOS 15.0, 15.1.
Fixes#5907.
The state machine for the Hide transition for the OSK was very complex,
and had some peculiarities which caused it to sometimes fail to
correctly hide. Once I simplified the state machine, it seems that those
edge cases have gone away, and the keyboard always hides correctly.
There is an unrelated issue with focus timeout which can be confusing
when testing this; see #5909.
Fixes#5779.
Fixes#5731 (I believe mitigation is sufficient to close this issue).
A variety of interrelated font and font size display issues resolved:
1. KVK font was not applied early enough for size calculations, which
meant that we were calculating font scaling per key based on a
default font when transforming from the KVK data for desktop devices
(defaultLayout.ts)
2. Font scaling for non-default layers was calculated when elements were
not visible and had no size information, giving incorrect values. To
resolve this, font scaling is now calculated when a layer is made
visible, which had performance impacts; resolved by reducing
unnecessary `layer.refreshLayout()` calls; see performance point 1
below (oskView.ts:layerChangeHandler())
3. `getViewportScale()` would return an incorrect scaled value when
emulating touch devices on a desktop browser (kmwutils.ts)
4. After switching keyboards, the device-specific scaling factor was not
maintained (oskView.ts:refreshLayout())
Related performance improvements:
1. Multiple calls to `layer.refreshLayout()` in `refreshLayout()` have
been eliminated, and only the currently visible layer is now
refreshed. This dramatically reduces the number of calls to
`getIdealFontSize()` which was the primary concern of #5731.
(visualKeyboards.ts)
2. Unnecessary use of `innerHTML` replaced with `innerText`
(oskBaseKey.ts)
Minor Keyman Developer performance improvement:
1. The web debugger no longer recalculates the OSK twice (test.js)
Fixes three issues:
* longpress positioning for inlined OSK views where not all elements
may be in the offset hierarchy; impacts particularly where an inlined
OSK may be within a flex-box.
* Tweaks position of keytip, to take into account the offsetTop of the
key element.
* Inverts test for display of longpress callout, matching Keyman 14.
