Fixes#3620.
Implements the Caps Lock layer support and the double-tap gesture on the
shift key to access it.
The double-tap gesture has been implemented with a view to extension to
support other multi-tap gestures in the future. However, for now, it is
limited to supporting the Shift key, if and only if the keyboard
includes a Caps layer.
The reason for this v15 limitation is that multi-tap on regular keys
would involve either rewinding the previous keystroke (the first tap),
or forcing keyboard developers to consider 'rota' style rules in their
keyboards to support the multi-tap gestures, as we need to make sure
that the first tap is accepted and processed for immediate feedback.
This needs more design, to avoid unnecessary complexity in the keyboards
and/or the rewinding of the keystroke (even though that is conceptually
supported in Keyman Engine for Web already). Basically, we don't want to
constrain the way that a keyboard author may use the multi-tap gesture
by hard-coding the rewind, but neither do we want to make all multi-tap
gestures needlessly complex to author.
The shift key (and other modifiers, potentially in future) needs special
support for multi-tap as the key that is being tapped changes with the
layer change. This is currently managed through recognising `K_SHIFT` in
the key id.
I have tried to follow the `PendingGesture` pattern for multi-tap, and
the gesture itself supports a series of taps, not just a double-tap. The
maximum time to complete the tap series is 125msec * number-of-taps, so
for a double-tap is 250msec.
The changes to support a Caps Lock layer itself were minimal; just
adding the `text.KeyboardProcessor.getStateFromLayer` function and
calling it during `KeyEvent` construction. The remaining changes relate
to the multi-tap gesture.
Minor changes:
* I moved `constructNullKeyEvent` to `KeyEvent` in order to make it
more accessible to other classes.
* The multi-tap gesture does not have a promise to complete, so that is
now an optional member of the `PendingGesture` interface.
This makes two corrections:
1. Removed an obsolete test for screen size based on dpi, which is
unreliable, meant that we had been treating many Android tablets as
phones (domManager.ts).
2. Font scale for Android tablets had some incorrect assumptions around
keyboard height and devicePixelRatio, which I was able to simplify
into a fixed ratio, and was much better across all devices I
tested (however, all emulated in Chrome).
Summary of these changes:
Two targeted improvements; more optimization is possible but IMO this is
a significant enough improvement to go through test:
1. In oskView.ts, avoid calling `VisualKeyboard.refreshLayout` twice
from `OSKView.refreshLayout`. This roughly halves the time spent in
`VisualKeyboard.refreshLayout` for a keyboard switch.
2. In modelManager.ts, we avoid a very expensive `unloadModel` /
`loadModel` sequence from `registerModel` by checking to see if the
model spec we are passed is already registered. This avoids an
expensive callback to Keyman for Android which causes the banner to
'bounce'.
Longer notes:
So `VisualKeyboard.refreshLayout` is called 20+ times during a keyboard
switch. It is a very expensive call. It is the bulk of the time spent
when you press the globe key in Keyman for Android. This seems ripe for
optimization; see the following functions in keyboard.html:
* `setKeymanLanguage` -> `setActiveKeyboard` -> 6 calls
* `enableSuggestions` -> `registerModel` -> 8 calls
* `stateChange` -> 6+ calls
`OSKView.refreshLayout` always seems to call
`VisualKeyboard.refreshLayout` twice: first with `setSize`, and then
again itself.
Keyman for Android and KeymanWeb seem to fight over who is responsible
for selecting models (KMW first calls `loadModel`, then Keyman for
Android calls `unloadModel`, `loadModel`, through `enableSuggestions`).
The fix for rotation (`correctOSKTextSize()`), as called from
`stateChange()`, is expensive as well, because it rebuilds the entire
keyboard from scratch.
Along with that, we have the size of the webview changing as the model
is unloaded and reloaded and lots of back-and-forth between the
KeymanWeb and KMEA which seems unnecessary.
Shouldn't KeymanWeb be responsible for loading and unloading models,
once they are all registered? We should only need to register the models
once at page load time (when config changes, do we just reload the
page?).
Fixes#5950.
If a user starts a longpress gesture and then flicks up, that
immediately brings up the longpress menu, and the timeout for display of
the menu should be cancelled -- otherwise the end result is a key that
stays 'stuck' on.
Fixes#5944.
If a custom OSK, such as sil_euro_latin, calls `keymanweb.KSF()` or
`keymanweb.saveFocus()`, this ensures that the next focus change is
ignored, which prevents the OSK from being dismissed as soon as it is
clicked on.
This is a regression, probably arising in version 10.0.
Note: `_SelectionChange` is no longer used or referenced, so I removed
it as part of this fix.
Fixes#5909.
The `setFocusTimer()` delay was 1000ms. However, it seems that a much
shorter delay accomplishes the same outcome for switching between
controls, as the important factor is to block asynchronously-arriving
events from cancelling the OSK / focus controls for touch alias
elements.
I have set the delay to 50ms. This is fast enough that a click in a
control followed by a click on the page area will always cancel the OSK,
but gives the space needed for queued focus/blur events have time to be
processed before cancelling the timer.
Fixes#5831.
iOS 15 has a significant crashing bug whereby use of a canvas element
completely crashes the browser in some contexts. In particular, we have
observed this when using a WKWebView in a keyboard extension without
'Allow Full Access' switched on.
Keyman uses canvas element to draw a nice looking key preview (key tip)
on iPhones. This is a critical issue for Keyman for iPhone. Keyman for
iPad is not affected because the iPad version does not use key previews.
The resolution here is to remove use of the canvas drawing for key tips
and use a simplified pure HTML/CSS shape. I have not conditioned this
fix on platform; I have currently opted to apply this to all platforms.
This issue will be reported to Apple for resolution. The issue applies
so far to iOS 15.0, 15.1.
Fixes#5907.
The state machine for the Hide transition for the OSK was very complex,
and had some peculiarities which caused it to sometimes fail to
correctly hide. Once I simplified the state machine, it seems that those
edge cases have gone away, and the keyboard always hides correctly.
There is an unrelated issue with focus timeout which can be confusing
when testing this; see #5909.
Relates to #5904.
If a call to api.keyman.com takes longer than 10 seconds, the timerid
would no longer be valid, even if the call returns successfully. This
threw a cascade of errors.
There was a secondary issue where keymanweb.com was passing in an
invalid timerid, causing the same error cascade. This is fixed in
keymanapp/keymanweb.com#38.
Fixes#5779.
Fixes#5731 (I believe mitigation is sufficient to close this issue).
A variety of interrelated font and font size display issues resolved:
1. KVK font was not applied early enough for size calculations, which
meant that we were calculating font scaling per key based on a
default font when transforming from the KVK data for desktop devices
(defaultLayout.ts)
2. Font scaling for non-default layers was calculated when elements were
not visible and had no size information, giving incorrect values. To
resolve this, font scaling is now calculated when a layer is made
visible, which had performance impacts; resolved by reducing
unnecessary `layer.refreshLayout()` calls; see performance point 1
below (oskView.ts:layerChangeHandler())
3. `getViewportScale()` would return an incorrect scaled value when
emulating touch devices on a desktop browser (kmwutils.ts)
4. After switching keyboards, the device-specific scaling factor was not
maintained (oskView.ts:refreshLayout())
Related performance improvements:
1. Multiple calls to `layer.refreshLayout()` in `refreshLayout()` have
been eliminated, and only the currently visible layer is now
refreshed. This dramatically reduces the number of calls to
`getIdealFontSize()` which was the primary concern of #5731.
(visualKeyboards.ts)
2. Unnecessary use of `innerHTML` replaced with `innerText`
(oskBaseKey.ts)
Minor Keyman Developer performance improvement:
1. The web debugger no longer recalculates the OSK twice (test.js)
Fixes#5353.
I made the promise usage consistent in the relevant addKeyboards
functions, instead of mixing promise and async/await style code, which
made it easier to spot the place(s) where promises had not been returned
as we needed.
