erabo/spiegel-keyman
1
0
Fork 0
mirror of https://github.com/keymanapp/keyman.git synced 2026-10-20 12:57:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 5

Merge pull request #13619 from keymanapp/chore/merge-master-into-web-core

Browse source 
chore: merge master into web-core 🎼
This commit is contained in:
Marc Durdin 2025-03-31 16:07:05 +07:00 • committed by GitHub
commit 7aab6cc4cb
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
171 changed files with 10481 additions and 1267 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
.clang-format
View file

@ -134,4 +134,3 @@ TabWidth: 8
UseCRLF: false
UseTab: Never
...

2
.github/workflows/deb-packaging.README.md vendored
View file

@ -1,4 +1,4 @@
# Debian Packaging GitHub Action
# Ubuntu Packaging GitHub Action
You can manually trigger a deb-packaging action on GitHub, e.g. to test changes:

15
.github/workflows/deb-packaging.yml vendored
View file

@ -1,3 +1,14 @@
#
# This workflow builds the source package for Keyman for Linux, and then builds
# the binary packages for the released versions of Ubuntu. It also signs the
# packages. Released Keyman versions get uploaded to the LLSO repository.
# It is triggered by a repository_dispatch event with the
# types `deb-release-packaging:*` and `deb-pr-packaging:*`.
#
# We use the filename `deb-packaging.yml` even though we only do Ubuntu
# packaging here, because we're packaging the .deb format, and because it's shorter.
#
name: "Ubuntu packaging"
run-name: "Ubuntu packaging - ${{ github.event.client_payload.branch }} (branch ${{ github.event.client_payload.baseBranch }}), by @${{ github.event.client_payload.user }}, testbuild: ${{ github.event.client_payload.isTestBuild }}"
on:
@ -16,7 +27,7 @@ on:
env:
COLOR_GREEN: "\e[32m"
GH_TOKEN: ${{ github.token }}
STATUS_CONTEXT: 'Debian Packaging'
STATUS_CONTEXT: 'Ubuntu Packaging'
DEBFULLNAME: 'Keyman GHA packager'
DEBEMAIL: 'support@keyman.com'
@ -45,7 +56,7 @@ jobs:
/repos/$GITHUB_REPOSITORY/statuses/${{ github.event.client_payload.buildSha }} \
-f state='pending' \
-f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" \
-f description='Debian packaging started' \
-f description='Ubuntu packaging started' \
-f context="$STATUS_CONTEXT"
- name: Install devscripts

74
HISTORY.md
View file

@ -1,5 +1,26 @@
# Keyman Version History
## 19.0.18 alpha 2025-03-25
* chore(developer): improve ldml code coverage (#13516)
## 19.0.17 alpha 2025-03-24
* chore(deps-dev): bump esbuild from 0.18.20 to 0.25.0 (#13191)
## 19.0.16 alpha 2025-03-19
* feat(core): check on load for valid kmx_plus unicode (#13486)
## 19.0.15 alpha 2025-03-18
* chore: merge master into linux-mcompile (#13506)
## 19.0.14 alpha 2025-03-14
* chore: B18S3 merge beta to master (#13498)
* feat(core): move kmx_plus.tests.cpp to GTest (#13494)
## 19.0.13 alpha 2025-03-12
* feat(windows): hard code key cap colour (#13444)
@ -59,6 +80,59 @@
* refactor(windows): rename `TKeymanMutex.MutexOwned` to `TakeOwnership` and add `ReleaseOwnership` (#13168)
* chore: increment to alpha 19.0 (#13187)
## 18.0.222 beta 2025-03-28
* fix(android): Handle banner overrides independently when device is locked and unlocked (#13546)
## 18.0.221 beta 2025-03-27
* fix(windows): configuration update tab text changes (#13555)
* chore(common): update language-subtag-registry to 2025-03-10 release (#13577)
* docs(mac): update help documentation due to changes in Settings in macOS 13.0 Ventura (#13501)
* fix(linux): improve emitting keystrokes (#13558)
* fix(linux): improve key mapping to allow Core to reset context (#13372)
## 18.0.220 beta 2025-03-24
* fix(windows,mac): Update Spanish configuration string and mac strings (#13564)
* chore(developer): use 'statement' instead of 'command' for messages about kmn statements (#13562)
## 18.0.219 beta 2025-03-21
* fix(windows): add exception handling for loading xml file in XSL Rendering (#13530)
* refactor(linux): rename `initialize_queue` method (#13554)
## 18.0.218 beta 2025-03-19
* fix(developer): add validation of `nul` usage in output part of rule (#13532)
* fix(web): model unload and load should use consistent, canEnable check (#13534)
* feat(web): adds gesture 'traces' for warning/error logging (#13360)
* fix(linux): fix building in Docker with different Ubuntu/Debian version (#13537)
* chore(linux): replace deprecated setup.py (#13524)
* chore(linux): rename "Debian packaging" to "Ubuntu packaging" (#13523)
## 18.0.217 beta 2025-03-18
* fix(windows): cherry pick hard code key cap (#13526)
* fix(android,windows): Update Russian language strings to use Cyrillic script (#13527)
* docs(web): add internal documentation for context state and keystroke processing (#13520)
* docs(common): add link to schemas (#13521)
* chore(linux): update URL for lcov package (#13535)
* chore(linux): allow to build and use Debian docker image (#13536)
## 18.0.216 beta 2025-03-17
* change(android): re-order host page script load so Sentry can report script load failures (#13333)
* chore(web): remove dead osk code (#13358)
* fix(web): cancel held backspace on new key input (#13359)
* fix(developer): handle non bmp numeric entities in xml reader (#13365)
## 18.0.215 beta 2025-03-14
* docs(common): Update Sentry links to keyman.sentry.io (#13491)
* chore(android,windows): Update Crowdin for Spanish and Spanish (Latin America) (#13499)
* chore(common): update crowdin strings for Italian (#13500)
## 18.0.214 beta 2025-03-13
* fix(mac): change text in privacy alert for macOS 13.0 and later (#13480)

2
VERSION.md
View file

@ -1 +1 @@
19.0.14
19.0.19

6
android/KMAPro/kMAPro/src/main/java/com/keyman/android/SystemKeyboard.java
View file

@ -159,9 +159,11 @@ public class SystemKeyboard extends InputMethodService implements OnKeyboardEven
Context appContext = getApplicationContext();
// Temporarily disable predictions on certain fields (e.g. hidden password field or numeric)
inputType = attribute.inputType;
KMManager.setMayPredictOverride(inputType);
if (KMManager.getMayPredictOverride()) {
KMManager.setPredictionsSuspended(inputType, KeyboardType.KEYBOARD_TYPE_SYSTEM);
if (KMManager.getPredictionsSuspended(KeyboardType.KEYBOARD_TYPE_SYSTEM)) {
KMManager.setBannerOptions(false);
// Set the system keyboard HTML banner
BannerController.setHTMLBanner(this, KeyboardType.KEYBOARD_TYPE_SYSTEM);
} else if (KMManager.isKeyboardLoaded(KeyboardType.KEYBOARD_TYPE_SYSTEM)){
// Check if predictions needs to be re-enabled per Settings preference
Keyboard kbInfo = KMManager.getCurrentKeyboardInfo(appContext);

6
android/KMAPro/kMAPro/src/main/java/com/tavultesoft/kmapro/MainActivity.java
View file

@ -244,6 +244,12 @@ public class MainActivity extends BaseActivity implements OnKeyboardEventListene
@Override
protected void onResume() {
super.onResume();
if (textView != null) {
// Reset inAppPredictionsSuspendedForSensitiveInput flag
KMManager.setPredictionsSuspended(textView.getInputType(), KeyboardType.KEYBOARD_TYPE_INAPP);
}
KMManager.onResume();
KMManager.hideSystemKeyboard();

6
android/KMAPro/kMAPro/src/main/res/values-b+es+419/strings.xml
View file

@ -126,11 +126,11 @@
<!-- Context: Adjust Keyboard Height menu -->
<string name="reset_to_defaults" comment="Button to reset to default heights">Restaurar valores predeterminados</string>
<!-- Context: Adjust Longpress Delay Time menu -->
<string name="longpress_delay_time" comment="Current longpress delay time (seconds)">Delay Time: %1$.1f seconds</string>
<string name="longpress_delay_time" comment="Current longpress delay time (seconds)">Tiempo de retraso: %1$.1f segundos</string>
<!-- Context: Adjust Longpress Delay Time menu -->
<string name="ic_delay_time_up" comment="Make longpress delay longer">Delay time longer</string>
<string name="ic_delay_time_up" comment="Make longpress delay longer">Tiempo de retraso más largo</string>
<!-- Context: Adjust Longpress Delay Time menu -->
<string name="ic_delay_time_down" comment="Make longpress delay shorter">Delay time shorter</string>
<string name="ic_delay_time_down" comment="Make longpress delay shorter">Tiempo de retraso más corto</string>
<!-- Context: Adjust Longpress Delay Time menu -->
<string name="ic_longpress_delay_slider" comment="Adjust longpress delay time">Longpress delay time slider</string>
<!-- Context: Keyman Web Browser -->

2
android/KMEA/app/src/main/assets/keyboard.html
View file

@ -9,9 +9,9 @@
<title>Keyman</title>
<!-- DOMRect only exists as of Chrome 61... thus coming later than ES6. -->
<script src="other-polyfills.js"></script>
<script src="keymanweb-webview.js"></script>
<script src="sentry.min.js"></script>
<script src="keyman-sentry.js"></script>
<script src="keymanweb-webview.js"></script>
<script src="android-host.js"></script>
<style type="text/css">
body {background-color:#000000;}

2
android/KMEA/app/src/main/java/com/keyman/engine/DisplayLanguages.java
View file

@ -66,7 +66,7 @@ public class DisplayLanguages {
new DisplayLanguageType("el", "Polytonic Greek"),
new DisplayLanguageType("pt-PT", "Português de Portugal"),
new DisplayLanguageType("ff-ZA", "Pulaar-Fulfulde"), // or Fulah
new DisplayLanguageType("ru-RU", "Pyccĸий (Russian)"),
new DisplayLanguageType("ru-RU", "Русский (Russian)"),
new DisplayLanguageType("shu-latn", "Shuwa (Latin)"),
new DisplayLanguageType("sv-SE", "svenska (Swedish)"),
new DisplayLanguageType("vi-VN", "Tiếng Việt (Vietnamese)"),

4
android/KMEA/app/src/main/java/com/keyman/engine/KMKeyboardWebViewClient.java
View file

@ -165,12 +165,12 @@ public final class KMKeyboardWebViewClient extends WebViewClient {
// appContext instead of context?
SharedPreferences prefs = context.getSharedPreferences(context.getString(R.string.kma_prefs_name), Context.MODE_PRIVATE);
boolean modelPredictionPref = false;
if (!KMManager.getMayPredictOverride() && KMManager.currentLexicalModel != null) {
if (!KMManager.getPredictionsSuspended(keyboardType) && KMManager.currentLexicalModel != null) {
modelPredictionPref = prefs.getInt(KMManager.getLanguageAutoCorrectionPreferenceKey(
KMManager.currentLexicalModel.get(KMManager.KMKey_LanguageID)), KMManager.KMDefault_Suggestion)
!= SuggestionType.SUGGESTIONS_DISABLED.toInt();
}
KMManager.setBannerOptions(modelPredictionPref);
KMManager.setBannerOptions(modelPredictionPref, keyboardType);
RelativeLayout.LayoutParams params = KMManager.getKeyboardLayoutParams();
kmKeyboard.setLayoutParams(params);
} else if (url.indexOf("suggestPopup") >= 0) {

85
android/KMEA/app/src/main/java/com/keyman/engine/KMManager.java
View file

@ -257,10 +257,13 @@ public final class KMManager {
// haptic feedback disabled for hardware keystrokes
private static boolean mayHaveHapticFeedback = false;
// Special override for when keyboard is entering a password text field.
// When mayPredictOverride is true, the option {'mayPredict' = false} is set in the lm-layer
// Special flags to temporarily disable predictions when editing a password field.
// These are maintained independently for inapp keyboard and system keyboard.
// When true, the suggestion banner passes the option {'mayPredict' = false} to KeymanWeb in the lm-layer
// regardless what the Settings preference is.
private static boolean mayPredictOverride = false;
// API added Keyman 18.0
private static boolean inAppPredictionsSuspendedForSensitiveInput = false;
private static boolean systemPredictionsSuspendedForSensitiveInput = false;
// Determine how system keyboard handles ENTER key
public static EnterModeType enterMode = EnterModeType.DEFAULT;
@ -1409,25 +1412,49 @@ public final class KMManager {
}
/**
* Sets mayPredictOverride true if the InputType field is a hidden password text field
* (either TYPE_TEXT_VARIATION_PASSWORD or TYPE_TEXT_VARIATION_WEB_PASSWORD
* but not TYPE_TEXT_VARIATION_VISIBLE_PASSWORD)
* Determine the predictions should be suspended for a given InputType field: hidden password text field
* (but not TYPE_TEXT_VARIATION_VISIBLE_PASSWORD).
* Also true for numeric text fields
* @param inputType android.text.InputType
* @return boolean
*/
public static void setMayPredictOverride(int inputType) {
mayPredictOverride =
((inputType == (InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_PASSWORD)) ||
(inputType == (InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_WEB_PASSWORD)) ||
isNumericField(inputType));
private static boolean shouldSuspendPredictions(int inputType) {
return ((inputType == (InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_PASSWORD)) ||
(inputType == (InputType.TYPE_CLASS_TEXT | InputType.TYPE_TEXT_VARIATION_WEB_PASSWORD)) ||
isNumericField(inputType));
}
/**
* Get the value of mayPredictOverride
* @return boolean
* Sets PredictionsSuspended true if the given InputType field for the keyboard is a hidden password text field
* (either TYPE_TEXT_VARIATION_PASSWORD or TYPE_TEXT_VARIATION_WEB_PASSWORD
* but not TYPE_TEXT_VARIATION_VISIBLE_PASSWORD)
* Also true for numeric text fields.
* This is to temporarily disable suggestions in these text fields.
* API added Keyman 18.0
* @param inputType android.text.InputType
* @param {KeyboardType} keyboard
*/
public static boolean getMayPredictOverride() {
return mayPredictOverride;
public static void setPredictionsSuspended(int inputType, KeyboardType keyboard) {
if (keyboard == KeyboardType.KEYBOARD_TYPE_INAPP) {
inAppPredictionsSuspendedForSensitiveInput = shouldSuspendPredictions(inputType);
} else if (keyboard == KeyboardType.KEYBOARD_TYPE_SYSTEM) {
systemPredictionsSuspendedForSensitiveInput = shouldSuspendPredictions(inputType);
}
}
/**
* Get the value of PredictionsSuspendedForSensitiveInput based on KeyboardType.
* API added in Keyman 18.0
* @param {KeyboardType} keyboard
* @return boolean of inAppPredictionsSuspendedForSensitiveInput or systemPredictionsSuspendedForSensitiveInput
*/
public static boolean getPredictionsSuspended(KeyboardType keyboard) {
if (keyboard == KeyboardType.KEYBOARD_TYPE_INAPP) {
return inAppPredictionsSuspendedForSensitiveInput;
} else if (keyboard == KeyboardType.KEYBOARD_TYPE_SYSTEM) {
return systemPredictionsSuspendedForSensitiveInput;
}
return false;
}
/**
@ -1556,20 +1583,23 @@ public final class KMManager {
// When entering password field, maySuggest should override to disabled
SharedPreferences prefs = appContext.getSharedPreferences(appContext.getString(R.string.kma_prefs_name), Context.MODE_PRIVATE);
int maySuggest = mayPredictOverride ? SuggestionType.SUGGESTIONS_DISABLED.toInt() :
prefs.getInt(getLanguageAutoCorrectionPreferenceKey(languageID), KMDefault_Suggestion);
int suggestionPreference = prefs.getInt(getLanguageAutoCorrectionPreferenceKey(languageID), KMDefault_Suggestion);
RelativeLayout.LayoutParams params;
if (isKeyboardLoaded(KeyboardType.KEYBOARD_TYPE_INAPP) && !InAppKeyboard.shouldIgnoreTextChange() && modelFileExists) {
params = getKeyboardLayoutParams();
// Do NOT re-layout here; it'll be triggered once the banner loads.
InAppKeyboard.loadJavascript(KMString.format("enableSuggestions(%s, %d)", model, maySuggest));
int inappMaySuggest = inAppPredictionsSuspendedForSensitiveInput ? SuggestionType.SUGGESTIONS_DISABLED.toInt() :
suggestionPreference;
InAppKeyboard.loadJavascript(KMString.format("enableSuggestions(%s, %d)", model, inappMaySuggest));
}
if (isKeyboardLoaded(KeyboardType.KEYBOARD_TYPE_SYSTEM) && !SystemKeyboard.shouldIgnoreTextChange() && modelFileExists) {
params = getKeyboardLayoutParams();
// Do NOT re-layout here; it'll be triggered once the banner loads.
int maySuggest = systemPredictionsSuspendedForSensitiveInput ? SuggestionType.SUGGESTIONS_DISABLED.toInt() :
suggestionPreference;
SystemKeyboard.loadJavascript(KMString.format("enableSuggestions(%s, %d)", model, maySuggest));
}
return true;
@ -1621,6 +1651,25 @@ public final class KMManager {
return true;
}
/**
* setBannerOptions - Update KMW for inapp/system keyboard whether to generate predictions.
* For now, also display banner
* @param mayPredict - boolean whether KMW should generate predictions
* @param {KeyboardType} keyboard
* @return boolean - Success
*/
public static boolean setBannerOptions(boolean mayPredict, KeyboardType keyboard) {
String url = KMString.format("setBannerOptions(%s)", mayPredict);
if (keyboard == KeyboardType.KEYBOARD_TYPE_INAPP && InAppKeyboard != null) {
InAppKeyboard.loadJavascript(url);
}
if (keyboard == KeyboardType.KEYBOARD_TYPE_SYSTEM && SystemKeyboard != null) {
SystemKeyboard.loadJavascript(url);
}
return true;
}
/**
* Update KeymanWeb banner type
* @param {KeyboardType} keyboard

10
android/KMEA/app/src/main/res/values-b+es+419/strings.xml
View file

@ -82,6 +82,14 @@
<string name="fatal_keyboard_error" comment="Fatal keyboard error (keyboard ID::packageID for language). Will load default keyboard"> Error fatal del teclado con %1$s:%2$s para el idioma %3$s. Cargando teclado predeterminado.</string>
<!-- Context: Fatal keyboard error. -->
<string name="fatal_keyboard_error_short" comment="Error in keyboard (keyboard ID::packageID for language)."> Error en el teclado %1$s:%2$s para el idioma %3$s.</string>
<!-- Context: Alert Dialog body: System WebView not installed -->
<string name="body_install_webview" comment="Notification when WebView not installed"> Keyman requiere que WebView esté instalado.\"</string>
<!-- Context: Alert Dialog body: System WebView installed but not enabled -->
<string name="body_enable_webview" comment="Notification when WebView installed but not enabled">Keyman requiere que WebView esté habilitado.\"</string>
<!-- Context Button label: Install WebView -->
<string name="label_install_webview" comment="Button label to install webview">Instalar WebView</string>
<!-- Context Button label: Enable WebView -->
<string name="label_enable_webview" comment="Button label to enable webview">Habilitar WebView</string>
<!-- Context: Query for associated dictionary -->
<string name="query_associated_model" comment="Check if there's an available dictionary to download">Buscando diccionario asociado para descargar</string>
<string name="cannot_query_associated_model" comment="Cannot reach server to check if there's an available dictionary to download"> No se pudo conectar al servidor de Keyman para buscar un diccionario asociado para descargar</string>
@ -115,6 +123,8 @@
<string name="failed_to_retrieve_file" comment="Failed to retrieve downloaded file">Error al recuperar el archivo descargado</string>
<!-- Context: General Updates -->
<string name="update_check_unavailable" comment="Error message when a Keyman server can't be reached">¡Error al acceder al servidor!</string>
<!-- Context: Update-check failure notification -->
<string name="update_check_download_manager_disabled" comment="Notification that a system service needed for updates is disabled">DownloadManager deshabilitado - No se pueden buscar actualizaciones</string>
<!-- Context: General Updates -->
<string name="update_check_current" comment="Notification that all resources are up to date">"¡Todos los recursos están actualizados!"</string>
<!-- Context: General Updates -->

10
android/KMEA/app/src/main/res/values-es-rES/strings.xml
View file

@ -82,6 +82,14 @@
<string name="fatal_keyboard_error" comment="Fatal keyboard error (keyboard ID::packageID for language). Will load default keyboard"> Error de teclado fatal con %1$s:%2$s para %3$s idioma. Cargando teclado predeterminado.</string>
<!-- Context: Fatal keyboard error. -->
<string name="fatal_keyboard_error_short" comment="Error in keyboard (keyboard ID::packageID for language)."> Error en el teclado %1$s:%2$s para %3$s idioma.</string>
<!-- Context: Alert Dialog body: System WebView not installed -->
<string name="body_install_webview" comment="Notification when WebView not installed"> Keyman requires WebView to be installed.\"</string>
<!-- Context: Alert Dialog body: System WebView installed but not enabled -->
<string name="body_enable_webview" comment="Notification when WebView installed but not enabled"> Keyman requires WebView to be enabled.\"</string>
<!-- Context Button label: Install WebView -->
<string name="label_install_webview" comment="Button label to install webview">Install WebView</string>
<!-- Context Button label: Enable WebView -->
<string name="label_enable_webview" comment="Button label to enable webview">Enable WebView</string>
<!-- Context: Query for associated dictionary -->
<string name="query_associated_model" comment="Check if there's an available dictionary to download">Comprobando si hay un diccionario asociado para descargar</string>
<string name="cannot_query_associated_model" comment="Cannot reach server to check if there's an available dictionary to download"> No se puede conectar al servidor Keyman para comprobar si hay un diccionario asociado para descargar</string>
@ -115,6 +123,8 @@
<string name="failed_to_retrieve_file" comment="Failed to retrieve downloaded file">Falló al recuperar el archivo descargado</string>
<!-- Context: General Updates -->
<string name="update_check_unavailable" comment="Error message when a Keyman server can't be reached">¡Falló al acceder al servidor!</string>
<!-- Context: Update-check failure notification -->
<string name="update_check_download_manager_disabled" comment="Notification that a system service needed for updates is disabled">DownloadManager disabled - cannot check for updates</string>
<!-- Context: General Updates -->
<string name="update_check_current" comment="Notification that all resources are up to date">"¡Todos los recursos están actualizados!"</string>
<!-- Context: General Updates -->

10
android/KMEA/app/src/main/res/values-it-rIT/strings.xml
View file

@ -82,6 +82,14 @@
<string name="fatal_keyboard_error" comment="Fatal keyboard error (keyboard ID::packageID for language). Will load default keyboard"> Errore fatale della tastiera con %1$s:%2$s per %3$s lingua. Caricamento tastiera predefinita.</string>
<!-- Context: Fatal keyboard error. -->
<string name="fatal_keyboard_error_short" comment="Error in keyboard (keyboard ID::packageID for language)."> Errore nella tastiera %1$s:%2$s per %3$s lingua.</string>
<!-- Context: Alert Dialog body: System WebView not installed -->
<string name="body_install_webview" comment="Notification when WebView not installed"> Keyman richiede che WebView sia installato.\"</string>
<!-- Context: Alert Dialog body: System WebView installed but not enabled -->
<string name="body_enable_webview" comment="Notification when WebView installed but not enabled"> Keyman richiede che WebView sia abilitato.\"</string>
<!-- Context Button label: Install WebView -->
<string name="label_install_webview" comment="Button label to install webview">Installa WebView</string>
<!-- Context Button label: Enable WebView -->
<string name="label_enable_webview" comment="Button label to enable webview">Abilita WebView</string>
<!-- Context: Query for associated dictionary -->
<string name="query_associated_model" comment="Check if there's an available dictionary to download">Controllo del dizionario associato da scaricare</string>
<string name="cannot_query_associated_model" comment="Cannot reach server to check if there's an available dictionary to download"> Non è possibile connettersi al server Keyman per verificare dizionari associati da scaricare</string>
@ -115,6 +123,8 @@
<string name="failed_to_retrieve_file" comment="Failed to retrieve downloaded file">Recupero del file scaricato non riuscito</string>
<!-- Context: General Updates -->
<string name="update_check_unavailable" comment="Error message when a Keyman server can't be reached">Accesso al server non riuscito!</string>
<!-- Context: Update-check failure notification -->
<string name="update_check_download_manager_disabled" comment="Notification that a system service needed for updates is disabled">DownloadManager disabilitato - impossibile controllare aggiornamenti</string>
<!-- Context: General Updates -->
<string name="update_check_current" comment="Notification that all resources are up to date">"Tutte le risorse sono aggiornate!"</string>
<!-- Context: General Updates -->

4
android/docs/engine/KMManager/getMaySendCrashReport.md
View file

@ -3,7 +3,7 @@ title: KMManager.getMaySendCrashReport()
---
## Summary
The **getMaySendCrashReport()** method returns whether Keyman Engine is allowed to send crash reports over the network to sentry.keyman.com.
The **getMaySendCrashReport()** method returns whether Keyman Engine is allowed to send crash reports over the network to Sentry.
## Syntax
```java
@ -11,7 +11,7 @@ KMManager.getMaySendCrashReport()
```
### Returns
Returns `true` if crash reports can be sent over the network to sentry.keyman.com, `false` otherwise.
Returns `true` if crash reports can be sent over the network to Sentry, `false` otherwise.
## Description
Use this method to check if Keyman Engine will be accessing the network to send crash reports.

29
android/docs/engine/KMManager/getPredictionsSuspended.md Normal file
View file

@ -0,0 +1,29 @@
---
title: KMManager.getPredictionsSuspended()
---
## Summary
The **getPredictionsSuspended()** method returns a flag that determines whether predictions are temporarily disabled because the currently selected text field is a hidden password text field or a numeric text field.
## Syntax
```java
KMManager.getPredictionsSuspended(KeyboardType keyboardType)
```
### Parameters
`keyboardType`
: The keyboard type. `KEYBOARD_TYPE_INAPP` or
`KEYBOARD_TYPE_SYSTEM`.
### Returns
Returns `true` if predictions are temporarily disabled because the currently selected text field is a hidden password text field or a numeric text field. `false` otherwise.
## Description
Use this method to check if predictions are temporarily disabled because of the type of currently selected text field. This value takes precedence over the user preference in the LanguageSettings menu.
## History
Added syntax in Keyman Engine for Android 18.0.
## See also
* [setPredictionsSuspended](setPredictionsSuspended)

10
android/docs/engine/KMManager/index.md
View file

@ -147,11 +147,14 @@ The KMManager is the core class which provides most of the methods and constants
: returns from stored preference the number of milliseconds to trigger a longpress gesture
[`getMaySendCrashReport()`](getMaySendCrashReport)
: returns whether Keyman Engine is allowed to send crash reports over the network to sentry.keyman.com
: returns whether Keyman Engine is allowed to send crash reports over the network to Sentry
[`getOrientation()`](getOrientation)
: returns the device's current orientation (Portrait vs Landscape)
[`getPredictionsSuspended()`](getPredictionsSuspended)
: returns whether predictions are temporarily disabled because the currently selected text field is a hidden password text field or a numeric text field.
[`getSpacebarText()`](getSpacebarText)
: returns the current text display pattern for the spacebar
@ -255,7 +258,10 @@ The KMManager is the core class which provides most of the methods and constants
: stores the longpress delay in milliseconds as a preference.
[`setMaySendCrashReport()`](setMaySendCrashReport)
: sets whether Keyman Engine can send crash reports over the network to sentry.keyman.com
: sets whether Keyman Engine can send crash reports over the network to Sentry
[`setPredictionsSuspended()`](setPredictionsSuspended)
: sets whether predictions are temporarily disabled because the currently selected text field is a hidden password text field or a numeric text field.
[`setShouldAllowSetKeyboard()`](setShouldAllowSetKeyboard)
: sets whether Keyman Engine allows setting a keyboard other than the default keyboard

2
android/docs/engine/KMManager/setMaySendCrashReport.md
View file

@ -4,7 +4,7 @@ title: KMManager.setMaySendCrashReport()
## Summary
The **setMaySendCrashReport()** enables or disables whether Keyman Engine can send crash reports over the
network to sentry.keyman.com.
network to Sentry.
## Syntax
```java

36
android/docs/engine/KMManager/setPredictionsSuspended.md Normal file
View file

@ -0,0 +1,36 @@
---
title: KMManager.setPredictionsSuspended()
---
## Summary
The **setPredictionsSuspended()** method sets a flag to temporarily disable predictions when the currently selected text field is a hidden password text field or a numeric text field.
## Syntax
```java
KMManager.setPredictionsSuspended(int inputType, KeyboardType keyboardType)
```
### Parameters
[`inputType`](https://developer.android.com/reference/android/text/InputType)
: An integer defining the basic context type of text being edited. If the text is one of the following InputTypes, this methods returns `true`:
* TYPE_CLASS_TEXT
* TYPE_TEXT_VARIATION_PASSWORD
* TYPE_CLASS_TEXT
* TYPE_TEXT_VARIATION_WEB_PASSWORD
* TYPE_CLASS_NUMBER
* TYPE_CLASS_PHONE
`keyboardType`
: The keyboard type. `KEYBOARD_TYPE_INAPP` or
`KEYBOARD_TYPE_SYSTEM`.
## Description
Use this method to temporarily disable predictions because of the currently selected text field. This value takes precedence over the user preference in the LanguageSettings menu.
## History
Added syntax in Keyman Engine for Android 18.0.
## See also
* [getPredictionsSuspended](getPredictionsSuspended)

2
android/docs/help/about/privacy.md
View file

@ -4,7 +4,7 @@ title: Data Privacy Policy
## Privacy Policy for Keyman
This application will send crash reporting information to https://sentry.keyman.com. No personally identifiable information or keyboard strokes are recorded or shared.
This application will send crash reporting information to https://keyman.sentry.io. No personally identifiable information or keyboard strokes are recorded or shared.
You can disable crash reporting on the Keyman
[settings menu](../basic/config/)

2
android/docs/help/basic/config/index.md
View file

@ -59,4 +59,4 @@ When enabled, the Keyman keyboard will provide haptic feedback (vibrate) when th
When enabled, the Keyman app will display the 'Get Started' screen on app startup.
## Allow sending crash reports over network
When enabled, the Keyman app will send crash reporting information to https://sentry.keyman.com. No personally identifiable information or keyboard strokes are recorded.
When enabled, the Keyman app will send crash reporting information to [Sentry](https://keyman.sentry.io). No personally identifiable information or keyboard strokes are recorded.

3
common/include/km_types.h
View file

@ -1,8 +1,5 @@
#pragma once
#include <stdint.h>
#include <stdint.h>
/*
#if defined(_WIN32) || defined(_WIN64)
#define snprintf _snprintf

20
common/include/test_assert.h
View file

@ -28,25 +28,39 @@
#ifdef try_status
#undef try_status
#endif
#if defined(EXPECT_EQ)
/* vector to GTest where available */
#define try_status(expr) EXPECT_EQ((expr), KM_CORE_STATUS_OK)
#else
#define try_status(expr) { \
auto __s = (expr); \
if (__s != KM_CORE_STATUS_OK) { \
_test_assert_failed(__s, u ## #expr); \
} \
}
#endif
#ifdef test_assert
#undef test_assert
#endif
#if defined(EXPECT_TRUE)
/* vector to GTest where available */
# define test_assert(expr) EXPECT_TRUE(expr) << "test_assert failed"
#else
#define test_assert(expr) { \
if (!(expr)) { \
_test_assert_failed(0, u ## #expr); \
} \
}
#endif
#ifdef test_assert_equal
#undef test_assert_equal
#endif
#if defined(EXPECT_EQ)
/* vector to GTest where available */
#define test_assert_equal(actual, expected) EXPECT_EQ(actual, expected)
#else
#define test_assert_equal(actual, expected) { \
if ((actual) != (expected)) { \
std::wcerr << console_color::fg(console_color::BRIGHT_RED) \
@ -58,10 +72,15 @@
std::exit(EXIT_FAILURE); \
} \
}
#endif
#ifdef test_assert_string_equal
#undef test_assert_string_equal
#endif
#if defined(EXPECT_EQ)
/* vector to GTest where available */
#define test_assert_string_equal(actual, expected) EXPECT_EQ(actual, expected)
#else
#define test_assert_string_equal(actual, expected) { \
if (u16cmp((actual), (expected)) != 0) { \
std::wcerr << console_color::fg(console_color::BRIGHT_RED) \
@ -73,3 +92,4 @@
std::exit(EXIT_FAILURE); \
} \
}
#endif

16
common/test/keyboards/baseline/k_055___deadkey_cancelled_by_arrow.kmn Normal file
View file

@ -0,0 +1,16 @@
store(&NAME) '050 - deadkey cancelled by arrow'
c Description: Tests that context gets reset if we encounter a right-arrow
c after a deadkey (#12968)
c keys: [SHIFT K_6][K_RIGHT][K_E][SHIFT K_6][K_LEFT][K_E][SHIFT K_6][K_UP][K_E][SHIFT K_6][K_DOWN][K_E]
c expected: PASS.PASS.PASS.PASS.
store(&version) '10.0'
begin Unicode > use(Main)
group(Main) using keys
+ '^' > dk(1)
dk(1) + 'e' > 'FAIL.'
+ 'e' > 'PASS.'

BIN
common/test/keyboards/baseline/k_055___deadkey_cancelled_by_arrow.kmx Normal file
View file

Binary file not shown.

514
core/include/keyman/keyman_core_api_vkeys.h
View file

@ -53,261 +53,265 @@ enum km_core_modifier_mask {
};
// These are Windows API VKEYs, using Keyman VKEY names.
// Underlying values from winuser.h (https://learn.microsoft.com/en-us/windows/win32/inputdev/virtual-key-codes)
// Note, some codes are defined here for parity with winuser.h, but are
// not available in .kmn language. These codes are marked as "internal"
// by using `__` in the name: `KM_CORE_VKEY__<name>__`
enum km_kpb_virtual_key {
KM_CORE_VKEY__00,
KM_CORE_VKEY_LBUTTON,
KM_CORE_VKEY_RBUTTON,
KM_CORE_VKEY_CANCEL,
KM_CORE_VKEY_MBUTTON,
KM_CORE_VKEY__05,
KM_CORE_VKEY__06,
KM_CORE_VKEY__07,
KM_CORE_VKEY_BKSP,
KM_CORE_VKEY_TAB,
KM_CORE_VKEY__0A,
KM_CORE_VKEY__0B,
KM_CORE_VKEY_KP5,
KM_CORE_VKEY_ENTER,
KM_CORE_VKEY__0E,
KM_CORE_VKEY__0F,
KM_CORE_VKEY_SHIFT,
KM_CORE_VKEY_CONTROL,
KM_CORE_VKEY_ALT,
KM_CORE_VKEY_PAUSE,
KM_CORE_VKEY_CAPS,
KM_CORE_VKEY__15,
KM_CORE_VKEY__16,
KM_CORE_VKEY__17,
KM_CORE_VKEY__18,
KM_CORE_VKEY__19,
KM_CORE_VKEY__1A,
KM_CORE_VKEY_ESC,
KM_CORE_VKEY__1C,
KM_CORE_VKEY__1D,
KM_CORE_VKEY__1E,
KM_CORE_VKEY__1F,
KM_CORE_VKEY_SPACE,
KM_CORE_VKEY_PGUP,
KM_CORE_VKEY_PGDN,
KM_CORE_VKEY_END,
KM_CORE_VKEY_HOME,
KM_CORE_VKEY_LEFT,
KM_CORE_VKEY_UP,
KM_CORE_VKEY_RIGHT,
KM_CORE_VKEY_DOWN,
KM_CORE_VKEY_SEL,
KM_CORE_VKEY_PRINT,
KM_CORE_VKEY_EXEC,
KM_CORE_VKEY_PRTSCN,
KM_CORE_VKEY_INS,
KM_CORE_VKEY_DEL,
KM_CORE_VKEY_HELP,
KM_CORE_VKEY_0,
KM_CORE_VKEY_1,
KM_CORE_VKEY_2,
KM_CORE_VKEY_3,
KM_CORE_VKEY_4,
KM_CORE_VKEY_5,
KM_CORE_VKEY_6,
KM_CORE_VKEY_7,
KM_CORE_VKEY_8,
KM_CORE_VKEY_9,
KM_CORE_VKEY__3A,
KM_CORE_VKEY__3B,
KM_CORE_VKEY__3C,
KM_CORE_VKEY__3D,
KM_CORE_VKEY__3E,
KM_CORE_VKEY__3F,
KM_CORE_VKEY__40,
KM_CORE_VKEY_A,
KM_CORE_VKEY_B,
KM_CORE_VKEY_C,
KM_CORE_VKEY_D,
KM_CORE_VKEY_E,
KM_CORE_VKEY_F,
KM_CORE_VKEY_G,
KM_CORE_VKEY_H,
KM_CORE_VKEY_I,
KM_CORE_VKEY_J,
KM_CORE_VKEY_K,
KM_CORE_VKEY_L,
KM_CORE_VKEY_M,
KM_CORE_VKEY_N,
KM_CORE_VKEY_O,
KM_CORE_VKEY_P,
KM_CORE_VKEY_Q,
KM_CORE_VKEY_R,
KM_CORE_VKEY_S,
KM_CORE_VKEY_T,
KM_CORE_VKEY_U,
KM_CORE_VKEY_V,
KM_CORE_VKEY_W,
KM_CORE_VKEY_X,
KM_CORE_VKEY_Y,
KM_CORE_VKEY_Z,
KM_CORE_VKEY__5B,
KM_CORE_VKEY__5C,
KM_CORE_VKEY__5D,
KM_CORE_VKEY__5E,
KM_CORE_VKEY__5F,
KM_CORE_VKEY_NP0,
KM_CORE_VKEY_NP1,
KM_CORE_VKEY_NP2,
KM_CORE_VKEY_NP3,
KM_CORE_VKEY_NP4,
KM_CORE_VKEY_NP5,
KM_CORE_VKEY_NP6,
KM_CORE_VKEY_NP7,
KM_CORE_VKEY_NP8,
KM_CORE_VKEY_NP9,
KM_CORE_VKEY_NPSTAR,
KM_CORE_VKEY_NPPLUS,
KM_CORE_VKEY_SEPARATOR,
KM_CORE_VKEY_NPMINUS,
KM_CORE_VKEY_NPDOT,
KM_CORE_VKEY_NPSLASH,
KM_CORE_VKEY_F1,
KM_CORE_VKEY_F2,
KM_CORE_VKEY_F3,
KM_CORE_VKEY_F4,
KM_CORE_VKEY_F5,
KM_CORE_VKEY_F6,
KM_CORE_VKEY_F7,
KM_CORE_VKEY_F8,
KM_CORE_VKEY_F9,
KM_CORE_VKEY_F10,
KM_CORE_VKEY_F11,
KM_CORE_VKEY_F12,
KM_CORE_VKEY_F13,
KM_CORE_VKEY_F14,
KM_CORE_VKEY_F15,
KM_CORE_VKEY_F16,
KM_CORE_VKEY_F17,
KM_CORE_VKEY_F18,
KM_CORE_VKEY_F19,
KM_CORE_VKEY_F20,
KM_CORE_VKEY_F21,
KM_CORE_VKEY_F22,
KM_CORE_VKEY_F23,
KM_CORE_VKEY_F24,
KM_CORE_VKEY__88,
KM_CORE_VKEY__89,
KM_CORE_VKEY__8A,
KM_CORE_VKEY__8B,
KM_CORE_VKEY__8C,
KM_CORE_VKEY__8D,
KM_CORE_VKEY__8E,
KM_CORE_VKEY__8F,
KM_CORE_VKEY_NUMLOCK,
KM_CORE_VKEY_SCROLL,
KM_CORE_VKEY__92,
KM_CORE_VKEY__93,
KM_CORE_VKEY__94,
KM_CORE_VKEY__95,
KM_CORE_VKEY__96,
KM_CORE_VKEY__97,
KM_CORE_VKEY__98,
KM_CORE_VKEY__99,
KM_CORE_VKEY__9A,
KM_CORE_VKEY__9B,
KM_CORE_VKEY__9C,
KM_CORE_VKEY__9D,
KM_CORE_VKEY__9E,
KM_CORE_VKEY__9F,
KM_CORE_VKEY__A0,
KM_CORE_VKEY__A1,
KM_CORE_VKEY__A2,
KM_CORE_VKEY__A3,
KM_CORE_VKEY__A4,
KM_CORE_VKEY__A5,
KM_CORE_VKEY__A6,
KM_CORE_VKEY__A7,
KM_CORE_VKEY__A8,
KM_CORE_VKEY__A9,
KM_CORE_VKEY__AA,
KM_CORE_VKEY__AB,
KM_CORE_VKEY__AC,
KM_CORE_VKEY__AD,
KM_CORE_VKEY__AE,
KM_CORE_VKEY__AF,
KM_CORE_VKEY__B0,
KM_CORE_VKEY__B1,
KM_CORE_VKEY__B2,
KM_CORE_VKEY__B3,
KM_CORE_VKEY__B4,
KM_CORE_VKEY__B5,
KM_CORE_VKEY__B6,
KM_CORE_VKEY__B7,
KM_CORE_VKEY__B8,
KM_CORE_VKEY__B9,
KM_CORE_VKEY_COLON,
KM_CORE_VKEY_EQUAL,
KM_CORE_VKEY_COMMA,
KM_CORE_VKEY_HYPHEN,
KM_CORE_VKEY_PERIOD,
KM_CORE_VKEY_SLASH,
KM_CORE_VKEY_BKQUOTE,
KM_CORE_VKEY__C1,
KM_CORE_VKEY__C2,
KM_CORE_VKEY__C3,
KM_CORE_VKEY__C4,
KM_CORE_VKEY__C5,
KM_CORE_VKEY__C6,
KM_CORE_VKEY__C7,
KM_CORE_VKEY__C8,
KM_CORE_VKEY__C9,
KM_CORE_VKEY__CA,
KM_CORE_VKEY__CB,
KM_CORE_VKEY__CC,
KM_CORE_VKEY__CD,
KM_CORE_VKEY__CE,
KM_CORE_VKEY__CF,
KM_CORE_VKEY__D0,
KM_CORE_VKEY__D1,
KM_CORE_VKEY__D2,
KM_CORE_VKEY__D3,
KM_CORE_VKEY__D4,
KM_CORE_VKEY__D5,
KM_CORE_VKEY__D6,
KM_CORE_VKEY__D7,
KM_CORE_VKEY__D8,
KM_CORE_VKEY__D9,
KM_CORE_VKEY__DA,
KM_CORE_VKEY_LBRKT,
KM_CORE_VKEY_BKSLASH,
KM_CORE_VKEY_RBRKT,
KM_CORE_VKEY_QUOTE,
KM_CORE_VKEY_oDF,
KM_CORE_VKEY_oE0,
KM_CORE_VKEY_oE1,
KM_CORE_VKEY_oE2, // 102nd key on European layouts
KM_CORE_VKEY_oE3,
KM_CORE_VKEY_oE4,
KM_CORE_VKEY__E5,
KM_CORE_VKEY_oE6,
KM_CORE_VKEY__E7,
KM_CORE_VKEY__E8,
KM_CORE_VKEY_oE9,
KM_CORE_VKEY_oEA,
KM_CORE_VKEY_oEB,
KM_CORE_VKEY_oEC,
KM_CORE_VKEY_oED,
KM_CORE_VKEY_oEE,
KM_CORE_VKEY_oEF,
KM_CORE_VKEY_oF0,
KM_CORE_VKEY_oF1,
KM_CORE_VKEY_oF2,
KM_CORE_VKEY_oF3,
KM_CORE_VKEY_oF4,
KM_CORE_VKEY_oF5,
KM_CORE_VKEY__F6,
KM_CORE_VKEY__F7,
KM_CORE_VKEY__F8,
KM_CORE_VKEY__F9,
KM_CORE_VKEY__FA,
KM_CORE_VKEY__FB,
KM_CORE_VKEY__FC,
KM_CORE_VKEY__FD,
KM_CORE_VKEY__FE,
KM_CORE_VKEY__FF,
KM_CORE_VKEY_LBUTTON, // 0x01
KM_CORE_VKEY_RBUTTON, // 0x02
KM_CORE_VKEY_CANCEL, // 0x03
KM_CORE_VKEY_MBUTTON, // 0x04
KM_CORE_VKEY__XBUTTON1__, // 0x05
KM_CORE_VKEY__XBUTTON2__, // 0x06
KM_CORE_VKEY__07, // 0x07 - Reserved
KM_CORE_VKEY_BKSP, // 0x08 - VK_BACK
KM_CORE_VKEY_TAB, // 0x09
KM_CORE_VKEY__0A, // 0x0a - Reserved
KM_CORE_VKEY__0B, // 0x0b - Reserved
KM_CORE_VKEY_KP5, // 0x0c - VK_CLEAR
KM_CORE_VKEY_ENTER, // 0x0d - VK_RETURN
KM_CORE_VKEY__0E, // 0x0e - Reserved
KM_CORE_VKEY__0F, // 0x0f - Reserved
KM_CORE_VKEY_SHIFT, // 0x10
KM_CORE_VKEY_CONTROL, // 0x11
KM_CORE_VKEY_ALT, // 0x12 - VK_MENU
KM_CORE_VKEY_PAUSE, // 0x13
KM_CORE_VKEY_CAPS, // 0x14 - VK_CAPITAL
KM_CORE_VKEY__KANA__, // 0x15
KM_CORE_VKEY__HANGUL__, // 0x16
KM_CORE_VKEY__JUNJA__, // 0x17
KM_CORE_VKEY__FINAL__, // 0x18
KM_CORE_VKEY__HANJA__, // 0x19 - or VK_KANJI
KM_CORE_VKEY__IMEOFF__, // 0x1a - VK_IME_OFF
KM_CORE_VKEY_ESC, // 0x1b - VK_ESCAPE
KM_CORE_VKEY__CONVERT__, // 0x1c
KM_CORE_VKEY__NONCONVERT__, // 0x1d
KM_CORE_VKEY__ACCEPT__, // 0x1e
KM_CORE_VKEY__MODECHANGE__, // 0x1f
KM_CORE_VKEY_SPACE, // 0x20
KM_CORE_VKEY_PGUP, // 0x21 - VK_PRIOR
KM_CORE_VKEY_PGDN, // 0x22 - VK_NEXT
KM_CORE_VKEY_END, // 0x23
KM_CORE_VKEY_HOME, // 0x24
KM_CORE_VKEY_LEFT, // 0x25
KM_CORE_VKEY_UP, // 0x26
KM_CORE_VKEY_RIGHT, // 0x27
KM_CORE_VKEY_DOWN, // 0x28
KM_CORE_VKEY_SEL, // 0x29 - VK_SELECT
KM_CORE_VKEY_PRINT, // 0x2a
KM_CORE_VKEY_EXEC, // 0x2b - VK_EXECUTE
KM_CORE_VKEY_PRTSCN, // 0x2c - VK_SNAPSHOT
KM_CORE_VKEY_INS, // 0x2d - VK_INSERT
KM_CORE_VKEY_DEL, // 0x2e - VK_DELETE
KM_CORE_VKEY_HELP, // 0x2f
KM_CORE_VKEY_0, // 0x30
KM_CORE_VKEY_1, // 0x31
KM_CORE_VKEY_2, // 0x32
KM_CORE_VKEY_3, // 0x33
KM_CORE_VKEY_4, // 0x34
KM_CORE_VKEY_5, // 0x35
KM_CORE_VKEY_6, // 0x36
KM_CORE_VKEY_7, // 0x37
KM_CORE_VKEY_8, // 0x38
KM_CORE_VKEY_9, // 0x39
KM_CORE_VKEY__3A, // 0x3a - undefined
KM_CORE_VKEY__3B, // 0x3b - undefined
KM_CORE_VKEY__3C, // 0x3c - undefined
KM_CORE_VKEY__3D, // 0x3d - undefined
KM_CORE_VKEY__3E, // 0x3e - undefined
KM_CORE_VKEY__3F, // 0x3f - undefined
KM_CORE_VKEY__40, // 0x40 - undefined
KM_CORE_VKEY_A, // 0x41
KM_CORE_VKEY_B, // 0x42
KM_CORE_VKEY_C, // 0x43
KM_CORE_VKEY_D, // 0x44
KM_CORE_VKEY_E, // 0x45
KM_CORE_VKEY_F, // 0x46
KM_CORE_VKEY_G, // 0x47
KM_CORE_VKEY_H, // 0x48
KM_CORE_VKEY_I, // 0x49
KM_CORE_VKEY_J, // 0x4a
KM_CORE_VKEY_K, // 0x4b
KM_CORE_VKEY_L, // 0x4c
KM_CORE_VKEY_M, // 0x4d
KM_CORE_VKEY_N, // 0x4e
KM_CORE_VKEY_O, // 0x4f
KM_CORE_VKEY_P, // 0x50
KM_CORE_VKEY_Q, // 0x51
KM_CORE_VKEY_R, // 0x52
KM_CORE_VKEY_S, // 0x53
KM_CORE_VKEY_T, // 0x54
KM_CORE_VKEY_U, // 0x55
KM_CORE_VKEY_V, // 0x56
KM_CORE_VKEY_W, // 0x57
KM_CORE_VKEY_X, // 0x58
KM_CORE_VKEY_Y, // 0x59
KM_CORE_VKEY_Z, // 0x5a
KM_CORE_VKEY__LWIN__, // 0x5b
KM_CORE_VKEY__RWIN__, // 0x5c
KM_CORE_VKEY__APPS__, // 0x5d
KM_CORE_VKEY__5E, // 0x5e - reserved
KM_CORE_VKEY__SLEEP__, // 0x5f
KM_CORE_VKEY_NP0, // 0x60 - VK_NUMPAD0
KM_CORE_VKEY_NP1, // 0x61 - VK_NUMPAD1
KM_CORE_VKEY_NP2, // 0x62 - VK_NUMPAD2
KM_CORE_VKEY_NP3, // 0x63 - VK_NUMPAD3
KM_CORE_VKEY_NP4, // 0x64 - VK_NUMPAD4
KM_CORE_VKEY_NP5, // 0x65 - VK_NUMPAD5
KM_CORE_VKEY_NP6, // 0x66 - VK_NUMPAD6
KM_CORE_VKEY_NP7, // 0x67 - VK_NUMPAD7
KM_CORE_VKEY_NP8, // 0x68 - VK_NUMPAD8
KM_CORE_VKEY_NP9, // 0x69 - VK_NUMPAD9
KM_CORE_VKEY_NPSTAR, // 0x6a - VK_MULTIPLY
KM_CORE_VKEY_NPPLUS, // 0x6b - VK_ADD
KM_CORE_VKEY_SEPARATOR, // 0x6c
KM_CORE_VKEY_NPMINUS, // 0x6d - VK_SUBTRACT
KM_CORE_VKEY_NPDOT, // 0x6e - VK_DECIMAL
KM_CORE_VKEY_NPSLASH, // 0x6f - VK_DIVIDE
KM_CORE_VKEY_F1, // 0x70
KM_CORE_VKEY_F2, // 0x71
KM_CORE_VKEY_F3, // 0x72
KM_CORE_VKEY_F4, // 0x73
KM_CORE_VKEY_F5, // 0x74
KM_CORE_VKEY_F6, // 0x75
KM_CORE_VKEY_F7, // 0x76
KM_CORE_VKEY_F8, // 0x77
KM_CORE_VKEY_F9, // 0x78
KM_CORE_VKEY_F10, // 0x79
KM_CORE_VKEY_F11, // 0x7a
KM_CORE_VKEY_F12, // 0x7b
KM_CORE_VKEY_F13, // 0x7c
KM_CORE_VKEY_F14, // 0x7d
KM_CORE_VKEY_F15, // 0x7e
KM_CORE_VKEY_F16, // 0x7f
KM_CORE_VKEY_F17, // 0x80
KM_CORE_VKEY_F18, // 0x81
KM_CORE_VKEY_F19, // 0x82
KM_CORE_VKEY_F20, // 0x83
KM_CORE_VKEY_F21, // 0x84
KM_CORE_VKEY_F22, // 0x85
KM_CORE_VKEY_F23, // 0x86
KM_CORE_VKEY_F24, // 0x87
KM_CORE_VKEY__88, // 0x88 - reserved
KM_CORE_VKEY__89, // 0x89 - reserved
KM_CORE_VKEY__8A, // 0x8a - reserved
KM_CORE_VKEY__8B, // 0x8b - reserved
KM_CORE_VKEY__8C, // 0x8c - reserved
KM_CORE_VKEY__8D, // 0x8d - reserved
KM_CORE_VKEY__8E, // 0x8e - reserved
KM_CORE_VKEY__8F, // 0x8f - reserved
KM_CORE_VKEY_NUMLOCK, // 0x90
KM_CORE_VKEY_SCROLL, // 0x91
KM_CORE_VKEY__92, // 0x92 - OEM specific
KM_CORE_VKEY__93, // 0x93 - OEM specific
KM_CORE_VKEY__94, // 0x94 - OEM specific
KM_CORE_VKEY__95, // 0x95 - OEM specific
KM_CORE_VKEY__96, // 0x96 - OEM specific
KM_CORE_VKEY__97, // 0x97 - unassigned
KM_CORE_VKEY__98, // 0x98 - unassigned
KM_CORE_VKEY__99, // 0x99 - unassigned
KM_CORE_VKEY__9A, // 0x9a - unassigned
KM_CORE_VKEY__9B, // 0x9b - unassigned
KM_CORE_VKEY__9C, // 0x9c - unassigned
KM_CORE_VKEY__9D, // 0x9d - unassigned
KM_CORE_VKEY__9E, // 0x9e - unassigned
KM_CORE_VKEY__9F, // 0x9f - unassigned
KM_CORE_VKEY__LSHIFT__, // 0xa0
KM_CORE_VKEY__RSHIFT__, // 0xa1
KM_CORE_VKEY__LCONTROL__, // 0xa2
KM_CORE_VKEY__RCONTROL__, // 0xa3
KM_CORE_VKEY__LMENU__, // 0xa4
KM_CORE_VKEY__RMENU__, // 0xa5
KM_CORE_VKEY__BROWSERBACK__, // 0xa6 - VK_BROWSER_BACK
KM_CORE_VKEY__BROWSERFORWARD__, // 0xa7 - VK_BROWSER_FORWARD
KM_CORE_VKEY__BROWSERREFRESH__, // 0xa8 - VK_BROWSER_REFRESH
KM_CORE_VKEY__BROWSERSTOP__, // 0xa9 - VK_BROWSER_STOP
KM_CORE_VKEY__BROWSERSEARCH__, // 0xaa - VK_BROWSER_SEARCH
KM_CORE_VKEY_BROWSERFAVORITES, // 0xab - VK_BROWSER_FAVORITES
KM_CORE_VKEY__BROWSERHOME__, // 0xac - VK_BROWSER_HOME
KM_CORE_VKEY__VOLUMEMUTE__, // 0xad - VK_VOLUME_MUTE
KM_CORE_VKEY__VOLUMEDOWN__, // 0xae - VK_VOLUME_DOWN
KM_CORE_VKEY__VOLUMEUP__, // 0xaf - VK_VOLUME_UP
KM_CORE_VKEY__MEDIANEXT__, // 0xb0 - VK_MEDIA_NEXT_TRACK
KM_CORE_VKEY__MEDIAPREV__, // 0xb1 - VK_MEDIA_PREV_TRACK
KM_CORE_VKEY__MEDIASTOP__, // 0xb2 - VK_MEDIA_STOP
KM_CORE_VKEY__MEDIAPLAY__, // 0xb3 - VK_MEDIA_PLAY_PAUSE
KM_CORE_VKEY__MAIL__, // 0xb4 - VK_LAUNCH_MAIL
KM_CORE_VKEY__MEDIA__, // 0xb5 - VK_LAUNCH_MEDIA_SELECT
KM_CORE_VKEY__APP1__, // 0xb6 - VK_LAUNCH_APP1
KM_CORE_VKEY__APP2__, // 0xb7 - VK_LAUNCH_APP2
KM_CORE_VKEY__B8, // 0xb8 - reserved
KM_CORE_VKEY__B9, // 0xb9 - reserved
KM_CORE_VKEY_COLON, // 0xba - VK_OEM_1
KM_CORE_VKEY_EQUAL, // 0xbb - VK_OEM_PLUS
KM_CORE_VKEY_COMMA, // 0xbc - VK_OEM_COMMA
KM_CORE_VKEY_HYPHEN, // 0xbd - VK_OEM_MINUS
KM_CORE_VKEY_PERIOD, // 0xbe - VK_OEM_PERIOD
KM_CORE_VKEY_SLASH, // 0xbf - VK_OEM_2
KM_CORE_VKEY_BKQUOTE, // 0xc0 - VK_OEM_3
KM_CORE_VKEY__C1, // 0xc1 - reserved
KM_CORE_VKEY__C2, // 0xc2 - reserved
KM_CORE_VKEY__C3, // 0xc3 - reserved
KM_CORE_VKEY__C4, // 0xc4 - reserved
KM_CORE_VKEY__C5, // 0xc5 - reserved
KM_CORE_VKEY__C6, // 0xc6 - reserved
KM_CORE_VKEY__C7, // 0xc7 - reserved
KM_CORE_VKEY__C8, // 0xc8 - reserved
KM_CORE_VKEY__C9, // 0xc9 - reserved
KM_CORE_VKEY__CA, // 0xca - reserved
KM_CORE_VKEY__CB, // 0xcb - reserved
KM_CORE_VKEY__CC, // 0xcc - reserved
KM_CORE_VKEY__CD, // 0xcd - reserved
KM_CORE_VKEY__CE, // 0xce - reserved
KM_CORE_VKEY__CF, // 0xcf - reserved
KM_CORE_VKEY__D0, // 0xd0 - reserved
KM_CORE_VKEY__D1, // 0xd1 - reserved
KM_CORE_VKEY__D2, // 0xd2 - reserved
KM_CORE_VKEY__D3, // 0xd3 - reserved
KM_CORE_VKEY__D4, // 0xd4 - reserved
KM_CORE_VKEY__D5, // 0xd5 - reserved
KM_CORE_VKEY__D6, // 0xd6 - reserved
KM_CORE_VKEY__D7, // 0xd7 - reserved
KM_CORE_VKEY__D8, // 0xd8 - reserved
KM_CORE_VKEY__D9, // 0xd9 - reserved
KM_CORE_VKEY__DA, // 0xda - reserved
KM_CORE_VKEY_LBRKT, // 0xdb - VK_OEM_4
KM_CORE_VKEY_BKSLASH, // 0xdc - VK_OEM_5
KM_CORE_VKEY_RBRKT, // 0xdd - VK_OEM_6
KM_CORE_VKEY_QUOTE, // 0xde - VK_OEM_7
KM_CORE_VKEY_oDF, // 0xdf - VK_OEM_8
KM_CORE_VKEY_oE0, // 0xe0 - reserved
KM_CORE_VKEY_oE1, // 0xe1 - reserved
KM_CORE_VKEY_oE2, // 0xe2 - VK_OEM_102 - 102nd key on European layouts
KM_CORE_VKEY_oE3, // 0xe3 - OEM specific
KM_CORE_VKEY_oE4, // 0xe4 - OEM specific
KM_CORE_VKEY__PROCESSKEY__, // 0xe5
KM_CORE_VKEY_oE6, // 0xe6 - OEM specific
KM_CORE_VKEY__PACKET__, // 0xe7
KM_CORE_VKEY__E8, // 0xe8 - unassigned
KM_CORE_VKEY_oE9, // 0xe9 - OEM specific
KM_CORE_VKEY_oEA, // 0xea - OEM specific
KM_CORE_VKEY_oEB, // 0xeb - OEM specific
KM_CORE_VKEY_oEC, // 0xec - OEM specific
KM_CORE_VKEY_oED, // 0xed - OEM specific
KM_CORE_VKEY_oEE, // 0xee - OEM specific
KM_CORE_VKEY_oEF, // 0xef - OEM specific
KM_CORE_VKEY_oF0, // 0xf0 - OEM specific
KM_CORE_VKEY_oF1, // 0xf1 - OEM specific
KM_CORE_VKEY_oF2, // 0xf2 - OEM specific
KM_CORE_VKEY_oF3, // 0xf3 - OEM specific
KM_CORE_VKEY_oF4, // 0xf4 - OEM specific
KM_CORE_VKEY_oF5, // 0xf5 - OEM specific
KM_CORE_VKEY__ATTN__, // 0xf6
KM_CORE_VKEY__CRSEL__, // 0xf7
KM_CORE_VKEY__EXSEL__, // 0xf8
KM_CORE_VKEY__EREOF__, // 0xf9
KM_CORE_VKEY__PLAY__, // 0xfa
KM_CORE_VKEY__ZOOM__, // 0xfb
KM_CORE_VKEY__NONAME__, // 0xfc
KM_CORE_VKEY__PA1__, // 0xfd
KM_CORE_VKEY__OEMCLEAR__, // 0xfe
KM_CORE_VKEY__FF, // 0xff
};

240
core/src/kmx/kmx_plus.cpp
View file

@ -129,42 +129,56 @@ header_from_bytes(const uint8_t *data, KMX_DWORD length, uint32_t ident) {
* @brief Accessor for a section based on bytes
*
* @tparam T
* @param data
* @param length
* @param data input pointer to the section to be validated. If nullptr, 'sect' will be nullptr, but will return true (valid).
* @param length length of input data. This function will not read past the end of this length.
* @param out on exit, will be set to the new section. On any failure, null data, or invalidity, will be nullptr.
* @return true if section is missing (nullptr) or valid
* @return const T*
*/
template <class T>
const T *section_from_bytes(const uint8_t *data, KMX_DWORD length) {
if (length < sizeof(T)) { // Does not include dynamic data. First check.
template <class T> bool section_from_bytes(const uint8_t* data, KMX_DWORD length, const T*& out) {
out = nullptr; // null unless valid.
if (data == nullptr) {
// missing section - not an error.
DebugLog("data was null (missing section)");
return true;
} else if (length < sizeof(T)) { // Does not include dynamic data. First check.
DebugLog("length < sizeof(section)");
assert(false);
return nullptr;
return false;
}
// verify the kmx+ header
const COMP_KMXPLUS_HEADER *header = header_from_bytes(data, length, T::IDENT);
// now it is safe to cast this to a T
const T *section = reinterpret_cast<const T *>(header);
if (section != nullptr && section->valid(length)) {
return section;
} else {
if (section == nullptr) {
// should not happen.
DebugLog("reinterpret_cast<> failed.");
assert(false);
return nullptr;
return false;
} else if (!section->valid(length)) {
return false; // validation failed.
} else {
out = section;
return true;
}
}
/**
* @param sect this is the pointer to the 'sect' section table. Should not be null!
* @param out on exit, will be set to the requested section, or nullptr if missing or invalid
* @returns true if section is valid or missing, false only if invalid
*/
template <class T>
const T *section_from_sect(const COMP_KMXPLUS_SECT* sect) {
const uint8_t *rawbytes = reinterpret_cast<const uint8_t *>(sect);
if (rawbytes == nullptr) {
DebugLog("section_from_sect(nullptr) == nullptr");
assert(false);
return nullptr;
bool get_section_from_sect(const COMP_KMXPLUS_SECT* sect, const T*& out) {
KMX_DWORD entryLength;
const uint8_t *rawbytes = sect->get(T::IDENT, entryLength);
if (rawbytes == nullptr) {
// just missing, not invalid
out = nullptr;
return true;
} else {
return section_from_bytes<T>(rawbytes, entryLength, out);
}
KMX_DWORD offset = sect->find(T::IDENT);
if (!offset) {
DebugLog("section_from_sect() - not found. section %c%c%c%c (0x%X)", DEBUG_IDENT(T::IDENT), T::IDENT);
return nullptr;
}
KMX_DWORD entrylength = sect->total - offset;
return section_from_bytes<T>(rawbytes+offset, entrylength);
}
bool
@ -261,8 +275,63 @@ COMP_KMXPLUS_STRS::valid(KMX_DWORD _kmn_unused(length)) const {
assert(start[length] == 0);
return false;
}
// TODO-LDML: validate valid UTF-16LE?
DebugLoad("strs #0x%X: '%s'", i, Debug_UnicodeString(start));
if (!COMP_KMXPLUS_STRS::valid_string(start, length)) {
DebugLog("#0x%X: String of length 0x%x invalid", i, length);
return false;
}
}
return true;
}
bool COMP_KMXPLUS_STRS::valid_string(const KMX_WCHAR* start, KMX_DWORD length) {
for (KMX_DWORD n = 0; n < length; n++) {
const auto& ch = start[n];
if (Uni_IsSurrogate2(ch)) {
DebugLog("String of length 0x%x @ 0x%x: Char U+%04X is a trailing (unpaired) surrogate", length, n, ch);
return false;
} else if (Uni_IsSurrogate1(ch)) {
n++;
if (n == length) {
DebugLog("String of length 0x%x @ 0x%x: Char U+%04X ends with leading (unpaired) surrogate", length, n, ch);
return false;
}
const auto& ch2 = start[n];
if (!Uni_IsSurrogate2(ch2)) {
DebugLog("String of length 0x%x @ 0x%x: Char U+%04X not a trailing surrogate", length, n, ch2);
return false;
}
const km_core_usv ch32 = Uni_SurrogateToUTF32(ch, ch2);
if (!Uni_IsValid(ch32)) {
DebugLog("String of length 0x%x @ 0x%x: Char U+%04X is illegal char", length, n, ch32);
return false;
}
} else if (ch == LDML_UC_SENTINEL) {
n++;
if (n == length) {
DebugLog("String of length 0x%x @ 0x%x: Sentinel value U+%04X at end of string", length, n, ch);
return false;
}
const auto& ch2 = start[n];
if (ch2 != LDML_MARKER_CODE) {
DebugLog("String of length 0x%x @ 0x%x: Sentinel value followed by U+%04X", length, n, ch2);
return false;
}
n++;
if (n == length) {
DebugLog("String of length 0x%x @ 0x%x: Sentinel value followed by U+%04X at end of string", length, n, ch);
return false;
}
const auto& ch3 = start[n];
if (!is_valid_marker(ch3)) {
DebugLog("String of length 0x%x @ 0x%x: Sentinel value + CODE followed by invalid marker U+%04X", length, n, ch);
return false;
}
// else OK (good marker)
} else if(!Uni_IsValid(ch)) {
DebugLog("String of length 0x%x @ 0x%x: Char U+%04X is illegal char", length, n, ch);
return false;
} // else OK (other char)
}
return true;
}
@ -316,6 +385,39 @@ COMP_KMXPLUS_SECT::valid(KMX_DWORD length) const {
return overall_valid;
}
KMX_DWORD COMP_KMXPLUS_SECT::find(KMX_DWORD ident) const {
for (KMX_DWORD i = 0; i < count; i++) {
if (ident == entries[i].sect) {
return entries[i].offset;
}
}
return 0;
}
const uint8_t *COMP_KMXPLUS_SECT::get(KMX_DWORD ident, KMX_DWORD &entryLength) const {
entryLength = 0;
// the section table is also the beginning of the file.
const uint8_t *rawbytes = reinterpret_cast<const uint8_t *>(this);
if (rawbytes == nullptr) {
// should not get here - null this pointer.
DebugLog("section_from_sect(nullptr) == nullptr");
assert(false);
return nullptr;
}
// lookup the offset from teh table
KMX_DWORD offset = find(ident);
if (!offset) {
DebugLog("section_from_sect() - not found. section %c%c%c%c (0x%X)", DEBUG_IDENT(ident), ident);
return nullptr;
}
// return the potential length, based on table.
// we approximate the entryLength here, to at least not run off the end of the file.
// we could take the offset of the next section, as an improvement.
entryLength = total - offset;
// return the pointer to the raw bytes
return rawbytes+offset;
}
// ---- transform related fcns
bool
COMP_KMXPLUS_ELEM::valid(KMX_DWORD _kmn_unused(length)) const {
@ -452,12 +554,11 @@ bool
COMP_KMXPLUS_TRAN_Helper::setTran(const COMP_KMXPLUS_TRAN *newTran) {
is_valid = true;
if (newTran == nullptr) {
DebugLog("tran helper: invalid, newTran=%p", newTran);
DebugLog("tran helper: missing, newTran=%p", newTran);
// Note: kmx_plus::kmx_plus has already called section_from_bytes()
// which validates this section's length. Will be nullptr here if invalid.
is_valid = false;
// No assert here: just a missing layer
return false;
// No assert here: just a missing section
return true;
}
DebugLog("tran helper: validating '%c%c%c%c' newTran=%p", DEBUG_IDENT(newTran->header.ident), newTran);
tran = newTran;
@ -582,9 +683,7 @@ COMP_KMXPLUS_LAYR_Helper::setLayr(const COMP_KMXPLUS_LAYR *newLayr) {
if (newLayr == nullptr) {
// Note: kmx_plus::kmx_plus has already called section_from_bytes()
// which validates this section's length. Will be nullptr here if invalid.
is_valid = false;
// No assert here: just a missing layer
return false;
return true; // not invalid, just missing
}
layr = newLayr;
const uint8_t *rawdata = reinterpret_cast<const uint8_t *>(newLayr);
@ -740,9 +839,7 @@ COMP_KMXPLUS_KEYS_Helper::setKeys(const COMP_KMXPLUS_KEYS *newKeys) {
if (newKeys == nullptr) {
// Note: kmx_plus::kmx_plus has already called section_from_bytes()
// which validates this section's length. Will be nullptr here if invalid.
is_valid = false;
// No assert here: just a missing layer
return false;
return true; // not invalid, just missing
}
key2 = newKeys;
const uint8_t *rawdata = reinterpret_cast<const uint8_t *>(newKeys);
@ -788,6 +885,14 @@ COMP_KMXPLUS_KEYS_Helper::setKeys(const COMP_KMXPLUS_KEYS *newKeys) {
is_valid = false;
assert(is_valid);
}
if (!(key.flags & LDML_KEYS_KEY_FLAGS_EXTEND)) {
// if extend flag is clear, then the 'to' field is a UTF-32 char
km_core_usv to = key.to;
if (!Uni_IsValid(to)) {
DebugLog("key[%d] has invalid non-extended UChar to U+%04X", i, key.to);
is_valid = false;
}
}
}
for(KMX_DWORD i = 0; is_valid && i < key2->flicksCount; i++) {
const auto &e = flickLists[i];
@ -838,7 +943,6 @@ COMP_KMXPLUS_KEYS_Helper::setKeys(const COMP_KMXPLUS_KEYS *newKeys) {
}
// Return results
DebugLog("COMP_KMXPLUS_KEYS_Helper.setKeys(): %s", is_valid ? "valid" : "invalid");
assert(is_valid);
return is_valid;
}
@ -933,9 +1037,7 @@ COMP_KMXPLUS_LIST_Helper::setList(const COMP_KMXPLUS_LIST *newList) {
if (newList == nullptr) {
// Note: kmx_plus::kmx_plus has already called section_from_bytes()
// which validates this section's length. Will be nullptr here if invalid.
is_valid = false;
// No assert here: just a missing layer
return false;
return true; // not invalid, just missing
}
list = newList;
const uint8_t *rawdata = reinterpret_cast<const uint8_t *>(newList);
@ -1036,9 +1138,7 @@ COMP_KMXPLUS_USET_Helper::setUset(const COMP_KMXPLUS_USET *newUset) {
if (newUset == nullptr) {
// Note: kmx_plus::kmx_plus has already called section_from_bytes()
// which validates this section's length. Will be nullptr here if invalid.
is_valid = false;
// No assert here: just a missing layer
return false;
return true; // not invalid, just missing
}
uset = newUset;
const uint8_t *rawdata = reinterpret_cast<const uint8_t *>(newUset);
@ -1193,50 +1293,42 @@ kmx_plus::kmx_plus(const COMP_KEYBOARD *keyboard, size_t length)
}
const uint8_t* rawdata = reinterpret_cast<const uint8_t*>(keyboard);
sect = section_from_bytes<COMP_KMXPLUS_SECT>(rawdata+ex->kmxplus.dpKMXPlus, ex->kmxplus.dwKMXPlusSize);
valid = true;
valid = section_from_bytes<COMP_KMXPLUS_SECT>(rawdata+ex->kmxplus.dpKMXPlus, ex->kmxplus.dwKMXPlusSize, sect) && valid;
if (sect == nullptr) {
DebugLog("kmx_plus(): 'sect' did not validate");
DebugLog("kmx_plus(): 'sect' missing or did not validate");
valid = false;
} else {
valid = true;
// load other sections, validating as we go
// each field will be set to nullptr if validation fails
bksp = section_from_sect<COMP_KMXPLUS_BKSP>(sect);
disp = section_from_sect<COMP_KMXPLUS_DISP>(sect);
elem = section_from_sect<COMP_KMXPLUS_ELEM>(sect);
key2 = section_from_sect<COMP_KMXPLUS_KEYS>(sect);
layr = section_from_sect<COMP_KMXPLUS_LAYR>(sect);
list = section_from_sect<COMP_KMXPLUS_LIST>(sect);
loca = section_from_sect<COMP_KMXPLUS_LOCA>(sect);
meta = section_from_sect<COMP_KMXPLUS_META>(sect);
strs = section_from_sect<COMP_KMXPLUS_STRS>(sect);
tran = section_from_sect<COMP_KMXPLUS_TRAN>(sect);
uset = section_from_sect<COMP_KMXPLUS_USET>(sect);
vars = section_from_sect<COMP_KMXPLUS_VARS>(sect);
// each field will be set to nullptr if validation fails or if the section is missing
valid = get_section_from_sect<COMP_KMXPLUS_BKSP>(sect, bksp) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_DISP>(sect, disp) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_ELEM>(sect, elem) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_KEYS>(sect, key2) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_LAYR>(sect, layr) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_LIST>(sect, list) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_LOCA>(sect, loca) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_META>(sect, meta) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_STRS>(sect, strs) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_TRAN>(sect, tran) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_USET>(sect, uset) && valid;
valid = get_section_from_sect<COMP_KMXPLUS_VARS>(sect, vars) && valid;
// Initialize the helper objects for sections with dynamic parts.
// Note: all of these setters will be passed 'nullptr'
// if any section had failed validation.
// if any section had failed validation, or was missing.
// A missing section does not invalidate the kmxplus.
// we attempt to initialize each one
(void)bkspHelper.setTran(bksp); // bksp handled by …TRAN_Helper
(void)key2Helper.setKeys(key2);
(void)layrHelper.setLayr(layr);
(void)listHelper.setList(list);
(void)tranHelper.setTran(tran);
(void)usetHelper.setUset(uset);
valid = bkspHelper.setTran(bksp) && valid; // bksp handled by …TRAN_Helper
valid = key2Helper.setKeys(key2) && valid;
valid = layrHelper.setLayr(layr) && valid;
valid = listHelper.setList(list) && valid;
valid = tranHelper.setTran(tran) && valid;
valid = usetHelper.setUset(uset) && valid;
}
}
KMX_DWORD COMP_KMXPLUS_SECT::find(KMX_DWORD ident) const {
for (KMX_DWORD i = 0; i < count; i++) {
if (ident == entries[i].sect) {
return entries[i].offset;
}
}
return 0;
}
std::u16string
COMP_KMXPLUS_STRS::get(KMX_DWORD entry) const {
assert(entry < count);

18
core/src/kmx/kmx_plus.h
View file

@ -75,12 +75,20 @@ struct COMP_KMXPLUS_SECT {
KMX_DWORD_unaligned count; // 000C number of section headers
COMP_KMXPLUS_SECT_ENTRY entries[]; // 0010 section entries
/**
* @brief Get the offset of a section, or 0
*
* @param ident section id such as 'strs'. Never 'sect'
* @return KMX_DWORD offset from beginning of kmxplus
* @brief Get the offset of a section
*
* @param ident section id such as 'strs'. Never 'sect' (the sect table does not list itself!)
* @return KMX_DWORD offset from beginning of kmxplus, or 0 if not found
*/
KMX_DWORD find(KMX_DWORD ident) const;
/**
* @brief Get the pointer to a specific section
*
* @param ident section id such as 'strs'. Never 'sect' (the sect table does not list itself!)
* @param entryLength on exit, will be set to the possible length of the section (based on the remainder of the KMX+ file)
* @return pointer to raw bytes of requested section, or nullptr if not found
*/
const uint8_t *get(KMX_DWORD ident, KMX_DWORD &entryLength) const;
/**
* @brief True if section is valid.
* Does not validate the entire file.
@ -237,6 +245,8 @@ struct COMP_KMXPLUS_STRS {
/** convert a single char to a string*/
static std::u16string str_from_char(KMX_DWORD v);
static bool valid_string(const KMX_WCHAR *start, KMX_DWORD length);
};
static_assert(sizeof(struct COMP_KMXPLUS_STRS) % 0x4 == 0, "Structs prior to variable part should align to 32-bit boundary");

22
core/src/ldml/ldml_processor.cpp
View file

@ -83,30 +83,39 @@ ldml_processor::ldml_processor(std::u16string const& kb_name, const std::vector<
}
keys.add((km_core_virtual_key)kmapEntry->vkey, kmapEntry->mod, str);
}
} // else: no keys! but still valid. Just, no keys.
} else {
DebugLog("Error, this keyboard has no 'key2' section on it!");
return;
}
// load transforms
if (kplus.tran != nullptr && kplus.tran->groupCount > 0) {
// Note that transforms::load depends on several other sections,
// and this loading will fail if they are missing.
transforms.reset(km::core::ldml::transforms::load(kplus, kplus.tran, kplus.tranHelper));
if (!transforms) {
DebugLog("Failed to load tran transforms");
return; // failed to load
}
}
} // else: no transforms
// load bksp transforms
if (kplus.bksp != nullptr && kplus.bksp->groupCount > 0) {
// Note that transforms::load depends on several other sections,
// and this loading will fail if they are missing.
bksp_transforms.reset(km::core::ldml::transforms::load(kplus, kplus.bksp, kplus.bkspHelper));
if (!bksp_transforms) {
DebugLog("Failed to load bksp transforms");
return; // failed to load
}
}
} // else: no bksp transforms
if (kplus.meta != nullptr) {
if (kplus.meta->settings & LDML_META_SETTINGS_NORMALIZATION_DISABLED) {
normalization_disabled = true;
}
}
} // else: no metadata
// Only valid if we reach here
DebugLog("_valid = true");
_valid = true;
@ -476,7 +485,10 @@ ldml_event_state::emit_text(const std::u32string &str) {
void
ldml_event_state::emit_text( km_core_usv ch) {
assert(ch != LDML_UC_SENTINEL);
// assert that no noncharacter gets through (which includes UC_SENTINEL)
// markers will go through emit_marker() and not through this function.
assert(km::core::kmx::Uni_IsValid(ch));
if(!km::core::kmx::Uni_IsValid(ch)) ch = 0xFFFD; // release build: replace invalid with substitution character
state->context().push_character(ch);
text.push_back(ch);
}

288
core/tests/unit/ldml/invalid-keyboards/ik_001_bad_key.txt Normal file
View file

@ -0,0 +1,288 @@
#
# Keyman is copyright (C) SIL International. MIT License.
#
# like basic.txt, but has a bad (illegal unicode) non-extended key
#
# For the format of this file, see “hextobin.ts”
#
# P.S. surprisingly, the Dart language highlighter in VSCode does a helpful job on this file.
block(kmxheader) # struct COMP_KEYBOARD {
4b 58 54 53 # KMX_DWORD dwIdentifier; // 0000 Keyman compiled keyboard id
00 11 00 00 # KMX_DWORD dwFileVersion; // 0004 Version of the file - Keyman 4.0 is 0x0400
00 00 00 00 # KMX_DWORD dwCheckSum; // 0008 deprecated in 16.0, always 0
00 00 00 00 # KMX_DWORD KeyboardID; // 000C as stored in HKEY_LOCAL_MACHINE//system//currentcontrolset//control//keyboard layouts
01 00 00 00 # KMX_DWORD IsRegistered; // 0010
00 00 00 00 # KMX_DWORD version; // 0014 keyboard version
sizeof(stores,12) # KMX_DWORD cxStoreArray; // 0018 in array entries
00 00 00 00 # KMX_DWORD cxGroupArray; // 001C in array entries
offset(stores) # KMX_DWORD dpStoreArray; // 0020 [LPSTORE] address of first item in store array
00 00 00 00 # KMX_DWORD dpGroupArray; // 0024 [LPGROUP] address of first item in group array
ff ff ff ff # KMX_DWORD StartGroup[2]; // 0028 index of starting groups [2 of them]
ff ff ff ff #
20 00 00 00 # KMX_DWORD dwFlags; // 0030 Flags for the keyboard file
00 00 00 00 # KMX_DWORD dwHotKey; // 0034 standard windows hotkey (hiword=shift/ctrl/alt stuff, loword=vkey)
00 00 00 00 # KMX_DWORD dpBitmapOffset; // 0038 offset of the bitmaps in the file
00 00 00 00 # KMX_DWORD dwBitmapSize; // 003C size in bytes of the bitmaps
# };
block(kmxplusinfo) # struct COMP_KEYBOARD_KMXPLUSINFO {
offset(sect) # KMX_DWORD dpKMXPlus; // 0040 offset of KMXPlus data from BOF, <sect> header is first
diff(sect,eof) # KMX_DWORD dwKMXPlusSize; // 0044 size in bytes of entire KMXPlus data
# };
block(stores) # struct COMP_STORE {
07 00 00 00 # KMX_DWORD dwSystemID; - TSS_NAME
offset(store_name_name) # KMX_DWORD dpName;
offset(store_name_string) # KMX_DWORD dpString;
# };
# excluding this so we don’t have compiled version changes
# 14 00 00 00 # KMX_DWORD dwSystemID; - TSS_COMPILEDVERSION
# offset(store_compiledversion_name) # KMX_DWORD dpName;
# offset(store_compiledversion_string) # KMX_DWORD dpString;
# };
24 00 00 00 # KMX_DWORD dwSystemID; - TSS_KEYBOARDVERSION
offset(store_keyboardversion_name) # KMX_DWORD dpName;
offset(store_keyboardversion_string) # KMX_DWORD dpString;
# };
26 00 00 00 # KMX_DWORD dwSystemID; - TSS_TARGETS
offset(store_targets_name) # KMX_DWORD dpName;
offset(store_targets_string) # KMX_DWORD dpString;
# };
block(store_name_name)
26 00 4e 00 41 00 4d 00 45 00 00 00 # '&NAME'
block(store_name_string)
54 00 65 00 73 00 74 00 4b 00 62 00 64 00 00 00 # 'TestKbd'
block(store_keyboardversion_name)
26 00 4b 00 45 00 59 00 42 00 4f 00 41 00 52 00
44 00 56 00 45 00 52 00 53 00 49 00 4f 00 4e 00
00 00 # '&KEYBOARDVERSION'
block(store_keyboardversion_string)
31 00 2e 00 30 00 2e 00 30 00 00 00 # '1.0.0'
block(store_targets_name)
26 00 54 00 41 00 52 00 47 00 45 00 54 00 53 00 00 00 # '&TARGETS'
block(store_targets_string)
64 00 65 00 73 00 6b 00 74 00 6f 00 70 00 00 00 # 'desktop'
block(sect) # struct COMP_KMXPLUS_SECT {
73 65 63 74 # KMX_DWORD header.ident; // 0000 Section name
diff(sect,endsect) # KMX_DWORD header.size; // 0004 Section length
diff(sect,eof) # KMX_DWORD total; // 0008 KMXPlus entire length
sizeof(sectitems,8) # KMX_DWORD count; // 000C number of section headers
# };
# Next sections are sect entries
# KMX_DWORD sect; // 0010+ Section identity
# KMX_DWORD offset; // 0014+ Section offset relative to dpKMXPlus of section
block(sectitems)
6b 65 79 73
diff(sect,keys)
6c 61 79 72
diff(sect,layr)
6c 6f 63 61
diff(sect,loca)
6d 65 74 61
diff(sect,meta)
73 74 72 73
diff(sect,strs)
block(endsect)
# ----------------------------------------------------------------------------------------------------
# keys
# ----------------------------------------------------------------------------------------------------
block(keys) # struct COMP_KMXPLUS_KEYS {
6b 65 79 73 # KMX_DWORD header.ident; // 0000 Section name - keys
sizeof(keys) # KMX_DWORD header.size; // 0004 Section length
01 00 00 00 # KMX_DWORD keyCount
01 00 00 00 # KMX_DWORD flicksCount
00 00 00 00 # KMX_DWORD flickCount
00 00 00 00 # KMX_DWORD kmapCount; // 0008 number of kmap - #48, one per key
# keys
# (#0000) BAD
FF FF 00 00 # U+FFFF
00 00 00 00 # KMX_DWORD flags = 0
index(strNull,strA,2) # KMXPLUS_STR 'a' (key id)
00 00 00 00 # KMXPLUS_STR switch
0A 00 00 00 # KMX_DWORD width*10
00 00 00 00 # TODO: index(listNull,listNull,4) # LIST longPress
00 00 00 00 # STR longPressDefault
00 00 00 00 # TODO: index(listNull,listNull,4) # LIST multiTap
00 00 00 00 # flicks
# flicks
# flicks 0 - null
00 00 00 00 # KMX_DWORD count
00 00 00 00 # KMX_DWORD flick
00 00 00 00 # KMX_STR id
# flick
# Right now there aren’t any flick elements.
#00 00 00 00 # LIST directions
#00 00 00 01 # flags
#00 00 00 00 # str: to
# kmapdata:
# none,no kmaps
# ----------------------------------------------------------------------------------------------------
# layr
# ----------------------------------------------------------------------------------------------------
block(layr) # struct COMP_KMXPLUS_LAYR {
6c 61 79 72 # KMX_DWORD header.ident; // 0000 Section name - layr
sizeof(layr) # KMX_DWORD header.size; // 0004 Section length
01 00 00 00 # KMX_DWORD listCount
01 00 00 00 # KMX_DWORD layerCount
01 00 00 00 # KMX_DWORD rowCount
01 00 00 00 # KMX_DWORD keyCount
# list 0
index(strNull,strUs,2) # KMXPLUS_STR hardware = 'us'
00 00 00 00 # KMX_DWORD layer;
01 00 00 00 # count
7B 00 00 00 # KMX_DWORD minDeviceWidth; // 123
# layers 0
00 00 00 00 # KMXPLUS_STR id;
00 00 00 00 # KMX_DWORD mod
00 00 00 00 # KMX_DWORD row index
01 00 00 00 # KMX_DWORD count
# rows 0
00 00 00 00 # KMX_DWORD key index
01 00 00 00 # KMX_DWORD count
# keys
index(strNull,strA,2) # KMXPLUS_STR locale; // 'that'
# ----------------------------------------------------------------------------------------------------
# loca
# ----------------------------------------------------------------------------------------------------
block(loca) # struct COMP_KMXPLUS_LOCA {
6c 6f 63 61 # KMX_DWORD header.ident; // 0000 Section name - loca
sizeof(loca) # KMX_DWORD header.size; // 0004 Section length
01 00 00 00 # KMX_DWORD count; // 0008 number of locales
index(strNull,strLocale,2) # KMXPLUS_STR locale; // 000C+ locale string entry = 'mt'
# };
# ----------------------------------------------------------------------------------------------------
# meta
# ----------------------------------------------------------------------------------------------------
block(meta) # struct COMP_KMXPLUS_META {
6d 65 74 61 # KMX_DWORD header.ident; // 0000 Section name - meta
sizeof(meta) # KMX_DWORD header.size; // 0004 Section length
index(strNull,strAuthor,2) # KMXPLUS_STR author;
index(strNull,strConformsTo,2) # KMXPLUS_STR conform;
index(strNull,strLayout,2) # KMXPLUS_STR layout;
index(strNull,strName,2) # KMXPLUS_STR name;
index(strNull,strIndicator,2) # KMXPLUS_STR indicator;
index(strNull,strVersion,2) # KMXPLUS_STR version;
00 00 00 00 # KMX_DWORD settings;
# };
# ----------------------------------------------------------------------------------------------------
# strs
# ----------------------------------------------------------------------------------------------------
# TODO: it’s the whole strs table from basic. could pare this down.
#
block(strs) # struct COMP_KMXPLUS_STRS {
73 74 72 73 # KMX_DWORD header.ident; // 0000 Section name - strs
diff(strs,endstrs) # KMX_DWORD header.size; // 0004 Section length
index(strNull,endstrs,2) # KMX_DWORD count; // 0008 count of str entries
# };
# Next sections are string entries
# KMX_DWORD offset; // 0010+ offset from this blob
# KMX_DWORD length; // 0014+ str length (UTF-16LE units)
diff(strs,strNull) sizeof(strNull,2)
diff(strs,strVersion) sizeof(strVersion,2)
diff(strs,strConformsTo) sizeof(strConformsTo,2)
diff(strs,strName) sizeof(strName,2)
diff(strs,strFromSet) sizeof(strFromSet,2)
diff(strs,strUSet) sizeof(strUSet,2)
diff(strs,strAmarker) sizeof(strAmarker,2)
diff(strs,strSentinel0001r) sizeof(strSentinel0001r,2)
diff(strs,strElemTranFrom1) sizeof(strElemTranFrom1,2)
diff(strs,strElemTranFrom1a) sizeof(strElemTranFrom1a,2)
diff(strs,strElemTranFrom1b) sizeof(strElemTranFrom1b,2)
diff(strs,strA) sizeof(strA,2)
diff(strs,strSet) sizeof(strSet,2)
diff(strs,strSet2) sizeof(strSet2,2)
diff(strs,strTranTo) sizeof(strTranTo,2)
diff(strs,strElemBkspFrom2) sizeof(strElemBkspFrom2,2)
diff(strs,strGapReserved) sizeof(strGapReserved,2)
diff(strs,strHmaqtugha) sizeof(strHmaqtugha,2)
diff(strs,strLocale) sizeof(strLocale,2)
diff(strs,strLayout) sizeof(strLayout,2)
diff(strs,strAuthor) sizeof(strAuthor,2)
diff(strs,strThat) sizeof(strThat,2)
diff(strs,strUs) sizeof(strUs,2)
diff(strs,strVse) sizeof(strVse,2)
diff(strs,strVst) sizeof(strVst,2)
diff(strs,strVus) sizeof(strVus,2)
diff(strs,strKeys) sizeof(strKeys,2)
diff(strs,strIndicator) sizeof(strIndicator,2)
diff(strs,strSentinel0001) sizeof(strSentinel0001,2)
# String table -- block(x) is used to store the null u16char at end of each string
# without interfering with sizeof() calculation above
#str #00
block(strNull) block(x) 00 00 # the zero-length string
block(strVersion) 31 00 2e 00 30 00 2e 00 30 00 block(x) 00 00 # '1.0.0'
block(strConformsTo) 34 00 35 00 block(x) 00 00 # '45'
block(strName) 54 00 65 00 73 00 74 00 4b 00 62 00 64 00 block(x) 00 00 # 'TestKbd'
block(strFromSet) 5B 00 5C 00 75 00 31 00 41 00 37 00 35 00 2D 00 5C 00 75 00 31 00 41 00 37 00 39 00 5D 00 block(x) 00 00 # [\u1a75-\u1a79]
block(strUSet) 5b 00 61 00 62 00 63 00 5d 00 block(x) 00 00 # '[abc]'
block(strAmarker) 5C 00 6D 00 7B 00 61 00 7D 00 block(x) 00 00 # '\m{a}'
block(strSentinel0001r) 5c 00 75 00 66 00 66 00 66 00 66 00 5c 00 75 00 30 00 30 00 30 00 38 00 5C 00 75 00 30 00 30 00 30 00 31 00 block(x) 00 00 # UC_SENTINEL CODE_DEADKEY \u0001 (regex form)
block(strElemTranFrom1) 5E 00 block(x) 00 00 # '^'
block(strElemTranFrom1a) 5E 00 61 00 block(x) 00 00 # '^a'
block(strElemTranFrom1b) 5E 00 65 00 block(x) 00 00 # '^e'
block(strA) 61 00 block(x) 00 00 # 'a'
block(strSet) 61 00 20 00 62 00 20 00 63 00 block(x) 00 00 # 'a b c'
#str #0A
block(strSet2) 61 00 62 00 63 00 block(x) 00 00 # 'abc'
block(strTranTo) 61 00 02 03 block(x) 00 00 # 'â' (U+0061 U+0302)
block(strElemBkspFrom2) 65 00 block(x) 00 00 # 'e'
block(strGapReserved) 67 00 61 00 70 00 20 00 28 00 72 00 65 00 73 00 65 00 72 00 76 00 65 00 64 00 29 00 block(x) 00 00 # 'gap (reserved)'
block(strHmaqtugha) 68 00 6d 00 61 00 71 00 74 00 75 00 67 00 68 00 61 00 block(x) 00 00 # 'hmaqtugha'
#str #10
block(strLocale) 6d 00 74 00 block(x) 00 00 # 'mt'
block(strLayout) 71 00 77 00 65 00 72 00 74 00 79 00 block(x) 00 00 # 'qwerty'
block(strAuthor) 73 00 72 00 6c 00 32 00 39 00 35 00 block(x) 00 00 # 'srl295'
block(strThat) 74 00 68 00 61 00 74 00 block(x) 00 00 # 'that'
block(strUs) 75 00 73 00 block(x) 00 00 # 'us' (layout)
block(strVse) 76 00 73 00 65 00 block(x) 00 00 # 'vse'
block(strVst) 76 00 73 00 74 00 block(x) 00 00 # 'vst'
block(strVus) 76 00 75 00 73 00 block(x) 00 00 # 'vus'
#str #1a
block(strKeys) 90 17 b6 17 block(x) 00 00 # 'ថា'
# <reorder before="ᩫ" from="᩠᩵ᩅ" order="10 55 10" />
block(strIndicator) 3d d8 40 de block(x) 00 00 # '🙀'
block(strSentinel0001) FF FF 08 00 01 00 block(x) 00 00 # UC_SENTINEL CODE_DEADKEY U+0001
block(endstrs) # end of strs block
block(eof) # end of file

21
core/tests/unit/ldml/invalid-keyboards/ik_001_bad_key.xml Normal file
View file

@ -0,0 +1,21 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
@@keys: [K_BKQUOTE]
@@expect-error: This keyboard is invalid (U+FFFF), so it will fail to load
-->
<keyboard3 xmlns="https://schemas.unicode.org/cldr/45/keyboard3" locale="mt" conformsTo="45">
<info author="srl295" indicator="🙀" layout="qwerty" name="TestKbd-UTF32"/>
<keys>
<key id="a" output="\u{FFFF}" /> <!-- invalid -->
</keys>
<layers formId="us">
<layer id="base">
<row keys="a" /> <!-- number row -->
</layer>
</layers>
</keyboard3>

98
core/tests/unit/ldml/invalid-keyboards/ik_002_no_keys.txt Normal file
View file

@ -0,0 +1,98 @@
#
# Keyman is copyright (C) SIL International. MIT License.
#
# like basic.txt, but has a bad (illegal unicode) non-extended key
#
# For the format of this file, see “hextobin.ts”
#
# P.S. surprisingly, the Dart language highlighter in VSCode does a helpful job on this file.
block(kmxheader) # struct COMP_KEYBOARD {
4b 58 54 53 # KMX_DWORD dwIdentifier; // 0000 Keyman compiled keyboard id
00 11 00 00 # KMX_DWORD dwFileVersion; // 0004 Version of the file - Keyman 4.0 is 0x0400
00 00 00 00 # KMX_DWORD dwCheckSum; // 0008 deprecated in 16.0, always 0
00 00 00 00 # KMX_DWORD KeyboardID; // 000C as stored in HKEY_LOCAL_MACHINE//system//currentcontrolset//control//keyboard layouts
01 00 00 00 # KMX_DWORD IsRegistered; // 0010
00 00 00 00 # KMX_DWORD version; // 0014 keyboard version
sizeof(stores,12) # KMX_DWORD cxStoreArray; // 0018 in array entries
00 00 00 00 # KMX_DWORD cxGroupArray; // 001C in array entries
offset(stores) # KMX_DWORD dpStoreArray; // 0020 [LPSTORE] address of first item in store array
00 00 00 00 # KMX_DWORD dpGroupArray; // 0024 [LPGROUP] address of first item in group array
ff ff ff ff # KMX_DWORD StartGroup[2]; // 0028 index of starting groups [2 of them]
ff ff ff ff #
20 00 00 00 # KMX_DWORD dwFlags; // 0030 Flags for the keyboard file
00 00 00 00 # KMX_DWORD dwHotKey; // 0034 standard windows hotkey (hiword=shift/ctrl/alt stuff, loword=vkey)
00 00 00 00 # KMX_DWORD dpBitmapOffset; // 0038 offset of the bitmaps in the file
00 00 00 00 # KMX_DWORD dwBitmapSize; // 003C size in bytes of the bitmaps
# };
block(kmxplusinfo) # struct COMP_KEYBOARD_KMXPLUSINFO {
offset(sect) # KMX_DWORD dpKMXPlus; // 0040 offset of KMXPlus data from BOF, <sect> header is first
diff(sect,eof) # KMX_DWORD dwKMXPlusSize; // 0044 size in bytes of entire KMXPlus data
# };
block(stores) # struct COMP_STORE {
07 00 00 00 # KMX_DWORD dwSystemID; - TSS_NAME
offset(store_name_name) # KMX_DWORD dpName;
offset(store_name_string) # KMX_DWORD dpString;
# };
# excluding this so we don’t have compiled version changes
# 14 00 00 00 # KMX_DWORD dwSystemID; - TSS_COMPILEDVERSION
# offset(store_compiledversion_name) # KMX_DWORD dpName;
# offset(store_compiledversion_string) # KMX_DWORD dpString;
# };
24 00 00 00 # KMX_DWORD dwSystemID; - TSS_KEYBOARDVERSION
offset(store_keyboardversion_name) # KMX_DWORD dpName;
offset(store_keyboardversion_string) # KMX_DWORD dpString;
# };
26 00 00 00 # KMX_DWORD dwSystemID; - TSS_TARGETS
offset(store_targets_name) # KMX_DWORD dpName;
offset(store_targets_string) # KMX_DWORD dpString;
# };
block(store_name_name)
26 00 4e 00 41 00 4d 00 45 00 00 00 # '&NAME'
block(store_name_string)
54 00 65 00 73 00 74 00 4b 00 62 00 64 00 00 00 # 'TestKbd'
block(store_keyboardversion_name)
26 00 4b 00 45 00 59 00 42 00 4f 00 41 00 52 00
44 00 56 00 45 00 52 00 53 00 49 00 4f 00 4e 00
00 00 # '&KEYBOARDVERSION'
block(store_keyboardversion_string)
31 00 2e 00 30 00 2e 00 30 00 00 00 # '1.0.0'
block(store_targets_name)
26 00 54 00 41 00 52 00 47 00 45 00 54 00 53 00 00 00 # '&TARGETS'
block(store_targets_string)
64 00 65 00 73 00 6b 00 74 00 6f 00 70 00 00 00 # 'desktop'
block(sect) # struct COMP_KMXPLUS_SECT {
73 65 63 74 # KMX_DWORD header.ident; // 0000 Section name
diff(sect,endsect) # KMX_DWORD header.size; // 0004 Section length
diff(sect,eof) # KMX_DWORD total; // 0008 KMXPlus entire length
sizeof(sectitems,8) # KMX_DWORD count; // 000C number of section headers
# };
# Next sections are sect entries
# KMX_DWORD sect; // 0010+ Section identity
# KMX_DWORD offset; // 0014+ Section offset relative to dpKMXPlus of section
block(sectitems)
# no keys!
# 6b 65 79 73
# diff(sect,keys)
block(endsect)
block(eof) # end of file

13
core/tests/unit/ldml/invalid-keyboards/ik_002_no_keys.xml Normal file
View file

@ -0,0 +1,13 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
@@keys: [K_BKQUOTE]
@@expect-error: This keyboard has no keys, so it will fail to load
-->
<keyboard3 xmlns="https://schemas.unicode.org/cldr/45/keyboard3" locale="mt" conformsTo="45">
<info author="srl295" indicator="🙀" layout="qwerty" name="TestKbd-UTF32"/>
<!-- no keys! -->
</keyboard3>

4
core/tests/unit/ldml/invalid-keyboards/meson.build
View file

@ -5,7 +5,9 @@
#
invalid_tests = [
'ik_000_null_invalid'
'ik_000_null_invalid',
'ik_001_bad_key',
'ik_002_no_keys',
]
# Build all keyboards in output folder

192
core/tests/unit/ldml/kmx_plus.tests.cpp
View file

@ -1,4 +1,5 @@
#include <cstdint>
#include <gtest/gtest.h>
#include <test_assert.h>
#include "kmx/kmx_plus.h"
#include "kmx/kmx_xstring.h"
@ -9,12 +10,9 @@
// needed for streaming operators
#include "utfcodec.hpp"
using namespace km::core::kmx;
int test_COMP_KMXPLUS_KEYS_KEY() {
std::cout << "== " << __FUNCTION__ << std::endl;
TEST(KMXPlusTest, test_COMP_KMXPLUS_KEYS_KEY) {
COMP_KMXPLUS_KEYS_KEY e[2] = {
{
0x00000127, // to = U+0127 = 295
@ -55,13 +53,9 @@ int test_COMP_KMXPLUS_KEYS_KEY() {
test_assert_equal(es1.at(0), 0xD83D);
test_assert_equal(es1.at(1), 0xDE40);
test_assert(es1 == std::u16string(u"🙀"));
return EXIT_SUCCESS;
}
int test_ldml_vkeys() {
std::cout << "== " << __FUNCTION__ << std::endl;
TEST(KMXPlusTest, test_ldml_vkeys) {
km::core::ldml::vkeys vk;
#define ADD_VKEY(k, m) { \
@ -151,13 +145,9 @@ int test_ldml_vkeys() {
"K_E"), RCTRLFLAG|LALTFLAG, found), u"K_E-K_ALTFLAG|RCTRLFLAG");
test_assert_equal(vk.lookup(km::tests::get_vk(
"K_E"), RCTRLFLAG|RALTFLAG, found), u"K_E-K_ALTFLAG|RCTRLFLAG");
return EXIT_SUCCESS;
}
int test_uset() {
std::cout << "== " << __FUNCTION__ << std::endl;
TEST(KMXPlusTest, test_uset) {
const COMP_KMXPLUS_USET_RANGE r[] = {
{0x61, 0x7a}, // [a-z]
{0x127, 0x127} // [ħ]
@ -171,31 +161,173 @@ int test_uset() {
SimpleUSet uempty;
test_assert_equal(uempty.contains(0x62), false);
test_assert_equal(uempty.contains(0x127), false);
return EXIT_SUCCESS;
}
int
main(int argc, const char *argv[]) {
int rc = EXIT_SUCCESS;
/** tests of the COMP_KMXPLUS_STRS::valid_string() */
TEST(KMXPlusTest, COMP_KMXPLUS_STRS_valid_string) {
// one simple case
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(
u"Hello", 5));
if (test_COMP_KMXPLUS_KEYS_KEY() != EXIT_SUCCESS) {
rc = EXIT_FAILURE;
{
// unpaired low surrogate
km_core_cu const s[] = {0xDC01, 0x0020};
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 1)); // at end of str
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 2)); // not followed by trailing surrogate
}
if (test_ldml_vkeys() != EXIT_SUCCESS) {
rc = EXIT_FAILURE;
{
// unpaired high surrogate
km_core_cu const s[] = { 0xD801, 0x0020};
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 1));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 2));
}
if (test_uset() != EXIT_SUCCESS) {
rc = EXIT_FAILURE;
{
// valid str
km_core_cu const s[] = { 0xD83D, 0xDE40, 0x0020}; // 🙀
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s, 2));
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s, 3));
}
if (rc == EXIT_FAILURE) {
std::cout << "== FAILURE" << std::endl;
} else {
std::cout << "== PASS" << std::endl;
{
// invalid str - noncharacter U+2FFFF
km_core_cu const s[] = { u'a', u'b', 0xD87F, 0xDFFF, u'c'};
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 5));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 4));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 3));
}
return rc;
{
// valid str with a marker - 'ab\m{x}c'
km_core_cu const s[] = {u'a', u'b', 0xFFFF, 0x0008, 0x0001, u'c'};
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s, 6)); // whole thing
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s, 5)); // whole thing minus 'c'
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s+2, 3)); // just the marker
// if we slice the marker, invalid
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 4));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 3));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s+2, 2));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s+2, 1));
}
{
// valid str with a marker - 'ab\m{x}c'
km_core_cu const s[] = {u'a', u'b', 0xFFFF, 0x0008, 0x0001, u'c'};
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s, 6)); // whole thing
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s, 5)); // whole thing minus 'c'
EXPECT_TRUE(COMP_KMXPLUS_STRS::valid_string(s + 2, 3)); // just the marker
// if we slice the marker, invalid
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 4));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 3));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 2));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 1));
}
{
// invalid str - incorrect char 0009 instead of 0008
km_core_cu const s[] = {u'a', u'b', 0xFFFF, 0x0009, 0x0001, u'c'};
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 6)); // whole thing
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 5)); // whole thing minus 'c'
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 3)); // just the marker
// if we slice the marker, invalid
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 4));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 3));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 2));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 1));
}
{
// invalid str - out of range merker 0xDDDD exceeds LDML_MARKER_MAX_INDEX
km_core_cu const s[] = {u'a', u'b', 0xFFFF, 0x0008, 0xDDDD, u'c'};
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 6)); // whole thing
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 5)); // whole thing minus 'c'
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 3)); // just the marker
// if we slice the marker, invalid
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 4));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 3));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 2));
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s + 2, 1));
}
{
// noncharacter FDD4
km_core_cu const s[] = {0xFDD4};
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 1));
}
{
// FFFE nonchar (mismatched BOM)
km_core_cu const s[] = {0xFFFE};
EXPECT_FALSE(COMP_KMXPLUS_STRS::valid_string(s, 1));
}
}
TEST(KMXPlusTest, COMP_KMXPLUS_STRS_withGoodStrings) {
const auto len = 10;
KMX_DWORD mystrs[len] = {
COMP_KMXPLUS_STRS::IDENT,
len * 4, // size
// ---
0x0002, // count
// #0
28, // offset
0x0000, // size
// #1
32, // offset
0x0001, // size
0x00000000, // null
// data @ +7
0x0041, // 'a'
0x0000, // null
};
const COMP_KMXPLUS_HEADER *header = reinterpret_cast<const COMP_KMXPLUS_HEADER *>(mystrs);
ASSERT_TRUE(header->valid(mystrs[1]));
const COMP_KMXPLUS_STRS *strs = reinterpret_cast<const COMP_KMXPLUS_STRS *>(mystrs);
ASSERT_TRUE(strs->valid(mystrs[1]));
KMX_DWORD_unaligned mycount = strs->count;
ASSERT_EQ(mycount, 2);
ASSERT_EQ(strs->get(0), u"");
ASSERT_EQ(strs->get(1), u"A");
}
TEST(KMXPlusTest, COMP_KMXPLUS_STRS_withBadStrings) {
const auto len = 10;
KMX_DWORD mystrs[len] = {
COMP_KMXPLUS_STRS::IDENT,
len * 4, // size
// ---
0x0002, // count
// #0
28, // offset
0x0000, // size
// #1
32, // offset
0x0001, // size
0x00000000, // null
// data @ +7
0xFFFF, // illegal nonchar at end (not a valid marker sequence)
0x0000, // null
};
const COMP_KMXPLUS_HEADER *header = reinterpret_cast<const COMP_KMXPLUS_HEADER *>(mystrs);
ASSERT_TRUE(header->valid(mystrs[1]));
const COMP_KMXPLUS_STRS *strs = reinterpret_cast<const COMP_KMXPLUS_STRS *>(mystrs);
ASSERT_FALSE(strs->valid(mystrs[1]));
}
GTEST_API_ int
main(int argc, char **argv) {
testing::InitGoogleTest(&argc, argv);
return RUN_ALL_TESTS();
}

6
core/tests/unit/ldml/meson.build
View file

@ -19,6 +19,10 @@ if node.found()
subdir('invalid-keyboards')
endif
gtest = subproject('gtest')
gtest_dep = gtest.get_variable('gtest_dep')
gmock_dep = gtest.get_variable('gmock_dep')
# Build ldml test executable
keyboard_build_path = meson.current_build_dir() / 'keyboards'
@ -87,7 +91,7 @@ e = executable('kmx_plus_tests', 'kmx_plus.tests.cpp',
cpp_args: defns + warns,
include_directories: [inc, libsrc, '../../../../developer/src/ext/json'],
link_args: links + tests_flags,
dependencies: [icu_uc, icu_i18n],
dependencies: [icu_uc, icu_i18n, gtest_dep, gmock_dep],
objects: lib.extract_all_objects(recursive: false))
test('kmx_plus_tests', e, suite: 'ldml')

6
developer/src/common/include/kmn_compiler_errors.h
View file

@ -275,8 +275,10 @@ namespace KmnCompilerMessages {
ERROR_ContextExCannotReferenceIf = SevError | 0x0B1,
ERROR_ContextExCannotReferenceNul = SevError | 0x0B2,
FATAL_BufferOverflow = SevFatal | 0x0C0
// FATAL_Break = SevFatal | 0x0C1, unused
ERROR_TextBeforeOrAfterNulInOutput = SevError | 0x0B3,
FATAL_BufferOverflow = SevFatal | 0x0C0
// FATAL_Break = SevFatal | 0x0C1, unused
};
}

2
developer/src/common/web/utils/package.json
View file

@ -12,7 +12,7 @@
"@keymanapp/common-types": "*",
"@sentry/node": "^7.57.0",
"eventemitter3": "^5.0.0",
"fast-xml-parser": "^4.5.0",
"fast-xml-parser": "^5.0.9",
"path-browserify": "^1.0.1",
"restructure": "^3.0.1",
"sax": ">=0.6.0",

106
developer/src/common/web/utils/src/xml-utils.ts
View file

@ -6,7 +6,7 @@
* Abstraction for XML reading and writing
*/
import { XMLParser, XMLBuilder } from 'fast-xml-parser';
import { XMLParser, XMLBuilder, XmlBuilderOptions, X2jOptions } from 'fast-xml-parser';
export type KeymanXMLType =
'keyboard3' // LDML <keyboard3>
@ -17,102 +17,87 @@ export type KeymanXMLType =
;
/** Bag of options, maximally one for each KeymanXMLType */
type KeymanXMLOptionsBag = {
[key in KeymanXMLType]?: any
type KeymanXMLParserOptionsBag = {
[key in KeymanXMLType]?: X2jOptions;
};
const PARSER_COMMON_OPTIONS: X2jOptions = {
attributeNamePrefix: '$', // causes remapping into $: { … } objects
htmlEntities: true,
ignoreAttributes: false,
ignorePiTags: true,
numberParseOptions: { // TODO: query is this option really necessary?
eNotation: null,
hex: null,
leadingZeros: null,
skipLike: /(?:)/, // parse numbers as strings
},
textNodeName: '_',
};
/** map of options for the XML parser */
const PARSER_OPTIONS: KeymanXMLOptionsBag = {
const PARSER_OPTIONS: KeymanXMLParserOptionsBag = {
'keyboard3': {
ignoreAttributes: false, // We'd like attributes, please
attributeNamePrefix: '@__', // We'll use this to convert attributes to strings and subobjects to arrays, when empty.
trimValues: false, // preserve spaces, but:
htmlEntities: true,
tagValueProcessor: (tagName: string, tagValue: string /*, jPath, hasAttributes, isLeafNode*/) => {
ignoreAttributes: false, // We'd like attributes, please
tagValueProcessor: (_tagName: string, tagValue: string /*, jPath, hasAttributes, isLeafNode*/) => {
// since trimValues: false, we need to zap any element values that would be trimmed.
// currently, the LDML spec doesn't have any element values, but this
// future-proofs us a little in that element values are allowed, just trimmed.
// if we do need elements in the future, we'd check the preserve-space attribute here.
return tagValue?.trim();
},
trimValues: false, // preserve spaces, but see tagValueProcessor
},
'keyboardTest3': {
ignorePiTags: true,
attributeNamePrefix: '', // avoid @_
htmlEntities: true,
ignoreAttributes: false, // We'd like attributes, please
attributeNamePrefix: '', // avoid @_
ignorePiTags: true,
preserveOrder: true, // Gives us a 'special' format
},
'kps': {
ignorePiTags: true,
ignoreAttributes: false,
htmlEntities: true,
attributeNamePrefix: '$', // causes remapping into $: { … } objects
textNodeName: '_',
numberParseOptions: {
skipLike: /(?:)/, // parse numbers as strings
hex: null,
leadingZeros: null,
eNotation: null,
},
...PARSER_COMMON_OPTIONS,
},
'kpj': {
ignorePiTags: true,
textNodeName: '_',
htmlEntities: true,
ignoreAttributes: false, // We'd like attributes, please
...PARSER_COMMON_OPTIONS,
attributeNamePrefix: '', // to avoid '@_' prefixes
numberParseOptions: {
skipLike: /(?:)/, // parse numbers as strings
hex: null,
leadingZeros: null,
eNotation: null,
},
},
'kvks': {
ignorePiTags: true,
textNodeName: '_',
htmlEntities: true,
ignoreAttributes: false, // We'd like attributes, please
attributeNamePrefix: '$', // causes remapping into $: { … } objects
numberParseOptions: {
skipLike: /(?:)/, // parse numbers as strings
hex: null,
leadingZeros: null,
eNotation: null,
},
trimValues: false, // preserve spaces, but:
tagValueProcessor: (tagName: string, tagValue: string, jPath: string, hasAttributes: string, isLeafNode: boolean) : string | undefined => {
...PARSER_COMMON_OPTIONS,
tagValueProcessor: (_tagName: string, tagValue: string, _jPath: string, _hasAttributes: boolean, isLeafNode: boolean) : string | undefined => {
if (!isLeafNode) {
return tagValue?.trim(); // trimmed value
} else {
return null; // no change to leaf nodes
}
},
trimValues: false, // preserve spaces
},
};
const GENERATOR_OPTIONS: KeymanXMLOptionsBag = {
type KeymanXMLGeneratorOptionsBag = {
[key in KeymanXMLType]?: XmlBuilderOptions
};
const GENERATOR_COMMON_OPTIONS: XmlBuilderOptions = {
attributeNamePrefix: '$',
ignoreAttributes: false,
format: true,
textNodeName: '_',
suppressEmptyNode: true,
};
const GENERATOR_OPTIONS: KeymanXMLGeneratorOptionsBag = {
kvks: {
attributeNamePrefix: '$',
ignoreAttributes: false,
format: true,
textNodeName: '_',
suppressEmptyNode: true,
...GENERATOR_COMMON_OPTIONS,
},
kpj: {
attributeNamePrefix: '$',
ignoreAttributes: false,
format: true,
textNodeName: '_',
suppressEmptyNode: true,
...GENERATOR_COMMON_OPTIONS,
},
kps: {
attributeNamePrefix: '$',
ignoreAttributes: false,
format: true,
textNodeName: '_',
suppressEmptyNode: true,
...GENERATOR_COMMON_OPTIONS,
},
keyboard3: {
attributeNamePrefix: '$',
@ -270,9 +255,6 @@ export class KeymanXMLReader {
throw Error(`Internal error: unhandled XML type ${this.type}`);
}
options = Object.assign({}, options); // TODO: xml2js likes to mutate the options here. Shallow clone the object.
if (options.emptyTag) {
options.emptyTag = {}; // TODO: xml2js likes to mutate the options here. Reset it.
}
return new XMLParser(options);
}
}

16
developer/src/common/web/utils/test/fixtures/kvks/hex_escape.kvks vendored Normal file
View file

@ -0,0 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<visualkeyboard>
<header>
<version>10.0</version>
<kbdname>hex_escape</kbdname>
<flags/>
</header>
<encoding name="unicode" fontname="arial" fontsize="-12">
<layer shift="">
<key vkey="K_1">&#x1234;</key>
<key vkey="K_2">&#256;</key>
<key vkey="K_3">&#65536;</key>
<key vkey="K_4">&#x12345;</key>
</layer>
</encoding>
</visualkeyboard>

10
developer/src/common/web/utils/test/fixtures/ldml-keyboard/numeric-id.xml vendored Normal file
View file

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="UTF-8"?>
<keyboard3 xmlns="https://schemas.unicode.org/cldr/45/keyboard3" locale="mt" conformsTo="46">
<info author="srl295" indicator="🙀" layout="qwerty" name="TestKbd"/>
<keys>
<key id="1" output="ħ" />
<key id="2" output="ថា" />
</keys>
</keyboard3>

2
developer/src/common/web/utils/test/fixtures/ldml-keyboard/test-fr.xml vendored
View file

@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE keyboardTest3 SYSTEM "../../../../../resources/standards-data/ldml-keyboards/46/dtd/ldmlKeyboardTest3.dtd">
<!DOCTYPE keyboardTest3 SYSTEM "../../../../../../../../resources/standards-data/ldml-keyboards/46/dtd/ldmlKeyboardTest3.dtd">
<keyboardTest3 conformsTo="techpreview">
<!--
Read by:

15
developer/src/common/web/utils/test/kvks/kvks-file.tests.ts
View file

@ -40,6 +40,21 @@ describe('kvks-file-reader', function() {
const reader = new KvksFileReader();
assert.throws(() => reader.read(input), 'File appears to be a binary .kvk file');
});
it('should read non-bmp hex escapes correctly', function() {
const path = makePathToFixture('kvks', 'hex_escape.kvks');
const input = fs.readFileSync(path);
const reader = new KvksFileReader();
const kvks = reader.read(input);
const invalidVkeys: string[] = [];
const vk = reader.transform(kvks, invalidVkeys);
assert.isEmpty(invalidVkeys);
assert.equal(vk.keys[0].text, '\u{1234}');
assert.equal(vk.keys[1].text, '\u{100}');
assert.equal(vk.keys[2].text, String.fromCodePoint(0x10000));
assert.equal(vk.keys[3].text, String.fromCodePoint(0x12345));
});
});
describe('kvks-file-writer', function() {

2
developer/src/common/web/utils/test/kmx/ldml-keyboard-testdata-reader.tests.ts → developer/src/common/web/utils/test/ldml/ldml-keyboard-testdata-reader.tests.ts
View file

@ -2,7 +2,7 @@ import { constants } from '@keymanapp/ldml-keyboard-constants';
import { assert } from 'chai';
import 'mocha';
import { testTestdataReaderCases } from '../helpers/reader-callback-test.js';
import { LKTAnyAction } from '../../src/types/ldml-keyboard/ldml-keyboard-testdata-xml.js';
import { LKTAnyAction } from '../types/ldml-keyboard/ldml-keyboard-testdata-xml.js';
describe('ldml keyboard xml reader tests', function () {
this.slow(500); // 0.5 sec -- json schema validation takes a while

25
developer/src/common/web/utils/test/kmx/ldml-keyboard-xml-reader.tests.ts → developer/src/common/web/utils/test/ldml/ldml-keyboard-xml-reader.tests.ts
View file

@ -1,9 +1,9 @@
import { LKKey, ImportStatus } from '../../src/types/ldml-keyboard/ldml-keyboard-xml.js';
import 'mocha';
import {assert} from 'chai';
import { CommonTypesMessages } from '../../src/common-messages.js';
import { testReaderCases } from '../helpers/reader-callback-test.js';
import { Constants } from '@keymanapp/common-types';
import { CommonTypesMessages } from '../../src/common-messages.js';
import { LKKey, ImportStatus } from '../../src/types/ldml-keyboard/ldml-keyboard-xml.js';
import { testReaderCases } from '../helpers/reader-callback-test.js';
import CLDRScanToVkey = Constants.CLDRScanToVkey;
import CLDRScanToKeyMap = Constants.CLDRScanToKeyMap;
@ -201,6 +201,25 @@ describe('ldml keyboard xml reader tests', function () {
}),
],
},
{
subpath: 'numeric-id.xml',
callback: (data, source, subpath, callbacks) => {
assert.ok(source?.keyboard3?.keys);
const k = pluckKeysFromKeybag(source?.keyboard3?.keys.key, ['1', '2']);
assert.sameDeepOrderedMembers(k.map((entry) => {
// Drop the Symbol members from the returned keys; assertions may expect their presence.
return {
id: entry.id,
output: entry.output
};
}), [
{id: '1', output: '1'}, //default import
{id: '2', output: '2'}, //default import
{id: '1', output: 'ħ'}, //override
{id: '2', output: 'ថា'}, //override
]);
},
},
]);
});

77
developer/src/kmc-kmn/src/compiler/kmn-compiler-messages.ts
View file

@ -227,40 +227,40 @@ export class KmnCompilerMessages {
static ERROR_InvalidToken = SevError | 0x00A;
static Error_InvalidToken = () => m(this.ERROR_InvalidToken, `Invalid token found`);
// kmcmplib: static Error_InvalidBegin = () => m(this.ERROR_InvalidBegin, `Invalid 'begin' command`);
// kmcmplib: static Error_InvalidBegin = () => m(this.ERROR_InvalidBegin, `Invalid 'begin' statement`);
static ERROR_InvalidBegin = SevError | 0x00B;
static Error_InvalidBegin = () => mw(this.ERROR_InvalidBegin,
`A "begin unicode" statement is required to compile a KeymanWeb keyboard`);
static ERROR_InvalidName = SevError | 0x00C;
static Error_InvalidName = () => m(this.ERROR_InvalidName, `Invalid 'name' command`);
static Error_InvalidName = () => m(this.ERROR_InvalidName, `Invalid 'name' statement`);
static ERROR_InvalidVersion = SevError | 0x00D;
static Error_InvalidVersion = () => m(this.ERROR_InvalidVersion, `Invalid 'version' command`);
static Error_InvalidVersion = () => m(this.ERROR_InvalidVersion, `Invalid 'version' statement`);
static ERROR_InvalidLanguageLine = SevError | 0x00E;
static Error_InvalidLanguageLine = () => m(this.ERROR_InvalidLanguageLine, `Invalid 'language' command`);
static Error_InvalidLanguageLine = () => m(this.ERROR_InvalidLanguageLine, `Invalid 'language' statement`);
static ERROR_LayoutButNoLanguage = SevError | 0x00F;
static Error_LayoutButNoLanguage = () => m(this.ERROR_LayoutButNoLanguage, `Layout command found but no language command`);
static Error_LayoutButNoLanguage = () => m(this.ERROR_LayoutButNoLanguage, `Layout statement found but no language statement`);
static ERROR_InvalidLayoutLine = SevError | 0x010;
static Error_InvalidLayoutLine = () => m(this.ERROR_InvalidLayoutLine, `Invalid 'layout' command`);
static Error_InvalidLayoutLine = () => m(this.ERROR_InvalidLayoutLine, `Invalid 'layout' statement`);
static ERROR_NoVersionLine = SevError | 0x011;
static Error_NoVersionLine = () => m(this.ERROR_NoVersionLine, `No version line found for file`);
static ERROR_InvalidGroupLine = SevError | 0x012;
static Error_InvalidGroupLine = () => m(this.ERROR_InvalidGroupLine, `Invalid 'group' command`);
static Error_InvalidGroupLine = () => m(this.ERROR_InvalidGroupLine, `Invalid 'group' statement`);
static ERROR_InvalidStoreLine = SevError | 0x013;
static Error_InvalidStoreLine = () => m(this.ERROR_InvalidStoreLine, `Invalid 'store' command`);
static Error_InvalidStoreLine = () => m(this.ERROR_InvalidStoreLine, `Invalid 'store' statement`);
static ERROR_InvalidCodeInKeyPartOfRule = SevError | 0x014;
static Error_InvalidCodeInKeyPartOfRule = () => m(this.ERROR_InvalidCodeInKeyPartOfRule, `Invalid command or code found in key part of rule`);
static Error_InvalidCodeInKeyPartOfRule = () => m(this.ERROR_InvalidCodeInKeyPartOfRule, `Invalid statement or code found in key part of rule`);
static ERROR_InvalidDeadkey = SevError | 0x015;
static Error_InvalidDeadkey = () => m(this.ERROR_InvalidDeadkey, `Invalid 'deadkey' or 'dk' command`);
static Error_InvalidDeadkey = () => m(this.ERROR_InvalidDeadkey, `Invalid 'deadkey' or 'dk' statement`);
static ERROR_InvalidValue = SevError | 0x016;
static Error_InvalidValue = () => m(this.ERROR_InvalidValue, `Invalid value in extended string`);
@ -269,7 +269,7 @@ export class KmnCompilerMessages {
static Error_ZeroLengthString = () => m(this.ERROR_ZeroLengthString, `A string of zero characters was found`);
static ERROR_TooManyIndexToKeyRefs = SevError | 0x018;
static Error_TooManyIndexToKeyRefs = () => m(this.ERROR_TooManyIndexToKeyRefs, `Too many index commands refering to key string`);
static Error_TooManyIndexToKeyRefs = () => m(this.ERROR_TooManyIndexToKeyRefs, `Too many index statements refering to key string`);
static ERROR_UnterminatedString = SevError | 0x019;
static Error_UnterminatedString = () => m(this.ERROR_UnterminatedString, `Unterminated string in line`);
@ -278,34 +278,34 @@ export class KmnCompilerMessages {
// static Error_StringInVirtualKeySection = () => m(this.ERROR_StringInVirtualKeySection, `extend string illegal in virtual key section`);
// static ERROR_AnyInVirtualKeySection = SevError | 0x01B; // no longer used, see #12612
// static Error_AnyInVirtualKeySection = () => m(this.ERROR_AnyInVirtualKeySection, `'any' command is illegal in virtual key section`);
// static Error_AnyInVirtualKeySection = () => m(this.ERROR_AnyInVirtualKeySection, `'any' statement is illegal in virtual key section`);
static ERROR_InvalidAny = SevError | 0x01C;
static Error_InvalidAny = () => m(this.ERROR_InvalidAny, `Invalid 'any' command`);
static Error_InvalidAny = () => m(this.ERROR_InvalidAny, `Invalid 'any' statement`);
static ERROR_StoreDoesNotExist = SevError | 0x01D;
static Error_StoreDoesNotExist = () => m(this.ERROR_StoreDoesNotExist, `Store referenced does not exist`);
// static ERROR_BeepInVirtualKeySection = SevError | 0x01E; // no longer used, see #12612
// static Error_BeepInVirtualKeySection = () => m(this.ERROR_BeepInVirtualKeySection, `'beep' command is illegal in virtual key section`);
// static Error_BeepInVirtualKeySection = () => m(this.ERROR_BeepInVirtualKeySection, `'beep' statement is illegal in virtual key section`);
// static ERROR_IndexInVirtualKeySection = SevError | 0x01F; // no longer used, see #12612
// static Error_IndexInVirtualKeySection = () => m(this.ERROR_IndexInVirtualKeySection, `'index' command is illegal in virtual key section`);
// static Error_IndexInVirtualKeySection = () => m(this.ERROR_IndexInVirtualKeySection, `'index' statement is illegal in virtual key section`);
static ERROR_InvalidIndex = SevError | 0x020;
static Error_InvalidIndex = () => m(this.ERROR_InvalidIndex, `Invalid 'index' command`);
static Error_InvalidIndex = () => m(this.ERROR_InvalidIndex, `Invalid 'index' statement`);
// static ERROR_OutsInVirtualKeySection = SevError | 0x021; // no longer used, see #12612
// static Error_OutsInVirtualKeySection = () => m(this.ERROR_OutsInVirtualKeySection, `'outs' command is illegal in virtual key section`);
// static Error_OutsInVirtualKeySection = () => m(this.ERROR_OutsInVirtualKeySection, `'outs' statement is illegal in virtual key section`);
static ERROR_InvalidOuts = SevError | 0x022;
static Error_InvalidOuts = () => m(this.ERROR_InvalidOuts, `Invalid 'outs' command`);
static Error_InvalidOuts = () => m(this.ERROR_InvalidOuts, `Invalid 'outs' statement`);
// static ERROR_ContextInVirtualKeySection = SevError | 0x024; // no longer used, see #12612
// static Error_ContextInVirtualKeySection = () => m(this.ERROR_ContextInVirtualKeySection, `'context' command is illegal in virtual key section`);
// static Error_ContextInVirtualKeySection = () => m(this.ERROR_ContextInVirtualKeySection, `'context' statement is illegal in virtual key section`);
static ERROR_InvalidUse = SevError | 0x025;
static Error_InvalidUse = () => m(this.ERROR_InvalidUse, `Invalid 'use' command`);
static Error_InvalidUse = () => m(this.ERROR_InvalidUse, `Invalid 'use' statement`);
static ERROR_GroupDoesNotExist = SevError | 0x026;
static Error_GroupDoesNotExist = () => m(this.ERROR_GroupDoesNotExist, `Group does not exist`);
@ -314,7 +314,7 @@ export class KmnCompilerMessages {
static Error_VirtualKeyNotAllowedHere = () => m(this.ERROR_VirtualKeyNotAllowedHere, `Virtual key is not allowed here`);
static ERROR_InvalidSwitch = SevError | 0x028;
static Error_InvalidSwitch = () => m(this.ERROR_InvalidSwitch, `Invalid 'switch' command`);
static Error_InvalidSwitch = () => m(this.ERROR_InvalidSwitch, `Invalid 'switch' statement`);
static ERROR_NoTokensFound = SevError | 0x029;
static Error_NoTokensFound = () => m(this.ERROR_NoTokensFound, `No tokens found in line`);
@ -326,19 +326,19 @@ export class KmnCompilerMessages {
static Error_LineTooLong = () => m(this.ERROR_LineTooLong, `Line too long`);
static ERROR_InvalidCopyright = SevError | 0x02C;
static Error_InvalidCopyright = () => m(this.ERROR_InvalidCopyright, `Invalid 'copyright' command`);
static Error_InvalidCopyright = () => m(this.ERROR_InvalidCopyright, `Invalid 'copyright' statement`);
static ERROR_CodeInvalidInThisSection = SevError | 0x02D;
static Error_CodeInvalidInThisSection = () => m(this.ERROR_CodeInvalidInThisSection, `This line is invalid in this section of the file`);
static ERROR_InvalidMessage = SevError | 0x02E;
static Error_InvalidMessage = () => m(this.ERROR_InvalidMessage, `Invalid 'message' command`);
static Error_InvalidMessage = () => m(this.ERROR_InvalidMessage, `Invalid 'message' statement`);
static ERROR_InvalidLanguageName = SevError | 0x02F;
static Error_InvalidLanguageName = () => m(this.ERROR_InvalidLanguageName, `Invalid 'languagename' command`);
static Error_InvalidLanguageName = () => m(this.ERROR_InvalidLanguageName, `Invalid 'languagename' statement`);
static ERROR_InvalidBitmapLine = SevError | 0x030;
static Error_InvalidBitmapLine = () => m(this.ERROR_InvalidBitmapLine, `Invalid 'bitmaps' command`);
static Error_InvalidBitmapLine = () => m(this.ERROR_InvalidBitmapLine, `Invalid 'bitmaps' statement`);
static ERROR_CannotReadBitmapFile = SevError | 0x031;
static Error_CannotReadBitmapFile = () => m(this.ERROR_CannotReadBitmapFile, `Cannot open the bitmap or icon file for reading`);
@ -353,13 +353,13 @@ export class KmnCompilerMessages {
static Error_InvalidCharacter = () => m(this.ERROR_InvalidCharacter, `A character was found that is outside the valid Unicode range (U+0000 - U+10FFFF)`);
static ERROR_InvalidCall = SevError | 0x035;
static Error_InvalidCall = () => m(this.ERROR_InvalidCall, `The 'call' command is invalid`);
static Error_InvalidCall = () => m(this.ERROR_InvalidCall, `The 'call' statement is invalid`);
// static ERROR_CallInVirtualKeySection = SevError | 0x036; // no longer used, see #12612
// static Error_CallInVirtualKeySection = () => m(this.ERROR_CallInVirtualKeySection, `'call' command is illegal in virtual key section`);
// static Error_CallInVirtualKeySection = () => m(this.ERROR_CallInVirtualKeySection, `'call' statement is illegal in virtual key section`);
static ERROR_CodeInvalidInKeyStore = SevError | 0x037;
static Error_CodeInvalidInKeyStore = () => m(this.ERROR_CodeInvalidInKeyStore, `The command is invalid inside a store that is used in a key part of the rule`);
static Error_CodeInvalidInKeyStore = () => m(this.ERROR_CodeInvalidInKeyStore, `The statement is invalid inside a store that is used in a key part of the rule`);
static ERROR_CannotLoadIncludeFile = SevError | 0x038;
static Error_CannotLoadIncludeFile = () => m(this.ERROR_CannotLoadIncludeFile, `Cannot load the included file: it is either invalid or does not exist`);
@ -747,11 +747,30 @@ export class KmnCompilerMessages {
);
static ERROR_ContextExCannotReferenceNul = SevError | 0x0B2;
static Error_ContextExCannotReferenceNul = () => m(
static Error_ContextExCannotReferenceNul = () => m(
this.ERROR_ContextExCannotReferenceNul,
`The offset in context() points to a non-character nul statement in the context`,
);
static ERROR_TextBeforeOrAfterNulInOutput = SevError | 0x0B3;
static Error_TextBeforeOrAfterNulInOutput = () => m(
this.ERROR_TextBeforeOrAfterNulInOutput,
`A rule with a nul statement in the output cannot also output text or text-emitting statements`,
`If a rule has [\`nul\`](https://help.keyman.com/developer/language/reference/nul)
in the output, it means that it deletes any context referenced on the left-hand
side of the rule, and has no other output, so it does not make sense to have any
content apart from the \`nul\` statement in the output.
The following statements can still be used with \`nul\` in the output:
* [\`beep()\`](https://help.keyman.com/developer/language/reference/beep)
* [\`call()\`](https://help.keyman.com/developer/language/reference/call)
* [\`reset()\`](https://help.keyman.com/developer/language/reference/reset)
* [\`return\`](https://help.keyman.com/developer/language/reference/return)
* [\`save()\`](https://help.keyman.com/developer/language/reference/save)
* [\`set()\`](https://help.keyman.com/developer/language/reference/set)
* [\`use()\`](https://help.keyman.com/developer/language/reference/use)`
);
static FATAL_BufferOverflow = SevFatal | 0x0C0;
static Fatal_BufferOverflow = () => m(this.FATAL_BufferOverflow, `The compiler memory buffer overflowed`);

9
developer/src/kmc-kmn/test/fixtures/invalid-keyboards/error_text_before_or_after_nul_in_output_after.kmn vendored Normal file
View file

@ -0,0 +1,9 @@
store(&NAME) 'ERROR_TextBeforeOrAfterNulInOutput'
store(&VERSION) '9.0'
begin Unicode > use(main)
group(main) using keys
c ERROR_TextBeforeOrAfterNulInOutput after nul
+ 'a' > nul 'a'

9
developer/src/kmc-kmn/test/fixtures/invalid-keyboards/error_text_before_or_after_nul_in_output_before.kmn vendored Normal file
View file

@ -0,0 +1,9 @@
store(&NAME) 'ERROR_TextBeforeOrAfterNulInOutput'
store(&VERSION) '9.0'
begin Unicode > use(main)
group(main) using keys
c ERROR_TextBeforeOrAfterNulInOutput before nul
+ 'b' > 'b' nul

15
developer/src/kmc-kmn/test/messages.tests.ts → developer/src/kmc-kmn/test/kmn-compiler-messages.tests.ts
View file

@ -1,10 +1,10 @@
import 'mocha';
import { assert } from 'chai';
import { KmnCompilerMessageRanges, KmnCompilerMessages } from '../src/compiler/kmn-compiler-messages.js';
import { TestCompilerCallbacks, verifyCompilerMessagesObject } from '@keymanapp/developer-test-helpers';
import { makePathToFixture } from './helpers/index.js';
import { KmnCompiler } from '../src/main.js';
import { CompilerErrorMask, CompilerErrorNamespace } from '@keymanapp/developer-utils';
import { TestCompilerCallbacks, verifyCompilerMessagesObject } from '@keymanapp/developer-test-helpers';
import { KmnCompiler } from '../src/main.js';
import { KmnCompilerMessageRanges, KmnCompilerMessages } from '../src/compiler/kmn-compiler-messages.js';
import { makePathToFixture } from './helpers/index.js';
describe('KmnCompilerMessages', function () {
const callbacks = new TestCompilerCallbacks();
@ -216,4 +216,11 @@ describe('KmnCompilerMessages', function () {
await testForMessage(this, ['invalid-keyboards', 'error_contextex_cannot_reference_nul.kmn'], KmnCompilerMessages.ERROR_ContextExCannotReferenceNul);
});
// ERROR_TextBeforeOrAfterNulInOutput
it('should generate ERROR_TextBeforeOrAfterNulInOutput if text before or after nul in output', async function() {
await testForMessage(this, ['invalid-keyboards', 'error_text_before_or_after_nul_in_output_before.kmn'], KmnCompilerMessages.ERROR_TextBeforeOrAfterNulInOutput);
await testForMessage(this, ['invalid-keyboards', 'error_text_before_or_after_nul_in_output_after.kmn'], KmnCompilerMessages.ERROR_TextBeforeOrAfterNulInOutput);
});
});

2
developer/src/kmc-kmn/test/kmw/kmw-messages.tests.ts
View file

@ -88,7 +88,7 @@ describe('KmwCompilerMessages', function () {
// ERROR_NotSupportedInKeymanWebOutput
it('should generate ERROR_NotSupportedInKeymanWebOutput if the command is not supported in output for KeymanWeb', async function() {
it('should generate ERROR_NotSupportedInKeymanWebOutput if the statement is not supported in output for KeymanWeb', async function() {
await testForMessage(this, ['kmw', 'error_not_supported_in_keyman_web_output.kmn'], KmwCompilerMessages.ERROR_NotSupportedInKeymanWebOutput);
});

7
developer/src/kmc-ldml/build.sh
View file

@ -75,6 +75,11 @@ function do_build_fixtures() {
# Generate a binary file from basic.txt for comparison purposes
node ../../../common/tools/hextobin/build/hextobin.js ./test/fixtures/basic.txt ./build/test/fixtures/basic-txt.kmx
if ! cmp ./build/test/fixtures/basic-xml.kmx ./build/test/fixtures/basic-txt.kmx; then
builder_die "basic-xml.kmx (compiled from basic.xml) and basic-txt.kmx (see basic.txt) are different!"
else
builder_echo grey "basic-xml.kmx and basic-txt.kmx are identical"
fi
}
builder_run_action clean do_clean
@ -82,7 +87,7 @@ builder_run_action configure do_configure
builder_run_action build do_build
builder_run_action build-fixtures do_build_fixtures
builder_run_action api api-extractor run --local --verbose
builder_run_action test builder_do_typescript_tests 75
builder_run_action test builder_do_typescript_tests 90
#-------------------------------------------------------------------------------------------------------------------

4
developer/src/kmc-ldml/package.json
View file

@ -59,7 +59,9 @@
],
"exclude-after-remap": true,
"exclude": [
"test/"
"test/",
"src/util/abnf/*/*.d.ts",
"src/util/abnf/*/*.js"
]
},
"repository": {

14
developer/src/kmc-ldml/src/compiler/ldml-compiler-messages.ts
View file

@ -58,8 +58,18 @@ export class LdmlCompilerMessages {
m(this.HINT_NoDisplayForMarker, `Key element with id "${def(o.id)}" has only marker output, but there is no matching display element by output or keyId. Keycap may be blank.`);
static ERROR_InvalidVersion = SevError | 0x000D;
static Error_InvalidVersion = (o:{version: string}) =>
m(this.ERROR_InvalidVersion, `Version number '${def(o.version)}' must be a semantic version format string.`);
static Error_InvalidVersion = (o:{version: string}) => m(
this.ERROR_InvalidVersion,
`Version number '${def(o.version)}' must be a semantic version format string with 'major.minor.patch' components.`,
`The version number in the LDML keyboard file must be a [semantic
version](https://semver.org) (semver) string. This string has a format of three
integers representing major, minor, and patch versions, separated by periods. In
the LDML keyboard specification, the full semver format is permitted, including
pre-release suffix strings, but the Keyman toolchain currently restricts the
format to the three integer components.
Example: \`"1.12.3"\``
);
static ERROR_MustBeAtLeastOneLayerElement = SevError | 0x000E;
static Error_MustBeAtLeastOneLayerElement = () =>

32
developer/src/kmc-ldml/test/compiler-e2e.tests.ts
View file

@ -1,8 +1,7 @@
import 'mocha';
import {assert} from 'chai';
import hextobin from '@keymanapp/hextobin';
import { KMXBuilder } from '@keymanapp/developer-utils';
import {checkMessages, compileKeyboard, compilerTestCallbacks, compilerTestOptions, makePathToFixture} from './helpers/index.js';
import {compileKeyboard, compilerTestCallbacks, compilerTestOptions, makePathToFixture} from './helpers/index.js';
import { compareXml } from './helpers/compareXml.js';
import { LdmlKeyboardCompiler } from '../src/compiler/compiler.js';
import { kmxToXml } from '../src/util/serialize.js';
@ -24,33 +23,29 @@ describe('compiler-tests', function() {
const inputFilename = makePathToFixture('basic.xml');
const binaryFilename = makePathToFixture('basic.txt');
// Compile the keyboard
const kmx = await compileKeyboard(inputFilename, {...compilerTestOptions, saveDebug: true, shouldAddCompilerVersion: false});
assert.isNotNull(kmx);
// Use the builder to generate the binary output file
const builder = new KMXBuilder(kmx, true);
const code = builder.compile();
checkMessages();
assert.isNotNull(code);
// Compare output
let expected = await hextobin(binaryFilename, undefined, {silent:true});
assert.deepEqual<Uint8Array>(code, expected);
// now compare it to use with run()
// Let's build basic.xml
// It should match basic.kmx (built from basic.txt)
const k = new LdmlKeyboardCompiler();
await k.init(compilerTestCallbacks, { ...compilerTestOptions, saveDebug: true, shouldAddCompilerVersion: false });
// now output it again as XML
const outputFilename = makePathToFixture('basic-serialized.xml');
const asXml = kmxToXml(kmx);
writeFileSync(outputFilename, asXml, 'utf-8');
const { artifacts } = await k.run(inputFilename, "basic-xml.kmx"); // need the exact name passed to build-fixtures
assert.isNotNull(artifacts);
const { kmx, kvk } = artifacts;
assert.isNotNull(kmx);
assert.deepEqual<Uint8Array>(kmx?.data, expected);
// TODO-LDML: compare the .kvk file to something else?
assert.isNotNull(kvk?.data);
});
it('should-serialize-kmx', async function() {
this.timeout(4000);
// Let's build basic.xml
// It should match basic.kmx (built from basic.txt)
const inputFilename = makePathToFixture('basic.xml');
// Compile the keyboard
@ -65,7 +60,6 @@ describe('compiler-tests', function() {
compareXml(outputFilename, inputFilename);
});
it('should handle non existent files', async () => {
const filename = 'DOES_NOT_EXIST.xml';
const k = new LdmlKeyboardCompiler();

6
developer/src/kmc-ldml/test/fixtures/basic.txt vendored
View file

@ -72,6 +72,10 @@ block(stores) # struct COMP_STORE {
offset(store_targets_string) # KMX_DWORD dpString;
# };
18 00 00 00 # TSS_VISUALKEYBOARD
00 00 00 00 # KMX_DWORD dpName;
offset(store_vk_path_string) # KMX_DWORD dpString;
block(store_name_name)
26 00 4e 00 41 00 4d 00 45 00 00 00 # '&NAME'
block(store_name_string)
@ -88,6 +92,8 @@ block(store_targets_name)
26 00 54 00 41 00 52 00 47 00 45 00 54 00 53 00 00 00 # '&TARGETS'
block(store_targets_string)
64 00 65 00 73 00 6b 00 74 00 6f 00 70 00 00 00 # 'desktop'
block(store_vk_path_string)
62 00 61 00 73 00 69 00 63 00 2d 00 78 00 6d 00 6c 00 2e 00 6b 00 76 00 6b 00 00 00 # 'basic-xml.kvk'
block(sect) # struct COMP_KMXPLUS_SECT {
73 65 63 74 # KMX_DWORD header.ident; // 0000 Section name

2
developer/src/kmc-model/package.json
View file

@ -41,7 +41,7 @@
"@types/node": "^20.4.1",
"c8": "^7.12.0",
"chalk": "^2.4.2",
"esbuild": "^0.18.9",
"esbuild": "^0.25.0",
"mocha": "^10.0.0"
},
"mocha": {

2
developer/src/kmc/package.json
View file

@ -60,7 +60,7 @@
"@types/mocha": "^5.2.7",
"@types/node": "^20.4.1",
"c8": "^7.12.0",
"esbuild": "^0.18.9",
"esbuild": "^0.25.0",
"mocha": "^8.4.0",
"typescript": "^5.4.5"
},

69
developer/src/kmcmplib/src/Compiler.cpp
View file

@ -1556,6 +1556,71 @@ KMX_DWORD CheckVirtualKeysInOutput(PKMX_WCHAR output) {
return STATUS_Success;
}
const KMX_BOOL CODE__IS_TEXTUAL[] = {
-1, // undefined 0x00
TRUE, // CODE_ANY 0x01
TRUE, // CODE_INDEX 0x02
TRUE, // CODE_CONTEXT 0x03
FALSE, // CODE_NUL 0x04
FALSE, // CODE_USE 0x05 // may trigger text effects but indirectly
FALSE, // CODE_RETURN 0x06
FALSE, // CODE_BEEP 0x07
TRUE, // CODE_DEADKEY 0x08
-1, // unused 0x09
TRUE, // CODE_EXTENDED 0x0A
-1, // CODE_EXTENDEDEND 0x0B (unused)
FALSE, // CODE_SWITCH 0x0C
-1, // CODE_KEY 0x0D (never used)
FALSE, // CODE_CLEARCONTEXT 0x0E
FALSE, // CODE_CALL 0x0F // may trigger text effects but indirectly
-1, // UC_SENTINEL_EXTENDEDEND 0x10 (not valid with UC_SENTINEL)
TRUE, // CODE_CONTEXTEX 0x11
TRUE, // CODE_NOTANY 0x12
FALSE, // CODE_SETOPT 0x13
FALSE, // CODE_IFOPT 0x14
FALSE, // CODE_SAVEOPT 0x15
FALSE, // CODE_RESETOPT 0x16
FALSE, // CODE_IFSYSTEMSTORE 0x17
FALSE // CODE_SETSYSTEMSTORE 0x18
};
static_assert(sizeof(CODE__IS_TEXTUAL) / sizeof(CODE__IS_TEXTUAL[0]) == (CODE_LASTCODE + 1), "Size of array CODE__EMITS_TEXT not correct");
KMX_BOOL IsTextualCode(KMX_WORD code) {
assert(code >= CODE_ANY && code <= CODE_LASTCODE);
if(!(code >= CODE_ANY && code <= CODE_LASTCODE)) {
return FALSE;
}
return CODE__IS_TEXTUAL[code];
}
/**
* Warn if output has text before or after a nul statement, which
* doesn't make sense -- if nul is used, it indicates no output
*/
KMX_DWORD CheckNulUsageInOutput(PKMX_WCHAR output) {
PKMX_WCHAR p;
KMX_BOOL hasNul = FALSE, hasOther = FALSE;
for (p = output; *p; p = incxstr(p)) {
if (*p == UC_SENTINEL) {
auto code = *(p+1);
if(code == CODE_NUL) {
hasNul = TRUE;
} else if(IsTextualCode(code)) {
hasOther = TRUE;
}
} else {
hasOther = TRUE;
}
}
if(hasNul && hasOther) {
return KmnCompilerMessages::ERROR_TextBeforeOrAfterNulInOutput;
}
return STATUS_Success;
}
/**
* Adds implicit `context` to start of output of rules for readonly groups
*/
@ -1694,6 +1759,10 @@ KMX_DWORD ProcessKeyLineImpl(PFILE_KEYBOARD fk, PKMX_WCHAR str, KMX_BOOL IsUnico
return msg;
}
if ((msg = CheckNulUsageInOutput(pklOut)) != STATUS_Success) {
return msg;
}
if (gp->fReadOnly) {
// Ensure no output is made from the rule, and that
// use() statements meet required readonly semantics

BIN
developer/src/kmcmplib/tests/fixtures/keyboards-repo/mozhi_malayalam.kmx vendored
View file

Binary file not shown.

2
docs/build/sentry-cli.md vendored
View file

@ -2,7 +2,7 @@
Contact the Keyman team if you need access to sentry.keyman.com for development.
You will also need to install [sentry-cli](https://docs.sentry.io/cli/installation/) for uploading Debug symbols.
After setting up your personal [Auth token](http://sentry.keyman.com/settings/account/api/auth-tokens/), add the following to **~/.bashrc**
After setting up your personal [Auth token](http://keyman.sentry.io/settings/account/api/auth-tokens/), add the following to **~/.bashrc**
```bash
export SENTRY_AUTH_TOKEN={your Sentry auth token}

4
docs/file-formats/README.md Normal file
View file

@ -0,0 +1,4 @@
# Description of file formats
In addition to the file formats described in this folder, see also
the schema files in `common/schemas` for additional file formats.

13
docs/file-formats/kmx-plus-file-format.md
View file

@ -36,9 +36,16 @@ Draft spec PR: <https://github.com/unicode-org/cldr/pull/1847>
- Other than the `sect` table itself, the rest of the sections follow in binary
order in the file. In other words, the binary ordering of the section
identifiers determines the order of the file layout.
- All sections other than the `sect` table are optional for the file
format to be valid, however, a valid keyboard will need most or
all of the sections present to be usable.
### Required sections
- The `sect` section is always required for a valid KMX Plus file.
- The `key2` section is required by core’s ldml_processor loader, for the
`key2.kmap` subtable.
- The `strs` section is effectively required in most cases.
- Both the `bksp` and `tran` sections require `elem`, `strs`, `vars`, and `meta`
sections.
- A valid keyboard will need most or all of the sections present to be usable.
### C7043.2.1 `sect`—Section Table of contents

2
docs/minimum-versions.md
View file

@ -72,7 +72,7 @@ https://help.keyman.com/developer/engine/android/latest-version/
| KEYMAN_VERSION_ISO639_3 | 2024-05-22 |
| KEYMAN_VERSION_JAVA | 11 |
| KEYMAN_VERSION_LANGTAGS | 2025-02-18 |
| KEYMAN_VERSION_LANGUAGE_SUBTAG_REGISTRY | 2024-05-16 |
| KEYMAN_VERSION_LANGUAGE_SUBTAG_REGISTRY | 2025-03-10 |
| KEYMAN_VERSION_UNICODE | 16.0.0 |
> ### This file is auto-generated by publish-minimum-versions.sh. Do not modify

2
ios/docs/help/basic/config/index.md
View file

@ -30,7 +30,7 @@ Click on this to [search for a keyboard or language](../../start/searching-for-k
When enabled, the Keyman app will display the 'Get Started' screen on app startup.
## Allow error reporting
When enabled, the Keyman app will send crash reporting information to https://sentry.keyman.com. No personally identifiable information or keyboard strokes are recorded.
When enabled, the Keyman app will send crash reporting information to Sentry. No personally identifiable information or keyboard strokes are recorded.
## Spacebar Caption
Click on this to [change the displayed keyboard name](spacebar-caption) on the spacebar.

18
ios/engine/KMEI/KeymanEngine/it.lproj/Localizable.strings
View file

@ -226,6 +226,24 @@
/* Text showing name of spacebar caption - language + keyboard */
"menu-settings-spacebar-item-languageKeyboard" = "Lingua e tastiera";
/* Label for the "Adjust Keyboard Height" item on the main settings screen */
"menu-settings-adjust-keyboard-height" = "Regola altezza tastiera";
/* Title for the "Adjust Keyboard Height" settings child screen */
"adjust-keyboard-height-title" = "Regola tastiera";
/* Label for "Reset to Default Keyboard Height" button on the adjust height screen */
"button-label-reset-default-keyboard-height" = "Reimposta l'altezza predefinita tastiera";
/* Instruction text to drag keyboard to resize */
"keyboard-drag-instructions" = "Trascina il controllo freccia per regolare l'altezza della tastiera.";
/* Instruction to rotate to adjust landscape keyboard height (displayed when device is portrait) */
"portrait-keyboard-rotate-instructions" = "Ruota il dispositivo per regolarlo in orizzontale.";
/* Instruction to rotate to adjust portrait keyboard height (displayed when device is landscape) */
"landscape-keyboard-rotate-instructions" = "Ruota il dispositivo per regolarlo in verticale.";
/* Short text for notification: download failure for keyboard */
"notification-download-failure-keyboard" = "Download tastiera non riuscito";

1
linux/build.sh
View file

@ -17,6 +17,7 @@ builder_describe \
":engine=ibus-keyman ibus-keyman" \
":help Online documentation" \
":service=keyman-system-service keyman-system-service" \
":mcompile=mcompile/keymap mnemonic layout recompiler for Linux" \
"clean" \
"configure" \
"build" \

2
linux/debian/control
View file

@ -25,6 +25,7 @@ Build-Depends:
perl,
pkgconf,
python3-all (>= 3.5),
python3-build,
python3-dbus,
python3-fonttools,
python3-gi,
@ -39,6 +40,7 @@ Build-Depends:
python3-requests,
python3-sentry-sdk (>= 1.1),
python3-setuptools,
python3-venv,
python3-xdg,
xserver-xephyr,
xvfb,

17
linux/ibus-keyman/src/engine.c
View file

@ -232,7 +232,7 @@ get_context_debug(IBusEngine *engine) {
static gchar *
debug_utf8_with_codepoints(const gchar *utf8) {
GString *output = g_string_new("");
g_string_append_printf(output, "|%s| len:%ld) [", utf8, g_utf8_strlen(utf8, -1));
g_string_append_printf(output, "|%s| (len:%ld) [", utf8, g_utf8_strlen(utf8, -1));
gunichar2 *utf16 = g_utf8_to_utf16(utf8, -1, NULL, NULL, NULL);
for (int i = 0; utf16[i] != '\0'; i++) {
g_string_append_printf(output, "U+%04x ", utf16[i]);
@ -290,7 +290,7 @@ set_context_if_needed(IBusEngine *engine) {
}
static void
initialize_queue(IBusKeymanEngine *keyman, int index, int count) {
initialize_queue_items(IBusKeymanEngine *keyman, int index, int count) {
g_assert(keyman != NULL);
g_assert(index >= 0 && index < MAX_QUEUE_SIZE);
g_assert(count > 0 && count <= MAX_QUEUE_SIZE);
@ -463,7 +463,7 @@ ibus_keyman_engine_constructor(
keyman->lctrl_pressed = FALSE;
keyman->ralt_pressed = FALSE;
keyman->rctrl_pressed = FALSE;
initialize_queue(keyman, 0, MAX_QUEUE_SIZE);
initialize_queue_items(keyman, 0, MAX_QUEUE_SIZE);
keyman->commit_item = &keyman->commit_queue[0];
gchar **split_name = g_strsplit(engine_name, ":", 2);
if (split_name[0] == NULL)
@ -703,18 +703,19 @@ process_persist_action(IBusEngine *engine, km_core_option_item *persist_options)
static void
process_emit_keystroke_action(IBusEngine *engine, km_core_bool emit_keystroke) {
if (!emit_keystroke) {
return;
}
IBusKeymanEngine *keyman = (IBusKeymanEngine *)engine;
if (client_supports_surrounding_text(engine)) {
// compliant app
if (!emit_keystroke) {
return;
}
ibus_engine_forward_key_event(engine, keyman->commit_item->keyval,
keyman->commit_item->keycode, keyman->commit_item->state);
return;
}
// non-compliant app
keyman->commit_item->emitting_keystroke = TRUE;
keyman->commit_item->emitting_keystroke = emit_keystroke;
}
static void
@ -768,7 +769,7 @@ commit_current_queue_item(IBusKeymanEngine *keyman) {
}
keyman->commit_item--;
memmove(keyman->commit_queue, &keyman->commit_queue[1], sizeof(commit_queue_item) * MAX_QUEUE_SIZE - 1);
initialize_queue(keyman, MAX_QUEUE_SIZE - 1, 1);
initialize_queue_items(keyman, MAX_QUEUE_SIZE - 1, 1);
}
static void

352
linux/ibus-keyman/src/keycodes.h
View file

@ -7,109 +7,255 @@
// uses kernel keycodes which are (X11 keycode - 8)
// see /usr/include/linux/input-event-codes.h
static km_core_virtual_key const keycode_to_vk[256] = {
0, // KEY_RESERVED = 0x00;
KM_CORE_VKEY_ESC, // KEY_ESC = 0x01;
KM_CORE_VKEY_1, // KEY_1 = 0x02;
KM_CORE_VKEY_2, // KEY_2 = 0x03;
KM_CORE_VKEY_3, // KEY_3 = 0x04;
KM_CORE_VKEY_4, // KEY_4 = 0x05;
KM_CORE_VKEY_5, // KEY_5 = 0x06;
KM_CORE_VKEY_6, // KEY_6 = 0x07;
KM_CORE_VKEY_7, // KEY_7 = 0x08;
KM_CORE_VKEY_8, // KEY_8 = 0x09;
KM_CORE_VKEY_9, // KEY_9 = 0x0A;
KM_CORE_VKEY_0, // KEY_0 = 0x0B;
KM_CORE_VKEY_HYPHEN, // KEY_MINUS = 0x0C;
KM_CORE_VKEY_EQUAL, // KEY_EQUALS = 0x0D;
KM_CORE_VKEY_BKSP, // KEY_BACKSPACE = 0x0E;
KM_CORE_VKEY_TAB, // KEY_TAB = 0x0F;
KM_CORE_VKEY_Q, // KEY_Q = 0x10;
KM_CORE_VKEY_W, // KEY_W = 0x11;
KM_CORE_VKEY_E, // KEY_E = 0x12;
KM_CORE_VKEY_R, // KEY_R = 0x13;
KM_CORE_VKEY_T, // KEY_T = 0x14;
KM_CORE_VKEY_Y, // KEY_Y = 0x15;
KM_CORE_VKEY_U, // KEY_U = 0x16;
KM_CORE_VKEY_I, // KEY_I = 0x17;
KM_CORE_VKEY_O, // KEY_O = 0x18;
KM_CORE_VKEY_P, // KEY_P = 0x19;
KM_CORE_VKEY_LBRKT, // KEY_LEFTBRACE = 0x1A;
KM_CORE_VKEY_RBRKT, // KEY_RIGHTBRACE = 0x1B;
KM_CORE_VKEY_ENTER, // KEY_ENTER = 0x1C;
KM_CORE_VKEY_CONTROL, // KEY_LEFTCTRL = 0x1D;
KM_CORE_VKEY_A, // KEY_A = 0x1E;
KM_CORE_VKEY_S, // KEY_S = 0x1F;
KM_CORE_VKEY_D, // KEY_D = 0x20;
KM_CORE_VKEY_F, // KEY_F = 0x21;
KM_CORE_VKEY_G, // KEY_G = 0x22;
KM_CORE_VKEY_H, // KEY_H = 0x23;
KM_CORE_VKEY_J, // KEY_J = 0x24;
KM_CORE_VKEY_K, // KEY_K = 0x25;
KM_CORE_VKEY_L, // KEY_L = 0x26;
KM_CORE_VKEY_COLON, // KEY_SEMICOLON = 0x27;
KM_CORE_VKEY_QUOTE, // KEY_APOSTROPHE = 0x28;
KM_CORE_VKEY_BKQUOTE, // KEY_GRAVE = 0x29;
KM_CORE_VKEY_SHIFT, // KEY_LEFTSHIFT = 0x2A;
KM_CORE_VKEY_BKSLASH, // KEY_BACKSLASH = 0x2B;
KM_CORE_VKEY_Z, // KEY_Z = 0x2C;
KM_CORE_VKEY_X, // KEY_X = 0x2D;
KM_CORE_VKEY_C, // KEY_C = 0x2E;
KM_CORE_VKEY_V, // KEY_V = 0x2F;
KM_CORE_VKEY_B, // KEY_B = 0x30;
KM_CORE_VKEY_N, // KEY_N = 0x31;
KM_CORE_VKEY_M, // KEY_M = 0x32;
KM_CORE_VKEY_COMMA, // KEY_COMMA = 0x33;
KM_CORE_VKEY_PERIOD, // KEY_DOT = 0x34;
KM_CORE_VKEY_SLASH, // KEY_SLASH = 0x35;
KM_CORE_VKEY_SHIFT, // KEY_RIGHTSHIFT = 0x36;
KM_CORE_VKEY_NPSTAR, // KEY_KPASTERISK = 0x37;
KM_CORE_VKEY_ALT, // KEY_LEFTALT = 0x38;
KM_CORE_VKEY_SPACE, // KEY_SPACE = 0x39;
KM_CORE_VKEY_CAPS, // KEY_CAPSLOCK = 0x3A;
KM_CORE_VKEY_F1, // KEY_F1 = 0x3B;
KM_CORE_VKEY_F2, // KEY_F2 = 0x3C;
KM_CORE_VKEY_F3, // KEY_F3 = 0x3D;
KM_CORE_VKEY_F4, // KEY_F4 = 0x3E;
KM_CORE_VKEY_F5, // KEY_F5 = 0x3F;
KM_CORE_VKEY_F6, // KEY_F6 = 0x40;
KM_CORE_VKEY_F7, // KEY_F7 = 0x41;
KM_CORE_VKEY_F8, // KEY_F8 = 0x42;
KM_CORE_VKEY_F9, // KEY_F9 = 0x43;
KM_CORE_VKEY_F10, // KEY_F10 = 0x44;
0, // KEY_NUMLOCK = 0x45;
0, // KEY_SCROLLLOCK = 0x46;
KM_CORE_VKEY_NP7, // KEY_KP7 = 0x47;
KM_CORE_VKEY_NP8, // KEY_KP8 = 0x48;
KM_CORE_VKEY_NP9, // KEY_KP9 = 0x49;
KM_CORE_VKEY_NPMINUS, // KEY_KPMINUS = 0x4A;
KM_CORE_VKEY_NP4, // KEY_KP4 = 0x4B;
KM_CORE_VKEY_NP5, // KEY_KP5 = 0x4C;
KM_CORE_VKEY_NP6, // KEY_KP6 = 0x4D;
KM_CORE_VKEY_NPPLUS, // KEY_KPPLUS = 0x4E;
KM_CORE_VKEY_NP1, // KEY_KP1 = 0x4F;
KM_CORE_VKEY_NP2, // KEY_KP2 = 0x50;
KM_CORE_VKEY_NP3, // KEY_KP3 = 0x51;
KM_CORE_VKEY_NP0, // KEY_KP0 = 0x52;
KM_CORE_VKEY_NPDOT, // KEY_KPDOT = 0x53;
0, // padding 0x54;
0, // KEY_ZENKAKUHANKAKU = 0x55;
KM_CORE_VKEY_oE2, // KEY_102ND = 0x56;
KM_CORE_VKEY_F11, // KEY_F11 = 0x57;
KM_CORE_VKEY_F12, // KEY_F12 = 0x58;
0, // KEY_RO = 0x59;
0, // KEY_KATAKANA = 0x5a;
0, // KEY_HIRAGANA = 0x5b;
0, // KEY_HENKAN = 0x5c;
0, // KEY_KATAKANAHIRAGANA = 0x5d;
0, // KEY_MUHENKAN = 0x5e;
0, // KEY_KPJPCOMMA = 0x5f;
0, // KEY_KPENTER = 0x60;
KM_CORE_VKEY_CONTROL, // KEY_RIGHTCTRL = 0x61;
0, // KEY_KPSLASH = 0x62;
0, // KEY_SYSRQ = 0x63;
KM_CORE_VKEY_ALT // KEY_RIGHTALT = 0x64;
// Many more KEYS currently not used by KMW...
0, // 0x00 - KEY_RESERVED
KM_CORE_VKEY_ESC, // 0x01 - KEY_ESC
KM_CORE_VKEY_1, // 0x02 - KEY_1
KM_CORE_VKEY_2, // 0x03 - KEY_2
KM_CORE_VKEY_3, // 0x04 - KEY_3
KM_CORE_VKEY_4, // 0x05 - KEY_4
KM_CORE_VKEY_5, // 0x06 - KEY_5
KM_CORE_VKEY_6, // 0x07 - KEY_6
KM_CORE_VKEY_7, // 0x08 - KEY_7
KM_CORE_VKEY_8, // 0x09 - KEY_8
KM_CORE_VKEY_9, // 0x0A - KEY_9
KM_CORE_VKEY_0, // 0x0B - KEY_0
KM_CORE_VKEY_HYPHEN, // 0x0C - KEY_MINUS
KM_CORE_VKEY_EQUAL, // 0x0D - KEY_EQUALS
KM_CORE_VKEY_BKSP, // 0x0E - KEY_BACKSPACE
KM_CORE_VKEY_TAB, // 0x0F - KEY_TAB
KM_CORE_VKEY_Q, // 0x10 - KEY_Q
KM_CORE_VKEY_W, // 0x11 - KEY_W
KM_CORE_VKEY_E, // 0x12 - KEY_E
KM_CORE_VKEY_R, // 0x13 - KEY_R
KM_CORE_VKEY_T, // 0x14 - KEY_T
KM_CORE_VKEY_Y, // 0x15 - KEY_Y
KM_CORE_VKEY_U, // 0x16 - KEY_U
KM_CORE_VKEY_I, // 0x17 - KEY_I
KM_CORE_VKEY_O, // 0x18 - KEY_O
KM_CORE_VKEY_P, // 0x19 - KEY_P
KM_CORE_VKEY_LBRKT, // 0x1A - KEY_LEFTBRACE
KM_CORE_VKEY_RBRKT, // 0x1B - KEY_RIGHTBRACE
KM_CORE_VKEY_ENTER, // 0x1C - KEY_ENTER
KM_CORE_VKEY_CONTROL, // 0x1D - KEY_LEFTCTRL
KM_CORE_VKEY_A, // 0x1E - KEY_A
KM_CORE_VKEY_S, // 0x1F - KEY_S
KM_CORE_VKEY_D, // 0x20 - KEY_D
KM_CORE_VKEY_F, // 0x21 - KEY_F
KM_CORE_VKEY_G, // 0x22 - KEY_G
KM_CORE_VKEY_H, // 0x23 - KEY_H
KM_CORE_VKEY_J, // 0x24 - KEY_J
KM_CORE_VKEY_K, // 0x25 - KEY_K
KM_CORE_VKEY_L, // 0x26 - KEY_L
KM_CORE_VKEY_COLON, // 0x27 - KEY_SEMICOLON
KM_CORE_VKEY_QUOTE, // 0x28 - KEY_APOSTROPHE
KM_CORE_VKEY_BKQUOTE, // 0x29 - KEY_GRAVE
KM_CORE_VKEY_SHIFT, // 0x2A - KEY_LEFTSHIFT
KM_CORE_VKEY_BKSLASH, // 0x2B - KEY_BACKSLASH
KM_CORE_VKEY_Z, // 0x2C - KEY_Z
KM_CORE_VKEY_X, // 0x2D - KEY_X
KM_CORE_VKEY_C, // 0x2E - KEY_C
KM_CORE_VKEY_V, // 0x2F - KEY_V
KM_CORE_VKEY_B, // 0x30 - KEY_B
KM_CORE_VKEY_N, // 0x31 - KEY_N
KM_CORE_VKEY_M, // 0x32 - KEY_M
KM_CORE_VKEY_COMMA, // 0x33 - KEY_COMMA
KM_CORE_VKEY_PERIOD, // 0x34 - KEY_DOT
KM_CORE_VKEY_SLASH, // 0x35 - KEY_SLASH
KM_CORE_VKEY_SHIFT, // 0x36 - KEY_RIGHTSHIFT
KM_CORE_VKEY_NPSTAR, // 0x37 - KEY_KPASTERISK
KM_CORE_VKEY_ALT, // 0x38 - KEY_LEFTALT
KM_CORE_VKEY_SPACE, // 0x39 - KEY_SPACE
KM_CORE_VKEY_CAPS, // 0x3A - KEY_CAPSLOCK
KM_CORE_VKEY_F1, // 0x3B - KEY_F1
KM_CORE_VKEY_F2, // 0x3C - KEY_F2
KM_CORE_VKEY_F3, // 0x3D - KEY_F3
KM_CORE_VKEY_F4, // 0x3E - KEY_F4
KM_CORE_VKEY_F5, // 0x3F - KEY_F5
KM_CORE_VKEY_F6, // 0x40 - KEY_F6
KM_CORE_VKEY_F7, // 0x41 - KEY_F7
KM_CORE_VKEY_F8, // 0x42 - KEY_F8
KM_CORE_VKEY_F9, // 0x43 - KEY_F9
KM_CORE_VKEY_F10, // 0x44 - KEY_F10
KM_CORE_VKEY_NUMLOCK, // 0x45 - KEY_NUMLOCK
KM_CORE_VKEY_SCROLL, // 0x46 - KEY_SCROLLLOCK
KM_CORE_VKEY_NP7, // 0x47 - KEY_KP7
KM_CORE_VKEY_NP8, // 0x48 - KEY_KP8
KM_CORE_VKEY_NP9, // 0x49 - KEY_KP9
KM_CORE_VKEY_NPMINUS, // 0x4A - KEY_KPMINUS
KM_CORE_VKEY_NP4, // 0x4B - KEY_KP4
KM_CORE_VKEY_NP5, // 0x4C - KEY_KP5
KM_CORE_VKEY_NP6, // 0x4D - KEY_KP6
KM_CORE_VKEY_NPPLUS, // 0x4E - KEY_KPPLUS
KM_CORE_VKEY_NP1, // 0x4F - KEY_KP1
KM_CORE_VKEY_NP2, // 0x50 - KEY_KP2
KM_CORE_VKEY_NP3, // 0x51 - KEY_KP3
KM_CORE_VKEY_NP0, // 0x52 - KEY_KP0
KM_CORE_VKEY_NPDOT, // 0x53 - KEY_KPDOT
0, // padding 0x54;
KM_CORE_VKEY__KANA__, // 0x55 - KEY_ZENKAKUHANKAKU
KM_CORE_VKEY_oE2, // 0x56 - KEY_102ND
KM_CORE_VKEY_F11, // 0x57 - KEY_F11
KM_CORE_VKEY_F12, // 0x58 - KEY_F12
0, // 0x59 - KEY_RO - ?
KM_CORE_VKEY__KANA__, // 0x5a - KEY_KATAKANA
KM_CORE_VKEY__KANA__, // 0x5b - KEY_HIRAGANA
KM_CORE_VKEY__KANA__, // 0x5c - KEY_HENKAN
KM_CORE_VKEY__KANA__, // 0x5d - KEY_KATAKANAHIRAGANA
KM_CORE_VKEY__KANA__, // 0x5e - KEY_MUHENKAN
KM_CORE_VKEY_COMMA, // 0x5f - KEY_KPJPCOMMA
KM_CORE_VKEY_ENTER, // 0x60 - KEY_KPENTER
KM_CORE_VKEY_CONTROL, // 0x61 - KEY_RIGHTCTRL
KM_CORE_VKEY_SLASH, // 0x62 - KEY_KPSLASH
KM_CORE_VKEY_PRTSCN, // 0x63 - KEY_SYSRQ
KM_CORE_VKEY_ALT, // 0x64 - KEY_RIGHTALT
0, // 0x65 - KEY_LINEFEED - ?
KM_CORE_VKEY_HOME, // 0x66 - KEY_HOME
KM_CORE_VKEY_UP, // 0x67 - KEY_UP
KM_CORE_VKEY_PGUP, // 0x68 - KEY_PAGEUP
KM_CORE_VKEY_LEFT, // 0x69 - KEY_LEFT
KM_CORE_VKEY_RIGHT, // 0x6A - KEY_RIGHT
KM_CORE_VKEY_END, // 0x6B - KEY_END
KM_CORE_VKEY_DOWN, // 0x6C - KEY_DOWN
KM_CORE_VKEY_PGDN, // 0x6D - KEY_PAGEDOWN
KM_CORE_VKEY_INS, // 0x6E - KEY_INSERT
KM_CORE_VKEY_DEL, // 0x6F - KEY_DELETE
0, // 0x70 - KEY_MACRO - ?
KM_CORE_VKEY__VOLUMEMUTE__, // 0x71 - KEY_MUTE
KM_CORE_VKEY__VOLUMEDOWN__, // 0x72 - KEY_VOLUMEDOWN
KM_CORE_VKEY__VOLUMEUP__, // 0x73 - KEY_VOLUMEUP
0, // 0x74 - KEY_POWER
KM_CORE_VKEY_EQUAL, // 0x75 - KEY_KPEQUAL
KM_CORE_VKEY_NPPLUS, // 0x76 - KEY_KPPLUSMINUS
KM_CORE_VKEY_PAUSE, // 0x77 - KEY_PAUSE
KM_CORE_VKEY__ZOOM__, // 0x78 - KEY_SCALE
KM_CORE_VKEY_NPDOT, // 0x79 - KEY_KPCOMMA
KM_CORE_VKEY__HANGUL__, // 0x7a - KEY_HANGEUL
KM_CORE_VKEY__HANJA__, // 0x7b - KEY_HANJA
0, // 0x7c - KEY_YEN - ?
KM_CORE_VKEY__LWIN__, // 0x7d - KEY_LEFTMETA
KM_CORE_VKEY__RWIN__, // 0x7e - KEY_RIGHTMETA
0, // 0x7f - KEY_COMPOSE - ?
KM_CORE_VKEY__BROWSERSTOP__, // 0x80 - KEY_STOP
KM_CORE_VKEY__BROWSERREFRESH__,// 0x81 - KEY_AGAIN
0, // 0x82 - KEY_PROPS - ?
0, // 0x83 - KEY_UNDO - ?
0, // 0x84 - KEY_FRONT - ?
0, // 0x85 - KEY_COPY - ?
0, // 0x86 - KEY_OPEN - ?
0, // 0x87 - KEY_PASTE - ?
KM_CORE_VKEY__BROWSERSEARCH__, // 0x88 - KEY_FIND
0, // 0x89 - KEY_CUT - ?
KM_CORE_VKEY_HELP, // 0x8a - KEY_HELP
KM_CORE_VKEY_ALT, // 0x8b - KEY_MENU
0, // 0x8c - KEY_CALC - ?
0, // 0x8d - KEY_SETUP - ?
KM_CORE_VKEY__SLEEP__, // 0x8e - KEY_SLEEP
0, // 0x8f - KEY_WAKEUP - ?
0, // 0x90 - KEY_FILE - ?
0, // 0x91 - KEY_SENDFILE - ?
0, // 0x92 - KEY_DELETEFILE - ?
0, // 0x93 - KEY_XFER - ?
KM_CORE_VKEY__APP1__, // 0x94 - KEY_PROG1
KM_CORE_VKEY__APP2__, // 0x95 - KEY_PROG2
KM_CORE_VKEY__BROWSERHOME__, // 0x96 - KEY_WWW
0, // 0x97 - KEY_MSDOS - ?
0, // 0x98 - KEY_COFFEE, KEY_SCREENLOCK - ?
0, // 0x99 - KEY_ROTATE_DISPLAY, KEY_DIRECTION - ?
0, // 0x9a - KEY_CYCLEWINDOWS - ?
KM_CORE_VKEY__MAIL__, // 0x9b - KEY_MAIL
0, // 0x9c - KEY_BOOKMARKS - ?
0, // 0x9d - KEY_COMPUTER - ?
KM_CORE_VKEY__BROWSERBACK__, // 0x9e - KEY_BACK
KM_CORE_VKEY__BROWSERFORWARD__,// 0x9f - KEY_FORWARD
0, // 0xa0 - KEY_CLOSECD - ?
0, // 0xa1 - KEY_EJECTCD - ?
0, // 0xa2 - KEY_EJECTCLOSECD - ?
KM_CORE_VKEY__MEDIANEXT__, // 0xa3 - KEY_NEXTSONG
KM_CORE_VKEY__MEDIAPLAY__, // 0xa4 - KEY_PLAYPAUSE
KM_CORE_VKEY__MEDIAPREV__, // 0xa5 - KEY_PREVIOUSSONG
0, // 0xa6 - KEY_STOPCD - ?
0, // 0xa7 - KEY_RECORD - ?
0, // 0xa8 - KEY_REWIND - ?
KM_CORE_VKEY__MEDIA__, // 0xa9 - KEY_PHONE
0, // 0xaa - KEY_ISO - ?
0, // 0xab - KEY_CONFIG - ?
KM_CORE_VKEY__BROWSERHOME__, // 0xac - KEY_HOMEPAGE
KM_CORE_VKEY__BROWSERREFRESH__,// 0xad - KEY_REFRESH
0, // 0xae - KEY_EXIT - ?
0, // 0xaf - KEY_MOVE - ?
0, // 0xb0 - KEY_EDIT - ?
0, // 0xb1 - KEY_SCROLLUP - ?
0, // 0xb2 - KEY_SCROLLDOWN - ?
0, // 0xb3 - KEY_KPLEFTPAREN - ?
0, // 0xb4 - KEY_KPRIGHTPAREN - ?
0, // 0xb5 - KEY_NEW - ?
0, // 0xb6 - KEY_REDO - ?
KM_CORE_VKEY_F13, // 0xb7 - KEY_F13
KM_CORE_VKEY_F14, // 0xb8 - KEY_F14
KM_CORE_VKEY_F15, // 0xb9 - KEY_F15
KM_CORE_VKEY_F16, // 0xba - KEY_F16
KM_CORE_VKEY_F17, // 0xbb - KEY_F17
KM_CORE_VKEY_F18, // 0xbc - KEY_F18
KM_CORE_VKEY_F19, // 0xbd - KEY_F19
KM_CORE_VKEY_F20, // 0xbe - KEY_F20
KM_CORE_VKEY_F21, // 0xbf - KEY_F21
KM_CORE_VKEY_F22, // 0xc0 - KEY_F22
KM_CORE_VKEY_F23, // 0xc1 - KEY_F23
KM_CORE_VKEY_F24, // 0xc2 - KEY_F24
0, // 0xc3 - unassigned
0, // 0xc4 - unassigned
0, // 0xc5 - unassigned
0, // 0xc6 - unassigned
0, // 0xc7 - unassigned
KM_CORE_VKEY__MEDIAPLAY__, // 0xc8 - KEY_PLAYCD
KM_CORE_VKEY__MEDIASTOP__, // 0xc9 - KEY_PAUSECD
0, // 0xca - KEY_PROG3 - ?
0, // 0xcb - KEY_PROG4 - ?
0, // 0xcc - KEY_ALL_APPLICATIONS, KEY_DASHBOARD - ?
0, // 0xcd - KEY_SUSPEND - ?
0, // 0xce - KEY_CLOSE - ?
KM_CORE_VKEY__MEDIAPLAY__, // 0xcf - KEY_PLAY
0, // 0xd0 - KEY_FASTFORWARD - ?
0, // 0xd1 - KEY_BASSBOOST - ?
KM_CORE_VKEY_PRINT, // 0xd2 - KEY_PRINT
0, // 0xd3 - KEY_HP - ?
0, // 0xd4 - KEY_CAMERA - ?
0, // 0xd5 - KEY_SOUND - ?
0, // 0xd6 - KEY_QUESTION - ?
KM_CORE_VKEY__MAIL__, // 0xd7 - KEY_EMAIL
0, // 0xd8 - KEY_CHAT - ?
0, // 0xd9 - KEY_SEARCH - ?
0, // 0xda - KEY_CONNECT - ?
0, // 0xdb - KEY_FINANCE - ?
0, // 0xdc - KEY_SPORT - ?
0, // 0xdd - KEY_SHOP - ?
0, // 0xde - KEY_ALTERASE - ?
0, // 0xdf - KEY_CANCEL - ?
0, // 0xe0 - KEY_BRIGHTNESSDOWN - ?
0, // 0xe1 - KEY_BRIGHTNESSUP - ?
KM_CORE_VKEY__MEDIA__, // 0xe2 - KEY_MEDIA
0, // 0xe3 - KEY_SWITCHVIDEOMODE - ?
0, // 0xe4 - KEY_KBDILLUMTOGGLE - ?
0, // 0xe5 - KEY_KBDILLUMDOWN - ?
0, // 0xe6 - KEY_KBDILLUMUP - ?
0, // 0xe7 - KEY_SEND - ?
0, // 0xe8 - KEY_REPLY - ?
0, // 0xe9 - KEY_FORWARDMAIL - ?
0, // 0xea - KEY_SAVE - ?
0, // 0xeb - KEY_DOCUMENTS - ?
0, // 0xec - KEY_BATTERY - ?
0, // 0xed - KEY_BLUETOOTH - ?
0, // 0xee - KEY_WLAN - ?
0, // 0xef - KEY_UWB - ?
0, // 0xf0 - KEY_UNKNOWN - ?
0, // 0xf1 - KEY_VIDEO_NEXT - ?
0, // 0xf2 - KEY_VIDEO_PREV - ?
0, // 0xf3 - KEY_BRIGHTNESS_CYCLE - ?
0, // 0xf4 - KEY_BRIGHTNESS_AUTO, KEY_BRIGHTNESS_ZERO - ?
0, // 0xf5 - KEY_DISPLAY_OFF - ?
0, // 0xf6 - KEY_WWAN, KEY_WIMAX - ?
0, // 0xf7 - KEY_RFKILL - ?
0, // 0xf8 - KEY_MICMUTE - ?
};
#endif // __KEYCODES_H__

10
linux/keyman-config/Makefile
View file

@ -15,11 +15,13 @@ clean:
check:
./build.sh test
devdist: version
python3 setup.py egg_info -b.`TZ=UTC git log -1 --pretty=format:%cd --date=format-local:%Y%m%d%H%M` sdist
dist: clean version
python3 setup.py sdist
# TODO: remove this if when we no longer support Ubuntu 22.04 Jammy
if dpkg --compare-versions "$(lsb_release -r -s)" lt 24.04; then \
python3 setup.py sdist ; \
else \
python3 -m build --outdir build --sdist ; \
fi
deb: dist
@VERSION=$(shell echo `basename dist/*.gz .tar.gz|cut -d "-" -f2` > /tmp/keyman_version)

16
linux/keyman-config/build.sh
View file

@ -83,7 +83,14 @@ build_action() {
builder_echo "Building man and help pages"
build_man_and_help_pages
builder_echo "Building keyman-config"
python3 setup.py build
# we use `dpkg --compare-versions` to compare the current Ubuntu version
if dpkg --compare-versions "$(lsb_release -r -s)" lt 24.04; then
# Ubuntu 22.04 Jammy has a buggy version of python3-build which doesn't work
# TODO: remove once we drop support for Ubuntu 22.04 Jammy
python3 setup.py build
else
python3 -m build --outdir build .
fi
}
test_action() {
@ -106,8 +113,7 @@ install_action() {
if [ -n "${SUDO_USER:-}" ]; then
# with sudo install into /usr/local
pip3 install qrcode sentry-sdk
# eventually change this to: pip3 install .
python3 setup.py install
pip3 install .
# install icons
mkdir -p /usr/local/share/keyman/icons
cp keyman_config/icons/* /usr/local/share/keyman/icons
@ -117,9 +123,7 @@ install_action() {
else
# without sudo install into /tmp/keyman (or $DESTDIR)
mkdir -p "/tmp/keyman/$(python3 -c 'import sys;import os;pythonver="python%d.%d" % (sys.version_info[0], sys.version_info[1]);sitedir = os.path.join("lib", pythonver, "site-packages");print(sitedir)')"
# when we no longer have to support old pip version (python > 3.6) change this to:
# pip3 install --prefix /tmp/keyman .
PYTHONUSERBASE=${DESTDIR:-/tmp/keyman} python3 setup.py install --user
pip3 install --prefix /tmp/keyman .
fi
}

3
linux/mcompile/Readme.md Normal file
View file

@ -0,0 +1,3 @@
This is a proposal to rewrite mcompile for Linux. For this we need to query the base keyboard data from the Linux platform, then rewriting the keyboard .kmx using the same approach as is done in mcompile for Windows, but working from the data from the x11 keyboard on Linux.
Ideally, we'd rewrite mcompile to be cross-platform (Windows, Linux, macOS), so that the keyboard interrogation would be separated from the .kmx rewriting, at least to some degree. Nevertheless it would probably be easiest to start from a standalone implementation.

2
linux/mcompile/keymap/.gitignore vendored Normal file
View file

@ -0,0 +1,2 @@
resources/
build/

7
linux/mcompile/keymap/README.md Normal file
View file

@ -0,0 +1,7 @@
This is a proposal to rewrite mcompile for Linux. For this we need to query the base keyboard data from the Linux platform, then rewriting the keyboard .kmx using the same approach as is done in mcompile for Windows, but working from the data from the x11 keyboard on Linux.
Ideally, we'd rewrite mcompile to be cross-platform (Windows, Linux, macOS), so that the keyboard interrogation would be separated from the .kmx rewriting, at least to some degree. Nevertheless it would probably be easiest to start from a standalone implementation.
Sample program that reads US basic keyboard and compares to key value group
# Keymap

33
linux/mcompile/keymap/build.sh Executable file
View file

@ -0,0 +1,33 @@
#!/usr/bin/env bash
## START STANDARD BUILD SCRIPT INCLUDE
# adjust relative paths as necessary
THIS_SCRIPT="$(readlink -f "${BASH_SOURCE[0]}")"
. "${THIS_SCRIPT%/*}/../../../resources/build/builder.inc.sh"
. "${THIS_SCRIPT%/*}/../../../resources/build/meson-utils.inc.sh"
## END STANDARD BUILD SCRIPT INCLUDE
#. "$KEYMAN_ROOT/resources/shellHelperFunctions.sh"
################################ Main script ################################
builder_describe \
"Mnemonic layout recompiler for Linux" \
"@/common/include" \
"clean" \
"configure" \
"build" \
"test"
builder_parse "$@"
builder_describe_outputs \
configure build/build.ninja \
build build/mcompile
TARGET_PATH="$THIS_SCRIPT_PATH/build"
builder_run_action clean do_meson_clean
builder_run_action configure do_meson_configure
builder_run_action build do_meson_build
builder_run_action test do_meson_test

331
linux/mcompile/keymap/deadkey.cpp Normal file
View file

@ -0,0 +1,331 @@
/*
* Keyman is copyright (C) 2004 - 2024 SIL International. MIT License.
*
* Mnemonic layout support for Linux
*/
#include "keymap.h"
#include "deadkey.h"
/**
* @brief create a Vector of DeadKey containing all combinations of deadkey + character for ALL possible Linux keyboards
* @return vector of Deadkey* that holds all combinations of deadkey + character
*/
std::vector<DeadKey*> create_deadkeys_by_basechar() {
std::vector<DeadKey*> alDead;
vec_dword_2D dk_ComposeTable;
create_DKTable(dk_ComposeTable);
for (int i = 0; i < (int)dk_ComposeTable.size() - 1; i++) {
DeadKey* dk2 = new DeadKey(dk_ComposeTable[i][0]);
for (int j = i; j < (int)dk_ComposeTable.size(); j++) {
if ((dk_ComposeTable[i][0] == dk_ComposeTable[j][0]) && (IsKeymanUsedChar(dk_ComposeTable[j][1])))
dk2->KMX_AddDeadKeyRow(dk_ComposeTable[j][1], dk_ComposeTable[j][2]);
}
alDead.push_back(dk2);
}
return alDead;
}
/**
* @brief filter entries for the currently used Linux Keyboard out of a vector of all existing deadKey combinations
* @param dk the deadkey for which all combinations will be found
* @param[in,out] dkVec combinations of deadkey + character for the currently used Linux Keyboard
* @param r_All_Vec all existing combinations of deadkey + character for ALL possible Linux keyboards
*/
void refine_alDead(KMX_WCHAR dk, std::vector<DeadKey*>& dkVec, std::vector<DeadKey*>& r_All_Vec) {
if (dk == 0)
return;
for (int j = 0; j < (int)r_All_Vec.size(); j++) {
if (dk == r_All_Vec[j]->KMX_GetDeadCharacter()) {
if (!found_dk_inVector(dk, dkVec)) {
dkVec.push_back(r_All_Vec[j]);
}
return;
}
}
}
/**
* @brief check whether a deadkey already exists in the deadkey vector
* @param dk the deadkey to be found
* @param dkVec vector containing combinations of deadkey + character
* @return true if deadkey alredy exists;
* false if not
*/
bool found_dk_inVector(KMX_WCHAR dk, std::vector<DeadKey*>& dkVec) {
for (int i = 0; i < (int)dkVec.size(); i++) {
if (dk == dkVec[i]->KMX_GetDeadCharacter())
return true;
}
return false;
}
/**
* @brief find all deadkey combinations for a certain deadkey in a vector of all deadkey combinations
* @param r_dk_ComposeTable vector containing all possible deadkey combinations
* @param dk deadkey of interest
* @param[in,out] dk_SingleTable vector containing all dk-character combinations for a specific deadkey dk
* @return true if successful;
* false if not
*/
bool query_dk_combinations_for_specific_dk(vec_dword_2D& r_dk_ComposeTable, KMX_DWORD dk, vec_dword_2D& dk_SingleTable) {
vec_dword_1D row;
for (int i = 0; i < (int)r_dk_ComposeTable.size(); i++) {
if (r_dk_ComposeTable[i][0] == dk && IsKeymanUsedChar(r_dk_ComposeTable[i][1])) {
row.push_back(r_dk_ComposeTable[i][0]);
row.push_back(r_dk_ComposeTable[i][1]);
row.push_back(r_dk_ComposeTable[i][2]);
dk_SingleTable.push_back(row);
row.clear();
}
}
if (dk_SingleTable.size() > 0)
return true;
else
return false;
}
/**
* @brief convert a character to the upper-case equivalent and find the corresponding shiftstate
* of the entered keyval: a(97) -> A(65) + Base A(65) -> A(65) + Shift
* @param kval keyval that might be changed
* @param[in,out] shift the shiftstate of the entered keyval
* @param keymap a pointer to the currently used (underlying) keyboard layout
* @return the upper case equivalent of the keyval
*/
KMX_DWORD KMX_change_keyname_to_capital(KMX_DWORD kVal, KMX_DWORD& shift, GdkKeymap* keymap) {
guint keyval = (guint)kVal;
GdkKeymapKey* keys;
gint n_keys;
KMX_DWORD capitalKeyval = (KMX_DWORD)gdk_keyval_to_upper(kVal);
if (keyval != 0) {
gdk_keymap_get_entries_for_keyval(keymap, keyval, &keys, &n_keys);
for (int i = 0; i < n_keys; i++) {
if (keys[i].group == 0) {
shift = keys[i].level;
return capitalKeyval;
}
}
}
return capitalKeyval;
}
/**
* @brief append a 1D-vector containing name, base character and unicode_value to a 2D-Vector
* holding all possible combinations of deadkey + character for all Linux keyboards
* @param[in,out] dk_ComposeTable 2D-Vector holding all possible combinations of deadkey + character
* @param diacritic_name the name of a diacritic
* @param base_char base character
* @param unicode_value Unicode-value of the combined character
*/
void add_deadkey_combination(vec_dword_2D& dk_ComposeTable, std::string diacritic_name, std::string base_char, KMX_DWORD unicode_value) {
vec_dword_1D line;
line.push_back(convertNamesTo_DWORD_Value(diacritic_name));
line.push_back(convertNamesTo_DWORD_Value(base_char));
line.push_back(unicode_value);
dk_ComposeTable.push_back(line);
}
/**
* @brief create a 2D-Vector containing all possible combinations of deadkey + character for all Linux keyboards
* the values are taken from from: https://help.ubuntu.com/community/GtkDeadKeyTable#Accents
* dk_ComposeTable[i][0] : diacritic_name (e.g. dead_circumflex)
* dk_ComposeTable[i][1] : base_char (e.g. a)
* dk_ComposeTable[i][2] : unicode_value-Value (e.g. 0x00E2)
* @param[in,out] dk_ComposeTable
*/
void create_DKTable(vec_dword_2D& dk_ComposeTable) {
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "a", 0x00E2); // small A with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "A", 0x00C2); // capital A with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "e", 0x00EA); // small E with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "E", 0x00CA); // capital E with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "i", 0x00EE); // small I with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "I", 0x00CE); // capital I with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "o", 0x00F4); // small O with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "O", 0x00D4); // capital O with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "u", 0x00FB); // small U with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "U", 0x00DB); // capital U with circumflex
add_deadkey_combination(dk_ComposeTable, "dead_acute", "a", 0x00E1); // small A with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "A", 0x00C1); // capital A with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "c", 0x0107); // small C with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "C", 0x0106); // capital C with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "e", 0x00E9); // small E with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "E", 0x00C9); // capital E with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "i", 0x00ED); // small I with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "I", 0x00CD); // capital I with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "l", 0x013A); // small L with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "L", 0x0139); // capital L with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "n", 0x0144); // small N with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "N", 0x0143); // capital N with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "o", 0x00F3); // small O with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "O", 0x00D3); // capital O with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "r", 0x0155); // small R with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "R", 0x0154); // capital R with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "s", 0x015B); // small S with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "S", 0x015A); // capital S with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "u", 0x00FA); // small U with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "U", 0x00DA); // capital U with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "y", 0x00FD); // small Y with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "Y", 0x00DD); // capital Y with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "z", 0x017A); // small Z with acute
add_deadkey_combination(dk_ComposeTable, "dead_acute", "Z", 0x0179); // capital Z with acute
add_deadkey_combination(dk_ComposeTable, "dead_grave", "a", 0x00E0); // small A with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "A", 0x00C0); // capital A with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "e", 0x00E8); // small E with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "E", 0x00C8); // capital E with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "i", 0x00EC); // small I with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "I", 0x00CC); // capital I with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "o", 0x00F2); // small O with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "O", 0x00D2); // capital O with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "u", 0x00F9); // small U with grave
add_deadkey_combination(dk_ComposeTable, "dead_grave", "U", 0x00D9); // capital U with grave
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "a", 0x00E3); // small A with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "A", 0x00C3); // capital A with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "i", 0x0129); // small I with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "I", 0x0128); // capital I with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "n", 0x00F1); // small N with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "N", 0x00D1); // capital N with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "o", 0x00F5); // small O with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "O", 0x00D5); // capital O with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "u", 0x0169); // small U with tilde
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "U", 0x0168); // capital U with tilde
add_deadkey_combination(dk_ComposeTable, "dead_macron", "a", 0x0101); // small A with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "A", 0x0100); // capital A with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "e", 0x0113); // small E with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "E", 0x0112); // capital E with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "i", 0x012B); // small I with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "I", 0x012A); // capital I with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "o", 0x014D); // small O with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "O", 0x014C); // capital O with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "u", 0x016B); // small U with macron
add_deadkey_combination(dk_ComposeTable, "dead_macron", "U", 0x016A); // capital U with macron
add_deadkey_combination(dk_ComposeTable, "dead_breve", "a", 0x0103); // small A with breve
add_deadkey_combination(dk_ComposeTable, "dead_breve", "A", 0x0102); // capital A with breve
add_deadkey_combination(dk_ComposeTable, "dead_breve", "g", 0x011F); // small G with breve
add_deadkey_combination(dk_ComposeTable, "dead_breve", "G", 0x011E); // capital G with breve
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "e", 0x0117); // small E with dot above
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "E", 0x0116); // capital E with dot above
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "i", 0x0131); // small DOTLESS_I
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "I", 0x0130); // capital I with dot above
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "z", 0x017C); // small Z with dot above
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "Z", 0x017B); // capital Z with dot above
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "a", 0x00E4); // small A with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "A", 0x00C4); // capital A with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "e", 0x00EB); // small E with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "E", 0x00CB); // capital E with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "i", 0x00EF); // small I with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "I", 0x00CF); // capital I with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "o", 0x00F6); // small O with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "O", 0x00D6); // capital O with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "u", 0x00FC); // small U with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "U", 0x00DC); // capital U with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "y", 0x00FF); // small Y with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "Y", 0x0178); // capital Y with diaeresis
add_deadkey_combination(dk_ComposeTable, "dead_abovering", "a", 0x00E5); // small A with ring above
add_deadkey_combination(dk_ComposeTable, "dead_abovering", "A", 0x00C5); // capital A with ring above
add_deadkey_combination(dk_ComposeTable, "dead_abovering", "u", 0x016F); // small U with ring above
add_deadkey_combination(dk_ComposeTable, "dead_abovering", "U", 0x016E); // capital U with ring above
add_deadkey_combination(dk_ComposeTable, "dead_doubleacute", "o", 0x0151); // small O with double acute
add_deadkey_combination(dk_ComposeTable, "dead_doubleacute", "O", 0x0150); // capital O with double acute
add_deadkey_combination(dk_ComposeTable, "dead_doubleacute", "u", 0x0171); // small U with double acute
add_deadkey_combination(dk_ComposeTable, "dead_doubleacute", "U", 0x0170); // capital U with double acute
add_deadkey_combination(dk_ComposeTable, "dead_caron", "c", 0x010D); // small C with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "C", 0x010C); // capital C with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "d", 0x010F); // small D with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "D", 0x010E); // capital D with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "e", 0x011B); // small E with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "E", 0x011A); // capital E with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "l", 0x013E); // small L with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "L", 0x013D); // capital L with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "n", 0x0148); // small N with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "N", 0x0147); // capital N with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "r", 0x0159); // small R with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "R", 0x0158); // capital R with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "s", 0x0161); // small S with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "S", 0x0160); // capital S with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "t", 0x0165); // small T with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "T", 0x0164); // capital T with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "z", 0x017E); // small Z with caron
add_deadkey_combination(dk_ComposeTable, "dead_caron", "Z", 0x017D); // capital Z with caron
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "c", 0x00E7); // small C with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "C", 0x00C7); // capital C with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "g", 0x0123); // small G with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "G", 0x0122); // capital G with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "k", 0x0137); // small K with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "K", 0x0136); // capital K with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "l", 0x013C); // small L with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "L", 0x013B); // capital L with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "n", 0x0146); // small N with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "N", 0x0145); // capital N with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "r", 0x0157); // small R with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "R", 0x0156); // capital R with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "s", 0x015F); // small S with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "S", 0x015E); // capital S with cedilla
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "a", 0x0105); // small A with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "A", 0x0104); // capital A with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "e", 0x0119); // small E with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "E", 0x0118); // capital E with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "i", 0x012F); // small I with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "I", 0x012E); // capital I with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "u", 0x0173); // small U with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "U", 0x0172); // capital U with ogonek
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "space", 0x005E); // CIRCUMFLEX_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_acute", "space", 0x0027); // APOSTROPHE
add_deadkey_combination(dk_ComposeTable, "dead_grave", "space", 0x0060); // GRAVE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_breve", "space", 0x02D8); // BREVE
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "space", 0x02D9); // DOT_ABOVE
add_deadkey_combination(dk_ComposeTable, "dead_abovering", "space", 0x02DA); // RING_ABOVE
add_deadkey_combination(dk_ComposeTable, "dead_doubleacute", "space", 0x02DD); // DOUBLE_ACUTE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_caron", "space", 0x02C7); // CARON
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "space", 0x00B8); // CEDILLA
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "space", 0x02DB); // OGONEK
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "space", 0x007E); // TILDE
add_deadkey_combination(dk_ComposeTable, "dead_breve", "dead_breve", 0x02D8); // BREVE
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "abovedot", 0x02D9); // DOT_ABOVE
add_deadkey_combination(dk_ComposeTable, "dead_abovedot", "dead_abovedot", 0x02D9); // DOT_ABOVE
add_deadkey_combination(dk_ComposeTable, "dead_abovering", "dead_abovering", 0x02DA); // RING_ABOVE
add_deadkey_combination(dk_ComposeTable, "dead_acute", "apostrophe", 0x00B4); // ACUTE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_acute", "acute", 0x00B4); // ACUTE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_acute", "dead_acute", 0x00B4); // ACUTE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_doubleacute", "dead_doubleacute", 0x02DD); // DOUBLE_ACUTE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_caron", "caron", 0x02C7); // CARON
add_deadkey_combination(dk_ComposeTable, "dead_caron", "dead_caron", 0x02C7); // CARON
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "comma", 0x00B8); // CEDILLA
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "cedilla", 0x00B8); // CEDILLA
add_deadkey_combination(dk_ComposeTable, "dead_cedilla", "dead_cedilla", 0x00B8); // CEDILLA
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "minus", 0x00AF); // MACRON
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "asciicircum", 0x005E); // CIRCUMFLEX_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "underscore", 0x00AF); // MACRON
add_deadkey_combination(dk_ComposeTable, "dead_circumflex", "dead_circumflex", 0x005E); // CIRCUMFLEX_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "quotedbl", 0x00A8); // DIAERESIS
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "diaeresis", 0x00A8); // DIAERESIS
add_deadkey_combination(dk_ComposeTable, "dead_diaeresis", "dead_diaeresis", 0x00A8); // DIAERESIS
add_deadkey_combination(dk_ComposeTable, "dead_grave", "grave", 0x0060); // GRAVE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_grave", "dead_grave", 0x0060); // GRAVE_ACCENT
add_deadkey_combination(dk_ComposeTable, "dead_macron", "macron", 0x00AF); // MACRON
add_deadkey_combination(dk_ComposeTable, "dead_macron", "dead_macron", 0x00AF); // MACRON
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "ogonek", 0x02DB); // OGONEK
add_deadkey_combination(dk_ComposeTable, "dead_ogonek", "dead_ogonek", 0x02DB); // OGONEK
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "asciitilde", 0x007E); // TILDE
add_deadkey_combination(dk_ComposeTable, "dead_tilde", "dead_tilde", 0x007E); // TILDE
}

30
linux/mcompile/keymap/deadkey.h Normal file
View file

@ -0,0 +1,30 @@
#pragma once
#ifndef DEADKEY_H
#define DEADKEY_H
#include "mc_import_rules.h"
#include <map>
/** @brief create a Vector of DeadKey containing all combinations of deadkey + character for ALL possible Linux keyboards */
std::vector<DeadKey*> create_deadkeys_by_basechar();
/** @brief filter entries for the currently used Linux Keyboard out of a vector of all existing deadKey combinations */
void refine_alDead(KMX_WCHAR dk, std::vector<DeadKey*>& dkVec, std::vector<DeadKey*>& r_All_Vec);
/** @brief check whether a deadkey already exists in the deadkey vector */
bool found_dk_inVector(KMX_WCHAR dk, std::vector<DeadKey*>& dkVec);
/** @brief find all deadkey combinations for a certain deadkey in a vector of all deadkey combinations */
bool query_dk_combinations_for_specific_dk(vec_dword_2D& r_dk_ComposeTable, KMX_DWORD dk, vec_dword_2D& dk_SingleTable);
/** @brief convert a character to the upper-case equivalent and find the corresponding shiftstate of the entered keyval */
KMX_DWORD KMX_change_keyname_to_capital(KMX_DWORD kVal, KMX_DWORD& shift, GdkKeymap* keymap);
/** @brief append a 1D-vector containing name, base character and unicode_value to a 2D-Vector */
void add_deadkey_combination(vec_dword_2D& dk_ComposeTable, std::string diacritic_name, std::string base_char, KMX_DWORD unicode_value);
/** @brief create a 2D-Vector containing all possible combinations of deadkey + character for all Linux keyboards */
void create_DKTable(vec_dword_2D& dk_ComposeTable);
#endif /*DEADKEY_H*/

1485
linux/mcompile/keymap/keymap.cpp Normal file
View file

File diff suppressed because it is too large Load diff

158
linux/mcompile/keymap/keymap.h Normal file
View file

@ -0,0 +1,158 @@
#pragma once
#ifndef KEYMAP_H
#define KEYMAP_H
#include <X11/XKBlib.h>
#include <X11/Xlib.h>
#include <gdk/gdk.h>
#include <map>
#include <algorithm>
#include <fstream>
#include <iomanip>
#include <iostream>
#include <string>
#include <vector>
#include "km_u16.h"
#include <cassert>
enum ShiftState {
Base = 0, // 0
Shft = 1, // 1
Ctrl = 2, // 2
ShftCtrl = Shft | Ctrl, // 3
Menu = 4, // 4 -- NOT USED
ShftMenu = Shft | Menu, // 5 -- NOT USED
MenuCtrl = Menu | Ctrl, // 6
ShftMenuCtrl = Shft | Menu | Ctrl, // 7
Xxxx = 8, // 8
ShftXxxx = Shft | Xxxx, // 9
};
#define VK_SPACE 0x20
#define VK_COLON 0xBA
#define VK_EQUAL 0xBB
#define VK_COMMA 0xBC
#define VK_HYPHEN 0xBD
#define VK_PERIOD 0xBE
#define VK_SLASH 0xBF
#define VK_ACCENT 0xC0
#define VK_LBRKT 0xDB
#define VK_BKSLASH 0xDC
#define VK_RBRKT 0xDD
#define VK_QUOTE 0xDE
#define VK_xDF 0xDF
#define VK_OEM_102 0xE2 // "<>" or "\|" on RT 102-key kbd.
#define VK_DIVIDE 0x6F
#define VK_CANCEL 3
#define VK_DECIMAL 0x2E
#define VK_OEM_CLEAR 0xFE
#define VK_LSHIFT 0xA0
#define VK_RSHIFT 0xA1
#define VK_LCONTROL 0xA2
#define VK_RCONTROL 0xA3
#define VK_LMENU 0xA4
#define VK_RMENU 0xA5
#define VK_SHIFT 0x10
#define VK_CONTROL 0x11
#define VK_MENU 0x12
#define VK_PAUSE 0x13
#define VK_CAPITAL 0x14
typedef std::vector<std::string> vec_string_1D;
typedef std::vector<KMX_DWORD> vec_dword_1D;
typedef std::vector<std::vector<KMX_DWORD> > vec_dword_2D;
typedef std::vector<std::vector<std::vector<KMX_DWORD> > > vec_dword_3D;
extern const KMX_DWORD INVALID_NAME;
extern const gint keycode_max;
extern const KMX_DWORD deadkey_min;
extern const KMX_DWORD deadkey_max;
/** @brief Map of all US English virtual key codes that we can translate */
extern const KMX_DWORD KMX_VKMap[];
/** @brief array of USVirtualKey-ScanCode-pairs */
extern const KMX_DWORD USVirtualKeyToScanCode[256];
/** @brief array of ScanCode-USVirtualKey-pairs */
extern const KMX_DWORD ScanCodeToUSVirtualKey[128];
/** @brief check if current machine uses little endian
* @return true if little endian is used;
* else false */
inline bool isLittleEndianSystem() {
char16_t test = 0x0102;
return (reinterpret_cast<char*>(&test))[0] == 0x02;
}
/** @brief map a shiftstate used on windows to a shiftstate suitable for gdk_keymap_translate_keyboard_state() on Linux */
int convert_Shiftstate_to_LinuxShiftstate(int shiftState);
/** @brief map a shiftstate used for rgkey to a shiftstate suitable for gdk_keymap_translate_keyboard_state() on Linux */
int convert_rgkey_Shiftstate_to_LinuxShiftstate(ShiftState shiftState);
/** @brief check for correct input parameter that will later be used in gdk_keymap_translate_keyboard_state() */
bool ensureValidInputForKeyboardTranslation(int shiftstate, gint keycode);
/** @brief convert names of keys stated in a symbol file to a keyvalue */
KMX_DWORD convertNamesTo_DWORD_Value(std::string tok_str);
/** @brief create a 3D-Vector containing data of the US keyboard and the currently used (underlying) keyboard */
int createOneVectorFromBothKeyboards(vec_dword_3D& all_vector, GdkKeymap* keymap);
/** @brief write data of the US keyboard into a 3D-Vector which later will contain data of the US keyboard and the currently used (underlying) keyboard */
int write_US_ToVector(vec_dword_3D& vec_us);
/** @brief create a 1D-Vector containing all relevant entries of the symbol file us basic */
bool createCompleteVector_US(vec_string_1D& complete_List);
/** @brief convert the key name obtained from symbol file to the matching keycode */
int get_keycode_from_keyname(std::string key_name);
/** @brief process each element of a 1D-Vector, split and write to a 3D-Vector */
int split_US_To_3D_Vector(vec_dword_3D& all_US, vec_string_1D completeList);
/** @brief create an 2D-Vector with all fields containing INVALID_NAME */
vec_dword_2D create_empty_2D_Vector(int dim_rows, int dim_ss);
/** @brief append a 2D-vector containing data of the currently used (underlying) keyboard to the 3D-vector */
int append_underlying_ToVector(vec_dword_3D& all_vector, GdkKeymap* keymap);
/** @brief create a pointer to pointer of the current keymap for later use */
bool InitializeGDK(GdkKeymap** keymap, int argc, gchar* argv[]);
/** @brief check if keyval correponds to a character used in mcompile */
bool IsKeymanUsedChar(int kv);
/** @brief convert a deadkey-value to a u16string if it is in the range of deadkeys used for mcompile */
std::u16string convert_DeadkeyValues_To_U16str(KMX_DWORD in);
/** @brief return the keyvalue for a given Keycode, shiftstate and caps of the currently used (underlying) keyboard layout. */
KMX_DWORD KMX_get_KeyVal_From_KeyCode(GdkKeymap* keymap, guint keycode, ShiftState ss, int caps);
/** @brief return the keyvalue for a given Keycode and shiftstate of the currently used (underlying) keyboard layout. */
KMX_DWORD KMX_get_KeyValUnderlying_From_KeyCodeUnderlying(GdkKeymap* keymap, guint keycode, int shiftState);
/** @brief return the keyvalue for a given Keycode and shiftstate of the currently used (underlying) keyboard layout. */
KMX_DWORD KMX_get_KeyValUnderlying_From_KeyCodeUnderlying(GdkKeymap* keymap, guint keycode, KMX_DWORD shiftState, PKMX_WCHAR deadkey);
/** @brief return the keyvalue of a key of the the currently used (underlying) keyboard for a given keyvalue of the US keyboard */
KMX_DWORD KMX_get_KeyValUnderlying_From_KeyValUS(vec_dword_3D& all_vector, KMX_DWORD kv_us);
/** @brief return the keycode of the currently used (underlying) keyboard for a given keycode of the US keyboard */
KMX_DWORD KMX_get_KeyCodeUnderlying_From_KeyCodeUS(GdkKeymap* keymap, vec_dword_3D& all_vector, KMX_DWORD kc_us, ShiftState ss, int caps);
/** @brief return the keycode of the currently used (underlying) keyboard for a given virtual key of the US keyboard */
KMX_DWORD KMX_get_KeyCodeUnderlying_From_VKUS(KMX_DWORD virtualKeyUS);
/** @brief return a virtual key of the US keyboard for a given keycode of the currently used (underlying) keyboard */
KMX_DWORD KMX_get_VKUS_From_KeyCodeUnderlying(KMX_DWORD keycode);
/** @brief convert a codepoint to a u16string */
std::u16string CodePointToU16String(unsigned int codepoint);
#endif /*KEYMAP_H*/

669
linux/mcompile/keymap/mc_import_rules.cpp Normal file
View file

@ -0,0 +1,669 @@
/*
* Keyman is copyright (C) 2004 - 2024 SIL International. MIT License.
*
* Mnemonic layout support for Linux
*/
#include <vector>
#include <string>
#include <stdio.h>
#include "mc_kmxfile.h"
#include "keymap.h"
const int KMX_ShiftStateMap[] = {
ISVIRTUALKEY,
ISVIRTUALKEY | K_SHIFTFLAG,
ISVIRTUALKEY | K_CTRLFLAG,
ISVIRTUALKEY | K_SHIFTFLAG | K_CTRLFLAG,
0,
0,
ISVIRTUALKEY | RALTFLAG,
ISVIRTUALKEY | RALTFLAG | K_SHIFTFLAG,
0,
0};
/**
* @brief Constructor
* @param deadCharacter a deadkey
*/
DeadKey::DeadKey(KMX_WCHAR deadCharacter) {
this->m_deadchar = deadCharacter;
}
/**
* @brief return dead character
* @return deadkey character
*/
KMX_WCHAR DeadKey::KMX_DeadCharacter() {
return this->m_deadchar;
}
/**
* @brief set Deadkey with values
* @param baseCharacter the base character
* @param combinedCharacter the combined character
*/
void DeadKey::KMX_AddDeadKeyRow(KMX_WCHAR baseCharacter, KMX_WCHAR combinedCharacter) {
this->m_rgbasechar.push_back(baseCharacter);
this->m_rgcombchar.push_back(combinedCharacter);
}
/**
* @brief check if character exists in DeadKey
* @param baseCharacter a character to be found
* @return true if found;
* false if not found
*/
bool DeadKey::KMX_ContainsBaseCharacter(KMX_WCHAR baseCharacter) {
std::vector<KMX_WCHAR>::iterator it;
for (it = this->m_rgbasechar.begin(); it < m_rgbasechar.end(); it++) {
if (*it == baseCharacter) {
return true;
}
}
return false;
}
/**
* @brief Find a keyvalue for given keycode, shiftstate and caps. A function similar to Window`s ToUnicodeEx() function.
*
* Contrary to what the function name might suggest, the function KMX_ToUnicodeEx does not process surrogate pairs.
* This is because it is used in mcompile only which only deals with latin scripts.
* In case this function should be used for surrogate pairs, they will be ignored and a message will be printed out
*
* @param keycode a key of the currently used keyboard Layout
* @param pwszBuff Buffer to store resulting character
* @param ss a shiftstate of the currently used keyboard Layout
* @param caps state of the caps key of the currently used keyboard Layout
* @param keymap the currently used (underlying)keyboard Layout
* @return -1 if a deadkey was found;
* 0 if no translation is available;
* +1 if character was found and written to pwszBuff
*/
int KMX_ToUnicodeEx(guint keycode, PKMX_WCHAR pwszBuff, ShiftState rgkey_ss, int caps, GdkKeymap* keymap) {
GdkKeymapKey* maps;
guint* keyvals;
gint count;
if (!gdk_keymap_get_entries_for_keycode(keymap, keycode, &maps, &keyvals, &count))
return 0;
if (!(ensureValidInputForKeyboardTranslation(convert_rgkey_Shiftstate_to_LinuxShiftstate(rgkey_ss), keycode))){
g_free(keyvals);
g_free(maps);
return 0;
}
KMX_DWORD keyVal = (KMX_DWORD)KMX_get_KeyVal_From_KeyCode(keymap, keycode, rgkey_ss, caps);
std::u16string str = convert_DeadkeyValues_To_U16str(keyVal);
KMX_WCHAR firstchar = *(PKMX_WCHAR)str.c_str();
if ((firstchar >= 0xD800) &&(firstchar <= 0xDFFF)) {
wprintf(L"Surrogate pair found that is not processed in KMX_ToUnicodeEx\n");
return 0;
}
pwszBuff[0] = firstchar;
g_free(keyvals);
g_free(maps);
if (u16len(pwszBuff) < 1)
return 0;
if ((keyVal >= deadkey_min) && (keyVal <= deadkey_max)) // deadkeys
return -1;
else if (gdk_keyval_to_unicode(keyVal) == 0) // NO UNICODE
return 0;
else // usable char
return 1;
}
KMX_WCHAR KMX_DeadKeyMap(int index, std::vector<DeadKey*>* deadkeys, int deadkeyBase, std::vector<KMX_DeadkeyMapping>* deadkeyMappings) { // I4327 // I4353
for (size_t i = 0; i < deadkeyMappings->size(); i++) {
if ((*deadkeyMappings)[i].deadkey == index) {
return (*deadkeyMappings)[i].dkid;
}
}
for (size_t i = 0; i < deadkeys->size(); i++) {
if ((*deadkeys)[i]->KMX_DeadCharacter() == index) {
return (KMX_WCHAR)(deadkeyBase + i);
}
}
return 0xFFFF;
}
/**
* @brief Base class for dealing with rgkey
*/
class KMX_VirtualKey {
private:
KMX_DWORD m_vk;
KMX_DWORD m_sc;
bool m_rgfDeadKey[10][2];
std::u16string m_rgss[10][2];
public:
KMX_VirtualKey(KMX_DWORD scanCode) {
this->m_vk = KMX_get_VKUS_From_KeyCodeUnderlying(scanCode);
this->m_sc = scanCode;
memset(this->m_rgfDeadKey, 0, sizeof(this->m_rgfDeadKey));
}
/**
* @brief return member variable virtual key
*/
KMX_DWORD VK() {
return this->m_vk;
}
/**
* @brief return member variable scancode
*/
KMX_DWORD SC() {
return this->m_sc;
}
std::u16string KMX_GetShiftState(ShiftState shiftState, bool capsLock) {
return this->m_rgss[(KMX_DWORD)shiftState][(capsLock ? 1 : 0)];
}
void KMX_SetShiftState(ShiftState shiftState, std::u16string value, bool isDeadKey, bool capsLock) {
this->m_rgfDeadKey[(KMX_DWORD)shiftState][(capsLock ? 1 : 0)] = isDeadKey;
this->m_rgss[(KMX_DWORD)shiftState][(capsLock ? 1 : 0)] = value;
}
bool KMX_IsSGCAPS() {
std::u16string stBase = this->KMX_GetShiftState(Base, false);
std::u16string stShift = this->KMX_GetShiftState(Shft, false);
std::u16string stCaps = this->KMX_GetShiftState(Base, true);
std::u16string stShiftCaps = this->KMX_GetShiftState(Shft, true);
return (
((stCaps.size() > 0) &&
(stBase.compare(stCaps) != 0) &&
(stShift.compare(stCaps) != 0)) ||
((stShiftCaps.size() > 0) &&
(stBase.compare(stShiftCaps) != 0) &&
(stShift.compare(stShiftCaps) != 0)));
}
bool KMX_IsCapsEqualToShift() {
std::u16string stBase = this->KMX_GetShiftState(Base, false);
std::u16string stShift = this->KMX_GetShiftState(Shft, false);
std::u16string stCaps = this->KMX_GetShiftState(Base, true);
return (
(stBase.size() > 0) &&
(stShift.size() > 0) &&
(stBase.compare(stShift) != 0) &&
(stShift.compare(stCaps) == 0));
}
bool KMX_IsAltGrCapsEqualToAltGrShift() {
std::u16string stBase = this->KMX_GetShiftState(MenuCtrl, false);
std::u16string stShift = this->KMX_GetShiftState(ShftMenuCtrl, false);
std::u16string stCaps = this->KMX_GetShiftState(MenuCtrl, true);
return (
(stBase.size() > 0) &&
(stShift.size() > 0) &&
(stBase.compare(stShift) != 0) &&
(stShift.compare(stCaps) == 0));
}
bool KMX_IsXxxxGrCapsEqualToXxxxShift() {
std::u16string stBase = this->KMX_GetShiftState(Xxxx, false);
std::u16string stShift = this->KMX_GetShiftState(ShftXxxx, false);
std::u16string stCaps = this->KMX_GetShiftState(Xxxx, true);
return (
(stBase.size() > 0) &&
(stShift.size() > 0) &&
(stBase.compare(stShift) != 0) &&
(stShift.compare(stCaps) == 0));
}
bool KMX_IsEmpty() {
for (int i = 0; i < 10; i++) {
for (int j = 0; j <= 1; j++) {
if (this->KMX_GetShiftState((ShiftState)i, (j == 1)).size() > 0) {
return (false);
}
}
}
return true;
}
/**
* @brief check if we use only keys used in mcompile
*/
bool KMX_IsKeymanUsedKey() {
return (this->m_vk >= 0x20 && this->m_vk <= 0x5F) || (this->m_vk >= 0x88);
}
KMX_DWORD KMX_GetShiftStateValue(int capslock, int caps, ShiftState ss) {
return KMX_ShiftStateMap[(int)ss] | (capslock ? (caps ? CAPITALFLAG : NOTCAPITALFLAG) : 0);
}
/**
* @brief count the number of keys
*/
int KMX_GetKeyCount(int MaxShiftState) {
int nkeys = 0;
// Get the CAPSLOCK value
for (int ss = 0; ss <= MaxShiftState; ss++) {
if (ss == Menu || ss == ShftMenu) {
// Alt and Shift+Alt don't work, so skip them
continue;
}
for (int caps = 0; caps <= 1; caps++) {
std::u16string st = this->KMX_GetShiftState((ShiftState)ss, (caps == 1));
// ctrl and shift+ctrl will be skipped since rgkey has no entries in m_rgss[2] m_rgss[3]
if (st.size() == 0) {
// No character assigned here
} else if (this->m_rgfDeadKey[(int)ss][caps]) {
// It's a dead key, append an @ sign.
nkeys++;
} else {
bool isvalid = true;
for (size_t ich = 0; ich < st.size(); ich++) {
if (st[ich] < 0x20 || st[ich] == 0x7F) {
isvalid = false;
printf("invalid for: %i\n", st[ich]);
break;
}
}
if (isvalid) {
nkeys++;
}
}
}
}
return nkeys;
}
bool KMX_LayoutRow(int MaxShiftState, LPKMX_KEY key, std::vector<DeadKey*>* deadkeys, int deadkeyBase, bool bDeadkeyConversion, vec_dword_3D& all_vector, GdkKeymap* keymap) { // I4552
// Get the CAPSLOCK value
/*int capslock =
(this->KMX_IsCapsEqualToShift() ? 1 : 0) |
(this->KMX_IsSGCAPS() ? 2 : 0) |
(this->KMX_IsAltGrCapsEqualToAltGrShift() ? 4 : 0) |
(this->KMX_IsXxxxGrCapsEqualToXxxxShift() ? 8 : 0);*/
int capslock = 1; // we do not use the equation to obtain capslock. On Linux we set capslock = 1
for (int ss = 0; ss <= MaxShiftState; ss++) {
if (ss == Menu || ss == ShftMenu) {
// Alt and Shift+Alt don't work, so skip them
continue;
}
for (int caps = 0; caps <= 1; caps++) {
std::u16string st = this->KMX_GetShiftState((ShiftState)ss, (caps == 1));
PKMX_WCHAR p;
if (st.size() == 0) {
// No character assigned here
} else if (this->m_rgfDeadKey[(int)ss][caps]) {
// It's a dead key, append an @ sign.
key->dpContext = new KMX_WCHAR[1];
*key->dpContext = 0;
key->ShiftFlags = this->KMX_GetShiftStateValue(capslock, caps, (ShiftState)ss);
// we already use VK_US so no need to convert it as we do on Windows
key->Key = this->VK();
key->Line = 0;
if (bDeadkeyConversion) { // I4552
p = key->dpOutput = new KMX_WCHAR[2];
*p++ = st[0];
*p = 0;
} else {
p = key->dpOutput = new KMX_WCHAR[4];
*p++ = UC_SENTINEL;
*p++ = CODE_DEADKEY;
*p++ = KMX_DeadKeyMap(st[0], deadkeys, deadkeyBase, &KMX_FDeadkeys); // I4353
*p = 0;
}
key++;
} else {
bool isvalid = true;
for (size_t ich = 0; ich < st.size(); ich++) {
if (st[ich] < 0x20 || st[ich] == 0x7F) {
isvalid = false;
printf("invalid 16 for: %i\n", st[ich]);
break;
}
}
if (isvalid) {
/*
* this is different to mcompile Windows !!!!
* this->m_sc stores SC-US = SCUnderlying
* this->m_vk stores VK-US ( not VK underlying !!)
* key->Key stores VK-US ( not VK underlying !!)
* key->dpOutput stores character Underlying
*/
KMX_DWORD sc_underlying = KMX_get_KeyCodeUnderlying_From_KeyCodeUS(keymap, all_vector, this->SC(), (ShiftState)ss, caps);
key->Key = KMX_get_VKUS_From_KeyCodeUnderlying(sc_underlying);
key->Line = 0;
key->ShiftFlags = this->KMX_GetShiftStateValue(capslock, caps, (ShiftState)ss);
key->dpContext = new KMX_WCHAR;
*key->dpContext = 0;
p = key->dpOutput = new KMX_WCHAR[st.size() + 1];
for (size_t ich = 0; ich < st.size(); ich++) {
*p++ = st[ich];
}
*p = 0;
key++;
}
}
}
}
return true;
}
};
/**
* @brief Base class for KMX_loader
*/
class KMX_Loader {
private:
KMX_BYTE lpKeyStateNull[256];
KMX_DWORD m_XxxxVk;
public:
KMX_Loader() {
m_XxxxVk = 0;
memset(lpKeyStateNull, 0, sizeof(lpKeyStateNull));
}
KMX_DWORD Get_XxxxVk() {
return m_XxxxVk;
}
void Set_XxxxVk(KMX_DWORD value) {
m_XxxxVk = value;
}
ShiftState KMX_MaxShiftState() {
return (Get_XxxxVk() == 0 ? ShftMenuCtrl : ShftXxxx);
}
bool KMX_IsControlChar(char16_t ch) {
return (ch < 0x0020) || (ch >= 0x007F && ch <= 0x009F);
}
};
/**
* @brief find the maximum index of a deadkey
@param p pointer to deadkey
* @return index of deadkey
*/
int KMX_GetMaxDeadkeyIndex(KMX_WCHAR* p) {
int n = 0;
while (p && *p) {
if (*p == UC_SENTINEL && *(p + 1) == CODE_DEADKEY)
n = std::max(n, (int)*(p + 2));
p = KMX_incxstr(p);
}
return n;
}
/**
* @brief Collect the key data, translate it to kmx and append to the existing keyboard
* It is important to understand that this function has different sorting order in rgkey compared to mcompile-windows!
* On Windows the values of rgkey are sorted according to the VK of the underlying keyboard
* On Linux the values of rgkey are sorted according to the VK of the the US keyboard
* Since Linux Keyboards do not use a VK mcompile uses the VK of the the US keyboard because
* these are available in mcompile through USVirtualKeyToScanCode/ScanCodeToUSVirtualKey and an offset of 8
* @param kp pointer to keyboard
* @param all_vector vector that holds the data of the US keyboard and the currently used (underlying) keyboard
* @param keymap the currently used (underlying)keyboard Layout
* @param FDeadkeys vector of all deadkeys for the currently used (underlying)keyboard Layout
* @param bDeadkeyConversion 1 to convert a deadkey to a character; 0 no conversion
* @return true in case of success
*/
bool KMX_ImportRules(LPKMX_KEYBOARD kp, vec_dword_3D& all_vector, GdkKeymap** keymap, std::vector<KMX_DeadkeyMapping>* FDeadkeys, KMX_BOOL bDeadkeyConversion) { // I4353 // I4552
KMX_Loader loader;
std::vector<KMX_VirtualKey*> rgKey; //= new VirtualKey[256];
std::vector<DeadKey*> alDead;
std::vector<DeadKey*> alDead_byBasechar = create_deadkeys_by_basechar();
rgKey.resize(256);
// Scroll through the Scan Code (SC) values and get the valid Virtual Key (VK)
// values in it. Then, store the SC in each valid VK so it can act as both a
// flag that the VK is valid, and it can store the SC value.
// Windows and Linux Keycodes start with 1; Mac keycodes start with 0
for (KMX_DWORD sc = 0x01; sc <= 0x7f; sc++) {
/* HERE IS A BIG DIFFERENCE COMPARED TO MCOMPILE FOR WINDOWS:
* mcompile on Windows fills rgkey.m_vk with the VK of the Underlying keyboard
* mcompile for Linux fills rgkey.m_vk with the VK of the US keyboard
* this results in a different sorting order in rgkey[] !
* Linux cannot get a VK for the underling Keyboard since this does not exist
* Linux can only get a VK for the US Keyboard (by using USVirtualKeyToScanCode/ScanCodeToUSVirtualKey)
* therefore we use VK_US in rgkey[ ] which we get from all_vector
*/
KMX_VirtualKey* key = new KMX_VirtualKey(sc);
if ((key->VK() != 0)) {
rgKey[key->VK()] = key;
} else {
delete key;
}
}
// in this part we skip shiftstates 4, 5, 8, 9
for (KMX_DWORD iKey = 0; iKey < rgKey.size(); iKey++) {
if (rgKey[iKey] != NULL) {
KMX_WCHAR sbBuffer[256]; // Scratchpad we use many places
for (ShiftState ss = Base; ss <= loader.KMX_MaxShiftState(); ss = (ShiftState)((int)ss + 1)) {
if (ss == Menu || ss == ShftMenu) {
// Alt and Shift+Alt don't work, so skip them (ss 4+5)
continue;
}
KMX_DWORD kc_underlying = KMX_get_KeyCodeUnderlying_From_VKUS(iKey);
for (int caps = 0; caps <= 1; caps++) {
int rc = KMX_ToUnicodeEx(kc_underlying, sbBuffer, ss, caps, *keymap);
if (rc > 0) {
if (*sbBuffer == 0) {
rgKey[iKey]->KMX_SetShiftState(ss, u"", false, (caps)); // different to windows since behavior on Linux is different (see above)
} else {
if ((ss == Ctrl || ss == ShftCtrl)) {
continue;
}
sbBuffer[rc] = 0;
rgKey[iKey]->KMX_SetShiftState(ss, sbBuffer, false, (caps)); // different to windows since behavior on Linux is different (see above)
}
} else if (rc < 0) {
sbBuffer[2] = 0;
rgKey[iKey]->KMX_SetShiftState(ss, sbBuffer, true, (caps)); // different to windows since behavior on Linux is different (see above)
refine_alDead(sbBuffer[0], alDead, alDead_byBasechar);
}
}
}
}
}
//-------------------------------------------------------------
// Now that we've collected the key data, we need to
// translate it to kmx and append to the existing keyboard
//-------------------------------------------------------------
int nDeadkey = 0;
LPKMX_GROUP gp = new KMX_GROUP[kp->cxGroupArray + 4]; // leave space for old
memcpy(gp, kp->dpGroupArray, sizeof(KMX_GROUP) * kp->cxGroupArray);
//
// Find the current highest deadkey index
//
kp->dpGroupArray = gp;
for (KMX_DWORD i = 0; i < kp->cxGroupArray; i++, gp++) {
LPKMX_KEY kkp = gp->dpKeyArray;
for (KMX_DWORD j = 0; j < gp->cxKeyArray; j++, kkp++) {
nDeadkey = std::max(nDeadkey, KMX_GetMaxDeadkeyIndex(kkp->dpContext));
nDeadkey = std::max(nDeadkey, KMX_GetMaxDeadkeyIndex(kkp->dpOutput));
}
}
kp->cxGroupArray++;
gp = &kp->dpGroupArray[kp->cxGroupArray - 1];
// calculate the required size of `gp->dpKeyArray`
KMX_DWORD nkeys = 0;
for (KMX_DWORD iKey = 0; iKey < rgKey.size(); iKey++) {
if ((rgKey[iKey] != NULL) && rgKey[iKey]->KMX_IsKeymanUsedKey() && (!rgKey[iKey]->KMX_IsEmpty())) {
nkeys += rgKey[iKey]->KMX_GetKeyCount(loader.KMX_MaxShiftState());
}
}
gp->fUsingKeys = TRUE;
gp->dpMatch = NULL;
gp->dpName = NULL;
gp->dpNoMatch = NULL;
gp->cxKeyArray = nkeys;
gp->dpKeyArray = new KMX_KEY[gp->cxKeyArray];
nDeadkey++; // ensure a 1-based index above the max deadkey value already in the keyboard
//
// Fill in the new rules
//
nkeys = 0;
for (KMX_DWORD iKey = 0; iKey < rgKey.size(); iKey++) {
if ((rgKey[iKey] != NULL) && rgKey[iKey]->KMX_IsKeymanUsedKey() && (!rgKey[iKey]->KMX_IsEmpty())) {
if (rgKey[iKey]->KMX_LayoutRow(loader.KMX_MaxShiftState(), &gp->dpKeyArray[nkeys], &alDead, nDeadkey, bDeadkeyConversion, all_vector, *keymap)) { // I4552
nkeys += rgKey[iKey]->KMX_GetKeyCount(loader.KMX_MaxShiftState());
}
}
}
gp->cxKeyArray = nkeys;
//
// Add nomatch control to each terminating 'using keys' group // I4550
//
LPKMX_GROUP gp2 = kp->dpGroupArray;
for (KMX_DWORD i = 0; i < kp->cxGroupArray - 1; i++, gp2++) {
if (gp2->fUsingKeys && gp2->dpNoMatch == NULL) {
KMX_WCHAR* p = gp2->dpNoMatch = new KMX_WCHAR[4];
*p++ = UC_SENTINEL;
*p++ = CODE_USE;
*p++ = (KMX_WCHAR)(kp->cxGroupArray);
*p = 0;
// I4550 - Each place we have a nomatch > use(baselayout) (this last group), we need to add all
// the AltGr and ShiftAltGr combinations as rules to allow them to be matched as well. Yes, this
// loop is not very efficient but it's not worthy of optimisation.
//
KMX_DWORD j;
LPKMX_KEY kkp;
for (j = 0, kkp = gp->dpKeyArray; j < gp->cxKeyArray; j++, kkp++) {
if ((kkp->ShiftFlags & (K_CTRLFLAG | K_ALTFLAG | LCTRLFLAG | LALTFLAG | RCTRLFLAG | RALTFLAG)) != 0) {
gp2->cxKeyArray++;
LPKMX_KEY kkp2 = new KMX_KEY[gp2->cxKeyArray];
memcpy(kkp2, gp2->dpKeyArray, sizeof(KMX_KEY) * (gp2->cxKeyArray - 1));
gp2->dpKeyArray = kkp2;
kkp2 = &kkp2[gp2->cxKeyArray - 1];
kkp2->dpContext = new KMX_WCHAR;
*kkp2->dpContext = 0;
kkp2->Key = kkp->Key;
kkp2->ShiftFlags = kkp->ShiftFlags;
kkp2->Line = 0;
KMX_WCHAR* p = kkp2->dpOutput = new KMX_WCHAR[4];
*p++ = UC_SENTINEL;
*p++ = CODE_USE;
*p++ = (KMX_WCHAR)(kp->cxGroupArray);
*p = 0;
}
}
}
}
// If we have deadkeys, then add a new group to translate the deadkeys per the deadkey tables
// We only do this if not in deadkey conversion mode
//
if (alDead.size() > 0 && !bDeadkeyConversion) { // I4552
kp->cxGroupArray++;
KMX_WCHAR* p = gp->dpMatch = new KMX_WCHAR[4];
*p++ = UC_SENTINEL;
*p++ = CODE_USE;
*p++ = (KMX_WCHAR)kp->cxGroupArray;
*p = 0;
gp++;
gp->fUsingKeys = FALSE;
gp->dpMatch = NULL;
gp->dpName = NULL;
gp->dpNoMatch = NULL;
gp->cxKeyArray = alDead.size();
LPKMX_KEY kkp = gp->dpKeyArray = new KMX_KEY[alDead.size()];
LPKMX_STORE sp = new KMX_STORE[kp->cxStoreArray + alDead.size() * 2];
memcpy(sp, kp->dpStoreArray, sizeof(KMX_STORE) * kp->cxStoreArray);
kp->dpStoreArray = sp;
sp = &sp[kp->cxStoreArray];
int nStoreBase = kp->cxStoreArray;
kp->cxStoreArray += alDead.size() * 2;
for (KMX_DWORD i = 0; i < alDead.size(); i++) {
DeadKey* dk = alDead[i];
sp->dpName = NULL;
sp->dwSystemID = 0;
sp->dpString = new KMX_WCHAR[dk->KMX_Count() + 1];
for (int j = 0; j < dk->KMX_Count(); j++)
sp->dpString[j] = dk->KMX_GetBaseCharacter(j);
sp->dpString[dk->KMX_Count()] = 0;
sp++;
sp->dpName = NULL;
sp->dwSystemID = 0;
sp->dpString = new KMX_WCHAR[dk->KMX_Count() + 1];
for (int j = 0; j < dk->KMX_Count(); j++)
sp->dpString[j] = dk->KMX_GetCombinedCharacter(j);
sp->dpString[dk->KMX_Count()] = 0;
sp++;
kkp->Line = 0;
kkp->ShiftFlags = 0;
kkp->Key = 0;
KMX_WCHAR* p = kkp->dpContext = new KMX_WCHAR[8];
*p++ = UC_SENTINEL;
*p++ = CODE_DEADKEY;
*p++ = KMX_DeadKeyMap(dk->KMX_DeadCharacter(), &alDead, nDeadkey, FDeadkeys); // I4353
// *p++ = nDeadkey+i;
*p++ = UC_SENTINEL;
*p++ = CODE_ANY;
*p++ = nStoreBase + i * 2 + 1;
*p = 0;
p = kkp->dpOutput = new KMX_WCHAR[5];
*p++ = UC_SENTINEL;
*p++ = CODE_INDEX;
*p++ = nStoreBase + i * 2 + 2;
*p++ = 2;
*p = 0;
kkp++;
}
}
return true;
}

43
linux/mcompile/keymap/mc_import_rules.h Normal file
View file

@ -0,0 +1,43 @@
#pragma once
#ifndef MC_IMPORT_RULES_H
#define MC_IMPORT_RULES_H
/** @brief Base class for Deadkey*/
class DeadKey {
private:
KMX_WCHAR m_deadchar;
std::vector<KMX_WCHAR> m_rgbasechar;
std::vector<KMX_WCHAR> m_rgcombchar;
public:
/** @brief Constructor */
DeadKey(KMX_WCHAR deadCharacter);
/** @brief return dead character */
KMX_WCHAR KMX_DeadCharacter();
/** @brief set Deadkey with values */
void KMX_AddDeadKeyRow(KMX_WCHAR baseCharacter, KMX_WCHAR combinedCharacter);
int KMX_Count() {
return this->m_rgbasechar.size();
}
KMX_WCHAR KMX_GetDeadCharacter() {
return this->m_deadchar;
}
KMX_WCHAR KMX_GetBaseCharacter(int index) {
return this->m_rgbasechar[index];
}
KMX_WCHAR KMX_GetCombinedCharacter(int index) {
return this->m_rgcombchar[index];
}
/** @brief check if character exists in DeadKey */
bool KMX_ContainsBaseCharacter(KMX_WCHAR baseCharacter);
};
#endif /*MC_IMPORT_RULES_H*/

584
linux/mcompile/keymap/mc_kmxfile.cpp Normal file
View file

@ -0,0 +1,584 @@
/*
* Keyman is copyright (C) 2004 - 2024 SIL International. MIT License.
*
* Mnemonic layout support for Linux
*/
#include "mc_kmxfile.h"
#include <typeinfo>
#define CERR_None 0x00000000
#define CERR_CannotAllocateMemory 0x00008004
#define CERR_UnableToWriteFully 0x00008007
#define CERR_SomewhereIGotItWrong 0x00008009
const int CODE__SIZE[] = {
-1, // undefined 0x00
1, // CODE_ANY 0x01
2, // CODE_INDEX 0x02
0, // CODE_CONTEXT 0x03
0, // CODE_NUL 0x04
1, // CODE_USE 0x05
0, // CODE_RETURN 0x06
0, // CODE_BEEP 0x07
1, // CODE_DEADKEY 0x08
-1, // unused 0x09
2, // CODE_EXTENDED 0x0A
-1, // CODE_EXTENDEDEND 0x0B (unused)
1, // CODE_SWITCH 0x0C
-1, // CODE_KEY 0x0D (never used)
0, // CODE_CLEARCONTEXT 0x0E
1, // CODE_CALL 0x0F
-1, // UC_SENTINEL_EXTENDEDEND 0x10 (not valid with UC_SENTINEL)
1, // CODE_CONTEXTEX 0x11
1, // CODE_NOTANY 0x12
2, // CODE_SETOPT 0x13
3, // CODE_IFOPT 0x14
1, // CODE_SAVEOPT 0x15
1, // CODE_RESETOPT 0x16
3, // CODE_IFSYSTEMSTORE 0x17
2 // CODE_SETSYSTEMSTORE 0x18
};
/** @brief check if the file has correct version */
KMX_BOOL KMX_VerifyKeyboard(PKMX_BYTE filebase, KMX_DWORD file_size);
/** @brief Fixup the keyboard by expanding pointers. */
LPKMX_KEYBOARD KMX_FixupKeyboard(PKMX_BYTE bufp, PKMX_BYTE base, KMX_DWORD dwFileSize);
/**
* @brief Save a Keyboard to a file
* @param fk pointer to the keyboard
* @param hOutfile pointer to the output file
* @param FSaveDebug
* @return an Error in case of failure
*/
KMX_DWORD KMX_WriteCompiledKeyboardToFile(LPKMX_KEYBOARD fk, FILE* hOutfile, KMX_BOOL FSaveDebug) {
LPKMX_GROUP fgp;
LPKMX_STORE fsp;
LPKMX_KEY fkp;
PCOMP_KEYBOARD ck;
PCOMP_GROUP gp;
PCOMP_STORE sp;
PCOMP_KEY kp;
PKMX_BYTE buf;
KMX_DWORD size, offset;
KMX_DWORD i, j;
// Calculate how much memory to allocate
size = sizeof(COMP_KEYBOARD) +
fk->cxGroupArray * sizeof(COMP_GROUP) +
fk->cxStoreArray * sizeof(COMP_STORE) +
// wcslen(fk->szName)*2 + 2 +
// wcslen(fk->szCopyright)*2 + 2 +
// wcslen(fk->szLanguageName)*2 + 2 +
// wcslen(fk->szMessage)*2 + 2 +
fk->dwBitmapSize;
for (i = 0, fgp = fk->dpGroupArray; i < fk->cxGroupArray; i++, fgp++) {
if (fgp->dpName)
size += (u16len(fgp->dpName) + 1) * sizeof(KMX_WCHAR);
size += fgp->cxKeyArray * sizeof(COMP_KEY);
for (j = 0, fkp = fgp->dpKeyArray; j < fgp->cxKeyArray; j++, fkp++) {
size += (u16len(fkp->dpOutput) + 1) * sizeof(KMX_WCHAR);
size += (u16len(fkp->dpContext) + 1) * sizeof(KMX_WCHAR);
}
if (fgp->dpMatch)
size += (u16len(fgp->dpMatch) + 1) * sizeof(KMX_WCHAR);
if (fgp->dpNoMatch)
size += (u16len(fgp->dpNoMatch) + 1) * sizeof(KMX_WCHAR);
}
for (i = 0; i < fk->cxStoreArray; i++) {
size += (u16len(fk->dpStoreArray[i].dpString) + 1) * sizeof(KMX_WCHAR);
if (fk->dpStoreArray[i].dpName)
size += (u16len(fk->dpStoreArray[i].dpName) + 1) * sizeof(KMX_WCHAR);
}
buf = new KMX_BYTE[size];
if (!buf)
return CERR_CannotAllocateMemory;
memset(buf, 0, size);
ck = (PCOMP_KEYBOARD)buf;
ck->dwIdentifier = FILEID_COMPILED;
ck->dwFileVersion = fk->dwFileVersion;
ck->dwCheckSum = 0; // No checksum in 16.0, see #7276
ck->KeyboardID = fk->xxkbdlayout;
ck->IsRegistered = fk->IsRegistered;
ck->cxStoreArray = fk->cxStoreArray;
ck->cxGroupArray = fk->cxGroupArray;
ck->StartGroup[0] = fk->StartGroup[0];
ck->StartGroup[1] = fk->StartGroup[1];
ck->dwHotKey = fk->dwHotKey;
ck->dwFlags = fk->dwFlags;
offset = sizeof(COMP_KEYBOARD);
ck->dpStoreArray = offset;
sp = (PCOMP_STORE)(buf + offset);
fsp = fk->dpStoreArray;
offset += sizeof(COMP_STORE) * ck->cxStoreArray;
for (i = 0; i < ck->cxStoreArray; i++, sp++, fsp++) {
sp->dwSystemID = fsp->dwSystemID;
sp->dpString = offset;
u16ncpy((PKMX_WCHAR)(buf + offset), fsp->dpString, (size - offset) / sizeof(KMX_WCHAR)); // I3481 // I3641
offset += (u16len(fsp->dpString) + 1) * sizeof(KMX_WCHAR);
if (!fsp->dpName) {
sp->dpName = 0;
} else {
sp->dpName = offset;
u16ncpy((PKMX_WCHAR)(buf + offset), fsp->dpName, (size - offset) / sizeof(KMX_WCHAR)); // I3481 // I3641
offset += (u16len(fsp->dpName) + 1) * sizeof(KMX_WCHAR);
}
}
ck->dpGroupArray = offset;
gp = (PCOMP_GROUP)(buf + offset);
offset += sizeof(COMP_GROUP) * ck->cxGroupArray;
for (i = 0, fgp = fk->dpGroupArray; i < ck->cxGroupArray; i++, gp++, fgp++) {
gp->cxKeyArray = fgp->cxKeyArray;
gp->fUsingKeys = fgp->fUsingKeys;
gp->dpMatch = gp->dpNoMatch = 0;
if (fgp->dpMatch) {
gp->dpMatch = offset;
u16ncpy((PKMX_WCHAR)(buf + offset), fgp->dpMatch, (size - offset) / sizeof(KMX_WCHAR)); // I3481 // I3641
offset += (u16len(fgp->dpMatch) + 1) * sizeof(KMX_WCHAR);
}
if (fgp->dpNoMatch) {
gp->dpNoMatch = offset;
u16ncpy((PKMX_WCHAR)(buf + offset), fgp->dpNoMatch, (size - offset) / sizeof(KMX_WCHAR)); // I3481 // I3641
offset += (u16len(fgp->dpNoMatch) + 1) * sizeof(KMX_WCHAR);
}
if (fgp->dpName) {
gp->dpName = offset;
u16ncpy((PKMX_WCHAR)(buf + offset), fgp->dpName, (size - offset) / sizeof(KMX_WCHAR)); // I3481 // I3641
offset += (u16len(fgp->dpName) + 1) * sizeof(KMX_WCHAR);
} else {
gp->dpName = 0;
}
gp->dpKeyArray = offset;
kp = (PCOMP_KEY)(buf + offset);
offset += gp->cxKeyArray * sizeof(COMP_KEY);
for (j = 0, fkp = fgp->dpKeyArray; j < gp->cxKeyArray; j++, kp++, fkp++) {
kp->Key = fkp->Key;
kp->Line = fkp->Line;
kp->ShiftFlags = fkp->ShiftFlags;
kp->dpOutput = offset;
u16ncpy((PKMX_WCHAR)(buf + offset), fkp->dpOutput, (size - offset) / sizeof(KMX_WCHAR)); // I3481 // I3641
offset += (u16len(fkp->dpOutput) + 1) * sizeof(KMX_WCHAR);
kp->dpContext = offset;
u16ncpy((PKMX_WCHAR)(buf + offset), fkp->dpContext, (size - offset) / sizeof(KMX_WCHAR)); // I3481 // I3641
offset += (u16len(fkp->dpContext) + 1) * sizeof(KMX_WCHAR);
}
}
if (fk->dwBitmapSize > 0) {
ck->dwBitmapSize = fk->dwBitmapSize;
ck->dpBitmapOffset = offset;
memcpy(buf + offset, ((PKMX_BYTE)fk) + fk->dpBitmapOffset, fk->dwBitmapSize);
offset += fk->dwBitmapSize;
} else {
ck->dwBitmapSize = 0;
ck->dpBitmapOffset = 0;
}
size_t nr_elements = fwrite(buf, size, 1, hOutfile);
if (nr_elements < 1) {
delete[] buf;
return CERR_SomewhereIGotItWrong;
}
if (offset != size) {
delete[] buf;
return CERR_UnableToWriteFully;
}
delete[] buf;
return CERR_None;
}
/**
* @brief save keyboard to file
* @param kbd pointer to the keyboard
* @param filename pointer to filename of a kmx-file
* @return TRUE on success;
* else FALSE
*/
KMX_BOOL KMX_SaveKeyboard(LPKMX_KEYBOARD kbd, KMX_CHAR* filename) {
FILE* fp;
fp = Open_File(filename, "wb");
if (fp == NULL) {
KMX_LogError(L"Failed to create output file (%d)", errno);
return FALSE;
}
KMX_DWORD err = KMX_WriteCompiledKeyboardToFile(kbd, fp, FALSE);
fclose(fp);
if (err != CERR_None) {
KMX_LogError(L"Failed to write compiled keyboard with error %d", err);
std::string s(filename);
remove(s.c_str());
return FALSE;
}
return TRUE;
}
/**
* @brief add an offset
* @param base pointer to starting point
* @param offset a given offset
* @return pointer to base + offset
*/
PKMX_WCHAR KMX_StringOffset(PKMX_BYTE base, KMX_DWORD offset) {
if (offset == 0)
return NULL;
return (PKMX_WCHAR)(base + offset);
}
#ifdef KMX_64BIT
/**
* @brief CopyKeyboard will copy the data into bufp from x86-sized structures into
* x64-sized structures starting at `base`. After this function finishes, we still
* need to keep the original data because we don't copy the strings. The method is
* used on 64-bit architectures.
* @param bufp pointer to buffer where data is copied into
* @param base pointer to starting point
* @return pointer to the keyboard
*/
LPKMX_KEYBOARD CopyKeyboard(PKMX_BYTE bufp, PKMX_BYTE base) {
PCOMP_KEYBOARD ckbp = (PCOMP_KEYBOARD)base;
/* Copy keyboard structure */
LPKMX_KEYBOARD kbp = (LPKMX_KEYBOARD)bufp;
bufp += sizeof(KMX_KEYBOARD);
kbp->dwIdentifier = ckbp->dwIdentifier;
kbp->dwFileVersion = ckbp->dwFileVersion;
kbp->dwCheckSum = ckbp->dwCheckSum;
kbp->xxkbdlayout = ckbp->KeyboardID;
kbp->IsRegistered = ckbp->IsRegistered;
kbp->version = ckbp->version;
kbp->cxStoreArray = ckbp->cxStoreArray;
kbp->cxGroupArray = ckbp->cxGroupArray;
kbp->StartGroup[0] = ckbp->StartGroup[0];
kbp->StartGroup[1] = ckbp->StartGroup[1];
kbp->dwFlags = ckbp->dwFlags;
kbp->dwHotKey = ckbp->dwHotKey;
kbp->dpBitmapOffset = ckbp->dpBitmapOffset;
kbp->dwBitmapSize = ckbp->dwBitmapSize;
kbp->dpStoreArray = (LPKMX_STORE)bufp;
bufp += sizeof(KMX_STORE) * kbp->cxStoreArray;
kbp->dpGroupArray = (LPKMX_GROUP)bufp;
bufp += sizeof(KMX_GROUP) * kbp->cxGroupArray;
PCOMP_STORE csp;
LPKMX_STORE sp;
KMX_DWORD i;
for (csp = (PCOMP_STORE)(base + ckbp->dpStoreArray), sp = kbp->dpStoreArray, i = 0; i < kbp->cxStoreArray; i++, sp++, csp++) {
sp->dwSystemID = csp->dwSystemID;
sp->dpName = KMX_StringOffset(base, csp->dpName);
sp->dpString = KMX_StringOffset(base, csp->dpString);
}
PCOMP_GROUP cgp;
LPKMX_GROUP gp;
for (cgp = (PCOMP_GROUP)(base + ckbp->dpGroupArray), gp = kbp->dpGroupArray, i = 0; i < kbp->cxGroupArray; i++, gp++, cgp++) {
gp->dpName = KMX_StringOffset(base, cgp->dpName);
gp->dpKeyArray = cgp->cxKeyArray > 0 ? (LPKMX_KEY)bufp : NULL;
gp->cxKeyArray = cgp->cxKeyArray;
bufp += sizeof(KMX_KEY) * gp->cxKeyArray;
gp->dpMatch = KMX_StringOffset(base, cgp->dpMatch);
gp->dpNoMatch = KMX_StringOffset(base, cgp->dpNoMatch);
gp->fUsingKeys = cgp->fUsingKeys;
PCOMP_KEY ckp;
LPKMX_KEY kp;
KMX_DWORD j;
for (ckp = (PCOMP_KEY)(base + cgp->dpKeyArray), kp = gp->dpKeyArray, j = 0; j < gp->cxKeyArray; j++, kp++, ckp++) {
kp->Key = ckp->Key;
kp->Line = ckp->Line;
kp->ShiftFlags = ckp->ShiftFlags;
kp->dpOutput = KMX_StringOffset(base, ckp->dpOutput);
kp->dpContext = KMX_StringOffset(base, ckp->dpContext);
}
}
return kbp;
}
// else KMX_FixupKeyboard
#else
/**
* @brief Fixup the keyboard by expanding pointers. On disk the pointers are stored relative to the
* beginning of the file, but we need real pointers. This method is used on 32-bit architectures.
* @param bufp pointer to buffer where data will be copied into
* @param base pointer to starting point
* @param dwFileSize size of the file
* @return pointer to the keyboard
*/
LPKMX_KEYBOARD KMX_FixupKeyboard(PKMX_BYTE bufp, PKMX_BYTE base, KMX_DWORD dwFileSize) {
UNREFERENCED_PARAMETER(dwFileSize);
KMX_DWORD i, j;
PCOMP_KEYBOARD ckbp = (PCOMP_KEYBOARD)base;
PCOMP_GROUP cgp;
PCOMP_STORE csp;
PCOMP_KEY ckp;
LPKMX_KEYBOARD kbp = (LPKMX_KEYBOARD)bufp;
LPKMX_STORE sp;
LPKMX_GROUP gp;
LPKMX_KEY kp;
kbp->dpStoreArray = (LPKMX_STORE)(base + ckbp->dpStoreArray);
kbp->dpGroupArray = (LPKMX_GROUP)(base + ckbp->dpGroupArray);
for (sp = kbp->dpStoreArray, csp = (PCOMP_STORE)sp, i = 0; i < kbp->cxStoreArray; i++, sp++, csp++) {
sp->dpName = KMX_StringOffset(base, csp->dpName);
sp->dpString = KMX_StringOffset(base, csp->dpString);
}
for (gp = kbp->dpGroupArray, cgp = (PCOMP_GROUP)gp, i = 0; i < kbp->cxGroupArray; i++, gp++, cgp++) {
gp->dpName = KMX_StringOffset(base, cgp->dpName);
gp->dpKeyArray = (LPKMX_KEY)(base + cgp->dpKeyArray);
if (cgp->dpMatch != NULL)
gp->dpMatch = (PKMX_WCHAR)(base + cgp->dpMatch);
if (cgp->dpNoMatch != NULL)
gp->dpNoMatch = (PKMX_WCHAR)(base + cgp->dpNoMatch);
for (kp = gp->dpKeyArray, ckp = (PCOMP_KEY)kp, j = 0; j < gp->cxKeyArray; j++, kp++, ckp++) {
kp->dpOutput = (PKMX_WCHAR)(base + ckp->dpOutput);
kp->dpContext = (PKMX_WCHAR)(base + ckp->dpContext);
}
}
return kbp;
}
#endif
/** @brief load a keyboard kmx-file */
KMX_BOOL KMX_LoadKeyboard(KMX_CHAR* fileName, LPKMX_KEYBOARD* lpKeyboard) {
*lpKeyboard = NULL;
PKMX_BYTE buf;
FILE* fp;
LPKMX_KEYBOARD kbp;
PKMX_BYTE filebase;
if (!fileName || !lpKeyboard) {
KMX_LogError(L"Bad Filename\n");
return FALSE;
}
fp = Open_File((const KMX_CHAR*)fileName, "rb");
if (fp == NULL) {
KMX_LogError(L"Could not open file\n");
return FALSE;
}
if (fseek(fp, 0, SEEK_END) != 0) {
fclose(fp);
KMX_LogError(L"Could not fseek file\n");
return FALSE;
}
auto file_size = ftell(fp);
if (file_size <= 0) {
fclose(fp);
return FALSE;
}
if (fseek(fp, 0, SEEK_SET) != 0) {
fclose(fp);
KMX_LogError(L"Could not fseek(set) file\n");
return FALSE;
}
#ifdef KMX_64BIT
/**
* allocate enough memory for expanded data structure + original data.
* Expanded data structure is double the size of data on disk (8-byte
* pointers) - on disk the "pointers" are relative to the beginning of
* the file.
* We save the original data at the end of buf; we don't copy strings, so
* those will remain in the location at the end of the buffer.
*/
buf = new KMX_BYTE[file_size * 3];
#else
buf = new KMX_BYTE[file_size];
#endif
if (!buf) {
delete[] buf;
fclose(fp);
KMX_LogError(L"Not allocmem\n");
return FALSE;
}
#ifdef KMX_64BIT
filebase = buf + file_size * 2;
#else
filebase = buf;
#endif
if (fread(filebase, 1, file_size, fp) < (size_t)file_size) {
KMX_LogError(L"Could not read file\n");
delete[] buf;
fclose(fp);
return FALSE;
}
fclose(fp);
if (*((PKMX_DWORD)filebase) != (KMX_DWORD)FILEID_COMPILED) {
delete[] buf;
KMX_LogError(L"Invalid file - signature is invalid\n");
return FALSE;
}
if (!KMX_VerifyKeyboard(filebase, file_size)) {
delete[] buf;
KMX_LogError(L"errVerifyKeyboard\n");
return FALSE;
}
#ifdef KMX_64BIT
kbp = CopyKeyboard(buf, filebase);
#else
kbp = KMX_FixupKeyboard(buf, filebase, file_size);
#endif
if (!kbp) {
delete[] buf;
KMX_LogError(L"errFixupKeyboard\n");
return FALSE;
}
if (kbp->dwIdentifier != FILEID_COMPILED) {
delete[] buf;
KMX_LogError(L"errNotFileID\n");
return FALSE;
}
*lpKeyboard = kbp;
return TRUE;
}
/**
* @brief check if the file has correct version
* @param filebase containing data of the input file
* @param file_size a size
* @return true if successful;
* false if not
*/
KMX_BOOL KMX_VerifyKeyboard(PKMX_BYTE filebase, KMX_DWORD file_size) {
KMX_DWORD i;
PCOMP_KEYBOARD ckbp = (PCOMP_KEYBOARD)filebase;
PCOMP_STORE csp;
// Check file version //
if (ckbp->dwFileVersion < VERSION_MIN || ckbp->dwFileVersion > VERSION_MAX) {
// Old or new version -- identify the desired program version //
for (csp = (PCOMP_STORE)(filebase + ckbp->dpStoreArray), i = 0; i < ckbp->cxStoreArray; i++, csp++) {
if (csp->dwSystemID == TSS_COMPILEDVERSION) {
if (csp->dpString == 0) {
KMX_LogError(L"errWrongFileVersion:NULL");
} else {
KMX_LogError(L"errWrongFileVersion:%10.10ls", (const PKMX_WCHAR)KMX_StringOffset((PKMX_BYTE)filebase, csp->dpString));
}
return FALSE;
}
}
KMX_LogError(L"errWrongFileVersion");
return FALSE;
}
return TRUE;
}
/**
* @brief increment in a string
* @param p pointer to a character
* @return pointer to the incremented character
*/
PKMX_WCHAR KMX_incxstr(PKMX_WCHAR p) {
if (p == NULL || *p == 0)
return p;
if (*p != UC_SENTINEL) {
if (*p >= 0xD800 && *p <= 0xDBFF && *(p + 1) >= 0xDC00 && *(p + 1) <= 0xDFFF)
return p + 2;
return p + 1;
}
// UC_SENTINEL(FFFF) with UC_SENTINEL_EXTENDEDEND(0x10) ==> variable length
if (*(p + 1) == CODE_EXTENDED) {
p += 2;
while (*p && *p != UC_SENTINEL_EXTENDEDEND)
p++;
if (*p == 0)
return p;
return p + 1;
}
if (*(p + 1) > CODE_LASTCODE || CODE__SIZE[*(p + 1)] == -1) {
return p + 1;
}
int deltaptr = 2 + CODE__SIZE[*(p + 1)];
// check for \0 between UC_SENTINEL(FFFF) and next printable character
for (int i = 0; i < deltaptr; i++) {
if (*p == 0)
return p;
p++;
}
return p;
}
/**
* @brief open a file
* @param filename name of the file
* @param mode same as mode in fopen
* @return pointer to file.
* On error returns a null pointer
*/
FILE* Open_File(const KMX_CHAR* filename, const KMX_CHAR* mode) {
#ifdef _MSC_VER
std::string cpath = filename; //, cmode = mode;
std::replace(cpath.begin(), cpath.end(), '/', '\\');
return fopen(cpath.c_str(), (const KMX_CHAR*)mode);
#else
return fopen(filename, mode);
std::string cpath, cmode;
cpath = (const KMX_CHAR*)filename;
cmode = (const KMX_CHAR*)mode;
return fopen(cpath.c_str(), cmode.c_str());
#endif
};

83
linux/mcompile/keymap/mc_kmxfile.h Normal file
View file

@ -0,0 +1,83 @@
#pragma once
#ifndef MC_KMXFILE_H
#define MC_KMXFILE_H
#include "km_types.h"
#include <kmx_file.h>
#include "mcompile.h"
#ifndef _KMXFILE_H
#define _KMXFILE_H
typedef struct KMX_tagSTORE {
KMX_DWORD dwSystemID;
PKMX_WCHAR dpName;
PKMX_WCHAR dpString;
} KMX_STORE, *LPKMX_STORE;
typedef struct KMX_tagKEY {
KMX_WCHAR Key;
KMX_DWORD Line;
KMX_DWORD ShiftFlags;
PKMX_WCHAR dpOutput;
PKMX_WCHAR dpContext;
} KMX_KEY, *LPKMX_KEY;
typedef struct KMX_tagGROUP {
KMX_WCHAR* dpName;
LPKMX_KEY dpKeyArray; // [LPKEY] address of first item in key array
PKMX_WCHAR dpMatch;
PKMX_WCHAR dpNoMatch;
KMX_DWORD cxKeyArray; // in array entries
int32_t fUsingKeys; // group(xx) [using keys] <-- specified or not
} KMX_GROUP, *LPKMX_GROUP;
typedef struct KMX_tagKEYBOARD {
KMX_DWORD dwIdentifier; // Keyman compiled keyboard id
KMX_DWORD dwFileVersion; // Version of the file - Keyman 4.0 is 0x0400
KMX_DWORD dwCheckSum; // As stored in keyboard. DEPRECATED as of 16.0
KMX_DWORD xxkbdlayout; // as stored in HKEY_LOCAL_MACHINE//system//currentcontrolset//control//keyboard layouts
KMX_DWORD IsRegistered; // layout id, from same registry key
KMX_DWORD version; // keyboard version
KMX_DWORD cxStoreArray; // in array entries
KMX_DWORD cxGroupArray; // in array entries
LPKMX_STORE dpStoreArray; // [LPSTORE] address of first item in store array, from start of file
LPKMX_GROUP dpGroupArray; // [LPGROUP] address of first item in group array, from start of file
KMX_DWORD StartGroup[2]; // index of starting groups [2 of them]
// Ansi=0, Unicode=1
KMX_DWORD dwFlags; // Flags for the keyboard file
KMX_DWORD dwHotKey; // standard windows hotkey (hiword=shift/ctrl/alt stuff, loword=vkey)
// PWSTR dpName; // offset of name
// PWSTR dpLanguageName; // offset of language name;
// PWSTR dpCopyright; // offset of copyright
// PWSTR dpMessage; // offset of message in Keyboard About box
KMX_DWORD dpBitmapOffset; // 0038 offset of the bitmaps in the file
KMX_DWORD dwBitmapSize; // 003C size in bytes of the bitmaps
//HBITMAP hBitmap; // handle to the bitmap in the file;
} KMX_KEYBOARD, *LPKMX_KEYBOARD;
/** @brief load a keyboard kmx-file */
KMX_BOOL KMX_LoadKeyboard(KMX_CHAR* fileName, LPKMX_KEYBOARD* lpKeyboard);
/** @brief save keyboard to file */
KMX_BOOL KMX_SaveKeyboard(LPKMX_KEYBOARD kbd, KMX_CHAR* filename);
/** @brief increment in a string */
PKMX_WCHAR KMX_incxstr(PKMX_WCHAR p);
/** @brief open a file */
FILE* Open_File(const KMX_CHAR* filename, const KMX_CHAR* mode);
#endif // _KMXFILE_H
#endif /*MC_KMXFILE_H*/

578
linux/mcompile/keymap/mcompile.cpp Normal file
View file

@ -0,0 +1,578 @@
/*
* Keyman is copyright (C) SIL International. MIT License.
*
* Mnemonic layout support for linux
*
* Defines the entry point for the console application.
*
* Note: this program deliberately leaks memory as it has a very short life cycle and managing the memory allocations
* for the subcomponents of the compiled keyboard is an unnecessary optimisation. Just so you know.
*/
#include "mcompile.h"
const int nr_DK_pairs = 1000;
static const int size_DK_array = (nr_DK_pairs + 1) *3;
/** @brief convert mnemonic keyboard layout to positional keyboard layout and translate keyboard */
KMX_BOOL KMX_DoConvert(LPKMX_KEYBOARD kbd, KMX_BOOL bDeadkeyConversion, gint argc, gchar* argv[]);
/** @brief Collect the key data, translate it to kmx and append to the existing keyboard */
bool KMX_ImportRules(LPKMX_KEYBOARD kp, vec_dword_3D& all_vector, GdkKeymap** keymap, std::vector<KMX_DeadkeyMapping>* KMX_FDeadkeys, KMX_BOOL bDeadkeyConversion); // I4353 // I4327
/** @brief return a vector of [usvk, ch_out] pairs: all existing combinations of a deadkey + character for the underlying keyboard */
int KMX_GetDeadkeys(vec_dword_2D& dk_Table, KMX_WORD deadkey, std::vector<KMX_WORD> &dk_vec, GdkKeymap* keymap);
std::vector<KMX_DeadkeyMapping> KMX_FDeadkeys; // I4353
#define _countof(a) (sizeof(a) / sizeof(*(a)))
/**
* @brief main function for mcompile for Linux
* @param argc number of commandline arguments
* @param argv pointer to commandline arguments: executable, inputfile, outputfile
* @return 0 on success
*/
int main(int argc, char* argv[]) {
int bDeadkeyConversion = 0;
if (argc > 1)
bDeadkeyConversion = (strcmp(argv[1], "-d") == 0); // I4552
int n = (bDeadkeyConversion ? 2 : 1);
if (argc < 3 || argc > 4 || (argc - n) != 2) { // I4273// I4273
printf(
"Usage: \tmcompile [-d] infile.kmx outfile.kmx\n"
" \tmcompile converts a Keyman mnemonic layout to\n"
" \ta positional one based on the currently used \n"
" \tLinux keyboard layout\n"
" \t(-d convert deadkeys to plain keys) \n \n"); // I4552
return 1;
}
// -u option is not available for Linux and macOS
KMX_CHAR* infile = argv[n];
KMX_CHAR* outfile = argv[n + 1];
printf("mcompile%s \"%s\" \"%s\"\n", bDeadkeyConversion ? " -d" : "", infile, outfile); // I4174
// 1. Load the keyman keyboard file
// 2. For each key on the system layout, determine its output character and perform a
// 1-1 replacement on the keyman keyboard of that character with the base VK + shift
// state. This fixup will transform the char to a vk, which will avoid any issues
// with the key.
//
//
// For each deadkey, we need to determine its possible outputs. Then we generate a VK
// rule for that deadkey, e.g. [K_LBRKT] > dk(c101)
//
// Next, update each rule that references the output from that deadkey to add an extra
// context deadkey at the end of the context match, e.g. 'a' dk(c101) + [K_SPACE] > 'b'.
// This will require a memory layout change for the .kmx file, plus fixups on the
// context+output index offsets
//
// --> virtual character keys
//
// [CTRL ' '] : we look at the character, and replace it in the same way, but merely
// switch the shift state from the VIRTUALCHARKEY to VIRTUALKEY, without changing any
// other properties of the key.
//
// 3. Write the new keyman keyboard file
LPKMX_KEYBOARD kmxfile;
if (!KMX_LoadKeyboard(infile, &kmxfile)) {
KMX_LogError(L"Failed to load keyboard (%d)\n", errno);
return 3;
}
if (KMX_DoConvert(kmxfile, bDeadkeyConversion, argc, (gchar**)argv)) { // I4552F
if(!KMX_SaveKeyboard(kmxfile, outfile)) {
KMX_LogError(L"Failed to save keyboard (%d)\n", errno);
return 3;
}
}
delete kmxfile;
return 0;
}
// Map of all shift states that we will work with
const KMX_DWORD VKShiftState[] = {0, K_SHIFTFLAG, LCTRLFLAG | RALTFLAG, K_SHIFTFLAG | LCTRLFLAG | RALTFLAG, 0xFFFF};
//
// TranslateKey
//
// For each key rule on the keyboard, remap its key to the
// correct shift state and key. Adjust the LCTRL+RALT -> RALT if necessary
//
/**
* @brief translate each key of a group: remap the content of a key (key->Key) of the US keyboard to a character (ch)
* @param key pointer to a key
* @param vk a keyvalue of the US keyboard
* @param shift shiftstate
* @param ch character of the underlying keyboard to be remapped
*/
void KMX_TranslateKey(LPKMX_KEY key, KMX_WORD vk, KMX_DWORD shift, KMX_WCHAR ch) {
// The weird LCTRL+RALT is Windows' way of mapping the AltGr key.
// We store that as just RALT, and use the option "Simulate RAlt with Ctrl+Alt"
// to provide an alternate..
if ((shift & (LCTRLFLAG | RALTFLAG)) == (LCTRLFLAG | RALTFLAG))
shift &= ~LCTRLFLAG;
if (key->ShiftFlags == 0 && key->Key == ch) {
// Key is a mnemonic key with no shift state defined.
// Remap the key according to the character on the key cap.
// KMX_LogError(L"Converted mnemonic rule on line %d, + '%c' TO + [%x K_%d]", key->Line, key->Key, shift, vk);
key->ShiftFlags = ISVIRTUALKEY | shift;
key->Key = vk;
} else if (key->ShiftFlags & VIRTUALCHARKEY && key->Key == ch) {
// Key is a virtual character key with a hard-coded shift state.
// Do not remap the shift state, just move the key.
// This will not result in 100% wonderful mappings as there could
// be overlap, depending on how keys are arranged on the target layout.
// But that is up to the designer.
// KMX_LogError(L"Converted mnemonic virtual char key rule on line %d, + [%x '%c'] TO + [%x K_%d]", key->Line, key->ShiftFlags, key->Key, key->ShiftFlags & ~VIRTUALCHARKEY, vk);
key->ShiftFlags &= ~VIRTUALCHARKEY;
key->Key = vk;
}
}
/**
* @brief translate a group of a keyboard
* @param group pointer to a keyboard group
* @param vk a keyvalue of the US keyboard
* @param shift shiftstate
* @param ch character of the underlying keyboard to be remapped
*/
void KMX_TranslateGroup(LPKMX_GROUP group, KMX_WORD vk, KMX_DWORD shift, KMX_WCHAR ch) {
for (unsigned int i = 0; i < group->cxKeyArray; i++) {
KMX_TranslateKey(&group->dpKeyArray[i], vk, shift, ch);
}
}
/**
* @brief translate a keyboard
* @param kbd pointer to the US keyboard
* @param vk a keyvalue of the US keyboard
* @param shift shiftstate
* @param ch character of the underlying keyboard to be remapped
*/
void KMX_TranslateKeyboard(LPKMX_KEYBOARD kbd, KMX_WORD vk, KMX_DWORD shift, KMX_WCHAR ch) {
for (unsigned int i = 0; i < kbd->cxGroupArray; i++) {
if (kbd->dpGroupArray[i].fUsingKeys) {
KMX_TranslateGroup(&kbd->dpGroupArray[i], vk, shift, ch);
}
}
}
/**
* @brief check key for unconverted key rules
* @param key pointer to a key
*/
void KMX_ReportUnconvertedKeyRule(LPKMX_KEY key) {
if (key->ShiftFlags == 0) {
// KMX_LogError(L"Did not find a match for mnemonic rule on line %d, + '%c' > ...", key->Line, key->Key);
} else if (key->ShiftFlags & VIRTUALCHARKEY) {
KMX_LogError(L"Did not find a match for mnemonic virtual character key rule on line %d, + [%x '%c'] > ...", key->Line, key->ShiftFlags, key->Key);
}
}
/**
* @brief check a group for unconverted rules
* @param group pointer to a keyboard group
*/
void KMX_ReportUnconvertedGroupRules(LPKMX_GROUP group) {
for (unsigned int i = 0; i < group->cxKeyArray; i++) {
KMX_ReportUnconvertedKeyRule(&group->dpKeyArray[i]);
}
}
/**
* @brief check a keyboard for unconverted rules
* @param kbd pointer to the US keyboard
*/
void KMX_ReportUnconvertedKeyboardRules(LPKMX_KEYBOARD kbd) {
for (unsigned int i = 0; i < kbd->cxGroupArray; i++) {
if (kbd->dpGroupArray[i].fUsingKeys) {
KMX_ReportUnconvertedGroupRules(&kbd->dpGroupArray[i]);
}
}
}
/**
* @brief remap the content of a key (key->dpContext) of the US keyboard to a deadkey sequence
* @param key pointer to a key
* @param deadkey a deadkey to be remapped
* @param vk a keyvalue of the US keyboard
* @param shift shiftstate
* @param ch character of the underlying keyboard
*/
void KMX_TranslateDeadkeyKey(LPKMX_KEY key, KMX_WCHAR deadkey, KMX_WORD vk, KMX_DWORD shift, KMX_WORD ch) {
if ((key->ShiftFlags == 0 || key->ShiftFlags & VIRTUALCHARKEY) && key->Key == ch) {
// The weird LCTRL+RALT is Windows' way of mapping the AltGr key.
// We store that as just RALT, and use the option "Simulate RAlt with Ctrl+Alt"
// to provide an alternate..
if ((shift & (LCTRLFLAG | RALTFLAG)) == (LCTRLFLAG | RALTFLAG)) // I4327
shift &= ~LCTRLFLAG;
if (key->ShiftFlags == 0) {
// KMX_LogError(L"Converted mnemonic rule on line %d, + '%c' TO dk(%d) + [%x K_%d]", key->Line, key->Key, deadkey, shift, vk);
key->ShiftFlags = ISVIRTUALKEY | shift;
} else {
// KMX_LogError(L"Converted mnemonic virtual char key rule on line %d, + [%x '%c'] TO dk(%d) + [%x K_%d]", key->Line, key->ShiftFlags, key->Key, deadkey, key->ShiftFlags & ~VIRTUALCHARKEY, vk);
key->ShiftFlags &= ~VIRTUALCHARKEY;
}
int len = u16len(key->dpContext);
PKMX_WCHAR context = new KMX_WCHAR[len + 4];
memcpy(context, key->dpContext, len * sizeof(KMX_WCHAR));
context[len] = UC_SENTINEL;
context[len + 1] = CODE_DEADKEY;
context[len + 2] = deadkey;
context[len + 3] = 0;
key->dpContext = context;
key->Key = vk;
}
}
/**
* @brief translate a group
* @param group pointer to a keyboard group
* @param deadkey deadkey to be remapped
* @param vk a keyvalue of the US keyboard
* @param shift shiftstate
* @param ch character of the underlying keyboard
*/
void KMX_TranslateDeadkeyGroup(LPKMX_GROUP group, KMX_WCHAR deadkey, KMX_WORD vk, KMX_DWORD shift, KMX_WORD ch) {
for (unsigned int i = 0; i < group->cxKeyArray; i++) {
KMX_TranslateDeadkeyKey(&group->dpKeyArray[i], deadkey, vk, shift, ch);
}
}
/**
* @brief translate a keyboard
* @param kbd pointer to the US keyboard
* @param deadkey a deadkey to be remapped
* @param vk a keyvalue of the US keyboard
* @param shift shiftstate
* @param ch character of the underlying keyboard
*/
void KMX_TranslateDeadkeyKeyboard(LPKMX_KEYBOARD kbd, KMX_WCHAR deadkey, KMX_WORD vk, KMX_DWORD shift, KMX_WORD ch) {
for (unsigned int i = 0; i < kbd->cxGroupArray; i++) {
if (kbd->dpGroupArray[i].fUsingKeys) {
KMX_TranslateDeadkeyGroup(&kbd->dpGroupArray[i], deadkey, vk, shift, ch);
}
}
}
/**
* @brief add a deadkey rule
* @param kbd pointer to the US keyboard
* @param deadkey a deadkey to be added
* @param vk a keyvalue of the US keyboard
* @param shift shiftstate
*/
void KMX_AddDeadkeyRule(LPKMX_KEYBOARD kbd, KMX_WCHAR deadkey, KMX_WORD vk, KMX_DWORD shift) {
// The weird LCTRL+RALT is Windows' way of mapping the AltGr key.
// We store that as just RALT, and use the option "Simulate RAlt with Ctrl+Alt"
// to provide an alternate..
if ((shift & (LCTRLFLAG | RALTFLAG)) == (LCTRLFLAG | RALTFLAG)) // I4549
shift &= ~LCTRLFLAG;
// If the first group is not a matching-keys group, then we need to add into
// each subgroup, otherwise just the match group
for (unsigned int i = 0; i < kbd->cxGroupArray; i++) {
if (kbd->dpGroupArray[i].fUsingKeys) {
LPKMX_KEY keys = new KMX_KEY[kbd->dpGroupArray[i].cxKeyArray + 1];
memcpy(keys + 1, kbd->dpGroupArray[i].dpKeyArray, kbd->dpGroupArray[i].cxKeyArray * sizeof(KMX_KEY));
keys[0].dpContext = new KMX_WCHAR[1];
keys[0].dpContext[0] = 0;
keys[0].dpOutput = new KMX_WCHAR[4];
keys[0].dpOutput[0] = UC_SENTINEL;
keys[0].dpOutput[1] = CODE_DEADKEY;
keys[0].dpOutput[2] = deadkey; // TODO: translate to unique index
keys[0].dpOutput[3] = 0;
keys[0].Key = vk;
keys[0].Line = 0;
keys[0].ShiftFlags = shift | ISVIRTUALKEY;
kbd->dpGroupArray[i].dpKeyArray = keys;
kbd->dpGroupArray[i].cxKeyArray++;
KMX_LogError(L"Add deadkey rule: + [%d K_%d] > dk(%d)", shift, vk, deadkey);
if (i == kbd->StartGroup[1])
break; // If this is the initial group, that's all we need to do.
}
}
}
/**
* @brief find the maximal deadkey id
* @param str the deadkey
* @return the maximum deadkey id
*/
KMX_WCHAR KMX_ScanXStringForMaxDeadkeyID(PKMX_WCHAR str) {
KMX_WCHAR dkid = 0;
while (str && *str) {
if (*str == UC_SENTINEL && *(str + 1) == CODE_DEADKEY) {
dkid = std::max(dkid, *(str + 2));
}
str = KMX_incxstr(str);
}
return dkid;
}
struct KMX_dkidmap {
KMX_WCHAR src_deadkey, dst_deadkey;
};
/**
* @brief find the deadkey id for a given deadkey
* @param kbd pointer to the keyboard
* @param deadkey for which an id is to be found
* @return 0 if failed;
* otherwise a deadkey-id
*/
KMX_WCHAR KMX_GetUniqueDeadkeyID(LPKMX_KEYBOARD kbd, KMX_WCHAR deadkey) {
LPKMX_GROUP gp;
LPKMX_KEY kp;
LPKMX_STORE sp;
KMX_DWORD i, j;
KMX_WCHAR dkid = 0;
static KMX_WCHAR s_next_dkid = 0;
static KMX_dkidmap* s_dkids = NULL;
static int s_ndkids = 0;
if (!kbd) {
if (s_dkids) {
delete s_dkids;
}
s_dkids = NULL;
s_ndkids = 0;
s_next_dkid = 0;
return 0;
}
for (int xi = 0; xi < s_ndkids; xi++) {
if (s_dkids[xi].src_deadkey == deadkey) {
return s_dkids[xi].dst_deadkey;
}
}
if (s_next_dkid != 0) {
s_dkids = (KMX_dkidmap*)realloc(s_dkids, sizeof(KMX_dkidmap) * (s_ndkids + 1));
s_dkids[s_ndkids].src_deadkey = deadkey;
return s_dkids[s_ndkids++].dst_deadkey = ++s_next_dkid;
}
for (i = 0, gp = kbd->dpGroupArray; i < kbd->cxGroupArray; i++, gp++) {
for (j = 0, kp = gp->dpKeyArray; j < gp->cxKeyArray; j++, kp++) {
dkid = std::max(dkid, KMX_ScanXStringForMaxDeadkeyID(kp->dpContext));
dkid = std::max(dkid, KMX_ScanXStringForMaxDeadkeyID(kp->dpOutput));
}
dkid = std::max(dkid, KMX_ScanXStringForMaxDeadkeyID(gp->dpMatch));
dkid = std::max(dkid, KMX_ScanXStringForMaxDeadkeyID(gp->dpNoMatch));
}
for (i = 0, sp = kbd->dpStoreArray; i < kbd->cxStoreArray; i++, sp++) {
dkid = std::max(dkid, KMX_ScanXStringForMaxDeadkeyID(sp->dpString));
}
s_dkids = (KMX_dkidmap*)realloc(s_dkids, sizeof(KMX_dkidmap) * (s_ndkids + 1));
s_dkids[s_ndkids].src_deadkey = deadkey;
return s_dkids[s_ndkids++].dst_deadkey = s_next_dkid = ++dkid;
}
/**
* @brief Lookup the deadkey table for the deadkey in the physical keyboard. Then for each character, go through and map it through
* @param kbd pointer to the keyboard
* @param vk_US virtual key of the us keyboard
* @param shift shiftstate
* @param deadkey character produced by a deadkey
* @param all_vector vector that holds the data of the US keyboard and the currently used (underlying) keyboard
* @param keymap pointer to the currently used (underlying) keyboard Layout
* @param dk_Table a vector of all possible deadkey combinations for all Linux keyboards
*/
void KMX_ConvertDeadkey(LPKMX_KEYBOARD kbd, KMX_WORD vk_US, KMX_DWORD shift, KMX_WCHAR deadkey, vec_dword_3D& all_vector, GdkKeymap* keymap, vec_dword_2D dk_Table) {
std::vector<KMX_WORD> vec_deadkeys;
// Lookup the deadkey table for the deadkey in the physical keyboard
// Then for each character, go through and map it through
KMX_WCHAR dkid = KMX_GetUniqueDeadkeyID(kbd, deadkey);
// Add the deadkey to the mapping table for use in the import rules phase
KMX_DeadkeyMapping KMX_deadkeyMapping = {deadkey, dkid, shift, vk_US}; // I4353
KMX_FDeadkeys.push_back(KMX_deadkeyMapping); // dkid, vk, shift); // I4353
KMX_AddDeadkeyRule(kbd, dkid, vk_US, shift);
KMX_GetDeadkeys(dk_Table, deadkey, vec_deadkeys, keymap); // returns vector of [usvk, ch_out] pairs
int n=0;
while (n < (int)vec_deadkeys.size() - 2) {
// Look up the ch
KMX_DWORD KeyValUnderlying = (KMX_DWORD) KMX_get_KeyValUnderlying_From_KeyValUS(all_vector, vec_deadkeys[n]);
KMX_TranslateDeadkeyKeyboard(kbd, dkid, KeyValUnderlying, vec_deadkeys[n + 1], vec_deadkeys[n + 2]);
n += 3;
}
}
/**
* @brief convert a mnemonic keyboard to a positional keyboard
* (i.e. setting *sp->dpString = '0' / TSS_MNEMONIC=0)
* @param kbd pointer to keyboard
* @return TRUE if conversion was successful;
* FALSE otherwise
*/
KMX_BOOL KMX_SetKeyboardToPositional(LPKMX_KEYBOARD kbd) {
LPKMX_STORE sp;
KMX_DWORD i;
for (i = 0, sp = kbd->dpStoreArray; i < kbd->cxStoreArray; i++, sp++) {
if (sp->dwSystemID == TSS_MNEMONIC) {
if (!sp->dpString) {
KMX_LogError(L"Invalid &mnemoniclayout system store");
return FALSE;
}
if (u16cmp((const KMX_WCHAR*)sp->dpString, u"1") != 0) {
KMX_LogError(L"Keyboard is not a mnemonic layout keyboard");
return FALSE;
}
*sp->dpString = '0';
return TRUE;
}
}
KMX_LogError(L"Keyboard is not a mnemonic layout keyboard");
return FALSE;
}
/**
* @brief convert mnemonic keyboard layout to positional keyboard layout and translate keyboard
* @param kbd pointer to US keyboard
* @param bDeadkeyConversion option for converting a deadkey to a character: 1 = dk conversion; 0 = no dk conversion
* @param argc number of command line arguments
* @param argv pointer to command line arguments
* @return TRUE if conversion was successful;
* FALSE if not
*/
KMX_BOOL KMX_DoConvert(LPKMX_KEYBOARD kbd, KMX_BOOL bDeadkeyConversion, gint argc, gchar* argv[]) {
KMX_WCHAR DeadKey = 0;
if (!KMX_SetKeyboardToPositional(kbd))
return FALSE;
// Go through each of the shift states - base, shift, ctrl+alt, ctrl+alt+shift, [caps vs ncaps?]
// Currently, we go in this order so the 102nd key works. But this is not ideal for keyboards without 102nd key: // I4651
// it catches only the first key that matches a given rule, but multiple keys may match that rule. This is particularly
// evident for the 102nd key on UK, for example, where \ can be generated with VK_OEM_102 or AltGr+VK_QUOTE.
// For now, we get the least shifted version, which is hopefully adequate.
GdkKeymap* keymap;
if (InitializeGDK(&keymap, argc, argv)) {
printf("ERROR: can't Initialize GDK\n");
return FALSE;
}
// create vector that contains Keycode, base, shift for US-KEyboard and underlying keyboard
vec_dword_3D all_vector;
if (createOneVectorFromBothKeyboards(all_vector, keymap)) {
printf("ERROR: can't create one vector from both keyboards\n");
return FALSE;
}
vec_dword_2D dk_Table;
create_DKTable(dk_Table);
for (int j = 0; VKShiftState[j] != 0xFFFF; j++) { // I4651
// Loop through each possible key on the keyboard
for (int i = 0; KMX_VKMap[i]; i++) { // I4651
// windows uses VK, Linux uses SC/Keycode
KMX_DWORD scUnderlying = (KMX_DWORD)KMX_get_KeyCodeUnderlying_From_VKUS(KMX_VKMap[i]);
KMX_WCHAR ch = KMX_get_KeyValUnderlying_From_KeyCodeUnderlying(keymap, scUnderlying, VKShiftState[j], &DeadKey);
// printf("--- VK_%d -> SC_ [%c] dk=%d ( ss %i) \n", KMX_VKMap[i], ch == 0 ? 32 : ch, DeadKey, VKShiftState[j]);
if (bDeadkeyConversion) { // I4552
if (ch == 0xFFFF) {
ch = DeadKey;
}
}
switch (ch) {
case 0x0000: break;
case 0xFFFF: KMX_ConvertDeadkey(kbd, KMX_VKMap[i], VKShiftState[j], DeadKey, all_vector, keymap, dk_Table); break;
default: KMX_TranslateKeyboard(kbd, KMX_VKMap[i], VKShiftState[j], ch);
}
}
}
KMX_ReportUnconvertedKeyboardRules(kbd);
if (!KMX_ImportRules(kbd, all_vector, &keymap, &KMX_FDeadkeys, bDeadkeyConversion)) { // I4353 // I4552
return FALSE;
}
return TRUE;
}
/**
* @brief return a vector of [usvk, ch_out] pairs: all existing combinations of a deadkey + character for the underlying keyboard
* @param dk_Table shiftstate of the deadkey
* @param deadkey deadkey character
* @param[out] dk_vec vector of [usvk, shiftstate, ch_out] triplets
* @param keymap pointer to the currently used (underlying) keyboard Layout
* @return size of vector of [usvk, shiftstate, ch_out] triplets
*/
int KMX_GetDeadkeys(vec_dword_2D& dk_Table, KMX_WORD deadkey, std::vector<KMX_WORD> &dk_vec, GdkKeymap* keymap) {
KMX_DWORD shift;
vec_dword_2D dk_SingleTable;
query_dk_combinations_for_specific_dk(dk_Table, deadkey, dk_SingleTable);
for (int i = 0; i < (int)dk_SingleTable.size(); i++) {
KMX_WORD vk = KMX_change_keyname_to_capital(dk_SingleTable[i][1], shift, keymap);
if (vk != 0) {
dk_vec.push_back(vk);
dk_vec.push_back(shift);
dk_vec.push_back(dk_SingleTable[i][2]);
} else {
KMX_LogError(L"Warning: complex deadkey not supported.");
}
}
return dk_vec.size();
}
/**
* @brief print (error) messages
* @param fmt text to print
*/
void KMX_LogError(const wchar_t* fmt, ...) {
wchar_t fmtbuf[256];
const wchar_t* end = L"\0";
const wchar_t* nl = L"\n";
va_list vars;
int j = 0;
va_start(vars, fmt);
vswprintf(fmtbuf, _countof(fmtbuf), fmt, vars);
fmtbuf[255] = 0;
do {
putwchar(fmtbuf[j]);
j++;
} while (fmtbuf[j] != *end);
putwchar(*nl);
}

25
linux/mcompile/keymap/mcompile.h Normal file
View file

@ -0,0 +1,25 @@
/*
* Keyman is copyright (C) SIL International. MIT License.
*
* Mnemonic layout support for linux
*/
#ifndef MCOMPILE_H
#define MCOMPILE_H
#include <vector>
#include "keymap.h"
#include "deadkey.h"
#include "mc_kmxfile.h"
struct KMX_DeadkeyMapping { // I4353
KMX_WCHAR deadkey, dkid;
KMX_DWORD shift;
KMX_WORD vk;
};
extern std::vector<KMX_DeadkeyMapping> KMX_FDeadkeys; // I4353
/** @brief print (error) messages */
void KMX_LogError(const wchar_t* fmt, ...);
#endif /*MCOMPILE_H*/

33
linux/mcompile/keymap/meson.build Normal file
View file

@ -0,0 +1,33 @@
project(
'mcompile', 'c', 'cpp',
license: 'MIT',
meson_version: '>=1.0',
)
gtk = dependency('gtk+-3.0', version: '>= 2.4')
xkb = dependency('xkbcommon')
deps = [gtk, xkb]
subdir('resources')
cpp_files = files(
'keymap.cpp',
'deadkey.cpp',
'mcompile.cpp',
'mc_kmxfile.cpp',
'mc_import_rules.cpp',
'../../../common/cpp/km_u16.cpp',
'../../../common/cpp/utfcodec.cpp',
)
comon_include_dir = [
include_directories('../../../common/include')
]
mcompile = executable(
'mcompile',
sources: [cpp_files],
dependencies: deps,
include_directories : comon_include_dir
)

4
mac/Keyman4MacIM/Keyman4MacIM/en.lproj/Localizable.strings
View file

@ -68,10 +68,10 @@
/* copyright label in the Package Information window */
"copyright-label" = "Copyright:";
/* message displayed to alert user to need grant accessibility permission in Preferences, pre-Ventura */
/* message displayed to alert user to grant accessibility permission in Preferences, pre-Ventura */
"privacy-alert-text" = "To function properly, Keyman requires accessibility features:\n\nGrant access in System Preferences, Security & Privacy.\nRestart your system.";
/* message displayed to alert user to need grant accessibility permission in Settings, for macOS Ventura and later */
/* message displayed to alert user to grant accessibility permission in Settings, for macOS Ventura and later */
"privacy-alert-text-ventura" = "To function properly, Keyman requires accessibility features:\n\nGrant access in System Settings, Privacy & Security.\nRestart your system.";
/* Text of menu item in Input Menu when no Keyboards are configured -- include parentheses */

5
mac/Keyman4MacIM/Keyman4MacIM/es.lproj/Localizable.strings
View file

@ -67,8 +67,11 @@
/* copyright label in the Package Information window */
"copyright-label" = "Derechos de autor:";
/* message displayed to alert user to need grant accessibility permission */
/* message displayed to alert user to grant accessibility permission in Preferences, pre-Ventura */
"privacy-alert-text" = "Para funcionar correctamente, Keyman requiere las siguientes características de accesibilidad:\nel acceso en Preferencias del Sistema, Seguridad y Privacidad.\nel reinicio del sistema.";
/* message displayed to alert user to grant accessibility permission in Settings, for macOS Ventura and later */
"privacy-alert-text-ventura" = "To function properly, Keyman requires accessibility features:\n\nGrant access in System Settings, Privacy & Security.\nRestart your system.";
/* Text of menu item in Input Menu when no Keyboards are configured -- include parentheses */
"no-keyboard-configured-menu-placeholder" = "(Sin teclado configurado)";

1
mac/build.sh
View file

@ -21,6 +21,7 @@ builder_describe "Builds Keyman for macOS." \
":engine KeymanEngine4Mac" \
":app Keyman4MacIM" \
":help Online documentation" \
":mcompile mnemonic layout recompiler- mac" \
":testapp Keyman4Mac (test harness)" \
"--quick,-q Bypasses notarization for $(builder_term install)"

7
mac/docs/help/basic/config/index.md
View file

@ -8,7 +8,7 @@ Keyman works, as well as access information about installed keyboards.
## Opening Keyman Configuration
To open Keyman Configuration:
1. ![mac OS menu bar](../../mac_images/menu_bar_keyman.png)
1. ![mac OS menu bar](../../mac_images/menu-bar-with-keyman.png)
Make sure that Keyman for macOS is the active input source (as shown above).
For more information, see [Getting Started with Keyman for macOS](../../start/tutorial).
@ -16,16 +16,15 @@ To open Keyman Configuration:
3. On the Input menu, in the section for Keyman, click **Configuration**. Note: If you just
installed Keyman and the Configuration menu is not available, please restart the computer.
![Keyman Configuration in menu](../../mac_images/keyman_menu_config.png)
![Keyman Configuration in menu](../../mac_images/keyman-configuration-menu.png)
4. The Keyman Configuration window appears:
![Keyman Configuration](../../mac_images/keyman_config.png)
![Keyman Configuration](../../mac_images/keyman-configuration-window.png)
## Using Keyman Configuration
Following are instructions for performing various tasks in the Configuration window:
- [Keyman Configuration - Keyboard Layouts Tab](keyboards)
- [Keyman Configuration - Options Tab](options)
- [Keyman Configuration - Support Tab](support)

BIN
mac/docs/help/mac_images/add_keyman.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 93 KiB

BIN
mac/docs/help/mac_images/apple_menu.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 27 KiB

BIN
mac/docs/help/mac_images/input-menu-keyman-selected.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 33 KiB

BIN
mac/docs/help/mac_images/input_menu.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 28 KiB

BIN
mac/docs/help/mac_images/keyboard-add-input-source.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 266 KiB

BIN
mac/docs/help/mac_images/keyboard-input-sources-annotated.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 223 KiB

BIN
mac/docs/help/mac_images/keyboard-settings-annotated.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 206 KiB

BIN
mac/docs/help/mac_images/keyman-configuration-menu.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 50 KiB

BIN
mac/docs/help/mac_images/keyman-configuration-window.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 114 KiB

Some files were not shown because too many files have changed in this diff Show more