Commit graph

202 commits

Author SHA1 Message Date
Marc Durdin
55d6efe482 fix(developer): prevent ANSI keyboards crashing debugger
Note: #11909 has some additional future cleanup we could do, but these
code paths are not currently accessible, so in the interest of moving
forward, am leaving them for now.

Fixes: #11909
Test-bot: skip
2025-10-24 14:45:17 +02:00
Steven R. Loomis
9989217f9e feat(core): improve kmxplus validation
- restructured validation calls to be more consistent

Fixes: #9446
2025-03-17 12:09:04 -05:00
Steven R. Loomis
0011a2d10d feat(core): improve kmxplus validation
- improve an output string
- catch BMP noncharacters besides U+FFFF (H/T @mcdurdin - this was the whole point of the PR) plus test

Fixes: #9446

Co-authored-by: Marc Durdin <marc@durdin.net>
2025-03-17 11:26:57 -05:00
Steven R. Loomis
f1b3555114 feat(core,docs): improve kmxplus validation
- improve how validation works - a missing section does not mean an invalid section. distingush these.
- propagate errors for invalid sections
- update documentation of required sections

Fixes: #9446
2025-03-14 15:37:23 -05:00
Steven R. Loomis
cd002441e5 feat(core): check non-extended keys for bad unicode
- add test case
- turn this into not be an assert, so we can test it

Fixes: #9446
2025-03-14 10:04:21 -05:00
Steven R. Loomis
2e7b5b1417 feat(core): check non-extended keys for bad unicode
- also, use U+FFFD in core if it's a bad char

Fixes: #9446
2025-03-14 09:31:38 -05:00
Steven R. Loomis
2e50282087
Merge branch 'master' into feat/core/9446-detect-bad-unicode-kmxplus 2025-03-14 09:19:03 -05:00
Steven R. Loomis
1132fe5845 feat(core): tests for check on load for valid kmx_plus unicode
- turn off some asserts- makes untestable
(there are asserts at 'higher levels' such as loading the entire kmx+)
- add a test with a synthesized COMP_KMXPLUS_STRS - a valid and an invalid one

Fixes: #9446
2025-03-13 17:15:21 -05:00
Steven R. Loomis
fba5f72844 feat(core): tests for check on load for valid kmx_plus unicode
Fixes: #9446
2025-03-13 16:44:31 -05:00
Steven R. Loomis
2c2bfe7f15 feat(core): check on load for valid kmx_plus unicode
- error out if failure

Fixes: #9446
2025-03-12 17:45:47 -05:00
Marc Durdin
577cbd3a07 fix(core): unalign usetCount and rangeCount
Fixes: #13419
2025-03-07 12:50:22 +07:00
Marc Durdin
b973bcc0d9 fix(core): include nul in context offset calculations in kmx processor
Two separate bugs addressed, with `index()` references and with
`context()` references -- both have the same root cause, of not taking
`nul` at the start of the context into account (as `nul` is not included
in the `m_miniContext` member, being a non-character). We already fixed
this issue for `if()` quite a long time ago, and some of the same
patterns can be with `m_miniContextIfLen` for example.

Fixes: #13304
Fixes: #13316
2025-02-21 17:17:19 +07:00
Steven R. Loomis
c9643642a8 Merge remote-tracking branch 'origin/master' into fix/core/12298-get-key-list 2024-11-29 15:33:53 -06:00
Eberhard Beilharz
b8c120dd5c
Merge pull request #12721 from keymanapp/chore/core/cherry-pick/load-from-blob-api
feat(core,linux,developer,windows): implement loading KMX from blob 🍒
2024-11-29 12:40:51 +01:00
Eberhard Beilharz
0958a2da23
fix(core): permanently disable logging
This change disables logging at compile time to work around #12661.
Logging can be enabled in the debugger, or by re-compiling with
`g_debug_KeymanLog` set to TRUE.

Related: #12661
Cherry-pick: #12674
2024-11-27 16:36:25 +01:00
Eberhard Beilharz
3dc3f040de
feat(core): implement loading KMX from blob
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a

Part-of: #11293
Part-of: #8093
2024-11-27 15:30:05 +01:00
Steven R. Loomis
6d1035a4ad feat(core): test improvements for get_key_list()
- move kmxplus processing into the base LdmlTestSource class
- add a function to traverse the layer list looking for keys to add
- The @@keylist keyword only has one example from each modifier set

Fixes: #12298
2024-11-25 12:54:02 -06:00
Marc Durdin
1be73a9c2e refactor(windows): clean up logging
* Remove unused parameters from SendDebugMessage functions
* Add SendDebugEntry and SendDebugExit functions for tracking
  function entry/exit
* Add indenting and function names to log entries
* Remove unused debug functions
* Eliminate now-unused hwnd parameter in initialization functions
* Replace Log,LogEntry,LogExit functions with SendDebug equivalents in
  kmtip

Many functions now have SendDebugEntry/SendDebugExit (or
return_SendDebugExit) pairs. It is important to SendDebugExit on all
returns from a function to keep the log indent depth consistent. In some
cases I chose not to add these logging calls, e.g. on frequently called
functions such as the message hooks.
2024-07-03 21:28:37 +10:00
Steven R. Loomis
5d420248eb feat(core): move more normalization logic into JS
- major redo of actions_normalize - into UTF-32 and not using ICU directly
- add some utilities: u32len, u32dup, context_items_from_utf32

#9467
2024-05-24 18:20:30 -05:00
Steven R. Loomis
02413c0251 chore(core): update core to C++17
- use utfcodec.hpp's convert<>() instead of codecvt

Fixes: #8800
2024-05-08 16:41:30 -05:00
Steven R. Loomis
6706ae0957 chore(core): km_core_cp -> km_core_cu
- km_core_cp represents a 16 bit code unit, not a code point.

Fixes: #11033
2024-05-02 17:39:15 -05:00
Steven R. Loomis
c91095f0ca fix(core): core to automatically reset context if a frame key pressed
- update ldml test with an exception for k_102_keytest
2024-04-04 17:12:26 -05:00
Steven R. Loomis
5b1c853557 fix(common,core): support other lookup for modifiers
- the 'other' keyword was incorrectly called 'default'

Fixes: #11072
2024-04-01 23:21:33 -05:00
Steven R. Loomis
bca037cbf3 chore(core): support default lookup for modifiers
- remove default flag from kmx_file.h

Fixes: #11072
2024-03-29 12:07:29 -05:00
Steven R. Loomis
89f5a0e090 fix(core): support default lookup for modifiers
Fixes: #11072
2024-03-28 13:12:01 -05:00
Steven R. Loomis
7572c9476e feat(developer): support normalization=disabled 🙀
- main core changes and test changes

#10554
2024-02-01 17:32:42 -06:00
Marc Durdin
671973baab refactor(core): split context API from Core primary API
Relates to #9999.
Fixes #10384.

The context API endpoints should no longer be considered as part of the
standard Core API. The only consumers that have a need to access these
APIs are the IMX integration in Engine for Windows, and the Keyman
Developer Debugger.

These symbols are currently used by Developer:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()`
* `km_core_context_set()`
* `km_core_context_clear()`

These symbols are currently used by Windows IMX:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_context_items_dispose()`
* `km_core_context_item_list_size()`
* `km_core_state_get_intermediate_context()`

The following functions and symbols are moving to
keyman_core_api_context.h:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()` function
* `km_core_state_get_intermediate_context()` function
* `km_core_context_set()` function
* `km_core_context_clear()` function
* `km_core_context_get()` function
* `km_core_context_items_from_utf16()` function
* `km_core_context_items_from_utf8()` function
* `km_core_context_items_to_utf8()` function
* `km_core_context_items_to_utf16()` function
* `km_core_context_items_to_utf32()` function
* `km_core_context_items_dispose()` function
* `km_core_context_length()` function
* `km_core_context_append()` function
* `km_core_context_shrink()` function
* `km_core_context_item_list_size()` function
2024-01-17 14:44:25 +11:00
Marc Durdin
036c32af1f feat(core): normalize input context
First half of #9999.

Adds support for normalization (to NFD) of input app context into the
cached context. The keyboard processor will work with the NFD cached
context.

Adds unit tests for the normalization as part of the LDML keyboard
processor test suite.

TODO:
* Comparing modified cached context to app context to determine the
  transform required to send to the app
* Handling illegal unicode and unpaired surrogates on input context
2024-01-15 11:50:16 +07:00
rc-swag
1cb288144c chore(core): Merge branch 'master' into fix/core/clear-core-context-invalidate 2023-12-19 09:13:43 +10:00
Steven R. Loomis
1cd97fadb9 feat(core): ldml improve key-not-found 🙀
- move vkey_to_contextreset  table into a common spot in core
- assert that vk < 0x100 to avoid running off the table

For: #9451
2023-12-12 08:56:33 -06:00
rc-swag
4a511bde1e fix(core): clear core context on invalidate cache 2023-12-12 23:49:04 +10:00
rc-swag
4480a05412 chore(windows): removed cached context windows engine
Removed the cached context from the windows engine
2023-11-23 16:58:41 +10:00
Marc Durdin
958fb6a5c9 chore: add comment 2023-11-23 11:58:41 +10:00
Marc Durdin
21e0254c95 chore(core): remove vkey output from kmx processor
Fixes #10049.

vkey output was never supported by Core, so this code effectively was a
no-op.
2023-11-23 11:51:22 +10:00
Steven R. Loomis
d349959dea chore(core): ldml update to prefinal v44 🙀
- drop name section
- drop flick flags
- drop normalization,add name to meta
- debug printf fix
- assertion fix for display

For feat(developer): ldml re-sync with CLDR v44 spec 🙀  #9838
2023-10-26 18:35:55 -05:00
Eberhard Beilharz
f4d8529789
chore(linux): rename keyboardprocessor_ldml.* to keyman_core_ldml 2023-10-20 17:30:10 +02:00
Eberhard Beilharz
604fbc1a75
chore(linux): Rename namespace kbp to core 2023-10-19 10:39:55 +02:00
Eberhard Beilharz
2f073f7778
chore(linux): Rename KMN_CORE to KM_CORE_LIBRARY 2023-10-19 08:51:29 +02:00
Eberhard Beilharz
b5c6f31b05
chore(linux): Rename KBP to CORE 2023-10-18 15:48:21 +02:00
Marc Durdin
db76e2541d chore: rename KM_KBP_ to KM_CORE_ 2023-10-09 14:53:22 +07:00
Marc Durdin
ba5f18defd chore: rename km_kbp_ to km_core_ 2023-10-09 14:53:13 +07:00
Marc Durdin
906a0a2ed7 chore: fix typo 2023-10-09 14:41:01 +07:00
Marc Durdin
4ef18f5e29 chore(core): rename keyboardprocessor.h to keyman_core_api.h
Fixes #9721.

Also tweaks some documentation references to keyboardprocessor, but does
not touch those relating to debian/control.
2023-10-09 14:21:01 +07:00
Steven R. Loomis
bdef1c48d3 fix(common): update keyman for removal of vkeys 🙀
dropped in upstream CLDR-17093

See #7135
2023-09-28 13:03:44 -05:00
Steven R. Loomis
ab29171b56 fix(core): regex set mapping support 🙀
- fix alignment

#9121
2023-08-24 14:09:31 -05:00
Steven R. Loomis
09772bb18d feat(core): regex set mapping support 🙀
- passes
- added some utilities on transform_element
- added more validation of transforms
- added tests of utility functions

#9121
2023-08-23 18:13:19 -05:00
Steven R. Loomis
935dec4304 feat(core): regex set mapping support 🙀
- restructured some things, loading and such seems to be in good shape

#9121
2023-08-23 15:03:44 -05:00
Steven R. Loomis
b6855e121f feat(core): ldml: support <display id=...> in the 'disp' section 🙀
- "recent" addition to CLDR DTD

Fixes: #8717
2023-08-18 17:02:35 -05:00
Steven R. Loomis
e5bb22aedd
Merge branch 'master' into feat/core/9121-regex-epic-ldml 2023-08-14 11:17:19 -05:00
Steven R. Loomis
9f033278d4 feat(core): marker validation 🙀
- validate marker number when going in or out of the context

#9119
2023-08-11 15:09:54 -05:00