First half of #9999.
Adds support for normalization (to NFD) of input app context into the
cached context. The keyboard processor will work with the NFD cached
context.
Adds unit tests for the normalization as part of the LDML keyboard
processor test suite.
TODO:
* Comparing modified cached context to app context to determine the
transform required to send to the app
* Handling illegal unicode and unpaired surrogates on input context
Adds the application context member of state and a corresponding,
failing unit test, which starts the process of verifying the handling
of the application context.
Fixes#10365.
Adds a `km_core_state_context_debug` function that returns a km_core_cp
string containing the current cached or intermediate context. The
function must be used only for debug logging purposes -- the string is
not intended to parsed for other purposes and the format may change in
the future.
This commit adds the function and corresponding unit tests. Subsequent
commits will update existing context debug functions in various engines
to use this API endpoint instead.
It was fairly surprising to have the unit tests for
`km_core_state_context_*` methods in `action_api.cpp`, so this change
moves them to a separate file.
- markers/normalization really needs its own file
- split out ldml_markers.* from ldml_transforms.*
- split out ldml_utils.hpp for a common assert macro
#10320
- renamed the single-segment functions to normalize_nfd_markers_segment() since they should only be run on a single segment
- commented out NFC for now, dead code
- add some additional checks/DebugLog around normalization
#10320
- more err reporting out of inner functions such as key not found
- helper functions for setting FAIL and SKIP actions with messages
- colorization
For: #9121
- the intermediate stages of transforms also need to use marker-safe normalization
- re-enable a test that was failing previously due to this
For: #9468
The unit tests needed to be updated to handle the cases
where the context is invalidated by a keystroke. However, they
still need to be able to verify the final string the application
should recieve. To do this a expected context token has been added
to the unit test format.
Test k_000___null_keyboard has been updated as the [RALT K_B] should
cause the context to be reset.
Test k_008___vkey_input__ctrl_alt_2_ has been updated as the [LCTRL LALT
K_B] does not match a keyboard rule the context is deemed invalid and should be
reset.
It could be a consideration to modify the text_store object to better
reflect the state of a applications context however this would mean
we would need two expected results one for the application and one
for the context
- also affects markers, feat(core): normalization per spec for transforms/etc 🙀#9468
- keep markers in nfd context string
- fix ldml test harness to handle context reset
- update test case
- still issues with overproduction of markers in the context
For: #9451
Adds state->set_actions(). This sets the Core's action list to match the
contents of the action struct. Note that markers are not supported and
backspace expected_values will be empty, as this information is not
available. As the intended consumer of the action struct does not need
to know this information, this should be adequate.
In the case when the cached context had been cleared the
km_core_state_context_set_if_needed call would just compare
the null terminations of both strings and not set the cached
context to the application context.
- distinguish between unmapped and 0-length output strings
- don't do any processing for 0-length output strings
- no change to developer, that will be next
- remove a comment referencing transform=no
For: #9451
Fixes#10067.
Management of memory for persisted options was wrong in the action
struct, as the members key and value would be freed immediately after
being added to the temporary vector (because the vector was of the
struct rather than of the class).
Given the struct is a C struct, we need the memory management to be
explicit, so we now release() each option into the vector as we create
it, which means that its member values will not be freed when the option
is then immediately deleted. (This allows us to use the initial copy of
the members of option that option() constructor does.)
Added the release() function as that was a relatively clear way of
indicating that the contents of the structure are now owned by the
caller, following the pattern from std::unique_ptr.
Finally, the unit test for persisted options was in the action_api.cpp
test module, but it was never called, so this was not being tested. Now
it is.
