Commit graph

157 commits

Author SHA1 Message Date
Marc Durdin
ee0fc0ce14 chore(developer): cleanup npm publish
Use a new flag `--npm-publish` in conjunction with `publish` action,
so that the default will always be to `npm pack` if the new flag is not
specified. This flag is also guarded in the actual npm publish code to
ensure that it can only run in the appropriate CI alpha/beta/stable
environment, and not in local or test.

This then also removes the separate `pack` action.

Also removes legacy boilerplate from a number of build scripts.
2024-05-20 06:34:57 +07:00
Steven R. Loomis
1fe76bcd26
Merge branch 'master' into chore/core/11033-points-not-units 2024-05-07 09:26:07 -05:00
Steven R. Loomis
6706ae0957 chore(core): km_core_cp -> km_core_cu
- km_core_cp represents a 16 bit code unit, not a code point.

Fixes: #11033
2024-05-02 17:39:15 -05:00
Marc Durdin
cd8121ef65 chore(common): builder scripts now use /resources/build/builder.inc.sh
Note: there is a bit of potential confusion about the difference between
/resources/builder.inc.sh (the full implementation for builder scripts),
and /resources/build/builder.inc.sh (the source script that builder
scripts should always use).

This allows us to make assumptions that will always be true for builder
scripts that may not be true for other scripts, such as setting base
folder.
2024-05-02 15:54:31 +07:00
Marc Durdin
0f390d47cc chore(common): maintenance on build scripts - cd
Fixes #11324.

* Always `cd "$THIS_SCRIPT_PATH"`
* Remove unnecessary `cd` from all build.sh
* Remove unnecessary `set -eu` from all build.sh (and `# set -x`)
* Replace old build-utils.sh incantation in a few build.sh scripts
2024-05-02 15:54:29 +07:00
Steven R. Loomis
5b1c853557 fix(common,core): support other lookup for modifiers
- the 'other' keyword was incorrectly called 'default'

Fixes: #11072
2024-04-01 23:21:33 -05:00
Steven R. Loomis
313bbd32a7 feat(core): kmx+ scaffolding for modifiers=default
- add a new value, 0x10000 to indicate 'default'

For: #11072
2024-03-28 12:19:41 -05:00
Marc Durdin
7f7bbe5030
chore: Apply suggestions from code review
Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2024-03-22 17:31:28 +11:00
Marc Durdin
7136a69ac8 chore(core): fix links and tweaks docs 2024-03-21 06:12:59 +07:00
Marc Durdin
7bfda9881f chore(core): fix typos 2024-03-20 13:30:05 +07:00
Marc Durdin
216d4afdef chore(core): fix typos 2024-03-20 13:09:09 +07:00
Marc Durdin
56e70b82d1 chore(core): add links to other markdown files for v17 2024-03-20 13:08:24 +07:00
Marc Durdin
f43d5aa53f chore: copy change docs into header as well 2024-03-20 12:58:42 +07:00
Marc Durdin
73daec43c1 chore(core): refresh documentation and add build tool
Refreshes all of the public API header documentation so we have a single
source for the documentation -- the keyman_core_api.h header. A lot of
reformatting but very little change in terms of content.

This is converted into Markdown by api-header-extractor, and the results
can be copied to help.keyman.com.

Will only do the main keyman_core_api.h for v17 release, in interests of
time. The other headers are primarily internal only and have many
deprecated functions also.

Will also leave the full automation of the documentation generation to a
future PR. The script will be there but it won't run during build for
now.
2024-03-20 12:56:08 +07:00
Marc Durdin
83ac78ab7c chore(core): refresh API docs for 17.0
Fixes #10580.
2024-03-13 09:23:07 -05:00
Steven R. Loomis
17a653fa19 chore(core): ldml update out of techpreview 🙀
- updates to support techpreview -> 45
- test changes

Fixes: #10900
2024-03-02 22:40:39 -06:00
Steven R. Loomis
8975a38109 chore(common): ldml update out of techpreview 🙀
- updates to support techpreview -> 45

Fixes: #10900
2024-03-02 22:40:38 -06:00
Marc Durdin
1b88ab99be fix(core): make km_core_state_get_actions() idempotent
Fixes #10582.

Returned struct from the `km_core_state_get_actions()` API is now owned
by the state object. This is a change in API contract. The corresponding
`km_core_actions_dispose()` API has been made private, because there is
never any need for API consumers to call it. As this change is happening
in alpha, we won't bump the ABI version.

`state->app_context` is now guaranteed to be in sync with
`state->context` after `km_core_process_event()`, with the actions
struct populated at that point.

The next and final step in this refactor is to remove the actions queue
altogether from the Core and make that a kmn-only concept, but that will
wait until 18.0. The only consumers of the actions queue are the
kmn-specific interactive debugger in Keyman Developer, and kmn-specific
Input Method eXtensions in Keyman Engine for Windows.
2024-02-01 11:28:08 +07:00
Marc Durdin
9ee5777e48 chore(core): move action apis into keyman_core_api_actions.h
Fixes #10520.

The definitions for `struct km_core_action_item`, `struct
km_core_backspace_item`, `enum km_core_backspace_type`, `enum
km_core_action_type`, `km_core_state_action_items()`,
`km_core_state_queue_action_items()`, `km_core_process_queued_actions()`
are now in keyman_core_api_actions.h rather than in the primary
keyman_core_api.h, as they should not be used by engines in general.

The current users of those APIs are the interactive kmx debugger, and
the IMX integration in Keyman Engine for Windows. Both of these depend
on the old action queue model rather than the actions struct.

In a future version, we may refactor these further to make them apply
directly to the KMX processor, and provide access via an interface to
the KMX processor rather than as a general Core API. Once this change is
made, then Core itself will no longer support action queues at all, and
will expect keyboard processors to fill in an action struct (this is
already done for LDML).
2024-01-31 11:32:09 +07:00
Marc Durdin
f546c44048
Merge pull request #10390 from keymanapp/epic/core/9999-normalization
epic: Keyman Core normalization 🌱
2024-01-31 10:45:51 +11:00
Marc Durdin
868247e9fe chore(core): address review comments 2024-01-30 07:36:52 +07:00
Marc Durdin
8501f2aa6d feat(core): add deleted_context to km_core_actions struct
Fixes #10530.

Adds the deleted_context member to the km_core_actions struct, and
associated unit tests. Simplifies integration by providing the consumer
with all the data they need in order to execute the transform,
specifically around number of delete operations required, without
needing to query the target application context again. The number of
delete operations will vary according to application compliance and
selected encoding; for example a UTF-16 string may require 2
delete-backs for surrogate pairs in the text buffer for a compliant app,
whereas there will be a single delete-back key event for a non-compliant
app.

The deleted_context member should also be used for debug assertions.
2024-01-29 13:21:25 +07:00
Marc Durdin
41cc2f4d1e feat(developer): make context_get and context_length public again
Rolls back the privatisation of the km_core_context_get and
km_core_context_length APIs because the debugger uses them.
2024-01-26 06:37:09 +07:00
Marc Durdin
4906545e31
Merge branch 'epic/core/9999-normalization' into refactor/core/10431-remove-unused-core-context-apis 2024-01-24 12:14:11 +11:00
Steven R. Loomis
f2082d79a6 feat(developer): ldml marker normalization in ts 🙀
- cleanup and move marker_no_index into the shared area

For: #10317
2024-01-19 18:07:51 -06:00
Steven R. Loomis
6ea83391a7 Merge remote-tracking branch 'origin/epic/core/9999-normalization' into feat/core/10410-ldml-really-use-action-struct 2024-01-19 10:40:42 -06:00
Marc Durdin
4f909dcc83
Merge pull request #10403 from keymanapp/feat/core/9999-normalize-output
feat(core): infrastructure for normalization of output 🌱
2024-01-19 13:51:00 +11:00
Marc Durdin
a26e21aba7 chore(core): address review comments 2024-01-19 08:01:45 +07:00
Steven R. Loomis
4c322ddb8a feat(core): ldml action struct 🌱
- update km_core_actions.output to be a const*
- for now, call state->actions().commit() from ldml_event_state::commit()

For: #10410
2024-01-18 22:26:46 -06:00
Marc Durdin
83f80c061a refactor(core): remove unused context APIs
Fixes #10431.

The `km_core_` prefix has been removed from internal-only functions, and
these function declarations moved to context.hpp.

The functions have not been moved from km_core_context_api.cpp at this
stage.

Rewrote the function documentation in Javadoc style comments for the
internal use functions.
2024-01-18 12:18:07 +07:00
Marc Durdin
c6acf86629
Merge pull request #10419 from keymanapp/chore/core/10410-ldml-just-say-no-to-action-queues
chore(core): ldml assert if queue functions are called 🌱
2024-01-18 14:47:04 +11:00
Marc Durdin
eece9716b7 refactor(core): additional comments 2024-01-18 10:42:18 +07:00
Steven R. Loomis
f484615450 chore(core): ldml don't use queue action functions 🌱 2024-01-17 16:16:56 -06:00
Marc Durdin
671973baab refactor(core): split context API from Core primary API
Relates to #9999.
Fixes #10384.

The context API endpoints should no longer be considered as part of the
standard Core API. The only consumers that have a need to access these
APIs are the IMX integration in Engine for Windows, and the Keyman
Developer Debugger.

These symbols are currently used by Developer:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()`
* `km_core_context_set()`
* `km_core_context_clear()`

These symbols are currently used by Windows IMX:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_context_items_dispose()`
* `km_core_context_item_list_size()`
* `km_core_state_get_intermediate_context()`

The following functions and symbols are moving to
keyman_core_api_context.h:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()` function
* `km_core_state_get_intermediate_context()` function
* `km_core_context_set()` function
* `km_core_context_clear()` function
* `km_core_context_get()` function
* `km_core_context_items_from_utf16()` function
* `km_core_context_items_from_utf8()` function
* `km_core_context_items_to_utf8()` function
* `km_core_context_items_to_utf16()` function
* `km_core_context_items_to_utf32()` function
* `km_core_context_items_dispose()` function
* `km_core_context_length()` function
* `km_core_context_append()` function
* `km_core_context_shrink()` function
* `km_core_context_item_list_size()` function
2024-01-17 14:44:25 +11:00
Marc Durdin
e39affe93b feat(core): infrastructure for normalization of output
Relates to #9999.

Establishes functions, unit test sources, normalization entry point and
an effectively no-op unit test for normalization support.
2024-01-16 13:46:46 +07:00
Marc Durdin
e5c2525241 refactor(core): split context API from Core primary API
Relates to #9999.
Fixes #10384.

The context API endpoints should no longer be considered as part of the
standard Core API. The only consumers that have a need to access these
APIs are the IMX integration in Engine for Windows, and the Keyman
Developer Debugger.

These symbols are currently used by Developer:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()`
* `km_core_context_set()`
* `km_core_context_clear()`

These symbols are currently used by Windows IMX:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_context_items_dispose()`
* `km_core_context_item_list_size()`
* `km_core_state_get_intermediate_context()`

The following functions and symbols are moving to
keyman_core_api_context.h:
* `km_core_context` struct
* `km_core_context_type` enum
* `km_core_context_item` struct
* `KM_CORE_CONTEXT_ITEM_END` macro
* `km_core_state_context()` function
* `km_core_state_get_intermediate_context()` function
* `km_core_context_set()` function
* `km_core_context_clear()` function
* `km_core_context_get()` function
* `km_core_context_items_from_utf16()` function
* `km_core_context_items_from_utf8()` function
* `km_core_context_items_to_utf8()` function
* `km_core_context_items_to_utf16()` function
* `km_core_context_items_to_utf32()` function
* `km_core_context_items_dispose()` function
* `km_core_context_length()` function
* `km_core_context_append()` function
* `km_core_context_shrink()` function
* `km_core_context_item_list_size()` function
2024-01-16 12:04:41 +07:00
Marc Durdin
cd69405960 feat(core): add app_context and failing tests
Adds the application context member of state and a corresponding,
failing unit test, which starts the process of verifying the handling
of the application context.
2024-01-15 11:50:15 +07:00
Marc Durdin
32ee4bc386 fix(core): typedef enum 2024-01-15 08:00:32 +07:00
Marc Durdin
bfb26eaf64 feat(core): adds km_core_state_context_debug
Fixes #10365.

Adds a `km_core_state_context_debug` function that returns a km_core_cp
string containing the current cached or intermediate context. The
function must be used only for debug logging purposes -- the string is
not intended to parsed for other purposes and the format may change in
the future.

This commit adds the function and corresponding unit tests. Subsequent
commits will update existing context debug functions in various engines
to use this API endpoint instead.
2024-01-15 07:19:31 +07:00
Eberhard Beilharz
9b768c57d7
refactor(linux): Use km_core_state_context_set_if_needed
Also update comment on `km_core_state_context_set_if_needed` to make it
clearer that it's expecting a UTF-16 string.

Closes #10212.
2024-01-10 18:57:41 +01:00
Marc Durdin
b6daac0bdb feat(core): action struct to action items conversion
Adds state->set_actions(). This sets the Core's action list to match the
contents of the action struct. Note that markers are not supported and
backspace expected_values will be empty, as this information is not
available. As the intended consumer of the action struct does not need
to know this information, this should be adequate.
2023-12-01 12:27:49 +07:00
Steven R. Loomis
f72c571696
Merge branch 'master' into chore/developer/9838-cldr44-epic-ldml 2023-10-26 18:38:14 -05:00
Steven R. Loomis
a8e2be5e9e chore(developer): ldml update to prefinal v44 🙀
- more tweaks

For feat(developer): ldml re-sync with CLDR v44 spec 🙀  #9838
2023-10-25 15:40:24 -05:00
Marc Durdin
8475b68248 feat(core): make persist_options always point to a valid array 2023-10-25 05:46:43 +07:00
Steven R. Loomis
1ff2bfc4d7 chore(developer,common,core): ldml update to prefinal v44 🙀
- more data file updates
- now updating some developer files
- updating the KMX+ spec with some changes.
 (vkey was dropped in #7135, remove it from the spec)

For feat(developer): ldml re-sync with CLDR v44 spec 🙀  #9838
2023-10-24 17:28:08 -05:00
Marc Durdin
82177013e9 chore(core): Merge branch 'master' into feat/core/9720-actions-apis 2023-10-24 20:55:10 +07:00
Marc Durdin
40ca1d1ed8 chore(core): address review comments 2023-10-24 20:53:44 +07:00
Marc Durdin
69e92f312a chore(core): rename to code_points_to_delete 2023-10-24 13:46:52 +07:00
Marc Durdin
cd428ae412 feat(core): address review comments
* Renamed various functions and types
* Clarified return values
* Updated doc comments
2023-10-24 12:22:34 +07:00
Marc Durdin
cbb708e74d chore(core): bool to int for cross-platform 2023-10-24 09:48:08 +07:00