Commit graph

1045 commits

Author SHA1 Message Date
Marc Durdin
9a01ac166d chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-12-01 16:17:21 +07:00
Eberhard Beilharz
b8c120dd5c
Merge pull request #12721 from keymanapp/chore/core/cherry-pick/load-from-blob-api
feat(core,linux,developer,windows): implement loading KMX from blob 🍒
2024-11-29 12:40:51 +01:00
Marc Durdin
b0e87cb85a chore(core): update assertion unit test for release builds 2024-11-28 12:24:54 +07:00
Marc Durdin
36ddd793a5 chore(core): ifdef variables only used in assertions 2024-11-28 10:58:51 +07:00
Marc Durdin
bcecedcb04 chore(developer): ifdef variables only used in assertions 2024-11-28 10:36:13 +07:00
Marc Durdin
b9b4ad4996 chore(developer): ifdef variables only used in assertions 2024-11-28 10:27:40 +07:00
Marc Durdin
e2b8222a6b chore(developer): ifdef variables only used in assertions 2024-11-28 10:12:28 +07:00
Marc Durdin
e640caaa9c chore(core): Merge branch 'master' into fix/core/12619-disable-assertions-vcwin-release-build 2024-11-28 09:53:45 +07:00
Marc Durdin
d71cb56ca7 fix(core): rename assert() to test_assert() in unit tests
Had a real yak shave this morning with disabling assertions in release
builds in our C/C++ code. It turns out that our unit tests use
`assert()` which we intended to use from `test_assert.h`, but in some
cases `cassert` or `assert.h` had been #included after `test_assert.h`,
overriding our special `assert()` macro. The chain of includes is
somewhat hard to puzzle out -- it's often buried several levels deep.
This meant that a release build would drop all test assertions, meaning
most tests passed, unsurprisingly, as there were no assertions left to
fail ... but some tests failed with crashes because we optimized out
important lines such as `assert(some_important_function())`.

I was quite unhappy with this fragility, so I have opted to rename
`assert()` to `test_assert()` in all of our home-grown C/C++ unit tests,
which further highlighted unit tests which were only using the C/C++
`assert()` and not ours, so then had to figure out which unit test
executables needed to have `test_assert` added, and then ... then ...
discovered a bug in `test_color.h`, where we were #including
`io.h`/`unistd.h` inside a `namespace console_color {}` block, which
just happened to be the first ref to those beautiful headers, and thus
(because `#pragma once`) meant that useful little functions like
`access()` were no longer accessible to us in the global namespace.

I have also audited Every Single Call to `assert()` to verify that we do
not do Important Work inside the parentheses, and, apart from those
offending unit tests, now resolved with `test_assert()`, it looks like
all is good.

I would like to present one very well-shaved yak in this commit.

Fixes: #12619
2024-11-28 09:46:01 +07:00
Eberhard Beilharz
0958a2da23
fix(core): permanently disable logging
This change disables logging at compile time to work around #12661.
Logging can be enabled in the debugger, or by re-compiling with
`g_debug_KeymanLog` set to TRUE.

Related: #12661
Cherry-pick: #12674
2024-11-27 16:36:25 +01:00
Eberhard Beilharz
3dc3f040de
feat(core): implement loading KMX from blob
- split keyboard loading into loading KMX file into blob and then
  loading the keyboard processor from the blob.
- deprecate `km_core_keyboard_load`
- move file access next to deprecated method. This is now the only place
  that loads a file in Core; unit tests have some more places that
  load files.
- introduce GTest and add unit tests for loading from blob

Cherry-picked from `epic/web-core` branch.

Cherry-Pick-Commit: 1deaa323ad
Cherry-Pick-Commit: 59019cc8b7
Cherry-Pick-Commit: bc46458368
Cherry-Pick-Commit: d06aa29956
Cherry-Pick-Commit: 1c88166f6e
Cherry-Pick-Commit: 069cd21ecd
Cherry-Pick-Commit: 052ae2ec35
Cherry-Pick-Commit: 11a2a3ba3a

Part-of: #11293
Part-of: #8093
2024-11-27 15:30:05 +01:00
Eberhard Beilharz
718881a027
chore(core): rename test files
This change rename the test files for Core according to the discussion
at the Keyman conference in November 2024.
2024-11-26 17:47:11 +01:00
Marc Durdin
fbab6eb8cf chore(core): move API docs from help.keyman.com
Reorganizes /core/doc to /core/docs and moves internal files
accordingly.

Depends-on: keymanapp/help.keyman.com#1684
2024-11-07 08:28:33 +07:00
Darcy Wong
e62d05cff7 fix(core): Disable test_unicode check for ICU version 2024-09-11 20:56:50 +07:00
Marc Durdin
96f288a83d fix(core): properly support 'other' modifier state with uint32_t type
While the modifier state property in core's API is 16-bit, internally
ldml_processor supports the modifier flag LDML_KEYS_MOD_OTHER with a
value of `0x10000`, which requires widening the value (we match the
32-bit size of the KMX_DWORD value from KMX+).

Note: this is not yet well unit-tested.

Relates-to: #11072
Fixes: #12057
2024-08-24 12:26:52 +08:00
Eberhard Beilharz
a98ee7040b
Merge pull request #12235 from keymanapp/fix/core/emcc_detect
fix(core): look for `emcc` instead of `emcc.py`
2024-08-20 22:41:46 +02:00
Eberhard Beilharz
ab3722ecfa
fix(core): look for emcc instead of emcc.py on Linux and Mac
`emcc.py` is not marked as executable in emcripten's git repo so the
build failed when trying to locate emscripten. However, it turns out
that `emcc` is marked as executable, so we use that instead.

On Windows however, we still need to use `emcc.py` because otherwise
Meson won't detect it as valid compiler.
2024-08-20 15:52:52 +02:00
Steven R. Loomis
7c0f51c486 fix(core): set mac build version for meson cli build to 10.13
- add as option to compiler
- Match version in #11302

Fixes: #11423
2024-08-19 10:22:10 -05:00
Marc Durdin
bad54e4fed
Merge pull request #12171 from keymanapp/refactor/core/make-utfcodec-common
refactor(core): move utfcodec to common
2024-08-14 19:10:16 +10:00
Marc Durdin
613bdb6c19 refactor(core): move utfcodec to common
Refactor to support codecvt cleanup work in kmcmplib.
2024-08-13 15:04:07 +02:00
Marc Durdin
859c88a6de docs: add .kmx specification
Fixes: #11407
2024-08-12 17:04:24 +07:00
Marc Durdin
05edc8a7ce
Merge branch 'master' into refactor/windows/cleanup-engine-logging 2024-07-23 17:29:55 +10:00
Eberhard Beilharz
88f6c95803
fix(core): allow to successfully build on Ubuntu 24.04
Without this change trying to build on Ubuntu 24.04 fails with
"error: 'intptr_t' has not been declared in '::'". I'm not sure
why that is happening, but this is an easy fix.
2024-07-04 13:07:10 +02:00
Marc Durdin
1be73a9c2e refactor(windows): clean up logging
* Remove unused parameters from SendDebugMessage functions
* Add SendDebugEntry and SendDebugExit functions for tracking
  function entry/exit
* Add indenting and function names to log entries
* Remove unused debug functions
* Eliminate now-unused hwnd parameter in initialization functions
* Replace Log,LogEntry,LogExit functions with SendDebug equivalents in
  kmtip

Many functions now have SendDebugEntry/SendDebugExit (or
return_SendDebugExit) pairs. It is important to SendDebugExit on all
returns from a function to keep the log indent depth consistent. In some
cases I chose not to add these logging calls, e.g. on frequently called
functions such as the message hooks.
2024-07-03 21:28:37 +10:00
Steven R. Loomis
a16608e229
Merge branch 'master' into test/core/10968-minimum-ldml-test 2024-06-17 08:24:11 -05:00
Steven R. Loomis
38c6fe1ba3
Merge pull request #11778 from keymanapp/feat/core/9467-remove-icu-for-wasm-epic-ldml
feat(core): remove ICU from core under wasm 🙀
2024-06-17 08:22:38 -05:00
Steven R. Loomis
e580043538
Merge pull request #11777 from keymanapp/feat/core/9467-devolve-regex-to-js2-epic-ldml
feat(core): devolve regex to javascript 🙀
2024-06-17 08:22:22 -05:00
Marc Durdin
d6992097fd
Merge pull request #11795 from keymanapp/fix/core/11794-serialize-core-tests-temporary-mitigation
fix(core): serialize tests for core/wasm on mac agents
2024-06-17 16:26:27 +10:00
Marc Durdin
db53ac1343 fix(core): serialize tests for core/wasm on mac agents
Mitigates: #11794
2024-06-16 16:42:38 +10:00
Steven R. Loomis
ed80095ab9
Merge branch 'master' into test/core/10968-minimum-ldml-test 2024-06-14 09:33:29 -05:00
Steven R. Loomis
2300571ad8 test(core): fix a type conversion issue in the minimal test
- need a null string for either narrow or wide chars

Fixes: #10968
2024-06-14 09:05:50 -05:00
Steven R. Loomis
ca34550a8d feat(core): update util_regex per review comments
- remove some TODOs that were obsolete
- copy/clarify/expand comments between the ICU and non-ICU sides

Fixes: #9467
2024-06-14 08:55:15 -05:00
Steven R. Loomis
3c82343218 test(core): Add a minimal test that exercises the core API
- this test doesn't actually run well, it compiles but fails with an assertion.
- the purpose is to exercise the linker.

Fixes: #10968
2024-06-13 16:56:36 -05:00
Steven R. Loomis
b2e3215b0b
Merge branch 'feat/core/9467-devolve-regex-to-js2-epic-ldml' into feat/core/9467-remove-icu-for-wasm-epic-ldml 2024-06-13 14:29:18 -05:00
Steven R. Loomis
ff3c9736f4
Merge branch 'master' into feat/core/9467-devolve-regex-to-js2-epic-ldml 2024-06-13 14:29:01 -05:00
Steven R. Loomis
edd42d1dcb
Merge pull request #11541 from keymanapp/feat/core/9467-devolve-norm-to-js3-epic-ldml
feat(core): devolve normalization to js 🙀
2024-06-13 14:28:45 -05:00
Steven R. Loomis
a744afc34f feat(core): remove ICU from core under wasm
Fixes: #9467
2024-06-13 14:20:58 -05:00
Steven R. Loomis
1a61a589b3 feat(core): devolve regex to js for wasm
- assert was wrong!

Fixes: #9467
2024-06-13 13:07:29 -05:00
Steven R. Loomis
263d0e32f2 chore(core): update comments and remove a raw numeric literal
- per review comments

Fixes: #9467

Co-authored-by: rc-swag <58423624+rc-swag@users.noreply.github.com>
2024-06-13 12:17:55 -05:00
Marc Durdin
577986bb50
Merge pull request #11523 from keymanapp/chore/common/8399-cleanup-meson-warnings
chore(common): cleanup meson deprecations and warnings
2024-06-08 10:46:41 +10:00
Steven R. Loomis
acd3cc5847 feat(core): devolve regex to JS
- use JS regex code
- almost there - some perhaps escaping differences

Fixes: #9467
2024-06-07 18:11:20 -05:00
Steven R. Loomis
c4b01f7715 test(core): detailed unit test for util_regex
Fixes: #9467
2024-06-07 16:27:45 -05:00
Steven R. Loomis
55a025c767 refactor(core): devolve regex to js for wasm
- new module, core/src/util_regex.hpp
- no wasm implementationyet

Fixes: #9467
2024-06-05 16:53:43 -05:00
Steven R. Loomis
f156a7264d feat(core): generator in core/src for util_normalize_table.h
- temporary header file generated by wasm during build
- built using icu
- test_unicode verifies the contents and synchronization with running ICU.
- this is used by util_normalize to provide normalization properties under wasm without needing to include ICU.

Fixes: #9467
2024-06-05 15:47:20 -05:00
Steven R. Loomis
fdb2e95d20 feat(core): build improvements for --update-unicode option
Fixes: #9467
2024-06-04 12:26:00 -05:00
Steven R. Loomis
e9673867fa feat(core): speedup NFD boundary table
- use RLE encoding, thanks @mcdurdin
- much smaller table and faster lookup

Fixes: #9467
2024-06-04 12:23:16 -05:00
Steven R. Loomis
7fbeea85cc Merge remote-tracking branch 'upstream/master' into feat/core/9467-devolve-norm-to-js3-epic-ldml 2024-06-04 11:15:31 -05:00
Steven R. Loomis
53a6638a70 feat(core): generate and use static table in wasm for NFD boundary
- add core/tools build tree with custom targets
- add to core/build.sh to generate nfd_table.h
- test_unicode to validate Unicode version and compare NFD to actual ICU
- currently, linear search of the table.
2024-06-04 11:15:17 -05:00
Steven R. Loomis
31dedb7ca3 test(core): comments in get_block_unicode_ver() 2024-05-31 16:14:05 -05:00
Steven R. Loomis
403d342e10 test(core): clarify output of test_unicode
- group output by functional area

#10183
2024-05-27 16:42:11 -05:00