If `nul` is used in the output part of a rule, then text or
text-emitting statements cannot also be included in the output of that
rule. This was not enforced in earlier versions of Keyman.
Adds a unit test for the enhanced validation.
Fixes: #13455
Relates-to: keymanapp/keyboards#3379
Add 4 test keyboards to validate `nul` and `if` used in conjunction with
`index` and `context` and corresponding references in core unit tests.
Add a script to rebuild baseline keyboards using a copy of kmcomp.exe
16.0.138; this is setup and tested only on Windows (YMMV on WINE, etc).
Add the 4 additional baseline test keyboards to kmcmplib unit tests for
build consistency between kmcomp 16 and kmcmplib 18 (all pass).
Note: two of the new Core tests currently fail. This is expected, see
issue #13304.
Fixes: #13303
Add validity checks to kmcmplib to verify that offsets provided to
`context(n)` are in range for the context, and do not point to
non-character elements (`nul` and `if`) in the context string.
Fixes: #13301Fixes: #13302
Relates-to: #13299
Relates-to: #13276
Had a real yak shave this morning with disabling assertions in release
builds in our C/C++ code. It turns out that our unit tests use
`assert()` which we intended to use from `test_assert.h`, but in some
cases `cassert` or `assert.h` had been #included after `test_assert.h`,
overriding our special `assert()` macro. The chain of includes is
somewhat hard to puzzle out -- it's often buried several levels deep.
This meant that a release build would drop all test assertions, meaning
most tests passed, unsurprisingly, as there were no assertions left to
fail ... but some tests failed with crashes because we optimized out
important lines such as `assert(some_important_function())`.
I was quite unhappy with this fragility, so I have opted to rename
`assert()` to `test_assert()` in all of our home-grown C/C++ unit tests,
which further highlighted unit tests which were only using the C/C++
`assert()` and not ours, so then had to figure out which unit test
executables needed to have `test_assert` added, and then ... then ...
discovered a bug in `test_color.h`, where we were #including
`io.h`/`unistd.h` inside a `namespace console_color {}` block, which
just happened to be the first ref to those beautiful headers, and thus
(because `#pragma once`) meant that useful little functions like
`access()` were no longer accessible to us in the global namespace.
I have also audited Every Single Call to `assert()` to verify that we do
not do Important Work inside the parentheses, and, apart from those
offending unit tests, now resolved with `test_assert()`, it looks like
all is good.
I would like to present one very well-shaved yak in this commit.
Fixes: #12619
anii.kmn and sil_kmhmu.kmn both have mismatching case in filename
references for icons. For now, we will disable tests for these two
keyboards, so that tests pass on Linux (which has case-sensitive
filesystem).
Note: the filename case was already addressed in the keyboards repo, so
when we realign to a more recent commit for the test fixtures, we should
be able to include these tests again.
Relates-to: #12623
On mac arch build, kmcmplib is failing to trim U+2002, unlike all other
platforms. Using UTF-8 locale should treat the ISO 30112 POSIX space
characters, which includes U+2002, as whitespace.
While kmc did this already in its callbacks, the unit test callbacks
did not, which meant that some keyboard tests failed on Linux and macOS.
Relates-to: #12623
The --full-test parameter in kmcmplib build.sh has not been working.
This PR reconnects the parameter and also enables it by default for CI
test builds (not release builds).
Fixes: #12623
masaram_gondi.kmx fixture is built from a known-bad source, which is
caught in kmcmplib 18.0.
This failure is because the keyboard source had non-BMP characters in
the key part of the rule on line 215, which has been picked up by the
18.0 compiler in #11806:
```
$keymanonly: if(opt1 = 'gondi') any(Vyanjana)"𑵄" + any(Vyanjana) > index(Vyanjana,2) U+11D45
```
The trick here is that the compiled fixture
developer/src/kmcmplib/tests/fixtures/keyboards-repo/masaram_gondi.kmx
is based on a [broken version of the keyboard
source](965ef1941f/release/m/masaram_gondi/source/masaram_gondi.kmn)
in the keyboards repo ([fixed
version](06cfd2d186/release/m/masaram_gondi/source/masaram_gondi.kmn)),
and updating to the fixed version will also trigger updates to a bunch
of other keyboards. I think the simplest resolution is probably to skip
masaram_gondi for 18.0.
Relates-to: #12623
Relates-to: #11806
