* Move the following files from common/windows/cpp/include/ to
common/include/:
* keymanversion.h
* vkeys.h
* Move the following file from common/windows/cpp/ to common/cpp/:
* vkeys.cpp
* Move crc32.cpp,crc32.h into kmcmplib, as they are only used there.
* Remove unused ConvertUTF.c and ConvertUTF.h.
* Update references to the above files.
* Remove precompiled header refs from vkeys.cpp and change data type to
`char*` for types in vkeys.cpp (unless `USE_CHAR16_T` is defined), so
that it is more cross-platform accessible, update usages accordingly,
fixup project definitions accordingly.
* Remove unused define `KMN_KBP_EXPORTING` from kmcmplib/meson.build.
Fixes: #15565
Test-bot: skip
This is basically the same test as `k_0201___ralt_2.kmn` but uses
ALT instead of RALT in the rule.
Also update `k_0201___ralt_2.kmn` to be a bit more extensive.
Note: since the change in `k_0201___ralt_2.kmn` is only in the test
data, i.e. the comments, the .js and .kmx files don't change.
Related-to: #15358
Test-bot: skip
sil_yi was impacted by the changes in #14746, as it had the group name
'Unicode Group', which is now illegal, so the compiler fails to build
the keyboard at the referenced commit. Easiest workaround currently is
to remove it from the set of compared keyboards.
The compiler has always been very ambiguous on which characters were
accepted in group and store names, even to the point of accepting
things like comma in a store name, which would then make it impossible
to reference in an `index` statement!
This commit clarifies the allowable characters in an identifier. While
it would have been possible to use UAX#31 for this, that would have
extended the requirements for this change substantially, and may have
caused us more trouble with legacy keyboards. Given kmcmplib is
end-of-life (see epic/ng-compiler), I have chosen a lower friction
approach. There are certainly other characters that could be excluded,
but in general I have chosen to exclude only those that will definitely
be problematic.
The set of allowable characters for deadkeys has actually been expanded
in this release to match the store and group name rules.
It is expected that there may be some impacted keyboards, but addressing
this change will be relatively straightforward, so I consider this to be
an acceptable back-compatibility trade-off, see
https://github.com/keymanapp/keyman/wiki/Principles-of-Keyman-Code-Changes#4-source-backward-compatibility-keyboard-model-and-package-source-file-formats-should-be-backward-compatibleFixes: #14604
Test-bot: skip
Build-bot: skip build:developer
Removes a number of unused functions from the file, renames remaining
functions to match our builder script prefix patterns, removes unused
imports from various build scripts (e.g. Android).
Fixes: #14453
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.
Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh
Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
android/build.sh.
* Renames functions in builder-basic.inc.sh
More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.
Fixes: #14065
Build-bot: build all
Test-bot: skip
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.
Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
We attempted to set the C++ version in standard.meson.build, but this
does not work well as the option name varies by platform, and it also
triggers a warning in meson. This fix moves the setting into each
meson.build project() default_options. As there are only a handful
of these, the maintenance burden is not high.
Fixes: #14432
Relates-to: #14355
Build-bot: build
Test-bot: skip
- add `builder_is_windows`, `builder_is_macos` and `builder_is_linux`
functions
- add/move `BUILDER_OS` env variable to `resources/builder.inc.sh`
- use the new functions and env variable
Fixes: #14109
Test-bot: skip
Use a shared function standard_meson_build to prep the
resources/meson.build folder for consistency. A future cleanup would be
to use meson-utils.inc.sh for all meson build scripts.
* Adds `VERSION_WITH_TAG` variable to standard.meson.build
* Consistently uses `file('VERSION.md')` to retrieve version number
Note that VERSION_WITH_TAG is not yet used but should be used for
version tagged files later.
If `nul` is used in the output part of a rule, then text or
text-emitting statements cannot also be included in the output of that
rule. This was not enforced in earlier versions of Keyman.
Adds a unit test for the enhanced validation.
Fixes: #13455
Relates-to: keymanapp/keyboards#3379
Add 4 test keyboards to validate `nul` and `if` used in conjunction with
`index` and `context` and corresponding references in core unit tests.
Add a script to rebuild baseline keyboards using a copy of kmcomp.exe
16.0.138; this is setup and tested only on Windows (YMMV on WINE, etc).
Add the 4 additional baseline test keyboards to kmcmplib unit tests for
build consistency between kmcomp 16 and kmcmplib 18 (all pass).
Note: two of the new Core tests currently fail. This is expected, see
issue #13304.
Fixes: #13303
Add validity checks to kmcmplib to verify that offsets provided to
`context(n)` are in range for the context, and do not point to
non-character elements (`nul` and `if`) in the context string.
Fixes: #13301Fixes: #13302
Relates-to: #13299
Relates-to: #13276
Had a real yak shave this morning with disabling assertions in release
builds in our C/C++ code. It turns out that our unit tests use
`assert()` which we intended to use from `test_assert.h`, but in some
cases `cassert` or `assert.h` had been #included after `test_assert.h`,
overriding our special `assert()` macro. The chain of includes is
somewhat hard to puzzle out -- it's often buried several levels deep.
This meant that a release build would drop all test assertions, meaning
most tests passed, unsurprisingly, as there were no assertions left to
fail ... but some tests failed with crashes because we optimized out
important lines such as `assert(some_important_function())`.
I was quite unhappy with this fragility, so I have opted to rename
`assert()` to `test_assert()` in all of our home-grown C/C++ unit tests,
which further highlighted unit tests which were only using the C/C++
`assert()` and not ours, so then had to figure out which unit test
executables needed to have `test_assert` added, and then ... then ...
discovered a bug in `test_color.h`, where we were #including
`io.h`/`unistd.h` inside a `namespace console_color {}` block, which
just happened to be the first ref to those beautiful headers, and thus
(because `#pragma once`) meant that useful little functions like
`access()` were no longer accessible to us in the global namespace.
I have also audited Every Single Call to `assert()` to verify that we do
not do Important Work inside the parentheses, and, apart from those
offending unit tests, now resolved with `test_assert()`, it looks like
all is good.
I would like to present one very well-shaved yak in this commit.
Fixes: #12619
anii.kmn and sil_kmhmu.kmn both have mismatching case in filename
references for icons. For now, we will disable tests for these two
keyboards, so that tests pass on Linux (which has case-sensitive
filesystem).
Note: the filename case was already addressed in the keyboards repo, so
when we realign to a more recent commit for the test fixtures, we should
be able to include these tests again.
Relates-to: #12623
On mac arch build, kmcmplib is failing to trim U+2002, unlike all other
platforms. Using UTF-8 locale should treat the ISO 30112 POSIX space
characters, which includes U+2002, as whitespace.
While kmc did this already in its callbacks, the unit test callbacks
did not, which meant that some keyboard tests failed on Linux and macOS.
Relates-to: #12623
The --full-test parameter in kmcmplib build.sh has not been working.
This PR reconnects the parameter and also enables it by default for CI
test builds (not release builds).
Fixes: #12623
masaram_gondi.kmx fixture is built from a known-bad source, which is
caught in kmcmplib 18.0.
This failure is because the keyboard source had non-BMP characters in
the key part of the rule on line 215, which has been picked up by the
18.0 compiler in #11806:
```
$keymanonly: if(opt1 = 'gondi') any(Vyanjana)"𑵄" + any(Vyanjana) > index(Vyanjana,2) U+11D45
```
The trick here is that the compiled fixture
developer/src/kmcmplib/tests/fixtures/keyboards-repo/masaram_gondi.kmx
is based on a [broken version of the keyboard
source](965ef1941f/release/m/masaram_gondi/source/masaram_gondi.kmn)
in the keyboards repo ([fixed
version](06cfd2d186/release/m/masaram_gondi/source/masaram_gondi.kmn)),
and updating to the fixed version will also trigger updates to a bunch
of other keyboards. I think the simplest resolution is probably to skip
masaram_gondi for 18.0.
Relates-to: #12623
Relates-to: #11806
