Commit graph

68 commits

Author SHA1 Message Date
Marc Durdin
0988a86fa4 maint(resources): improve builder_launch to allow for arbitrary parameters
Also cleans up reporting of child scripts and links the list of reserved
parameters across a few locations. Address review comments.
2025-11-14 10:44:35 +01:00
Marc Durdin
c573709ee6 maint(resources): add build timing report to builder
This is enabled automatically for CI builds, and can be enabled with
`--timing` parameter otherwise. Also, adds `builder_launch`,
documentation to come.

Test-bot: skip
2025-11-10 11:03:45 +01:00
Eberhard Beilharz
ab33d638f5
maint(linux): check that command is at beginning of line
Addresses code review comments.

Co-Authored-By: Marc Durdin <marc@durdin.net>
2025-09-08 16:52:56 +02:00
Eberhard Beilharz
612cd01743
maint(linux): allow to skip API check
This change adds an option to skip the API checks during a build.
This can be done by adding the trailer `Keyman-Api-Check: skip`
to the PR description.

Fixes: #10454
Test-bot: skip
Keyman-Api-Check: skip
2025-09-05 19:04:23 +02:00
Marc Durdin
f75aaf0efe chore(resources): Merge branch 'master' into maint/resources/14565-fix-build-bot-parse-and-add-logging 2025-08-28 04:45:53 +02:00
Eberhard Beilharz
64e6fcd467
Merge pull request #14580 from keymanapp/refactor/common/shellcheck 2025-08-27 18:27:54 +02:00
Eberhard Beilharz
1e7ba23668
refactor(common): fix build-bot trigger script
Also rename variables to make it clearer that `buildBotCommand` contains
multiple commands.
2025-08-27 17:32:05 +02:00
Marc Durdin
6c2ef33492 maint(resources): fix parse bug in build-bot and add extra logging
Fixes: #14565
Build-bot: skip build:common
Test-bot: skip
2025-08-27 14:12:51 +02:00
Eberhard Beilharz
d16a46d5ad
feat(resources): add option to suppress output to bash testing framework
This change adds the `--quiet` option to the `run_tests` function of
the testing framework. This will redirect the regular output to a log
file (`/tmp/<testname>.log`) and only output the pass/failed result.
2025-08-25 17:35:27 +02:00
Eberhard Beilharz
4c57967e79
refactor(common): fix shellcheck warnings
Test-bot: skip
2025-08-22 17:52:09 +02:00
Marc Durdin
bb1f2006f1
Merge branch 'master' into maint/resources/14506-dont-fallback-to-build-on-unsupported-targets-in-child-builds 2025-08-19 17:39:29 +07:00
Marc Durdin
50a03b502a maint(resources): don't fall back to 'build' action
When an unsupported action is provided to a child build, then don't fall
back to the default action 'build', because generally that is not what
we want -- we want a no-op

Fixes: #14506
Build-bot: skip:all build:common_linux,common_mac,common_windows
Test-bot: skip
2025-08-19 12:32:46 +02:00
Marc Durdin
81c807c257 chore(common): Merge branch 'master' into maint/common/14453-move-ci-scripts 2025-08-19 10:46:11 +02:00
Marc Durdin
a81138c62b
Merge pull request #14449 from keymanapp/maint/common/14065-consolidate-builder-scripts
maint(common): consolidate builder scripts 🐸
2025-08-12 10:41:09 +07:00
Marc Durdin
f95b26ac1f
Merge pull request #14446 from keymanapp/maint/common/14275-rename-and-cleanup-shellHelperFunctions.sh
maint(common): rename shellHelperFunctions.sh to utils.inc.sh 🐸
2025-08-12 10:40:59 +07:00
Marc Durdin
3292c0add7
Merge branch 'docs/resources/14266-clarify-build-bot' into maint/resources/14419-build-sh-style-build-bot 2025-08-07 11:43:17 +07:00
Marc Durdin
e6aa384e07 maint(resources): build.sh-style commands for build-bot
Legacy format will still work, but preferred format is now
`Build-bot: level:platform[,...platform] ...`

Fixes: #14419
Build-bot: build
Test-bot: skip
2025-08-05 19:58:25 +10:00
Marc Durdin
e46828b339 maint(common): move trigger*.inc.sh and sentry-control.inc.sh to ci/
Relates-to: #14453
2025-08-05 11:06:14 +10:00
Marc Durdin
f394245636 maint(common): consolidate builder scripts
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.

Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh

Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
  android/build.sh.
* Renames functions in builder-basic.inc.sh

More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.

Fixes: #14065
Build-bot: build all
Test-bot: skip
2025-08-02 08:11:24 +10:00
Marc Durdin
65158a0526 maint(common): rename shellHelperFunctions.sh to utils.inc.sh
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.

Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
2025-08-02 06:58:40 +10:00
Eberhard Beilharz
0f9d4e488a
Merge pull request #14347 from keymanapp/maint/linux/13464_emscripten
If the `$EMSCRIPTEN_BASE` dir is missing (which usually points to `emsdk/upstream/emscripten`), but we have the `emsdk` root directory with `emsdk/emsdk`, we can have everything we need to install the desired emscripten version.

Issue #13464 describes an error where the `upstream/emscripten` directory was missing - probably because the build got interrupted while installing emscripten. This change fixes this.

Fixes: #13464
2025-07-23 16:15:13 +02:00
Eberhard Beilharz
7732520e68
Merge pull request #14346 from keymanapp/maint/common/locate_emscr_tests 2025-07-23 15:59:23 +02:00
Eberhard Beilharz
71af4a9636
chore(windows): Merge branch 'master' into maint/windows/addzip 2025-07-22 17:29:56 +02:00
Eberhard Beilharz
1fe8b893c6
chore(common): Merge branch 'master' into maint/common/locate_emscr_tests 2025-07-22 16:49:54 +02:00
Eberhard Beilharz
d9e788d20d
maint(common): address code review comments
- update doc in the function header
- add unit tests to show that flags are allowed anywhere now
2025-07-22 15:08:07 +02:00
Eberhard Beilharz
e7ce49a295
maint(common): fix add_zip_files if flags come before zip filename
Previously `add_zip_files` didn't work if the zip filename was not
specified as first parameter. This change allows it to come after flags.

This will fix web release builds.

Test-bot: skip
2025-07-22 10:48:27 +02:00
Eberhard Beilharz
cb6e62531e
test(common): improve zip tests
Some cleanup and better use of functionality of testing framework.
2025-07-22 10:46:35 +02:00
Eberhard Beilharz
61d30ae0b7
Merge pull request #14380 from keymanapp/refactor/common/ziptests
This makes of the testing framework added in #14345 and also renames the tests to better separate the parts of the test name.
2025-07-22 09:55:49 +02:00
Eberhard Beilharz
68031e7b78
maint(common): address code review comments 2025-07-21 17:33:08 +02:00
Eberhard Beilharz
67a7c69324
refactor(common): use test framework for zip tests
This makes of the testing framework added in #14345 and also renames
the tests to better separate the parts of the test name.

Build-bot: skip
Test-bot: skip
2025-07-21 16:50:49 +02:00
Eberhard Beilharz
1ed4af1f7f
test(common): add unit test for reporting test failures
I added a unit test as part of #14345 but then forgot to commit it.

Follow-up-of: #14345
Build-bot: skip
Test-bot: skip
2025-07-21 16:44:07 +02:00
Eberhard Beilharz
501c666d7d
maint(windows): consider GO_FAST env variable
This changes makes use of the logic that used to be in `win/zip.inc.sh`:
if the environment variable `GO_FAST` is set to 1, we use a low
compression level, resulting in faster builds. If it is set to 0 we
use maximum compression, resulting in slower builds. If `GO_FAST` is
not set we don't specify the compression level and rely on the defaults.

Test-bot: skip
2025-07-21 10:26:30 +02:00
Eberhard Beilharz
dc83c013e5
Merge pull request #14345 from keymanapp/maint/resources/testframework 2025-07-21 09:17:37 +02:00
Eberhard Beilharz
fe62983cb4
Merge pull request #14352 from keymanapp/maint/common/dierelative
`7z` and `zip` behave differently if adding files that are not in the current directory. For files with relative paths, zip will add them relative to the current directory which is not what we want and can cause problems for the user. Therefore this change disallows files that are not in the current directory and aborts the build.
2025-07-21 08:35:54 +02:00
Eberhard Beilharz
aceb74c4aa
maint(resources): address code review comments
- return number of failed tests from `run_tests`
- add unit test for `run_tests`
- some cleanup
2025-07-18 20:23:33 +02:00
Eberhard Beilharz
33f77656de
Merge pull request #14333 from keymanapp/maint/common/zipfunc
This change adds support for the `-xr!wildcard` parameter to the `add_zip_files` function, e.g. `-xr!build.sh` will exclude all `build.sh` files from the archive. This is an alternative to adding a file list.

Also add unit tests for `zip.inc.sh`.
2025-07-15 09:37:33 +02:00
Eberhard Beilharz
39bff3e504
maint(common): fail build if adding relative file to zip
`7z` and `zip` behave differently if adding files that are not in the
current directory. For files with relative paths, zip will add them
relative to the current directory which is not what we want and can
cause problems for the user. Therefore this change disallows files that
are not in the current directory and aborts the build.
2025-07-14 15:21:10 +02:00
Eberhard Beilharz
a1452d6ab8
maint(common): add simple unit testing framework for bash
Build-bot: skip
Test-bot: skip
2025-07-14 14:43:40 +02:00
Eberhard Beilharz
9be6128377
maint(resources): clarify description
Addresses code review comment.
2025-07-14 11:12:49 +02:00
Eberhard Beilharz
e634f20313
maint(linux): fix locate_emscripten if $EMSCRIPTEN_BASE dir missing
If the `$EMSCRIPTEN_BASE` dir is missing (which usually points to
`emsdk/upstream/emscripten`), but we have the `emsdk` root directory
with `emsdk/emsdk`, we can have everything we need to install the
desired emscripten version.

Issue #13464 describes an error where the `upstream/emscripten` directory
was missing - probably because the build got interrupted while installing
emscripten. This change fixes this.

Fixes: #13464
Test-bot: skip
2025-07-11 21:37:26 +02:00
Eberhard Beilharz
30836e750b
maint(common): add unit tests for locate_emscripten.inc.sh
Build-bot: skip
Test-bot: skip
2025-07-11 21:16:32 +02:00
Eberhard Beilharz
784941b03c
maint(common): add simple unit testing framework for bash
Build-bot: skip
Test-bot: skip
2025-07-11 20:19:48 +02:00
Eberhard Beilharz
1f88da7deb
maint(resources): add build.sh to run all resource/build/test tests
Also adjust scripts to output standard pass message.
2025-07-10 15:22:50 +02:00
Eberhard Beilharz
d863fc4aa7
maint(common): add unit tests for zip.inc.sh 2025-07-10 14:27:30 +02:00
Marc Durdin
dde272ac27
Merge branch 'master' into feat/resources/14200-builder-ignore-unknown-for-script-integrations 2025-06-24 06:10:12 +07:00
Marc Durdin
8145c017d7 feat(resources): support ignoring unknown options in builder
Fixes: #14200
Test-bot: skip
2025-06-24 07:12:09 +10:00
Marc Durdin
94cdf834df maint(common): polish docs and add tests for build-bot 2025-06-23 05:05:01 +10:00
Marc Durdin
d9378aebb2 maint(common): Build-bot now available to control test builds
The Build-bot command is available now to control which platforms are
built. See the wiki for instructions on how to use Build-bot.

Build-bot: build
Test-bot: skip
2025-06-05 15:05:35 +07:00
Eberhard Beilharz
d307051836
Merge pull request #13854 from keymanapp/maint/common/versionvars
Previously the builder scripts defined a readonly `VERSION` environment variable for the Keyman version. That caused problems when another (external) script tried to define a `VERSION` variable. We encountered this problem when trying to move the TC build steps of a configuration into a single script (#13399) when we tried to source `~/.nvm/nvm.sh`.

This change uses a Keyman specific prefix for the version variables and renames `VERSION` → `KEYMAN_VERSION` etc. as well as `TIER` → `KEYMAN_TIER`. Unfortunately these variables are used in a lot of places, so this turned out to be a bit of a yak shave.
2025-05-12 08:58:19 +02:00
Eberhard Beilharz
907a16012c
maint(common): output start/end messages if not running on TC
If we're not running on TC we should output the start/end messages as
headings unless it's a child build. This was the previous behavior
before we added support for TC blocks.
2025-05-08 17:01:00 +02:00