Commit graph

84 commits

Author SHA1 Message Date
Eberhard Beilharz
8f0206329b
Update resources/shellHelperFunctions.sh
Co-authored-by: Darcy Wong <darcy_wong@sil.org>
2025-07-09 16:09:02 +02:00
Eberhard Beilharz
479a0da739
maint(windows): fix platform name for Windows 🩹
The .download_info files use the platform `win` instead of `windows`.
This PR changes this, and also renames the `_verify_project` function
to `_verify_platform` to better match what it's checking. Fixes Windows
release builds.

Test-bot: skip
2025-07-09 12:01:13 +02:00
Eberhard Beilharz
a25202530e
maint(common): consolidate write_download_info functions
Fixes: #14259
Build-bot: skip
Test-bot: skip
2025-07-07 18:51:37 +02:00
Eberhard Beilharz
38f43d84cf
refactor(common): some reformattings in shellHelperFunctions.sh
Also always include `jq.inc.sh`. The include file already has everything
we did here.

Test-bot: skip
2025-07-07 18:44:28 +02:00
Eberhard Beilharz
4d81589f48
maint(common): use unique names for Keyman version variables
Previously the builder scripts defined a readonly `VERSION` environment
variable for the Keyman version. That caused problems when another
(external) script tried to define a `VERSION` variable. We encountered
this problem when trying to move the TC build steps of a configuration
into a single script (#13399) when we tried to source `~/.nvm/nvm.sh`.

This change uses a Keyman specific prefix for the version variables and
renames `VERSION` → `KEYMAN_VERSION` etc. Unfortunately these variables
are used in a lot of places, so this turned out to be a bit of a yak
shave.

Test-bot: skip
2025-05-07 18:46:51 +02:00
Eberhard Beilharz
af1f2629dd
maint(common): use function to detect if running on TC
This change makes use of the new builder script function to detect if
the script is running on TeamCity.
2025-05-06 17:42:25 +02:00
Darcy Wong
8d0b183788 fix(common): Fix logic 2025-05-05 10:26:42 +07:00
Darcy Wong
875cc303cf fix(common): Test for BSD vs coreutils 2025-05-05 10:24:45 +07:00
Darcy Wong
15fd72eadd maint(common): Handle coreutils stat 2025-05-03 09:12:25 +07:00
Darcy Wong
85a25446cb fix(common): Tweak BUILDER_OS check 2025-05-02 10:07:05 +07:00
Darcy Wong
afe9b9fe16 test(common): Debug statement on BUILDER_OS 2025-05-02 09:34:57 +07:00
Darcy Wong
bc318915bb
Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-05-02 08:26:28 +07:00
Darcy Wong
e8ba5f8016 maint(common): Update write_download_info() to be cross-platform
* Handle stat flags based on `BUILDER_OS`
* Change from md5 to md5sum
2025-05-01 14:45:27 +07:00
Marc Durdin
37cb3767ac chore(developer): lower threshold for kmc-model-info coverage, work around TeamCity reporter bug 2025-03-05 01:00:03 +03:00
Marc Durdin
ebdb043ba7 fix(common): fail builds with low test coverage for all typescript projects
This is probably my fault, when I consolidated the typescript test
scripts, I didn't spot the `--check-coverage=false` flag which meant
that although the coverage tests were run, the builds would not fail if
they were too low.

Fixes: #13417
2025-03-04 23:59:04 +03:00
Eberhard Beilharz
0df5992102
chore(linux): Merge branch 'master' into chore/linux/cicontainer 2025-01-08 12:39:10 +01:00
Eberhard Beilharz
afa47e08ec
Update resources/shellHelperFunctions.sh
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-12-06 09:09:24 +01:00
Eberhard Beilharz
fbc3790043 chore(core): wait for nvm use to finish
This solves the underlying problem of builds not working on my machine.
Seems the environment isn't fully set after calling `nvm use`, so we have
to either `sleep 1` or wait for the process to end (which this change
does).
2024-12-05 11:03:01 +01:00
Eberhard Beilharz
7caf60789e chore(core): only install node on Windows if not available
This fixes the build on my Windows machine. Previously `nvm install/use`
for whatever reason caused Windows no longer be able to find `node`
thus causing the build to fail. This change when run on Windows first
checks the current node version and installs it only if it is not
installed.
2024-12-04 20:31:37 +01:00
Marc Durdin
5d06465388
docs(common): add builder_do_typescript_tests docs 2024-11-29 02:46:05 +01:00
Marc Durdin
70a32f00d7 refactor(developer): unify test action 2024-11-28 15:16:25 +07:00
Marc Durdin
640d3e5096 test(developer): add markdown link check test for product documentation
Adds check-markdown package that parses .md files, looks for links and
images, and verifies that they are valid links (for internal links).
2024-09-25 15:51:32 -07:00
Eberhard Beilharz
389fb4f9ee
chore(common): Allow to build offline
A lot of the `configure` actions rely on node dependencies and so call
`npm install`. Without a internet connection this hangs forever, even
if you had successfully installed the dependencies before and nothing
changed since then. This change adds a `--offline` parameter that passes
`--prefer-offline` to npm, which causes npm to use the cached
dependencies.

Also sets the `MESON_PACKAGE_CACHE_DIR` environment variable so that all
(sub-)projects share the same package cache dir. This will speed up regular
builds a bit (e.g. `developer/src/kmcmplib` and `core` both have `icu4c`
as a dependency), but will also help with offline builds.

`MESON_PACKAGE_CACHE_DIR` requires Meson 1.3, however older Meson
versions ignore this environment variable and simply continue to use a
cache per subproject, so will continue to work (minus offline builds).
2024-09-18 22:40:20 +02:00
Eberhard Beilharz
d6c5320325
chore(linux): address code review comments
- use KEYMAN_USE_NVM and KEYMAN_USE_EMSDK
- pre-install node (where necessary) to prevent having to do it on each
  build
- adjust to current `master`
- fix a few bugs in the Dockerfile
- get required node version from package.json (through
  `shellHelperFunctions.sh`)
2024-08-31 22:33:56 +07:00
Marc Durdin
fc4df037bb chore(common): use nvm to select version of node for builds
Used by default only on build agents. See docs/build/node.md and
resources/build/_builder_nvm.sh for documentation of this change.

Fixes: #11376
2024-08-01 05:09:08 +07:00
Joshua A. Horton
c91ffb95cd fix(common): better error-code capture 2024-06-05 13:03:13 +07:00
Joshua Horton
e2e8ed3491
change(common): emit error-code on retry fail
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-06-05 12:46:14 +07:00
Joshua Horton
66bdc29f6e fix(common): revert accidental test-code inclusion 2024-05-31 15:08:36 +07:00
Joshua Horton
b6a06c6626 change(common): simplifies retry command-execution syntax 2024-05-31 15:01:33 +07:00
Joshua Horton
6251faa23d fix(common): only retry the npm ci call itself 2024-05-31 14:52:31 +07:00
Joshua Horton
e382dddb5b
chore(common): Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2024-05-16 16:03:14 +07:00
Joshua A. Horton
470588933e chore(common): adjustments per code review 2024-05-15 14:26:27 +07:00
Joshua A. Horton
5a0e66ce65 chore(web): adds retry mechanism for build script npm ci calls 2024-05-15 12:12:58 +07:00
Marc Durdin
4cc90f2722
chore(common): use mac /usr/bin/stat rather than homebrew version 2023-03-23 16:53:32 +11:00
Joshua A. Horton
42f03916ad feat(common): adds warning if described output does not exist when finishing action 2023-02-23 16:18:18 +07:00
Marc Durdin
6df6b0ef18 chore: remove log_error 2023-02-20 09:19:03 +07:00
Marc Durdin
501c317b24 chore: use builder_warn, remove log_warning 2023-02-20 09:18:26 +07:00
Marc Durdin
a004978863 chore: use builder_heading, remove echo_heading 2023-02-20 09:17:51 +07:00
Marc Durdin
4d53a277f3 chore: use builder_die, remove fail 2023-02-20 09:15:38 +07:00
Marc Durdin
318b4d4478 chore: use builder_warn, remove warn 2023-02-20 08:56:31 +07:00
Marc Durdin
8f5997a3e6 chore: rename os_id to BUILDER_OS 2023-02-20 08:44:03 +07:00
Marc Durdin
f61ef08f05 chore: cleanup shellHelperFunctions 2023-02-20 08:33:23 +07:00
Marc Durdin
1cecb87f7f feat(common): builder dependency support 2022-10-12 12:50:26 +11:00
Marc Durdin
683d8b9e68 chore(common): build-utils.sh minor tweaks
Fixing up a number of minor bits and pieces identified during demo:

* invalid action or :target would not be picked up if paired with a
  valid one (e.g.: build:foo should not have worked)
* colors defined in both shellHelperFunctions and build-utils were
  problematic; able to remove them from shellHelperFunctions because
  all references included both scripts
* builder_has_action and builder_report now allow both `action:target`
  and `action :target`
* keyman-version/build.sh did not have working test action...
2022-07-29 17:37:19 +10:00
Marc Durdin
c1118d908c
Merge branch 'master' into chore/common/builder-script-target-and-option-support 2022-07-29 10:37:18 +10:00
Marc Durdin
ba6bc677cd chore(common): tweak build help and color support 2022-07-29 10:35:18 +10:00
Eberhard Beilharz
9f7b85834a
chore(linux): Update resources/shellHelperFunctions.sh
Somehow missed this with the previous commit.

Co-authored-by: Marc Durdin <marc@durdin.net>
2022-07-28 09:20:35 +02:00
Eberhard Beilharz
420cc0814d
chore(linux): Apply suggestions from code review
Co-authored-by: Marc Durdin <marc@durdin.net>
2022-07-28 09:19:25 +02:00
Eberhard Beilharz
9232c20569
chore(linux): address review comments
This change also slightly modifies `echo_heading` and outputs a
mark. This will show to the left of the line as well as in the
scroll bar in a terminal in VSCode [[1]] and will help to find the
headings if you have a lot of output in between.

[1]:https://code.visualstudio.com/updates/v1_69#_setmark-sequence-support
2022-07-27 12:22:55 +02:00
Marc Durdin
767927bf16 chore(common): use /usr/bin/env bash
Fixes up scripts (except under /linux) to use `#!/usr/bin/env bash`
instead of `#!/bin/bash` or `#!/bin/sh` so that we don't end up with
the ancient version of bash supplied with macOS.

This became urgent with this PR, because of bash-4.xisms in
build-utils.sh, for example on line 572:

```
if [[ -v _builder_params[$e] ]]; then
```
2022-07-25 14:59:37 +10:00