Commit graph

78 commits

Author SHA1 Message Date
Markus Greiner
473b177050 Handle uid and gid the same way as in the other Dockerfiles 2025-11-04 22:53:26 +01:00
Markus Greiner
e86c419a42
Update resources/docker-images/android/Dockerfile
Forgotten to remove when CHANGE_PERMISSIONS was invented.

Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2025-11-04 22:48:00 +01:00
Markus Greiner
d9b4ec9baf Weakening condition for gid and uid (after review
Test-bot: skip
Build-bot: skip
2025-10-22 18:02:17 +02:00
Markus Greiner
c9b44d33c7 Changing permissions only in the case if building and testing on GitHub 2025-10-22 15:39:17 +02:00
Markus Greiner
54f5fc1ea3 Privileged mode is not required. 2025-10-21 15:59:06 +02:00
Markus Greiner
fb14a67db6 Docker changes permissions to match the file owner uid and gid
There is an issue that the Docker containers cannot modify the
files on KEYMAN_ROOT, since the user running build and tests on
Docker does not have the same gid and uid as the host user starting
the container.
The issue is solved by temporarily changing the uid and gid to
match the container user's uid and gid.

Test-bot: skip
Build-bot: skip
2025-10-21 15:59:06 +02:00
Eberhard Beilharz
6c2237d800
Merge pull request #14817 from keymanapp/maint/linux/13887_integrtests
maint(linux): fix integration tests in Docker

This PR allows the integration tests to work when running them in a Docker container. Also fixes things when running on TC's Ubuntu 24.04 Noble build agent, and improves the behavior when running directly on a developer's machine (previously typing stopped working after the tests...).

Also fixes building mcompile in a Docker container.

Fixes: #13887
2025-10-06 12:01:38 +02:00
Markus Greiner
49d8acf16b running the container for core in privileged mode
this is expected to fix a permission issue when running in
GitHub Actions.

Test-bot: skip
Build-bot: skip
2025-09-30 12:57:16 +02:00
Eberhard Beilharz
167ec33d54
chore(linux): Merge remote-tracking branch 'origin/master' into maint/linux/remotedbgdocker 2025-09-30 08:29:02 +02:00
Eberhard Beilharz
cca4a502f1
docs(linux): document --remote-debug option for run.sh 2025-09-29 17:49:00 +02:00
Eberhard Beilharz
16727304b5
chore(linux): fix building mcompile in Docker container 2025-09-25 19:34:22 +02:00
Markus Greiner
b8007beedc fix(common): registry clean-up enabled, no terminal in github action
No longer dryrun for the registry clean-up GHA workflow
docker run doesn't use a terminl (erasing -t flag)

Test-bot: skip
Build-bot: skip
2025-09-24 17:24:16 +02:00
Eberhard Beilharz
db3cd5092d
maint(linux): remove several shellcheck directives and warnings
This change improves our `.shellcheckrc` file so that several directives
are no longer needed and source files are still found. Also remove some
directives that are no longer necessary, either because the changes to
`.shellcheckrc` makes them obsolete, or because the lines changed so
that they no longer contain the problem.

Also remove some shellcheck warnings.

Build-bot: skip
Test-bot: skip
2025-09-23 18:30:16 +02:00
Eberhard Beilharz
23f8c59c07
maint(linux): allow to remote debug integration tests in Docker
This adds a `--remote-debug` option to `resources/docker-images/run.sh`.
If specified the container will expose `localhost:2345` for remote
debugging inside of the container.

This is useful for debugging integration tests in the container.

Build-bot: skip
Test-bot: skip
2025-09-23 18:20:32 +02:00
Markus Greiner
ea77b0cb5f Merge remote-tracking branch 'origin/master' into fix/common/tests-running-in-container
adding "-t" when Docker containers are not started by a Docker container
2025-09-22 14:04:26 +02:00
Eberhard Beilharz
14d47cc14d
refactor(resources): use array for Docker arguments
Test-bot: skip
2025-09-19 18:45:31 +02:00
Markus Greiner
bae371bb50 fix(common): some more tests running successfully in a container
setting KEYMAN_ROOT
ignoring test:linux until #13887 is fixed
removing docker "-t" option, because there is no terminal to attach
while running on the GitHub server

Test-bot: skip
Build-bot: skip
2025-09-18 11:24:53 +02:00
Markus Greiner
535f7aa153 maint(resources): added trigger for Docker images build, pubish, test
moving example script to readme file
adding delete job for container images in ghcr

Part-of: #13594
Test-bot: skip
Build-bot: skip
2025-09-08 17:28:11 +02:00
Markus Greiner
5a4635c883 Update resources/docker-images/README.md
Co-authored-by: Eberhard Beilharz <ermshiperete@users.noreply.github.com>
2025-09-08 15:26:44 +02:00
Markus Greiner
f8f6626f55 main(resources): #13954 adding GitHub actions workflow for Docker builds
Test-bot: skip
Build-bot: skip
2025-09-08 15:26:44 +02:00
Markus Greiner
ef8297b352
Merge pull request #14648 from keymanapp/maint/common/13954-registry-usage-for-docker-images
maint: add registry usage for Docker images
2025-09-02 13:54:41 +02:00
Markus Greiner
62eabcc0f0 maint: 13954 simplified if clause after review 2025-09-01 17:53:00 +02:00
Markus Greiner
8152731de7
Update resources/docker-images/docker-build.inc.sh
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-09-01 17:46:52 +02:00
Markus Greiner
2d72727ca5
Update resources/docker-images/README.md
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-09-01 17:46:17 +02:00
Markus Greiner
4e6ca0d332 maint: add registry usage for Docker images
It is now possible to keep the Docker images for building and testing
in a Docker container registry.
This is the first part of changes for #13954. The GitHub actions part is
missing.

Fixes: #13954
Test-bot: skip
Build-bot: skip
2025-08-29 21:53:00 +02:00
Markus Greiner
5610cab8f3 maint: reverting change for Dockerfile for building and testing web
KEYMAN_ROOT should not be set in the docker run command
Test-bot: skip
Build-bot: skip
2025-08-29 19:11:00 +02:00
Markus Greiner
4a8aeb9b45 fix: fixing some issues for bulding and testing with Docker containers
Fixed some errors found during tests for #13954 when running
resources/docker-images/run.sh with targets ":linux" and ":web".

Build-bot: skip
Test-bot: skip
2025-08-29 10:55:57 +02:00
Markus Greiner
68b5c3463b
Merge pull request #14568 from keymanapp/maint/common/move-node-related-functions-out-of-utils.inc.sh-missing-parts
maint(common): rename and move node-related script functions into node.inc.sh
2025-08-21 11:00:46 +02:00
Markus Greiner
acaeb6aa02 maint(common): rename and move node-related script functions into node.inc.sh
Consolidates the node-related script functions into node.inc.sh, as part
        of cleaning up the build scripts and making them easier to maintain into
        the future. In the earlier commite some changes were missing.

        Follows: #14540
        Fixes: #14447

        Build-bot: skip
        Test-bot: skip
2025-08-20 10:44:56 +02:00
Eberhard Beilharz
c8cc5518df
Update resources/docker-images/README.md
Co-authored-by: Marc Durdin <marc@durdin.net>
2025-08-20 08:46:02 +02:00
Eberhard Beilharz
e98f2bb53c
maint(common): add warning about not mixing build platforms
Developers have to be consistent if they build on Windows/git-bash or
in WSL/Docker. Mixing builds without cleaning the repo first will fail.
See https://github.com/keymanapp/keyman/pull/14154#issuecomment-3182076088.

Build-bot: skip
Test-bot: skip
2025-08-18 17:51:00 +02:00
Eberhard Beilharz
72d8b705ff
maint(common): on Windows build as root inside docker container
This fixes a permission problem when we run docker from git bash.
On Windows we want to build as _root_ inside of the docker container.
This gives the necessary permissions due to the way the Windows user
is mapped to git bash/mingw and the docker container. On Linux
however we want to build as _build_ user inside the container so that
files we create are owned by the host user.

See https://github.com/keymanapp/keyman/pull/14154#issuecomment-2970577312
for further explanation.
2025-08-18 17:50:29 +02:00
Eberhard Beilharz
57a9583093
maint(common): disable path conversions on Windows
By default Git Bash converts paths starting with `/`, so running the
docker image fails with errors or files mentioning `\Program Files\Git\`.
This change adds a wrapper function that prevents the conversion
and thus uses the paths in the correct form when calling docker.
2025-08-18 17:50:28 +02:00
Markus Greiner
72214187c5 maint(common): rename and move node-related script functions into node.inc.sh
Consolidates the node-related script functions into node.inc.sh, as part
    of cleaning up the build scripts and making them easier to maintain into
    the future. In the earlier commite some changes were missing.

    Fixes: #14447
2025-08-18 11:42:54 +02:00
Marc Durdin
f394245636 maint(common): consolidate builder scripts
Clarifies the confusing builder.inc.sh / build-utils.sh distinction by
giving the scripts more appropriate names. Most build scripts should use
builder-full.inc.sh; some helper scripts can use builder-basic.inc.sh.
Documented in resources/build/README.md.

Renames:
* resources/build/builder.inc.sh to resources/build/builder-full.inc.sh
* resources/build/build-utils.sh to resources/build/builder-basic.inc.sh

Other changes:
* Moves Android-specific functions out of builder-basic.inc.sh and into
  android/build.sh.
* Renames functions in builder-basic.inc.sh

More functions may be moved from builder-basic.inc.sh into utils.inc.sh
or other scripts in the future.

Fixes: #14065
Build-bot: build all
Test-bot: skip
2025-08-02 08:11:24 +10:00
Marc Durdin
65158a0526 maint(common): rename shellHelperFunctions.sh to utils.inc.sh
Adds documentation for various functions, removes unused functions (a
couple of simple, very lightly used functions were unDRYed; these could
go back the other way across all shell scripts if necessary). TODO items
noted for follow-up refactoring.

Fixes: #14275
Relates-to: #14269
Build-bot: build all
Test-bot: skip
2025-08-02 06:58:40 +10:00
Eberhard Beilharz
3b136fcec9
maint(common): consolidate functions to determine OS
- add `builder_is_windows`, `builder_is_macos` and `builder_is_linux`
  functions
- add/move `BUILDER_OS` env variable to `resources/builder.inc.sh`
- use the new functions and env variable

Fixes: #14109
Test-bot: skip
2025-07-09 18:33:09 +02:00
Eberhard Beilharz
33fe775aa7
maint(developer): use docker buildx to build docker images
This change verifies that `docker buildx` is available and then uses that
for building the images. For that we apparently need the magic line at
the top of the `Dockerfile`s and set the environment variable
`DOCKER_BUILDKIT`. This solves a problem building docker images on
Linux with an older Docker version where the heredocs didn't work.
2025-06-06 18:10:53 +02:00
Eberhard Beilharz
f13134100c
maint(developer): specify platforms as targets instead of actions in run.sh
This adjusts `resources/docker-images/run.sh` to more closely match
the syntax in `build.sh`.
2025-06-06 17:07:02 +02:00
Eberhard Beilharz
6b3956ecaa
docs(developer): update README.md for docker images 2025-06-04 11:53:11 +02:00
Eberhard Beilharz
ea8eea1cfa
maint(developer): add docker image to build developer node modules
Also update build documentation.

Fixes: #14045
Test-bot: skip
2025-05-27 18:17:16 +02:00
Eberhard Beilharz
9d2ab35531
maint(common): move TC build configs for Linux to scripts
This change adds scripts for the TeamCity build configurations for
Linux builds.

Part-of: #13399
Test-bot: skip
2025-05-12 17:42:25 +02:00
Eberhard Beilharz
47407a3164
maint(common): remove obsolete code from docker-images run.sh
PR #13861 missed the fact that docker-images' `run.sh` script also
contains code that got re-added during a faulty merge. This change cleans
this up.
2025-05-09 18:07:16 +02:00
Eberhard Beilharz
d59291c062
maint(common): remove obsolete code from docker-images build.sh
Looks like a merge undid some of the changes from #13537 so that we
ended up with duplicate code. This change cleans this up.
2025-05-08 09:22:26 +02:00
Marc Durdin
a52437c407 chore: Merge branch 'master' into chore/b18s4-merge-beta-to-master 2025-03-29 05:38:08 +07:00
Eberhard Beilharz
4609abaaaf
chore(linux): use bash by default when building containers 2025-03-18 18:37:58 +01:00
Eberhard Beilharz
d85cd135db
chore(linux): update comment 2025-03-18 18:34:26 +01:00
Eberhard Beilharz
e649222a1c
fix(linux): fix building in Docker with different Ubuntu/Debian version
Also some general improvements for building and running Docker container.
2025-03-18 17:25:10 +01:00
Eberhard Beilharz
da305bab57
chore(linux): deal with default values
When `_convert_parameters_to_build_args` sets the defaults, our previous
implementation to check for default values no longer works.
2025-03-18 16:18:55 +01:00
Eberhard Beilharz
18a600be6a
chore(linux): initialize variables with default values if not set
This changes `convert_parameters_to_build_args` to set the variable to
the default values unless set already. Then we don't have to deal with
unset variables and setting default values when we use the variables.
2025-03-18 16:18:47 +01:00