The journald_unavailable() helper was testing 'journalctl -n 1' which
checks the general system journal, but quadlet and auto-update tests
actually need to read user systemd unit logs.
On some RHEL systems, the general journal works but reading user unit
logs fails with 'No journal files were found'.
Update the function to test the actual use case: reading logs from
user systemd units with 'journalctl --user --unit=...'. This properly
detects when user unit logging is unavailable so tests skip correctly.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
After[1] c/image no longer prints "Storing signatures" so we should
not check for it.
[1] https://github.com/containers/image/pull/2001
(partial cherry-pick from commit 6eaf8a271d)
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
The buildah buil kill trick is bad as we have to sleep and wait to aboid
flakes which takes time. Instead it is possible to redo this build part
manually with buildah commands. It is not trival and harder to
understand but it safes 2-3s so I think it is worth it.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
(cherry picked from commit 350dfabf66)
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
Make safename() invocations consistent within the same
test. This puts the onus on the caller to add a unique
element when calling multiple times, e.g. "ctr1-$(safename)".
This is not too much of a burden. Major benefit is making
it easy for a reader to associate containers, pods, volumes,
images within a given test.
And, use dashes, not underscores. "podman generate kube"
removes underscores, making it very difficult to do
things like "podman inspect $podname" (because we need
to generate "$podname_with_underscores_removed")
Signed-off-by: Ed Santiago <santiago@redhat.com>
(cherry picked from commit 380ed3a40d)
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
Many system tests use hardcoded names for containers, images,
and everything. This has worked because system tests run
serially. It will not work if we ever run in parallel.
Create a new safename() helper, and use it as follows:
myctr=c_$(safename)
myvol1=v1_$(safename)
...
Find current instances of hardcoded names, and replace
with safe ones.
Whether or not we ever end up parallelizing system tests,
this is simply good practice.
There are far too many instances to fix in one (reviewable) PR.
This is commit 1 of N.
(partial cherry-pick from commit b28027148b)
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
The test was using old expectations that required all containers to
have system_r role. The updated version from main branch correctly
expects that privileged/pid=host/ipc=host/label=disable containers
run with the current user and role, not system_r. This is intentional
behavior to support confined users (see commit 5b7dce8a3d).
This brings the test expectations in line with the actual behavior.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
The test framework was not parsing the '+w' suffix in expected exit
codes (e.g., '0+w' meaning "expect exit code 0 and warnings allowed").
This caused the literal string to be passed as a command argument to
podman, resulting in "unrecognized command" errors.
Added pattern matching for:
- [0-9]+w (single digit with warnings)
- [1-9][0-9]+w (double digit with warnings)
- [12][0-9][0-9]+w (triple digit with warnings)
The '+w' suffix is stripped off and only the numeric exit code is used
for validation. Note: The actual warning validation is not implemented
yet, but this allows tests using the '+w' notation to run correctly.
This fixes the "podman-remote: non-default connection" test failure.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
The events file backend writes events as they're generated, which may
not match chronological order due to race conditions during concurrent
operations in rmi -f. The remove and untag events from rmi can appear
in any order in the events file.
This matches the approach used in main branch (see commit 111a4bbe71)
which recognizes that event ordering is not guaranteed for concurrent
operations, especially in podman-remote.
This fix:
- Simplifies initial validation to check for event presence (not order)
- Checks deterministic events (0-6) in exact order
- Validates rmi events (7-9) are present without requiring specific order
- Checks final event (10) is the second loadfromarchive
- Does NOT modify runtime code to enforce ordering (events remain as-generated)
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
... Podman
cli flags couldn't override the active-destination when env variables were set. As a remedy, the precedence of cli flags has been changed.
Note: This commit is from #19997 and it brought in some pretty massive changes
to how the remote connections are created.
Signed-off-by: Chetan Giradkar <cgiradka@redhat.com>
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Unfortunately on a number of occasions, Podman has been released
officially with a `-dev` suffix in the version number. Assist in
catching this mistake at release time by the addition of a simple
conditional test. Note that it must be positively enabled by a
magic env. var. before executing the system tests.
Ref. original PR: https://github.com/containers/podman/pull/26540
Signed-off-by: Chris Evich <cevich@redhat.com>
Checking for the mountdir is not relevent, a recent c/storage change[1] no
longer deletes the mount point directory so the check will cause a false
positive. findmnt exits 1 when the given path is not a mountpoint so
let's use that to check.
[1] 3f2e81abb3
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The scenario for inducing this is as follows:
1. Start a container with a long stop timeout and a PID1 that
ignores SIGTERM
2. Use `podman stop` to stop that container
3. Simultaneously, in another terminal, kill -9 `pidof podman`
(the container is now in ContainerStateStopping)
4. Now kill that container's Conmon with SIGKILL.
5. No commands are able to move the container from Stopping to
Stopped now.
The cause is a logic bug in our exit-file handling logic. Conmon
being dead without an exit file causes no change to the state.
Add handling for this case that tries to clean up, including
stopping the container if it still seems to be running.
Fixes#19629
Addresses: https://issues.redhat.com/browse/ACCELFIX-250
Signed-off-by: Matt Heon <mheon@redhat.com>
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Add a wait_for_ready() to one kube-play test, to make sure
container output has made it to the journal.
Probably does not fix#18501, but I think it might fix its
most common presentation.
Signed-off-by: Ed Santiago <santiago@redhat.com>
The long term goal was to provide the customer with a way to turn on the
preexec_hooks processing of scripts by having some kind of configuration
that could be read. I had tried putting it into containers.conf to
start, but that turned out to be unyieldly quickly, and time is of
the essence for this fix. That is mostly due to the fact that this
code is preexecution and in C, the containers.conf file is read in
Go much further down the stack.
After first trying this process using an ENVVAR, I have
thought it over and chatted with others, and will now look for a
/etc/containers/podman_preexec_hooks.txt file to exist. If the admin
had put one in there, we will then process the files in the
directories /usr/libexec/podman/pre-exec-hooks
and /etc/containers/pre-exec-hooks.
Thoughts/suggestions gratefully accepted. This will be a 8.8/9.2 ZeroDay
fix and will need to be backported to the v4.4.1-rhel branch.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
When a userns is set we setup the network after the bind mounts, at the
point where resolv.conf is generated we do not yet know the subnet.
Just like the other dns servers for bridge networks we need to add the
ip later in completeNetworkSetup()
Addresses: https://bugzilla.redhat.com/show_bug.cgi?id=2182492 and
https://bugzilla.redhat.com/show_bug.cgi?id=2182491
This is targeted to RHEL 8.8 and 9.2 ZeroDay
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
skip in setup() if journald unavailable.
To be pedantic, this is overkill: some quadlet tests pass
because they don't run journald. Too bad.
Also skip a play-kube test that requires journal
Signed-off-by: Ed Santiago <santiago@redhat.com>
As described in #17777, the `restart` on-failure action did not behave
correctly when the health check is being run by a transient systemd
unit. It ran just fine when being executed outside such a unit, for
instance, manually or, as done in the system tests, in a scripted
fashion.
There were two issue causing the `restart` on-failure action to
misbehave:
1) The transient systemd units used the default `KillMode=cgroup` which
will nuke all processes in the specific cgroup including the recently
restarted container/conmon once the main `podman healthcheck run`
process exits.
2) Podman attempted to remove the transient systemd unit and timer
during restart. That is perfectly fine when manually restarting the
container but not when the restart itself is being executed inside
such a transient unit. Ultimately, Podman tried to shoot itself in
the foot.
Fix both issues by moving the restart logic in the cleanup process.
Instead of restarting the container, the `healthcheck run` will just
stop the container and the cleanup process will restart the container
once it has turned unhealthy.
Backport of commit 9563415430.
Fixes: #17777
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2180125
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2180126
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Only enforce the passthrough log driver for Quadlet. Commit 68fbebf
introduced a regression on the `podman-kube@` template as `podman logs`
stopped working and settings from containers.conf were ignored.
Fixes: #17482
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Primarily, add skip_if_journald_unavailable because RHEL.
Secondarily, reverse a flipped actual/expect assertion
that made it difficult to understand the RHEL failure.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Commit 2f29639bd3 added a UX improvement to cleanup/teardown when
running the specified YAML has failed. However, the teardown happens
unconditionally such that rerunning the same YAML file will teardown the
previously created workload instead of just failing with a name-conflict
error (e.g., "pod already exists"). The regression popped up testing
the Ansible system role with Podman v4.4.0.
For now, do not teardown at all on error to quickly fix this regression
for the upcoming Podman v4.4.1 release. The UX improvement is still
desired but must be conditional and only happen on newly created
resources, which probably requires moving it down to the backend.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Reserved annotations are used internally by Podman and would effect
nothing when run with Kubernetes so we should not be generating these
annotations.
Fixes: https://github.com/containers/podman/issues/17105
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
...and instrument with 'podman ps'es for debugging failures.
Test flakes pretty regularly in Fedora gating. If the increased
timeout doesn't help, at least we should be able to see if the
container is stopping or failed or something.
Signed-off-by: Ed Santiago <santiago@redhat.com>
to not give a false sense of security since these are not a security
mechanism but a hook to run arbitrary code before executing a
command.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
Until Podman v4.3, privileged rootfull containers would expose all the
host devices to the container while rootless ones would exclude
`/dev/ptmx` and `/dev/tty*`.
When 5a2405ae1b ("Don't mount /dev/tty* inside privileged containers
running systemd") landed, rootfull containers started excluding all the
`/dev/tty*` devices when the container would be running in systemd
mode, reducing the disparity between rootless and rootfull containers
when running in this mode.
However, this commit regressed some legitimate use cases: exposing
non-virtual-terminal tty devices (modems, arduinos, serial
consoles, ...) to the container, and the regression was addressed in
f4c81b0aa5 ("Only prevent VTs to be mounted inside privileged
systemd containers").
This now calls into question why all tty devices were historically
prevented from being shared to the rootless non-privileged containers.
A look at the podman git history reveals that the code was introduced
as part of ba430bfe5e ("podman v2 remove bloat v2"), and obviously
was copy-pasted from some other code I couldn't find.
In any case, we can easily guess that this check was put for the same
reason 5a2405ae1b was introduced: to prevent breaking the host
environment's consoles. This also means that excluding *all* tty
devices is overbearing, and should instead be limited to just virtual
terminals like we do on the rootfull path.
This is what this commit does, thus making the rootless codepath behave
like the rootfull one when in systemd mode.
This leaves `/dev/ptmx` as the main difference between the two
codepath. Based on the blog post from the then-runC maintainer[1] and
this Red Hat bug[2], I believe that this is intentional and a needed
difference for the rootless path.
Closes: #16925
Suggested-by: Fabian Holler <mail@fholler.de>
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
[1]: https://www.cyphar.com/blog/post/20160627-rootless-containers-with-runc
[2]: https://bugzilla.redhat.com/show_bug.cgi?id=501718
Increase the loop range from 5 to 20 to make sure we give the service
enough time to transition to inactive. Other tests have the same range
with 0.5 seconds sleeps, so I expect the new value to be sufficient and
consistent.
Fixes: #17093
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Make sure that the specs of containers generated by `kube play` are
correctly completed. They have not before which surfaced in default
environment variables not being set.
Fixes: #17016
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
The kube-play test file was a rat's nest of long complicated
yaml strings all differing only slightly. Clean it up, by
adding a helper function with optional parameters. The
helper is ugly, but the actual test code (the important
stuff) is cleaner.
Signed-off-by: Ed Santiago <santiago@redhat.com>
While manually playing with --service-container, I encountered a number
of too verbose logs. For instance, there's no need to error-log when
the service-container has already been stopped.
For testing, add a new kube test with a multi-pod YAML which will
implicitly show that #17024 is now working.
Fixes: #17024
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
While mounting virtual console devices in a systemd container is a
recipe for disaster (I experienced it first hand), mounting serial
console devices, modems, and others should still be done by default
for privileged systemd-based containers.
v2, addressing the review from @fho:
- use backticks in the regular expression to remove backslashes
- pre-compile the regex at the package level
- drop IsVirtualTerminalDevice (not needed for a one-liner)
v3, addressing the review from @fho and @rhatdan:
- re-introduce a private function for matching the device names
- use path.Match rather than a regex not to slow down startup time
Closes#16925.
Fixes: 5a2405ae1b ("Don't mount /dev/tty* inside privileged...")
Signed-off-by: Martin Roukala (né Peres) <martin.roukala@mupuf.org>
If the run errors, cobra does not execute post runs. It is a somehow
known issue (https://github.com/spf13/cobra/issues/914) but problematic
for Podmand as the runtime is shutdown during post run.
Since some commands overwrite the post run and a general lack in cobra
of post runs on errors, move the shutting down the engines directly into
Execute. Fixing the issue may fix a number of flakes.
Note that the shutdowns are NOPs for the remote client.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
follow-up to 6886e80b45
when "podman -rm -f" is used on a container in "stopping" state, also
make sure it is terminated before removing it from the local storage.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
if /sys is bind mounted from the host then also add an explicit mount
for /sys/fs/cgroup so that 'ro' is honored.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
