Commit graph

671 commits

Author SHA1 Message Date
cdoern
d28e85741f InfraContainer Rework
InfraContainer should go through the same creation process as regular containers. This change was from the cmd level
down, involving new container CLI opts and specgen creating functions. What now happens is that both container and pod
cli options are populated in cmd and used to create a podSpecgen and a containerSpecgen. The process then goes as follows

FillOutSpecGen (infra) -> MapSpec (podOpts -> infraOpts) -> PodCreate -> MakePod -> createPodOptions -> NewPod -> CompleteSpec (infra) -> MakeContainer -> NewContainer -> newContainer -> AddInfra (to pod state)

Signed-off-by: cdoern <cdoern@redhat.com>
2021-08-26 16:05:16 -04:00
OpenShift Merge Robot
94c37d7d47
Merge pull request #11298 from baude/kubeupdown
teardown play kube
2021-08-26 13:58:44 -04:00
OpenShift Merge Robot
49cfed756f
Merge pull request #11103 from jwhonce/wip/bindings
Fix file descriptor leaks in bindings and add test
2021-08-25 14:42:12 -04:00
Jhon Honce
1dc6d14735 Fix file descriptor leaks and add test
* Add response.Body.Close() where needed to release HTTP
  connections to API server.
* Add tests to ensure no general leaks occur. 100% coverage would be
  required to ensure no leaks on any call.
* Update code comments to be godoc correct

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-08-24 16:36:10 -07:00
Brent Baude
1e176923b1 teardown play kube
add the ability for play kube to tear down based on the yaml used to
play it.  it is indicated by --down in the play kube command.  volumes
are NOT deleted during the teardown.  pods and their containers are
stopped and removed.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-24 14:26:14 -05:00
Nalin Dahyabhai
21f396de6f logs: adjust handling around partial log messages
In libpod/logs.LogLine.Write(), don't write a newline to stdout/stderr
when the log message is only part of a line.

In libpod.ConmonOCIRuntime.HTTPAttach(), don't send a newline over the
HTTP connection when the log message is only part of a line.

In pkg/api/handlers/compat.LogsFromContainer(), don't send a newline
over the HTTP connection when the log message is only part of a line,
and don't make doing so conditional on whether or not the client used
the docker or podman endpoint.

In pkg/domain/infra/tunnel.ContainerEngine.ContainerLogs(), don't add
our own newline to log messages, since they already come through from
the server when they need to.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-08-23 18:00:18 -04:00
flouthoc
edddfe8c4f volumes: Add support for exporting volumes to external tar
Adds support for transferring data between systems and backing up systems.
Use cases: recover from disasters or move data between machines.

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-23 20:42:41 +05:30
Brent Baude
cd40c875ac Add ability to build images in play kube
When playing a kube YAML file, it can be desirable to be able to build
an image on the fly.  This is good for development of an image and YAML
files and somewhat mocks what compose does.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-18 09:55:24 -05:00
Valentin Rothberg
fe2be7f886 make sure that signal buffers are sufficiently big
Dealing with os.Signal channels seems more like an art than science
since signals may get lost.  os.Notify doesn't block on an unbuffered
channel, so users are expected to know what they're doing or hope for
the best.

In the recent past, I've seen a number of flakes and BZs on non-amd64
architectures where I was under the impression that signals may got
lost, for instance, during stop and exec.

[NO TESTS NEEDED] since this is art.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-17 12:10:20 +02:00
openshift-ci[bot]
1f0a24437d
Merge pull request #11011 from baude/initcontainers
implement init containers in podman
2021-08-05 13:52:15 +00:00
Brent Baude
3c3fa6fac4 implement init containers in podman
this is the first pass at implementing init containers for podman pods.
init containersare made popular by k8s as a way to run setup for pods
before the pods standard containers run.

unlike k8s, we support two styles of init containers: always and
oneshot.  always means the container stays in the pod and starts
whenever a pod is started.  this does not apply to pods restarting.
oneshot means the container runs onetime when the pod starts and then is
removed.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-04 14:14:36 -05:00
Daniel J Walsh
41f94a4dc1
Fix podman unpause,pause,kill --all to work like podman stop --all
Currently if you execute podman unpause --all, podman pause --all
Podman shows attempts to unpause containers that are not paused
and prints an error.  This PR catches this error and only prints errors if
a paused container was not able to be unpaused.

Currently if you execute podman pause --all or podman kill --all, Podman
Podman shows attempts to pause or kill containers that are not running
and prints an error.  This PR catches this error and only prints errors if
a running container was not able to be paused or killed.

Also change printing of multiple errors to go to stderr and to prefix
"Error: " in front to match the output of the last error.

Fixes: https://github.com/containers/podman/issues/11098

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-04 12:50:00 -04:00
openshift-ci[bot]
8aa869e628
Merge pull request #10973 from rhatdan/quota
Support size options on builtin volumes
2021-08-04 11:59:58 +00:00
openshift-ci[bot]
3a922cbc99
Merge pull request #11003 from pascomnet/f_stats
stats: add a interval parameter to cli and api stats streaming
2021-08-04 09:56:57 +00:00
Daniel J Walsh
c0952c7334
Support size and inode options on builtin volumes
[NO TESTS NEEDED] Since it is difficult to setup xfs quota

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1982164

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-02 10:32:45 -04:00
OpenShift Merge Robot
f9395ddc5a
Merge pull request #11067 from vrothberg/fix-10154-2
remote build: fix streaming and error handling
2021-07-28 16:17:45 +02:00
OpenShift Merge Robot
6c5966cf3c
Merge pull request #10910 from adrianreber/2021-07-12-checkpoint-restore-into-pod
Add support for checkpoint/restore into and out of pods
2021-07-28 14:48:28 +02:00
Valentin Rothberg
4df6e31ccb remote build: fix streaming and error handling
Address a number of issues in the streaming logic in remote build, most
importantly an error in using buffered channels on the server side.

The pattern below does not guarantee that the channel is entirely read
before the context fires.

for {
	select {
		case <- bufferedChannel:
		...
		case <- ctx.Done():
		...
	}
}

Fixes: #10154
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-28 14:46:15 +02:00
OpenShift Merge Robot
a5de831418
Merge pull request #11049 from vrothberg/fix-7370
support container to container copy
2021-07-27 22:20:45 +02:00
OpenShift Merge Robot
a6a0f6c192
Merge pull request #11061 from giuseppe/migrate-no-move-to-cgroup
command: migrate doesn't move process to cgroup
2021-07-27 20:46:46 +02:00
Adrian Reber
eb94467780
Support checkpoint/restore with pods
This adds support to checkpoint containers out of pods and restore
container into pods.

It is only possible to restore a container into a pod if it has been
checkpointed out of pod. It is also not possible to restore a non pod
container into a pod.

The main reason this does not work is the PID namespace. If a non pod
container is being restored in a pod with a shared PID namespace, at
least one process in the restored container uses PID 1 which is already
in use by the infrastructure container. If someone tries to restore
container from a pod with a shared PID namespace without a shared PID
namespace it will also fail because the resulting PID namespace will not
have a PID 1.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-07-27 16:10:44 +02:00
Mehul Arora
6fe03b25ab support container to container copy
Implement container to container copy.  Previously data could only be
copied from/to the host.

Fixes: #7370
Co-authored-by: Mehul Arora <aroram18@mcmaster.ca>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-27 15:32:23 +02:00
Giuseppe Scrivano
63ef5576ed
command: migrate doesn't move process to cgroup
add a new annotation for the "system migrate" command to not move the
pause process to a separate cgroup.

The operation is not needed since "system migrate" destroys the pause
process, so there won't be any process left to move to a cgroup.

[NO TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-07-27 14:26:23 +02:00
Giuseppe Scrivano
a0313ef92b
rootless: do not overwrite err variable
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-07-27 14:26:22 +02:00
Valentin Rothberg
7689783ae5 exec: fix cleanup
Commit 341e6a1 made sure that all exec sessions are getting cleaned up.
But it also came with a peformance penalty.  Fix that penalty by
spawning the cleanup process to really only cleanup the exec session
without attempting to remove the container.

[NO TESTS NEEDED] since we have no means to test such performance
issues in CI.

Fixes: #10701
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-27 12:00:15 +02:00
OpenShift Merge Robot
d7b2f03f8a
Merge pull request #10983 from vrothberg/fix-dangling
refine dangling checks
2021-07-27 10:57:21 +02:00
Thomas Weber
cdbbd79155 stats: add a interval parameter to cli and api stream mode
podman stats polled by default in a 1 sec period.
This can put quite some load on a machine if you run many containers.

The default value is now 5 seconds.
You can change this interval with a new, optional, --interval, -i cli flag.
The api request got also a interval query parameter for the same purpose.

Additionally a unused const was removed.
Api and cli will fail the request if a 0 or negative value is passed in.

Signed-off-by: Thomas Weber <towe75@googlemail.com>
2021-07-27 07:20:47 +02:00
OpenShift Merge Robot
89853a30eb
Merge pull request #11036 from caarlos0/fix-podman-manifest-push
fix: podman manifest push respect --tls-verify flag
2021-07-26 16:01:21 +02:00
Carlos Alexandro Becker
5473490c6e fix: podman manifest push respect --tls-verify flag
[NO TESTS NEEDED]

Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2021-07-26 08:56:30 -03:00
Valentin Rothberg
c197d19fe8 play kube: support capitalized pull policy
Pull policies in K8s yaml may be capitalized, so lower them before
parsing.

Fixes: bugzilla.redhat.com/show_bug.cgi?id=1985905
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-26 10:28:45 +02:00
Valentin Rothberg
1b6423e9f1 refine dangling checks
By proxy by vendoring containers/common. Previously, a "dangling" image
was an untagged image; just a described in the Docker docs. The
definition of dangling has now been refined to an untagged image without
children to be compatible with Docker.

Further update a redundant image-prune test.

Fixes: #10998
Fixes: #10832
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-26 09:28:17 +02:00
cdoern
0f708efd8b Implemented --until flag for libpod's container logs
compat containers/logs was missing actual usage of until query param.
This led me to implement the until param for libpod's container logs as well. Added e2e tests.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-07-22 10:56:56 -04:00
OpenShift Merge Robot
68585b0168
Merge pull request #10848 from vrothberg/update-libimage
vendor containers/common@main
2021-07-16 11:12:44 -04:00
Valentin Rothberg
e1ac0c3033 vendor containers/common@main
The `IgnorePlatform` options has been removed from the
`LookupImageOptions` in libimage to properly support multi-arch images.

Skip one buildah-bud test which requires updated CI images.  This is
currently being done in github.com/containers/podman/pull/10829 but
we need to unblock merging common and buildah into podman.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 14:19:56 +02:00
Valentin Rothberg
9924c57d4c podman start: remove containers configured for auto removal
Make sure that containers configured for auto removal
(e.g., via `podman create --rm`) are removed in `podman start`
if starting the container failed.

Fixes: #10935
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 10:57:40 +02:00
OpenShift Merge Robot
d24fc6b843
Merge pull request #10939 from Luap99/rootless-cni
Fix race conditions in rootless cni setup
2021-07-15 11:11:10 -04:00
Paul Holzinger
0007c98ddb
Fix race conditions in rootless cni setup
There was an race condition when calling `GetRootlessCNINetNs()`. It
created the rootless cni directory before it got locked. Therefore
another process could have called cleanup and removed this directory
before it was used resulting in errors. The lockfile got moved into the
XDG_RUNTIME_DIR directory to prevent a panic when the parent dir was
removed by cleanup.

Fixes #10930
Fixes #10922

To make this even more robust `GetRootlessCNINetNs()` will now return
locked. This guarantees that we can run `Do()` after `GetRootlessCNINetNs()`
before another process could have called `Cleanup()` in between.

[NO TESTS NEEDED] CI is flaking, hopefully this will fix it.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-07-15 14:33:56 +02:00
Valentin Rothberg
59f31d86ac auto-update: add --dry-run
Add a --dry-run flag to `podman auto-update` which will look for new
images but won't perform any pull or restart any service or container.

The "UPDATED" column will now indicate the availability of a newer image
via "pending".

```
$ podman auto-update --dry-run
UNIT                    CONTAINER            IMAGE                   POLICY      UPDATED
container-test.service  08fd34e533fd (test)  localhost:5000/busybox  registry    false
```

Fixes: #9949
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-15 11:39:50 +02:00
Valentin Rothberg
01cfb51fe9 auto-update: make output more user friendly
The rather raw and scarce output of `podman auto-update` has been a
thorn in my eyes for a longer while.  So far, Podman would only print
updated systemd units, one per line, without further formatting.

Motivated by issue #9949 which is asking for some more useful
information in combination with a dry-run feature, I sat down and
reflected which information may come in handy.

Running `podman auto-update` will now look as follows:

```
$ podman auto-update
Trying to pull [...]

UNIT                    CONTAINER            IMAGE                   POLICY      UPDATED
container-test.service  08fd34e533fd (test)  localhost:5000/busybox  registry    false
```

Also refactor the spaghetti code in the backend a bit to make it easier
to digest and maintain.

For easier testing and for the sake of consistency with other commands
listing output, add a `--format` flag.

The man page will get an overhaul in a follow up commit.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-14 16:23:51 +02:00
Valentin Rothberg
6fcf0b2f32 auto update: minor style nits
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-13 13:52:46 +02:00
Valentin Rothberg
a90a4ec7c6 auto update: pass through a context
Pass a single context.Context through the call stack.  If auto-updates
will ever be made available for REST calls, the context will help
supporting disconnected clients.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-13 13:52:46 +02:00
Valentin Rothberg
38bef70b30 manifest push --rm: use libimage for removal
Use libimage for removing the manifest instead of going directly
through the store.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-07 15:55:36 +02:00
OpenShift Merge Robot
895b815188
Merge pull request #10836 from Luap99/diff
podman diff accept two images or containers
2021-07-03 06:17:03 -04:00
Paul Holzinger
8f6a0243f4
podman diff accept two images or containers
First, make podman diff accept optionally a second argument. This allows
the user to specify a second image/container to compare the first with.
If it is not set the parent layer will be used as before.

Second, podman container diff should only use containers and podman
image diff should only use images. Previously, podman container diff
would use the image when both an image and container with this name
exists.

To make this work two new parameters have been added to the api. If they
are not used the previous behaviour is used. The same applies to the
bindings.

Fixes #10649

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-07-02 17:11:56 +02:00
Matej Vasek
86c6014145 Implement --archive flag for podman cp
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-07-01 12:01:46 +02:00
Daniel J Walsh
092902b455
Handle advanced --network options in podman play kube
Since Podman create/run can support this, so should play.

Fixes: https://github.com/containers/podman/issues/10807

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-06-30 13:32:27 -04:00
flouthoc
2243b60203 reset: remove external containers on podman system reset
[NO TESTS NEEDED]

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-06-30 09:04:08 +05:30
Tino Rusch
e7507fe7cf
make DriverOpts name consistent.
Signed-off-by: Tino Rusch <tino.rusch@gmail.com>
2021-06-25 08:33:15 +02:00
Tino Rusch
b56b4b5374
read secret config from config file if no user data.
feat: read secret config from config file if the user hasn't entered
explicit config values

feat: allow to specify `--driver-opts opt1=val1,opt2=val2` in the secret
create command to allow overriding the default values

fix: show driver options in `podman secret inspect`

Signed-off-by: Tino Rusch <tino.rusch@gmail.com>
2021-06-24 12:31:14 +02:00
Valentin Rothberg
5fc622f945 create: support images with invalid platform
Much to my regret, there is a number of images in the wild with invalid
platforms breaking the platform checks in libimage that want to make
sure that a local image is matching the expected platform.

Imagine a `podman run --arch=arm64 fedora` with a local amd64 fedora
image.  We really shouldn't use the local one in this case and pull down
the arm64 one.

The strict platform checks in libimage in combination with invalid
platforms in images surfaced in Podman being able to pull an image but
failing to look it up in subsequent presence checks.  A `podman run`
would hence pull such an image but fail to create the container.

Support images with invalid platforms by vendoring the latest HEAD from
containers/common.  Also remove the partially implemented pull-policy
logic from Podman and let libimage handle that entirely.  However,
whenever --arch, --os or --platform are specified, the pull policy will
be forced to "newer".  This way, we pessimistically assume that the
local image has an invalid platform and we reach out to the registry.
If there's a newer image (i.e., one with a different digest), we'll pull
it down.

Please note that most of the logic has either already been implemented
in libimage or been moved down which allows for removing some clutter
from Podman.

[NO TESTS NEEDED] since c/common has new tests.  Podman can rely on the
existing tests.

Fixes: #10648
Fixes: #10682
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-06-23 15:42:13 +02:00