Commit graph

765 commits

Author SHA1 Message Date
cdoern
d28e85741f InfraContainer Rework
InfraContainer should go through the same creation process as regular containers. This change was from the cmd level
down, involving new container CLI opts and specgen creating functions. What now happens is that both container and pod
cli options are populated in cmd and used to create a podSpecgen and a containerSpecgen. The process then goes as follows

FillOutSpecGen (infra) -> MapSpec (podOpts -> infraOpts) -> PodCreate -> MakePod -> createPodOptions -> NewPod -> CompleteSpec (infra) -> MakeContainer -> NewContainer -> newContainer -> AddInfra (to pod state)

Signed-off-by: cdoern <cdoern@redhat.com>
2021-08-26 16:05:16 -04:00
OpenShift Merge Robot
94c37d7d47
Merge pull request #11298 from baude/kubeupdown
teardown play kube
2021-08-26 13:58:44 -04:00
OpenShift Merge Robot
49cfed756f
Merge pull request #11103 from jwhonce/wip/bindings
Fix file descriptor leaks in bindings and add test
2021-08-25 14:42:12 -04:00
Jhon Honce
1dc6d14735 Fix file descriptor leaks and add test
* Add response.Body.Close() where needed to release HTTP
  connections to API server.
* Add tests to ensure no general leaks occur. 100% coverage would be
  required to ensure no leaks on any call.
* Update code comments to be godoc correct

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-08-24 16:36:10 -07:00
Brent Baude
1e176923b1 teardown play kube
add the ability for play kube to tear down based on the yaml used to
play it.  it is indicated by --down in the play kube command.  volumes
are NOT deleted during the teardown.  pods and their containers are
stopped and removed.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-24 14:26:14 -05:00
OpenShift Merge Robot
23f9565547
Merge pull request #11263 from nalind/journal-read
libpod/Container.readFromJournal(): don't skip the first entry
2021-08-24 14:16:22 -04:00
Paul Holzinger
c0b1edd6a4
Network interface
Implement a new network interface to abstract CNI from libpod. The
interface is implemented for the CNI backend but in the future we can
add more backends.

The code is structured in three new packages:
- `libpod/network/types`: contains the interface definition
  and the necessary types for it.
- `libpod/network/cni` contains the interface implementation for the CNI
  backend.
- `libpod/network/util` a set of utility functions related to
  networking.

The CNI package uses ginkgo style unit tests. To test Setup/Teardown the
test must be run as root. Each test will run in their own namespace to
make the test independent from the host environment.

New features with the CNI backend:
- The default network will be created in memory if it does not exists on
  disk.
- It can set more than one static IP per container network.
- Networks are loaded once from disk and only if this interface is
  used, e.g. for commands such as `podman info` networks are not loaded.
  This reduces unnecessary disk IO.

This commit only adds the interface it is not wired into libpod. This
requires a lot of breaking changes which will be done in a followup
commit.

Once this is integrated into libpod the current network code under
`libpod/network` should be removed. Also the dependency on OCICNI
should be dropped.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-08-24 12:28:03 +02:00
Nalin Dahyabhai
21f396de6f logs: adjust handling around partial log messages
In libpod/logs.LogLine.Write(), don't write a newline to stdout/stderr
when the log message is only part of a line.

In libpod.ConmonOCIRuntime.HTTPAttach(), don't send a newline over the
HTTP connection when the log message is only part of a line.

In pkg/api/handlers/compat.LogsFromContainer(), don't send a newline
over the HTTP connection when the log message is only part of a line,
and don't make doing so conditional on whether or not the client used
the docker or podman endpoint.

In pkg/domain/infra/tunnel.ContainerEngine.ContainerLogs(), don't add
our own newline to log messages, since they already come through from
the server when they need to.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-08-23 18:00:18 -04:00
OpenShift Merge Robot
90cf78b199
Merge pull request #11290 from flouthoc/volume-export
volumes: Add support for `volume export` which allows exporting content to  external path.
2021-08-23 14:01:20 -04:00
flouthoc
edddfe8c4f volumes: Add support for exporting volumes to external tar
Adds support for transferring data between systems and backing up systems.
Use cases: recover from disasters or move data between machines.

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-23 20:42:41 +05:30
Valentin Rothberg
33fee83dea add flag to record memory profiles
Add a new flag `--memory-profile=$path` which creates a memory profile.
The generated profile can later be analyzed via `go tool pprof`.

[NO TESTS NEEDED] since it's a hidden flag, devs-only.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-20 15:35:58 +02:00
Brent Baude
cd40c875ac Add ability to build images in play kube
When playing a kube YAML file, it can be desirable to be able to build
an image on the fly.  This is good for development of an image and YAML
files and somewhat mocks what compose does.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-18 09:55:24 -05:00
Valentin Rothberg
fe2be7f886 make sure that signal buffers are sufficiently big
Dealing with os.Signal channels seems more like an art than science
since signals may get lost.  os.Notify doesn't block on an unbuffered
channel, so users are expected to know what they're doing or hope for
the best.

In the recent past, I've seen a number of flakes and BZs on non-amd64
architectures where I was under the impression that signals may got
lost, for instance, during stop and exec.

[NO TESTS NEEDED] since this is art.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-17 12:10:20 +02:00
Daniel J Walsh
404488a087
Run codespell to fix spelling
[NO TESTS NEEDED] Just fixing spelling.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-11 16:41:45 -04:00
openshift-ci[bot]
1968fdc874
Merge pull request #11173 from jmguzik/pod-ps-until-filter
Add until filter to podman pod ps
2021-08-11 08:38:20 +00:00
Jakub Guzik
ed30ae4a8a Add until filter to podman pod ps
This commit adds additional until filter to podman pod ps (ls/list).
Additionally, it also adds descriptions for podman pod ps filters available
via http api.

Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-08-10 22:10:40 +02:00
Daniel J Walsh
221b1add74 Add support for pod inside of user namespace.
Add the --userns flag to podman pod create and keep
track of the userns setting that pod was created with
so that all containers created within the pod will inherit
that userns setting.

Specifically we need to be able to launch a pod with
--userns=keep-id

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-08-09 15:17:22 -04:00
openshift-ci[bot]
8409817733
Merge pull request #11074 from vrothberg/auto-update-rollback
auto-update: simple rollback
2021-08-06 12:26:31 +00:00
openshift-ci[bot]
1f0a24437d
Merge pull request #11011 from baude/initcontainers
implement init containers in podman
2021-08-05 13:52:15 +00:00
Valentin Rothberg
30df551bde auto-update: simple rollback
Add support for simple rollbacks during `podman auto-update`.  Rollbacks
are enabled by default.  If a systemd unit cannot be restarted after an
update, the previous image will be retagged and the unit will be
restarted a second time.

Add system tests for rollbacks.  Also fix a bug in the restart sequence;
we have to use the channel to actually know whether the restart was
successful or not.

NOTE: To make rollbacks really useful, users must run their containers
with `--sdnotify=container` such that the containers send the ready
message over the (mounted) socket.  This way, restarting the systemd
units during auto update will block until the message has been received
(or a timeout kicked in).

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-05 15:20:38 +02:00
Brent Baude
3c3fa6fac4 implement init containers in podman
this is the first pass at implementing init containers for podman pods.
init containersare made popular by k8s as a way to run setup for pods
before the pods standard containers run.

unlike k8s, we support two styles of init containers: always and
oneshot.  always means the container stays in the pod and starts
whenever a pod is started.  this does not apply to pods restarting.
oneshot means the container runs onetime when the pod starts and then is
removed.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-04 14:14:36 -05:00
Daniel J Walsh
41f94a4dc1
Fix podman unpause,pause,kill --all to work like podman stop --all
Currently if you execute podman unpause --all, podman pause --all
Podman shows attempts to unpause containers that are not paused
and prints an error.  This PR catches this error and only prints errors if
a paused container was not able to be unpaused.

Currently if you execute podman pause --all or podman kill --all, Podman
Podman shows attempts to pause or kill containers that are not running
and prints an error.  This PR catches this error and only prints errors if
a running container was not able to be paused or killed.

Also change printing of multiple errors to go to stderr and to prefix
"Error: " in front to match the output of the last error.

Fixes: https://github.com/containers/podman/issues/11098

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-04 12:50:00 -04:00
openshift-ci[bot]
8aa869e628
Merge pull request #10973 from rhatdan/quota
Support size options on builtin volumes
2021-08-04 11:59:58 +00:00
openshift-ci[bot]
3a922cbc99
Merge pull request #11003 from pascomnet/f_stats
stats: add a interval parameter to cli and api stats streaming
2021-08-04 09:56:57 +00:00
openshift-ci[bot]
bdbc21095a
Merge pull request #10828 from cdoern/scp
Created image scp feature
2021-08-02 17:15:54 +00:00
Daniel J Walsh
c0952c7334
Support size and inode options on builtin volumes
[NO TESTS NEEDED] Since it is difficult to setup xfs quota

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1982164

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-02 10:32:45 -04:00
cdoern
1d10ca739f Created scp.go image_scp_test.go and podman-image-scp.1.md
added functionality for image secure copying from local to remote.
Also moved system connection add code around a bit so functions within that file
can be used by scp.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-07-30 17:19:24 -04:00
flouthoc
2a484e782a ps: support the container notation for ps --filter network=...
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-07-30 19:31:05 +05:30
OpenShift Merge Robot
f9395ddc5a
Merge pull request #11067 from vrothberg/fix-10154-2
remote build: fix streaming and error handling
2021-07-28 16:17:45 +02:00
OpenShift Merge Robot
6c5966cf3c
Merge pull request #10910 from adrianreber/2021-07-12-checkpoint-restore-into-pod
Add support for checkpoint/restore into and out of pods
2021-07-28 14:48:28 +02:00
Valentin Rothberg
4df6e31ccb remote build: fix streaming and error handling
Address a number of issues in the streaming logic in remote build, most
importantly an error in using buffered channels on the server side.

The pattern below does not guarantee that the channel is entirely read
before the context fires.

for {
	select {
		case <- bufferedChannel:
		...
		case <- ctx.Done():
		...
	}
}

Fixes: #10154
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-28 14:46:15 +02:00
OpenShift Merge Robot
a5de831418
Merge pull request #11049 from vrothberg/fix-7370
support container to container copy
2021-07-27 22:20:45 +02:00
OpenShift Merge Robot
a6a0f6c192
Merge pull request #11061 from giuseppe/migrate-no-move-to-cgroup
command: migrate doesn't move process to cgroup
2021-07-27 20:46:46 +02:00
Adrian Reber
eb94467780
Support checkpoint/restore with pods
This adds support to checkpoint containers out of pods and restore
container into pods.

It is only possible to restore a container into a pod if it has been
checkpointed out of pod. It is also not possible to restore a non pod
container into a pod.

The main reason this does not work is the PID namespace. If a non pod
container is being restored in a pod with a shared PID namespace, at
least one process in the restored container uses PID 1 which is already
in use by the infrastructure container. If someone tries to restore
container from a pod with a shared PID namespace without a shared PID
namespace it will also fail because the resulting PID namespace will not
have a PID 1.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-07-27 16:10:44 +02:00
Mehul Arora
6fe03b25ab support container to container copy
Implement container to container copy.  Previously data could only be
copied from/to the host.

Fixes: #7370
Co-authored-by: Mehul Arora <aroram18@mcmaster.ca>
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-27 15:32:23 +02:00
Giuseppe Scrivano
63ef5576ed
command: migrate doesn't move process to cgroup
add a new annotation for the "system migrate" command to not move the
pause process to a separate cgroup.

The operation is not needed since "system migrate" destroys the pause
process, so there won't be any process left to move to a cgroup.

[NO TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-07-27 14:26:23 +02:00
Giuseppe Scrivano
a0313ef92b
rootless: do not overwrite err variable
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-07-27 14:26:22 +02:00
Valentin Rothberg
7689783ae5 exec: fix cleanup
Commit 341e6a1 made sure that all exec sessions are getting cleaned up.
But it also came with a peformance penalty.  Fix that penalty by
spawning the cleanup process to really only cleanup the exec session
without attempting to remove the container.

[NO TESTS NEEDED] since we have no means to test such performance
issues in CI.

Fixes: #10701
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-27 12:00:15 +02:00
OpenShift Merge Robot
d7b2f03f8a
Merge pull request #10983 from vrothberg/fix-dangling
refine dangling checks
2021-07-27 10:57:21 +02:00
Thomas Weber
cdbbd79155 stats: add a interval parameter to cli and api stream mode
podman stats polled by default in a 1 sec period.
This can put quite some load on a machine if you run many containers.

The default value is now 5 seconds.
You can change this interval with a new, optional, --interval, -i cli flag.
The api request got also a interval query parameter for the same purpose.

Additionally a unused const was removed.
Api and cli will fail the request if a 0 or negative value is passed in.

Signed-off-by: Thomas Weber <towe75@googlemail.com>
2021-07-27 07:20:47 +02:00
OpenShift Merge Robot
89853a30eb
Merge pull request #11036 from caarlos0/fix-podman-manifest-push
fix: podman manifest push respect --tls-verify flag
2021-07-26 16:01:21 +02:00
Carlos Alexandro Becker
5473490c6e fix: podman manifest push respect --tls-verify flag
[NO TESTS NEEDED]

Signed-off-by: Carlos Alexandro Becker <caarlos0@gmail.com>
2021-07-26 08:56:30 -03:00
Valentin Rothberg
c197d19fe8 play kube: support capitalized pull policy
Pull policies in K8s yaml may be capitalized, so lower them before
parsing.

Fixes: bugzilla.redhat.com/show_bug.cgi?id=1985905
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-26 10:28:45 +02:00
Valentin Rothberg
1b6423e9f1 refine dangling checks
By proxy by vendoring containers/common. Previously, a "dangling" image
was an untagged image; just a described in the Docker docs. The
definition of dangling has now been refined to an untagged image without
children to be compatible with Docker.

Further update a redundant image-prune test.

Fixes: #10998
Fixes: #10832
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-26 09:28:17 +02:00
OpenShift Merge Robot
d956500743
Merge pull request #10996 from cdoern/untilLog
Implemented --until flag for Libpod's Container Logs
2021-07-24 05:56:18 -04:00
cdoern
0f708efd8b Implemented --until flag for libpod's container logs
compat containers/logs was missing actual usage of until query param.
This led me to implement the until param for libpod's container logs as well. Added e2e tests.

Signed-off-by: cdoern <cdoern@redhat.com>
2021-07-22 10:56:56 -04:00
Jakub Guzik
2d8e837a9b Add until filter to volume ls filters list
As a conclusion of a discussion in #10861, until filter is added
by this commit to volume ls filters.

Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-07-22 00:01:07 +02:00
OpenShift Merge Robot
68585b0168
Merge pull request #10848 from vrothberg/update-libimage
vendor containers/common@main
2021-07-16 11:12:44 -04:00
Valentin Rothberg
e1ac0c3033 vendor containers/common@main
The `IgnorePlatform` options has been removed from the
`LookupImageOptions` in libimage to properly support multi-arch images.

Skip one buildah-bud test which requires updated CI images.  This is
currently being done in github.com/containers/podman/pull/10829 but
we need to unblock merging common and buildah into podman.

[NO TESTS NEEDED]

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-07-16 14:19:56 +02:00
OpenShift Merge Robot
ce28dc3c4c
Merge pull request #10820 from jvanz/indfra-container-name-issue-10794
--infra-name command line argument
2021-07-16 07:53:43 -04:00