Commit graph

2145 commits

Author SHA1 Message Date
Valentin Rothberg
fceecc3a5b remote checkpoint/restore: more fixes
* Support `checkpoint --pre-checkpoint`

* Support `checkpoint --with-previous`

* Disable `restore --import-previous` for the remote client since we had
  to send two files which in turn would require to tar them up and hence
  be a breaking change.  Podman 4.0 would be the chance and I hope we'll
  find time before that to remote-restore prettier.

Note that I did not run over swagger yet to check whether all parameters
are actually documented due to time constraints.

Fixes: #12334
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-18 16:52:15 +01:00
Valentin Rothberg
6f6a6925b2 fix CI
Our fedora-minimal image on Quay bases on fedora-minimal:latest which
starting with F35 removed a number of binaries that our CI depends on.
Fix that by pulling `fedora-minimal:34` from the Fedora registry
directly.

Once the build bot on Quay has been disabled, we move the image over
there to make sure that it will not change over time.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-18 15:19:14 +01:00
OpenShift Merge Robot
85733e0b4a
Merge pull request #12281 from vrothberg/fix-12007
fix remote checkpoint/restore
2021-11-17 12:02:37 +01:00
OpenShift Merge Robot
197ebe8b5e
Merge pull request #12218 from vrothberg/pause-catatonit
infra container: replace pause with catatonit
2021-11-16 18:16:34 +01:00
Valentin Rothberg
33ec8c6698 fix remote checkpoint/restore
Nothing was working before, and it's too much to summarize.  To make
sure we're not regressing in the future again, enable the remote e2e
tests.

Fixes: #12007
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-16 14:41:18 +01:00
OpenShift Merge Robot
be681ab518
Merge pull request #12294 from flouthoc/secret-mount-target
secret: honor custom `target=` for secrets with `type=mount` for ctr.
2021-11-16 01:45:27 +01:00
Aditya Rajan
014cc4b9d9
secret: honor custom target for secrets with run
Honor custom `target` if specified while running or creating containers
with secret `type=mount`.

Example:
`podman run -it --secret token,type=mount,target=TOKEN ubi8/ubi:latest
bash`

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-15 23:19:27 +05:30
Valentin Rothberg
5934e4c9b5 infra container: replace pause with catatonit
Podman has been using catatonit for a number of years already.
Thanks to @giuseppe, catatonit is now able to run as a pause
process which allows us to replace the pause binary entirely.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-15 12:53:25 +01:00
Adrian Reber
d28b39a90d
Added test for checkpoint/restore --print-stats
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
OpenShift Merge Robot
cca6df428c
Merge pull request #12272 from hshiina/memory-swappiness
Enable 'podman run --memory-swappiness=0'
2021-11-13 13:12:48 +01:00
OpenShift Merge Robot
78bc2390f4
Merge pull request #12271 from Luap99/play-kube-net
Fix network mode in play kube
2021-11-12 19:59:44 +01:00
Hironori Shiina
9226ccb59f Enable 'podman run --memory-swappiness=0'
'--memory-swappiness=0' used to work. This patch fixes the regression
issue, which was caused by the change of infra container creation
process.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-11-12 13:20:20 -05:00
OpenShift Merge Robot
0aecacb865
Merge pull request #12224 from cdoern/scp
Podman Image SCP transfer patch
2021-11-12 18:44:44 +01:00
Paul Holzinger
164c42b485
Fix network mode in play kube
We need to use the config network mode when no network mode was set. To
do so we have to keep the nsmode empty, MakeContainer() will use the
correct network mode from the config when needed.

Fixes #12248

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-12 18:32:01 +01:00
cdoern
ac38eca3fd Podman Image SCP transfer patch
Fixed syntax so that podman image scp transfer works with no user specified.
This command can only be executed as root so to obtain the default user, I searched for
the SUDO_USER environmental variable. If that is not found, we error out and inform the user
to set this variable and make sure they are running as root

Signed-off-by: cdoern <cdoern@redhat.com>
Signed-off-by: cdoern <cbdoer23@g.holycross.edu>
2021-11-11 20:14:38 -05:00
OpenShift Merge Robot
fa8184c807
Merge pull request #12240 from umohnani8/table
Print headers for system connection ls
2021-11-10 23:29:36 +01:00
Paul Holzinger
27de152b5a
network reload without ports should not reload ports
When run as rootless the podman network reload command tries to reload
the rootlessport ports because the childIP could have changed.
However if the containers has no ports we should skip this instead of
printing a warning.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-10 21:16:08 +01:00
Urvashi Mohnani
a55fdbb49c Print headers for system connection ls
Print out the headers even if the system connection list
is empty to match the behavior of other list commands.

Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-11-10 14:48:08 -05:00
Valentin Rothberg
faf450ea18 support health checks from image configs
Health checks may be defined in the container config or the config of an
image.  So far, Podman only looked at the container config.

The plumbing happened in libimage but add a regression test to Podman as
well to make sure the glue code will not regress.

Note that I am pinning github.com/onsi/gomega to v1.16.0 since v1.17.0
requires go 1.16 which in turn is breaking CI.

Fixes: #12226
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-09 15:32:36 +01:00
OpenShift Merge Robot
6d56ac7e0f
Merge pull request #12222 from Luap99/pod-create-net
pod create: read network mode from config
2021-11-09 02:08:56 +01:00
Paul Holzinger
8b7b0b7caf
pod create: read network mode from config
When we create a pod we have to parse the network mode form the config
file. This is a regression in commit d28e85741f.

Fixes #12207

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-08 18:59:28 +01:00
Jhon Honce
e907f095b2 test connection add
* Fix connection JSON encoding
* Add custom ginkgo matchers for connection testing
* Cleanup code

Fixes #11984

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-08 09:20:58 -07:00
OpenShift Merge Robot
22ef488d24
Merge pull request #12213 from flouthoc/system-connection-rm-all
system: Adds support for removing all named destination via `--all`
2021-11-08 16:50:59 +01:00
OpenShift Merge Robot
c9ba1fb7d4
Merge pull request #11958 from cdoern/scp
Podman Image SCP rootful to rootless transfer
2021-11-08 16:19:57 +01:00
Aditya Rajan
338eb9d75e
system: Adds support for removing all named destination via --all
Adds support of dropping all named destination from system connections via `--all`.

Closes: https://github.com/containers/podman/issues/12018

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-08 19:42:45 +05:30
OpenShift Merge Robot
865653b661
Merge pull request #12184 from adrianreber/2021-11-05-stats-dump
Add 'stats-dump' file to exported checkpoint
2021-11-08 09:29:56 +01:00
OpenShift Merge Robot
abbd6c167e
Merge pull request #11890 from Luap99/ports
libpod: deduplicate ports in db
2021-11-06 10:39:16 +01:00
Adrian Reber
3e1940a8e4
Test to check for presence of 'stats-dump' in exported checkpoints
Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-05 16:15:01 +00:00
cdoern
ffa5ed0e0e Podman Image SCP rootful to rootless transfer
Added functionality for users to transfer images from root storage to rootless storage without using sshd. This is
done through rootful podman by running `sudo podman image scp root@localhost::image user@localhost:: the user is needed
in order to find and use their uid/gid to exec a new process.

added necessary tests, and functions for this implementation. Created new image function Transfer so that
the underlying code is majorly removed from CLI

Signed-off-by: cdoern <cdoern@redhat.com>
2021-11-05 12:04:20 -04:00
Paul Holzinger
7f433df7e7
rename rootless cni ns to rootless netns
Since we want to use the rootless cni ns also for netavark we should
pick a more generic name. The name is now "rootless network namespace"
or short "rootless netns".

The rename might cause some issues after the update but when the
all containers are restarted or the host is rebooted it should work
correctly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 15:44:37 +01:00
OpenShift Merge Robot
c0351a75ae
Merge pull request #12162 from giuseppe/run-split-test-in-separate-cgroup
test: run --cgroups=split in new cgroup
2021-11-04 15:43:31 +01:00
Giuseppe Scrivano
0234b153cc
test: run --cgroups=split in new cgroup
the --cgroups=split test changes the current cgroup as it creates a
sub-cgroup.  This can cause a race condition in tests that are reading
the current cgroup.

Closes: https://github.com/containers/podman/issues/11191

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-04 13:16:17 +01:00
OpenShift Merge Robot
8fdde67008
Merge pull request #12060 from mtrmac/podman-trust-show-f35
Fix `Podman image trust` tests
2021-11-03 21:49:21 +01:00
Praveen Kumar
e69eae6458 Fix help message case for podman version
This is a cosmetic change. The help message for `podman version` is in
title case whereas all other command help messages are not in title
case. This stands out as inconsistent when looking at the output of
`podman help`.

Signed-off-by: Praveen Kumar <praveen+git@kumar.in>
2021-10-29 11:49:49 -07:00
Giuseppe Scrivano
4e9e6f21ff
volumes: allow more options for devpts
allow to pass down more options that are supported by the kernel.

Discussion here: https://github.com/containers/toolbox/issues/568

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 15:30:06 +02:00
Paul Holzinger
0136a66a83
libpod: deduplicate ports in db
The OCICNI port format has one big problem: It does not support ranges.
So if a users forwards a range of 1k ports with podman run -p 1001-2000
we have to store each of the thousand ports individually as array element.
This bloats the db and makes the JSON encoding and decoding much slower.
In many places we already use a better port struct type which supports
ranges, e.g. `pkg/specgen` or the new network interface.

Because of this we have to do many runtime conversions between the two
port formats. If everything uses the new format we can skip the runtime
conversions.

This commit adds logic to replace all occurrences of the old format
with the new one. The database will automatically migrate the ports
to new format when the container config is read for the first time
after the update.

The `ParsePortMapping` function is `pkg/specgen/generate` has been
reworked to better work with the new format. The new logic is able
to deduplicate the given ports. This is necessary the ensure we
store them efficiently in the DB. The new code should also be more
performant than the old one.

To prove that the code is fast enough I added go benchmarks. Parsing
1 million ports took less than 0.5 seconds on my laptop.

Benchmark normalize PortMappings in specgen:
Please note that the 1 million ports are actually 20x 50k ranges
because we cannot have bigger ranges than 65535 ports.
```
$ go test -bench=. -benchmem  ./pkg/specgen/generate/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/pkg/specgen/generate
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
BenchmarkParsePortMappingNoPorts-12             480821532                2.230 ns/op           0 B/op          0 allocs/op
BenchmarkParsePortMapping1-12                      38972             30183 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMapping100-12                    18752             60688 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMapping1k-12                      3104            331719 ns/op          223840 B/op       3018 allocs/op
BenchmarkParsePortMapping10k-12                      376           3122930 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMapping1m-12                         3         390869926 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingReverse100-12             18940             63414 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMappingReverse1k-12               3015            362500 ns/op          223841 B/op       3018 allocs/op
BenchmarkParsePortMappingReverse10k-12               343           3318135 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMappingReverse1m-12                  3         403392469 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingRange1-12                 37635             28756 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange100-12               39604             28935 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1k-12                38384             29921 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange10k-12               29479             40381 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1m-12                  927           1279369 ns/op          143022 B/op        164 allocs/op
PASS
ok      github.com/containers/podman/v3/pkg/specgen/generate    25.492s
```

Benchmark convert old port format to new one:
```
go test -bench=. -benchmem  ./libpod/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/libpod
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Benchmark_ocicniPortsToNetTypesPortsNoPorts-12          663526126                1.663 ns/op           0 B/op          0 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1-12                 7858082               141.9 ns/op            72 B/op          2 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10-12                2065347               571.0 ns/op           536 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts100-12                138478              8641 ns/op            4216 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1k-12                   9414            120964 ns/op           41080 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10k-12                   781           1490526 ns/op          401528 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1m-12                      4         250579010 ns/op        40001656 B/op          4 allocs/op
PASS
ok      github.com/containers/podman/v3/libpod  11.727s
```

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-27 18:59:56 +02:00
OpenShift Merge Robot
d908da51fb
Merge pull request #12064 from vrothberg/fix-11933
container create: fix --tls-verify parsing
2021-10-27 15:24:23 +00:00
OpenShift Merge Robot
6caf5e3b7c
Merge pull request #12111 from giuseppe/fix-warning-move-pause-process
runtime: check for pause pid existence
2021-10-27 15:07:59 +00:00
OpenShift Merge Robot
b29dc1bde3
Merge pull request #12110 from cevich/fix_systemd_pid1
Fix systemd PID1 test
2021-10-27 14:34:58 +00:00
Valentin Rothberg
ff31f2264d container create: fix --tls-verify parsing
Make sure that the value is only set if specified on the CLI.  c/image
already defaults to true but if set in the system context, we'd skip
settings in the registries.conf.

Fixes: #11933
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-27 14:36:25 +02:00
OpenShift Merge Robot
979b631228
Merge pull request #11956 from vrothberg/pause
remove need to download pause image
2021-10-27 10:22:56 +00:00
Giuseppe Scrivano
6b3b0a17c6
runtime: check for pause pid existence
check that the pause pid exists before trying to move it to a separate
scope.

Closes: https://github.com/containers/podman/issues/12065

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-27 11:24:50 +02:00
OpenShift Merge Robot
ed3aa2acaf
Merge pull request #12098 from Luap99/slirp-dad
Slirp4netns with ipv6 set net.ipv6.conf.default.accept_dad=0
2021-10-26 20:54:27 +00:00
Chris Evich
d53789068a
Fix systemd PID1 test
Previously this test used an ad-hoc timeout mechanism to synchronize
with output of the container ID.  However, depending on runtime
conditions this may not correctly correspond with complete startup
of the systemd process.  Consequently this test fails under some
conditions with an error like:

`System has not been booted with systemd as init system (PID 1). Can't
operate. Failed to connect to bus: Host is down`

Fix this by using the more appropriate `WaitContainerReady()`
against output from system startup, close to finalization.  In this way,
the test status command cannot run until systemd is fully operational.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-10-26 16:39:14 -04:00
OpenShift Merge Robot
1243954372
Merge pull request #12067 from hshiina/logs-journal-tail
Fix a few problems in 'podman logs --tail' with journald driver
2021-10-26 20:33:26 +00:00
OpenShift Merge Robot
420ac5d13d
Merge pull request #12088 from adrianreber/2021-10-25-fix-label-ipc-host
Allow 'container restore' with '--ipc host'
2021-10-26 16:38:54 +00:00
Paul Holzinger
008075ce54
Slirp4netns with ipv6 set net.ipv6.conf.default.accept_dad=0
Duplicate Address Detection slows the ipv6 setup down for 1-2 seconds.
Since slirp4netns is run it is own namespace and not directly routed
we can skip this to make the ipv6 address immediately available.
We change the default to make sure the slirp tap interface gets the
correct value assigned so DAD is disabled for it.
Also make sure to change this value back to the original after slirp4netns
is ready in case users rely on this sysctl.

Fixes #11062

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-26 18:27:30 +02:00
Hironori Shiina
c723e6b978 Fix a few problems in 'podman logs --tail' with journald driver
The following problems regarding `logs --tail` with the journald log
driver are fixed:
- One more line than a specified value is displayed.
- '--tail 0' displays all lines while the other log drivers displays
  nothing.
- Partial lines are not considered.
- If the journald events backend is used and a container has exited,
  nothing is displayed.

Integration tests that should have detected the bugs are also fixed. The
tests are executed with json-file log driver three times without this
fix.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-10-26 12:18:57 -04:00
Adrian Reber
bf8fd943ef
Allow 'container restore' with '--ipc host'
Trying to restore a container that was started with '--ipc host' fails
with:

Error: error creating container storage: ProcessLabel and Mountlabel must either not be specified or both specified

We already fixed this exact same error message for containers started
with '--privileged'. The previous fix was to check if the to be restored
container is a privileged container (c.config.Privileged). Unfortunately
this does not work for containers started with '--ipc host'.

This commit changes the check for a privileged container to check if
both the ProcessLabel and the MountLabel is actually set and only then
re-uses those labels.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-10-26 14:42:32 +00:00
Valentin Rothberg
75f478c08b pod create: remove need for pause image
So far, the infra containers of pods required pulling down an image
rendering pods not usable in disconnected environments.  Instead, build
an image locally which uses local pause binary.

Fixes: #10354
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-26 13:51:45 +02:00