Commit graph

6757 commits

Author SHA1 Message Date
Paul Holzinger
7f433df7e7
rename rootless cni ns to rootless netns
Since we want to use the rootless cni ns also for netavark we should
pick a more generic name. The name is now "rootless network namespace"
or short "rootless netns".

The rename might cause some issues after the update but when the
all containers are restarted or the host is rebooted it should work
correctly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 15:44:37 +01:00
Markus Thömmes
756dda298c Keep error semantics intact
Signed-off-by: Markus Thömmes <markusthoemmes@me.com>
2021-11-05 09:34:22 +01:00
OpenShift Merge Robot
c0351a75ae
Merge pull request #12162 from giuseppe/run-split-test-in-separate-cgroup
test: run --cgroups=split in new cgroup
2021-11-04 15:43:31 +01:00
Giuseppe Scrivano
0234b153cc
test: run --cgroups=split in new cgroup
the --cgroups=split test changes the current cgroup as it creates a
sub-cgroup.  This can cause a race condition in tests that are reading
the current cgroup.

Closes: https://github.com/containers/podman/issues/11191

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-04 13:16:17 +01:00
OpenShift Merge Robot
8fdde67008
Merge pull request #12060 from mtrmac/podman-trust-show-f35
Fix `Podman image trust` tests
2021-11-03 21:49:21 +01:00
OpenShift Merge Robot
4105b025d6
Merge pull request #12166 from Luap99/mac
MAC address json unmarshal should allow strings
2021-11-03 17:15:46 +01:00
Paul Holzinger
001d48929d
MAC address json unmarshal should allow strings
Create a new mac address type which supports json marshal/unmarshal from
and to string. This change is backwards compatible with the previous
versions as the unmarshal method still accepts the old byte array or
base64 encoded string.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-03 15:30:16 +01:00
OpenShift Merge Robot
33643f4b09
Merge pull request #12159 from jwhonce/issues/12115
Implement top streaming for containers and pods
2021-11-02 22:28:54 +01:00
OpenShift Merge Robot
82dba97547
Merge pull request #12158 from edsantiago/more_bats
System tests: enhance volume test, add debug prints
2021-11-02 18:51:24 +01:00
OpenShift Merge Robot
e63e90999c
Merge pull request #12156 from matejvasek/docker-api-zero-value-fixes
Fix libpod API conformance to swagger
2021-11-02 18:19:24 +01:00
Jhon Honce
449cc7a5c2 Implement top streaming for containers and pods
* Implement API query parameter stream and delay for containers and
  pods top endpoints
* Update swagger with breaking changes
* Add python API tests for endpoints

Fixes #12115

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-02 08:11:14 -07:00
OpenShift Merge Robot
0686f0bb2f
Merge pull request #12118 from hshiina/log-f-journald
Set flags to test 'logs -f' with journald driver
2021-11-02 13:18:26 +01:00
Ed Santiago
7b2531c135 System tests: enhance volume test, add debug prints
Volume test: add a sequence of stat()s to confirm that volumes
are mounted as a different device than root.

Network test: add debugging code for #11825 (dnsmasq inotify
failure in bodhi only).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-11-01 13:03:05 -06:00
Matej Vasek
218d91d76d Fix libpod API conformance to swagger
* Return empty array when nothing has been pruned.
* Use correct return type swagger doc-comment.

Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 16:53:49 +01:00
Praveen Kumar
e69eae6458 Fix help message case for podman version
This is a cosmetic change. The help message for `podman version` is in
title case whereas all other command help messages are not in title
case. This stands out as inconsistent when looking at the output of
`podman help`.

Signed-off-by: Praveen Kumar <praveen+git@kumar.in>
2021-10-29 11:49:49 -07:00
OpenShift Merge Robot
1305902ff4
Merge pull request #12127 from vrothberg/bz-2014149
volumes: be more tolerant and fix infinite loop
2021-10-29 13:30:29 +00:00
Boaz Shuster
7494876000 Use systemctl in local system test
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-10-29 09:37:15 +03:00
Jhon Honce
98506c961b Allow label and labels when creating volumes
JSON payload may have either key. Labels will override any values set
via Label.

Fixes #12102

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-28 16:02:22 -07:00
Valentin Rothberg
c5f0a5d788 volumes: be more tolerant and fix infinite loop
Make Podman more tolerant when parsing image volumes during container
creation and further fix an infinite loop when checking them.

Consider `VOLUME ['/etc/foo', '/etc/bar']` in a Containerfile.  While
it looks correct to the human eye, the single quotes are wrong and yield
the two volumes to be `[/etc/foo,` and `/etc/bar]` in Podman and Docker.

When running the container, it'll create a directory `bar]` in `/etc`
and a directory `[` in `/` with two subdirectories `etc/foo,`.  This
behavior is surprising to me but how Docker behaves.  We may improve on
that in the future.  Note that the correct way to syntax for volumes in
a Containerfile is `VOLUME /A /B /C` or `VOLUME ["/A", "/B", "/C"]`;
single quotes are not supported.

This change restores this behavior without breaking container creation
or ending up in an infinite loop.

BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2014149
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-28 16:37:33 +02:00
Giuseppe Scrivano
4e9e6f21ff
volumes: allow more options for devpts
allow to pass down more options that are supported by the kernel.

Discussion here: https://github.com/containers/toolbox/issues/568

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 15:30:06 +02:00
Paul Holzinger
0136a66a83
libpod: deduplicate ports in db
The OCICNI port format has one big problem: It does not support ranges.
So if a users forwards a range of 1k ports with podman run -p 1001-2000
we have to store each of the thousand ports individually as array element.
This bloats the db and makes the JSON encoding and decoding much slower.
In many places we already use a better port struct type which supports
ranges, e.g. `pkg/specgen` or the new network interface.

Because of this we have to do many runtime conversions between the two
port formats. If everything uses the new format we can skip the runtime
conversions.

This commit adds logic to replace all occurrences of the old format
with the new one. The database will automatically migrate the ports
to new format when the container config is read for the first time
after the update.

The `ParsePortMapping` function is `pkg/specgen/generate` has been
reworked to better work with the new format. The new logic is able
to deduplicate the given ports. This is necessary the ensure we
store them efficiently in the DB. The new code should also be more
performant than the old one.

To prove that the code is fast enough I added go benchmarks. Parsing
1 million ports took less than 0.5 seconds on my laptop.

Benchmark normalize PortMappings in specgen:
Please note that the 1 million ports are actually 20x 50k ranges
because we cannot have bigger ranges than 65535 ports.
```
$ go test -bench=. -benchmem  ./pkg/specgen/generate/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/pkg/specgen/generate
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
BenchmarkParsePortMappingNoPorts-12             480821532                2.230 ns/op           0 B/op          0 allocs/op
BenchmarkParsePortMapping1-12                      38972             30183 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMapping100-12                    18752             60688 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMapping1k-12                      3104            331719 ns/op          223840 B/op       3018 allocs/op
BenchmarkParsePortMapping10k-12                      376           3122930 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMapping1m-12                         3         390869926 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingReverse100-12             18940             63414 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMappingReverse1k-12               3015            362500 ns/op          223841 B/op       3018 allocs/op
BenchmarkParsePortMappingReverse10k-12               343           3318135 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMappingReverse1m-12                  3         403392469 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingRange1-12                 37635             28756 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange100-12               39604             28935 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1k-12                38384             29921 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange10k-12               29479             40381 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1m-12                  927           1279369 ns/op          143022 B/op        164 allocs/op
PASS
ok      github.com/containers/podman/v3/pkg/specgen/generate    25.492s
```

Benchmark convert old port format to new one:
```
go test -bench=. -benchmem  ./libpod/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/libpod
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Benchmark_ocicniPortsToNetTypesPortsNoPorts-12          663526126                1.663 ns/op           0 B/op          0 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1-12                 7858082               141.9 ns/op            72 B/op          2 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10-12                2065347               571.0 ns/op           536 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts100-12                138478              8641 ns/op            4216 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1k-12                   9414            120964 ns/op           41080 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10k-12                   781           1490526 ns/op          401528 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1m-12                      4         250579010 ns/op        40001656 B/op          4 allocs/op
PASS
ok      github.com/containers/podman/v3/libpod  11.727s
```

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-27 18:59:56 +02:00
OpenShift Merge Robot
d908da51fb
Merge pull request #12064 from vrothberg/fix-11933
container create: fix --tls-verify parsing
2021-10-27 15:24:23 +00:00
OpenShift Merge Robot
6caf5e3b7c
Merge pull request #12111 from giuseppe/fix-warning-move-pause-process
runtime: check for pause pid existence
2021-10-27 15:07:59 +00:00
OpenShift Merge Robot
b29dc1bde3
Merge pull request #12110 from cevich/fix_systemd_pid1
Fix systemd PID1 test
2021-10-27 14:34:58 +00:00
Hironori Shiina
e68fbf03aa Set flags to test 'logs -f' with journald driver
`logs -f` with `journald` is supported only when `journald` events
backend is used. To pass system tests using `logs -f` in an environment
where `events_logger` is not set to `journald` in `containers.conf`,
this fix sets `--events-backend` or `--log-driver` temporally.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-10-27 10:28:16 -04:00
Valentin Rothberg
ff31f2264d container create: fix --tls-verify parsing
Make sure that the value is only set if specified on the CLI.  c/image
already defaults to true but if set in the system context, we'd skip
settings in the registries.conf.

Fixes: #11933
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-27 14:36:25 +02:00
OpenShift Merge Robot
979b631228
Merge pull request #11956 from vrothberg/pause
remove need to download pause image
2021-10-27 10:22:56 +00:00
Giuseppe Scrivano
6b3b0a17c6
runtime: check for pause pid existence
check that the pause pid exists before trying to move it to a separate
scope.

Closes: https://github.com/containers/podman/issues/12065

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-27 11:24:50 +02:00
OpenShift Merge Robot
ed3aa2acaf
Merge pull request #12098 from Luap99/slirp-dad
Slirp4netns with ipv6 set net.ipv6.conf.default.accept_dad=0
2021-10-26 20:54:27 +00:00
Chris Evich
d53789068a
Fix systemd PID1 test
Previously this test used an ad-hoc timeout mechanism to synchronize
with output of the container ID.  However, depending on runtime
conditions this may not correctly correspond with complete startup
of the systemd process.  Consequently this test fails under some
conditions with an error like:

`System has not been booted with systemd as init system (PID 1). Can't
operate. Failed to connect to bus: Host is down`

Fix this by using the more appropriate `WaitContainerReady()`
against output from system startup, close to finalization.  In this way,
the test status command cannot run until systemd is fully operational.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-10-26 16:39:14 -04:00
OpenShift Merge Robot
1243954372
Merge pull request #12067 from hshiina/logs-journal-tail
Fix a few problems in 'podman logs --tail' with journald driver
2021-10-26 20:33:26 +00:00
OpenShift Merge Robot
420ac5d13d
Merge pull request #12088 from adrianreber/2021-10-25-fix-label-ipc-host
Allow 'container restore' with '--ipc host'
2021-10-26 16:38:54 +00:00
Paul Holzinger
008075ce54
Slirp4netns with ipv6 set net.ipv6.conf.default.accept_dad=0
Duplicate Address Detection slows the ipv6 setup down for 1-2 seconds.
Since slirp4netns is run it is own namespace and not directly routed
we can skip this to make the ipv6 address immediately available.
We change the default to make sure the slirp tap interface gets the
correct value assigned so DAD is disabled for it.
Also make sure to change this value back to the original after slirp4netns
is ready in case users rely on this sysctl.

Fixes #11062

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-26 18:27:30 +02:00
Hironori Shiina
c723e6b978 Fix a few problems in 'podman logs --tail' with journald driver
The following problems regarding `logs --tail` with the journald log
driver are fixed:
- One more line than a specified value is displayed.
- '--tail 0' displays all lines while the other log drivers displays
  nothing.
- Partial lines are not considered.
- If the journald events backend is used and a container has exited,
  nothing is displayed.

Integration tests that should have detected the bugs are also fixed. The
tests are executed with json-file log driver three times without this
fix.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-10-26 12:18:57 -04:00
Adrian Reber
bf8fd943ef
Allow 'container restore' with '--ipc host'
Trying to restore a container that was started with '--ipc host' fails
with:

Error: error creating container storage: ProcessLabel and Mountlabel must either not be specified or both specified

We already fixed this exact same error message for containers started
with '--privileged'. The previous fix was to check if the to be restored
container is a privileged container (c.config.Privileged). Unfortunately
this does not work for containers started with '--ipc host'.

This commit changes the check for a privileged container to check if
both the ProcessLabel and the MountLabel is actually set and only then
re-uses those labels.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-10-26 14:42:32 +00:00
Valentin Rothberg
75f478c08b pod create: remove need for pause image
So far, the infra containers of pods required pulling down an image
rendering pods not usable in disconnected environments.  Instead, build
an image locally which uses local pause binary.

Fixes: #10354
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-26 13:51:45 +02:00
Miloslav Trmač
adee084d65 Fix tests of podman image trust --raw and --json
Instead using the OS-wide system default policy, use
the one in this repo, and adjust the expected results
(as well as making the test stricter).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-10-25 18:02:41 +02:00
Miloslav Trmač
53ff49237b Tighten the expected output of the "podman image trust show" test
... to include all fields.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-10-25 18:02:41 +02:00
Miloslav Trmač
c872788e4e Use INTEGRATION_ROOT instead of current directory
Should not change behavior, just to set a consistent
precedent for code introduced in future commits.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-10-25 18:02:41 +02:00
Daniel J Walsh
acd8b49000
Add support to play kube for --log-opt
Fixes: https://github.com/containers/podman/issues/11727

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-25 11:40:42 -04:00
OpenShift Merge Robot
833d92d709
Merge pull request #12021 from rhatdan/kube
Generate Kube should not print default structs
2021-10-22 14:12:44 +00:00
Boaz Shuster
ece0c7e5d3 Support template unit files in podman generate systemd
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-10-22 04:19:18 +03:00
OpenShift Merge Robot
54f2c9a540
Merge pull request #12057 from flouthoc/allow-tagging-manifest-list
tag: Support tagging manifest list instead of resolving to images
2021-10-21 13:45:28 +00:00
OpenShift Merge Robot
6338e74cc9
Merge pull request #12028 from edsantiago/test_system_connection
Add test for system connection
2021-10-21 13:44:28 +00:00
Aditya Rajan
f4f96962c0
tag: Support tagging manifest list instead of resolving to images
Following commit makes sure when buildah tag is invoked on a manifest
list, it tags the same manifest list instead of resolving to an image and
tagging it.

Port of: https://github.com/containers/buildah/pull/3483

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-10-21 17:44:31 +05:30
Ed Santiago
70a5d8cd1d System tests: confirm that -a and -l clash
...and fix one instance where there was no check

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-10-20 11:09:32 -06:00
OpenShift Merge Robot
eba281c3e0
Merge pull request #11851 from cdoern/podRm
Pod Rm Infra Handling Improvements
2021-10-20 13:20:12 +00:00
OpenShift Merge Robot
2c6c801067
Merge pull request #12041 from edsantiago/container_env
system tests: CONTAINER_* and --help: cleanup
2021-10-20 08:40:10 +00:00
Ed Santiago
960a55c09d system tests: CONTAINER_* and --help: cleanup
A small part of this test was written in a confusing and fragile
way: it was very hard to understand, and in fact only worked
through pure luck (using 'echo $output', which emitted everything
in one long line, vs the standard quoted 'echo "$output"' which
would've kept the formatting and caused the test to pass,
incorrectly, no matter whether --remote was in the output
or not). Plus, the '$?' check in the next line would never
trigger on failure anyway, so the failure message would've
been unhelpful if the test were ever to fail.

Anyhow. Make it readable and make it work.

(Followup to #11990)

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-10-19 14:22:50 -06:00
Daniel J Walsh
20b5a8599b
podman run --memory=0 ... should not set memory limit
On Docker this is ignored, and it should be on Podman as
well. This is documented in the man page.

Fixes: https://github.com/containers/podman/issues/12002

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-10-19 14:06:33 -04:00