Commit graph

15354 commits

Author SHA1 Message Date
Giuseppe Scrivano
0234b153cc
test: run --cgroups=split in new cgroup
the --cgroups=split test changes the current cgroup as it creates a
sub-cgroup.  This can cause a race condition in tests that are reading
the current cgroup.

Closes: https://github.com/containers/podman/issues/11191

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-04 13:16:17 +01:00
OpenShift Merge Robot
8fdde67008
Merge pull request #12060 from mtrmac/podman-trust-show-f35
Fix `Podman image trust` tests
2021-11-03 21:49:21 +01:00
OpenShift Merge Robot
4105b025d6
Merge pull request #12166 from Luap99/mac
MAC address json unmarshal should allow strings
2021-11-03 17:15:46 +01:00
Paul Holzinger
001d48929d
MAC address json unmarshal should allow strings
Create a new mac address type which supports json marshal/unmarshal from
and to string. This change is backwards compatible with the previous
versions as the unmarshal method still accepts the old byte array or
base64 encoded string.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-03 15:30:16 +01:00
OpenShift Merge Robot
a245da1e52
Merge pull request #12157 from afbjorklund/stop-message
Make stop message more similar to start
2021-11-03 01:35:54 +01:00
OpenShift Merge Robot
33643f4b09
Merge pull request #12159 from jwhonce/issues/12115
Implement top streaming for containers and pods
2021-11-02 22:28:54 +01:00
Anders F Björklund
6e6388eac4 Make stop message more similar to start
[NO TESTS NEEDED]

Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
2021-11-02 22:26:04 +01:00
OpenShift Merge Robot
82dba97547
Merge pull request #12158 from edsantiago/more_bats
System tests: enhance volume test, add debug prints
2021-11-02 18:51:24 +01:00
OpenShift Merge Robot
e63e90999c
Merge pull request #12156 from matejvasek/docker-api-zero-value-fixes
Fix libpod API conformance to swagger
2021-11-02 18:19:24 +01:00
OpenShift Merge Robot
0d5aef47d3
Merge pull request #12051 from machacekondra/fix_http409_errorhandling
Handle HTTP 409 error messages properly
2021-11-02 16:11:28 +01:00
Jhon Honce
449cc7a5c2 Implement top streaming for containers and pods
* Implement API query parameter stream and delay for containers and
  pods top endpoints
* Update swagger with breaking changes
* Add python API tests for endpoints

Fixes #12115

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-02 08:11:14 -07:00
OpenShift Merge Robot
0686f0bb2f
Merge pull request #12118 from hshiina/log-f-journald
Set flags to test 'logs -f' with journald driver
2021-11-02 13:18:26 +01:00
OpenShift Merge Robot
24e5cbd9a5
Merge pull request #12160 from matejvasek/fix-swagger-api
Fix swagger definitions
2021-11-02 12:46:25 +01:00
Ondra Machacek
f2115471dd Handle HTTP 409 error messages properly for Pod actions
This PR fixes the case when the API return HTTP 409 response. Where the
API return the body format different then for other HTTP error codes.

Signed-off-by: Ondra Machacek <omachace@redhat.com>
2021-11-02 12:28:52 +01:00
Matej Vasek
d0dfc5e223 Add tests
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 23:01:17 +01:00
Matej Vasek
48d0d2b4a3 Fix swagger definitions
[NO TESTS NEEDED]

Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 22:37:35 +01:00
OpenShift Merge Robot
3147ff829b
Merge pull request #12139 from cevich/add_rootless_sshkey
Cirrus: Authorize rootless user self-ssh
2021-11-01 22:23:32 +01:00
Matej Vasek
48e1cca9f8 More conforming libpod API and swagger types
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 20:25:58 +01:00
Matej Vasek
62ee24bb7c More conforming libpod API and swagger types
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 20:19:06 +01:00
Matej Vasek
a845613d03 Better emptiness test for custom JSON serializer
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 20:12:35 +01:00
Ed Santiago
7b2531c135 System tests: enhance volume test, add debug prints
Volume test: add a sequence of stat()s to confirm that volumes
are mounted as a different device than root.

Network test: add debugging code for #11825 (dnsmasq inotify
failure in bodhi only).

Signed-off-by: Ed Santiago <santiago@redhat.com>
2021-11-01 13:03:05 -06:00
Boaz Shuster
15eb016017 add unit test to containers_test
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-11-01 19:53:20 +02:00
Matej Vasek
120ad2d3ca Use correct swagger type in doc-comment
Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 17:09:35 +01:00
Chris Evich
9c34bd52fd
Cirrus: Authorize rootless user self-ssh
Future testing needs dictate rootless (in addition to root) users are
able to ssh to localhost.  Add ssh-key generation commands for the
rootless user, and authorize their public key.

Minor: Also remove update of `/etc/sub{uid,gid}` files, since this is
now done automatically by `{user,group}add` commands.

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-11-01 12:05:05 -04:00
Matej Vasek
218d91d76d Fix libpod API conformance to swagger
* Return empty array when nothing has been pruned.
* Use correct return type swagger doc-comment.

Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-01 16:53:49 +01:00
OpenShift Merge Robot
85bad0cc7c
Merge pull request #12119 from ashley-cui/updates
[CI:DOCS] Add information on how podman machine is updated
2021-10-30 14:44:43 +00:00
OpenShift Merge Robot
7cd317e9c3
Merge pull request #12141 from kprav33n/help-doc
Fix help message case for `podman version`
2021-10-30 08:56:45 +00:00
OpenShift Merge Robot
09efcd48a8
Merge pull request #12090 from afbjorklund/image-stream
Record the image stream along with the path
2021-10-29 20:22:29 +00:00
OpenShift Merge Robot
e3c45ab9cc
Merge pull request #12128 from containers/dependabot/go_modules/k8s.io/api-0.22.3
Bump k8s.io/api from 0.22.2 to 0.22.3
2021-10-29 20:21:29 +00:00
Praveen Kumar
e69eae6458 Fix help message case for podman version
This is a cosmetic change. The help message for `podman version` is in
title case whereas all other command help messages are not in title
case. This stands out as inconsistent when looking at the output of
`podman help`.

Signed-off-by: Praveen Kumar <praveen+git@kumar.in>
2021-10-29 11:49:49 -07:00
OpenShift Merge Robot
1305902ff4
Merge pull request #12127 from vrothberg/bz-2014149
volumes: be more tolerant and fix infinite loop
2021-10-29 13:30:29 +00:00
OpenShift Merge Robot
584049325b
Merge pull request #12138 from gsanchietti/doc_pause_typo
[CI:DOCS] Fix pause usage example
2021-10-29 13:28:28 +00:00
Giacomo Sanchietti
9fc98f265a
Fix pause usage example
The page contains a wrong 'stop' command example.

Signed-off-by: Giacomo Sanchietti <giacomo.sanchietti@nethesis.it>
2021-10-29 14:24:10 +02:00
OpenShift Merge Robot
197152b02b
Merge pull request #12133 from jwhonce/issues/12102
Allow label and labels when creating volumes
2021-10-29 09:36:28 +00:00
Boaz Shuster
7494876000 Use systemctl in local system test
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-10-29 09:37:15 +03:00
Jhon Honce
98506c961b Allow label and labels when creating volumes
JSON payload may have either key. Labels will override any values set
via Label.

Fixes #12102

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-10-28 16:02:22 -07:00
OpenShift Merge Robot
f7ca045737
Merge pull request #12124 from giuseppe/allow-devpts-options
volumes: allow more options for devpts
2021-10-28 18:52:24 +00:00
OpenShift Merge Robot
d2147bada6
Merge pull request #12117 from adrianreber/2021-10-27-set-checkpointed-false-after-restore
Set Checkpointed state to false after restore
2021-10-28 18:06:25 +00:00
Valentin Rothberg
c5f0a5d788 volumes: be more tolerant and fix infinite loop
Make Podman more tolerant when parsing image volumes during container
creation and further fix an infinite loop when checking them.

Consider `VOLUME ['/etc/foo', '/etc/bar']` in a Containerfile.  While
it looks correct to the human eye, the single quotes are wrong and yield
the two volumes to be `[/etc/foo,` and `/etc/bar]` in Podman and Docker.

When running the container, it'll create a directory `bar]` in `/etc`
and a directory `[` in `/` with two subdirectories `etc/foo,`.  This
behavior is surprising to me but how Docker behaves.  We may improve on
that in the future.  Note that the correct way to syntax for volumes in
a Containerfile is `VOLUME /A /B /C` or `VOLUME ["/A", "/B", "/C"]`;
single quotes are not supported.

This change restores this behavior without breaking container creation
or ending up in an infinite loop.

BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2014149
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-10-28 16:37:33 +02:00
Ashley Cui
3c79202fb9 Add information on how podman machine is updated
Update documentation on how the default podman machine distribution,
FCOS, is updated.

Signed-off-by: Ashley Cui <acui@redhat.com>
2021-10-28 09:43:00 -04:00
Giuseppe Scrivano
4e9e6f21ff
volumes: allow more options for devpts
allow to pass down more options that are supported by the kernel.

Discussion here: https://github.com/containers/toolbox/issues/568

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 15:30:06 +02:00
Giuseppe Scrivano
e6286fbace
volumes: do not pass mount opt as formatter string
otherwise passing a formatter string as an option causes a weird
error message:

$ podman run --mount type=devpts,destination=/dev/pts,%sfoo ...
Error: %!s(MISSING)foo: invalid mount option

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 15:30:05 +02:00
dependabot[bot]
feebf1bd81
Bump k8s.io/api from 0.22.2 to 0.22.3
Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.22.2 to 0.22.3.
- [Release notes](https://github.com/kubernetes/api/releases)
- [Commits](https://github.com/kubernetes/api/compare/v0.22.2...v0.22.3)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-28 12:27:17 +00:00
OpenShift Merge Robot
3bc449371c
Merge pull request #12126 from giuseppe/fix-race-warning-message
runtime: change PID existence check
2021-10-28 12:16:24 +00:00
OpenShift Merge Robot
f16b133664
Merge pull request #12120 from giuseppe/rename-cgroup-subtree
oci: rename sub-cgroup to runtime instead of supervisor
2021-10-28 12:15:26 +00:00
Giuseppe Scrivano
960831f9c8
runtime: change PID existence check
commit 6b3b0a17c6 introduced a check for
the PID file before attempting to move the PID to a new scope.

This is still vulnerable to TOCTOU race condition though, since the
PID file or the PID can be removed/killed after the check was
successful but before it was used.

Closes: https://github.com/containers/podman/issues/12065

[NO NEW TESTS NEEDED] it fixes a CI flake

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 11:18:48 +02:00
Giuseppe Scrivano
9e5cd32056
oci: rename sub-cgroup to runtime instead of supervisor
we are having a hard time figuring out a failure in the CI:

https://github.com/containers/podman/issues/11191

Rename the sub-cgroup created here, so we can be certain the error is
caused by this part.

[NO NEW TESTS NEEDED] we need this for the CI.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-10-28 09:18:08 +02:00
Paul Holzinger
0136a66a83
libpod: deduplicate ports in db
The OCICNI port format has one big problem: It does not support ranges.
So if a users forwards a range of 1k ports with podman run -p 1001-2000
we have to store each of the thousand ports individually as array element.
This bloats the db and makes the JSON encoding and decoding much slower.
In many places we already use a better port struct type which supports
ranges, e.g. `pkg/specgen` or the new network interface.

Because of this we have to do many runtime conversions between the two
port formats. If everything uses the new format we can skip the runtime
conversions.

This commit adds logic to replace all occurrences of the old format
with the new one. The database will automatically migrate the ports
to new format when the container config is read for the first time
after the update.

The `ParsePortMapping` function is `pkg/specgen/generate` has been
reworked to better work with the new format. The new logic is able
to deduplicate the given ports. This is necessary the ensure we
store them efficiently in the DB. The new code should also be more
performant than the old one.

To prove that the code is fast enough I added go benchmarks. Parsing
1 million ports took less than 0.5 seconds on my laptop.

Benchmark normalize PortMappings in specgen:
Please note that the 1 million ports are actually 20x 50k ranges
because we cannot have bigger ranges than 65535 ports.
```
$ go test -bench=. -benchmem  ./pkg/specgen/generate/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/pkg/specgen/generate
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
BenchmarkParsePortMappingNoPorts-12             480821532                2.230 ns/op           0 B/op          0 allocs/op
BenchmarkParsePortMapping1-12                      38972             30183 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMapping100-12                    18752             60688 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMapping1k-12                      3104            331719 ns/op          223840 B/op       3018 allocs/op
BenchmarkParsePortMapping10k-12                      376           3122930 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMapping1m-12                         3         390869926 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingReverse100-12             18940             63414 ns/op          141088 B/op        315 allocs/op
BenchmarkParsePortMappingReverse1k-12               3015            362500 ns/op          223841 B/op       3018 allocs/op
BenchmarkParsePortMappingReverse10k-12               343           3318135 ns/op         1223650 B/op      30027 allocs/op
BenchmarkParsePortMappingReverse1m-12                  3         403392469 ns/op        124593840 B/op   4000624 allocs/op
BenchmarkParsePortMappingRange1-12                 37635             28756 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange100-12               39604             28935 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1k-12                38384             29921 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange10k-12               29479             40381 ns/op          131584 B/op          9 allocs/op
BenchmarkParsePortMappingRange1m-12                  927           1279369 ns/op          143022 B/op        164 allocs/op
PASS
ok      github.com/containers/podman/v3/pkg/specgen/generate    25.492s
```

Benchmark convert old port format to new one:
```
go test -bench=. -benchmem  ./libpod/
goos: linux
goarch: amd64
pkg: github.com/containers/podman/v3/libpod
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Benchmark_ocicniPortsToNetTypesPortsNoPorts-12          663526126                1.663 ns/op           0 B/op          0 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1-12                 7858082               141.9 ns/op            72 B/op          2 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10-12                2065347               571.0 ns/op           536 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts100-12                138478              8641 ns/op            4216 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1k-12                   9414            120964 ns/op           41080 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts10k-12                   781           1490526 ns/op          401528 B/op          4 allocs/op
Benchmark_ocicniPortsToNetTypesPorts1m-12                      4         250579010 ns/op        40001656 B/op          4 allocs/op
PASS
ok      github.com/containers/podman/v3/libpod  11.727s
```

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-10-27 18:59:56 +02:00
OpenShift Merge Robot
a05a445f48
Merge pull request #12066 from matejvasek/set-docker-host
Set DOCKER_HOST in the VM
2021-10-27 15:25:21 +00:00
OpenShift Merge Robot
d908da51fb
Merge pull request #12064 from vrothberg/fix-11933
container create: fix --tls-verify parsing
2021-10-27 15:24:23 +00:00