Commit graph

4133 commits

Author SHA1 Message Date
Valentin Rothberg
33ec8c6698 fix remote checkpoint/restore
Nothing was working before, and it's too much to summarize.  To make
sure we're not regressing in the future again, enable the remote e2e
tests.

Fixes: #12007
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-16 14:41:18 +01:00
OpenShift Merge Robot
be681ab518
Merge pull request #12294 from flouthoc/secret-mount-target
secret: honor custom `target=` for secrets with `type=mount` for ctr.
2021-11-16 01:45:27 +01:00
Daniel J Walsh
44d1618dd7
Add --unsetenv & --unsetenv-all to remove def environment variables
Podman adds a few environment variables by default, and
currently there is no way to get rid of them from your container.
This option will allow  you to specify which defaults you don't
want.

--unsetenv-all will remove all default environment variables.

Default environment variables can come from podman builtin,
containers.conf or from the container image.

Fixes: https://github.com/containers/podman/issues/11836

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-15 15:10:12 -05:00
Aditya Rajan
014cc4b9d9
secret: honor custom target for secrets with run
Honor custom `target` if specified while running or creating containers
with secret `type=mount`.

Example:
`podman run -it --secret token,type=mount,target=TOKEN ubi8/ubi:latest
bash`

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-15 23:19:27 +05:30
OpenShift Merge Robot
47eaf23039
Merge pull request #12300 from WanzenBug/fix-bindings-context
bindings: reuse context for API requests
2021-11-15 18:31:37 +01:00
OpenShift Merge Robot
e9d8ca22b8
Merge pull request #11076 from boaz0/closes_10275
Support template unit files in podman generate systemd
2021-11-15 16:30:38 +01:00
Moritz "WanzenBug" Wanzenböck
5df883e87d bindings: reuse context for API requests
One of the main uses of context.Context is to provide cancellation for
go-routines, including API requests. While all user-facing bindings
already used a context parameter, it was only used to pass the client
information around.

This commit changes the internal DoRequest wrapper to take an additional
context argument, and pass that to the http request. Previously, the context
was derived from context.Background(), which made it impossible to cancel
once started.

All the convenience wrappers already supported the context parameter, so the
only user facing change is that cancelling those context now works as one
would expect.

Signed-off-by: Moritz "WanzenBug" Wanzenböck <moritz@wanzenbug.xyz>
2021-11-15 15:42:39 +01:00
Paul Holzinger
295d87bb0b
podman machine improve port forwarding
This commits adds port forwarding logic directly into podman. The
podman-machine cni plugin is no longer needed.

The following new features are supported:
 - works with cni, netavark and slirp4netns
 - ports can use the hostIP to bind instead of hard coding 0.0.0.0
 - gvproxy no longer listens on 0.0.0.0:7777 (requires a new gvproxy
   version)
 - support the udp protocol

With this we no longer need podman-machine-cni and should remove it from
the packaging. There is also a change to make sure we are backwards
compatible with old config which include this plugin.

Fixes #11528
Fixes #11728

[NO NEW TESTS NEEDED] We have no podman machine test at the moment.
Please test this manually on your system.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-15 15:20:47 +01:00
Aditya Rajan
c050f05ccf
filter: add basic pattern matching for label keys
Following PR adds basic pattern matching to filter by labels for `keys`.
Adds support for use-cases like `--filter label=some.prefix.com/key/*`
where end-users want to match a pattern for keys as compared to exact
value.

Signed-off-by: Aditya Rajan <arajan@redhat.com>
2021-11-15 17:49:54 +05:30
Valentin Rothberg
5934e4c9b5 infra container: replace pause with catatonit
Podman has been using catatonit for a number of years already.
Thanks to @giuseppe, catatonit is now able to run as a pause
process which allows us to replace the pause binary entirely.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-15 12:53:25 +01:00
Adrian Reber
80e56fa12b
Added optional container restore statistics
This adds the parameter '--print-stats' to 'podman container restore'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to restore a checkpoint and print out these
information. CRIU already creates process restore statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the restored container, Podman will now print
out JSON:

 # podman container restore --latest --print-stats
 {
     "podman_restore_duration": 305871,
     "container_statistics": [
         {
             "Id": "47b02e1d474b5d5fe917825e91ac653efa757c91e5a81a368d771a78f6b5ed20",
             "runtime_restore_duration": 140614,
             "criu_statistics": {
                 "forking_time": 5,
                 "restore_time": 67672,
                 "pages_restored": 14
             }
         }
     ]
 }

The output contains 'podman_restore_duration' which contains the
number of microseconds Podman required to restore the checkpoint. The
output also includes 'runtime_restore_duration' which is the time
the runtime needed to restore that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:25 +00:00
Adrian Reber
6202e8102b
Added optional container checkpointing statistics
This adds the parameter '--print-stats' to 'podman container checkpoint'.
With '--print-stats' Podman will measure how long Podman itself, the OCI
runtime and CRIU requires to create a checkpoint and print out these
information. CRIU already creates checkpointing statistics which are
just read in addition to the added measurements. In contrast to just
printing out the ID of the checkpointed container, Podman will now print
out JSON:

 # podman container checkpoint --latest --print-stats
 {
     "podman_checkpoint_duration": 360749,
     "container_statistics": [
         {
             "Id": "25244244bf2efbef30fb6857ddea8cb2e5489f07eb6659e20dda117f0c466808",
             "runtime_checkpoint_duration": 177222,
             "criu_statistics": {
                 "freezing_time": 100657,
                 "frozen_time": 60700,
                 "memdump_time": 8162,
                 "memwrite_time": 4224,
                 "pages_scanned": 20561,
                 "pages_written": 2129
             }
         }
     ]
 }

The output contains 'podman_checkpoint_duration' which contains the
number of microseconds Podman required to create the checkpoint. The
output also includes 'runtime_checkpoint_duration' which is the time
the runtime needed to checkpoint that specific container. Each container
also includes 'criu_statistics' which displays the timing information
collected by CRIU.

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-15 11:50:24 +00:00
OpenShift Merge Robot
cca6df428c
Merge pull request #12272 from hshiina/memory-swappiness
Enable 'podman run --memory-swappiness=0'
2021-11-13 13:12:48 +01:00
OpenShift Merge Robot
8f3fb743ee
Merge pull request #12270 from rhatdan/auth
--authfile command line argument for image sign command.
2021-11-13 13:10:48 +01:00
OpenShift Merge Robot
78bc2390f4
Merge pull request #12271 from Luap99/play-kube-net
Fix network mode in play kube
2021-11-12 19:59:44 +01:00
Hironori Shiina
9226ccb59f Enable 'podman run --memory-swappiness=0'
'--memory-swappiness=0' used to work. This patch fixes the regression
issue, which was caused by the change of infra container creation
process.

Signed-off-by: Hironori Shiina <shiina.hironori@jp.fujitsu.com>
2021-11-12 13:20:20 -05:00
Paul Holzinger
164c42b485
Fix network mode in play kube
We need to use the config network mode when no network mode was set. To
do so we have to keep the nsmode empty, MakeContainer() will use the
correct network mode from the config when needed.

Fixes #12248

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-12 18:32:01 +01:00
OpenShift Merge Robot
07f5a94711
Merge pull request #12279 from mscherer/fix_11842
Always create working directory when using compat API
2021-11-12 16:55:24 +01:00
Michael Scherer
b1c0023424 Always create working directory when using compat API
Docker/Moby always create the working directory, and some tools
rely on that behavior (example, woodpecker/drone).

Fixes #11842

Signed-off-by: Michael Scherer <misc@redhat.com>
2021-11-12 14:18:26 +01:00
Valentin Rothberg
f517510bc8 play kube: don't force-pull infra image
Do not force-pull the infra image in `play kube` but let the backend
take care of that when creating the pod(s) which may build a local
`podman-pause` image instead of using the default infra image.

Fixes: #12254
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-11-12 13:44:13 +01:00
José Guilherme Vanz
6762d5e238
--authfile command line argument for image sign command.
Adds the --authfile command line argument to allow users to use
alternative authfile paths when signing images.

Replaces: https://github.com/containers/podman/pull/10975
Fixes: https://github.com/containers/podman/issues/10866

Signed-off-by: José Guilherme Vanz <jvanz@jvanz.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-11-11 15:11:19 -05:00
OpenShift Merge Robot
69d2e78b3b
Merge pull request #12214 from giuseppe/fix-rootless-error-message
rootless: adjust error message
2021-11-11 20:42:43 +01:00
Paul Holzinger
fe90a45e0d
Add flag to overwrite network backend from config
To make testing easier we can overwrite the network backend with the
global `--network-backend` option.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-11 17:30:27 +01:00
Boaz Shuster
73e1cdfe9e export adding id-specifier code to setContainerNameForTemplate
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-11-11 11:45:36 +02:00
Giuseppe Scrivano
9877280f5a
rootless: adjust error message
since we now support reading additional IDs with libsubid, clarify
that the /etc/subuid and /etc/subgid files are honored only when
shadow-utils is configured to use them.

[NO TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2021-11-10 09:26:22 +01:00
OpenShift Merge Robot
43bd57c7fb
Merge pull request #12195 from boaz0/closes_11998
podman-generate-kube - remove empty structs from YAML
2021-11-09 19:46:31 +01:00
Boaz Shuster
6ee3b33d38 change from run to create in 250-systemd.bats
Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-11-09 11:54:18 +02:00
OpenShift Merge Robot
c9ba1fb7d4
Merge pull request #11958 from cdoern/scp
Podman Image SCP rootful to rootless transfer
2021-11-08 16:19:57 +01:00
OpenShift Merge Robot
865653b661
Merge pull request #12184 from adrianreber/2021-11-05-stats-dump
Add 'stats-dump' file to exported checkpoint
2021-11-08 09:29:56 +01:00
Boaz Shuster
f3fab1e17c podman-generate-kube - remove empty structs from YAML
[NO NEW TESTS NEEDED]

Signed-off-by: Boaz Shuster <boaz.shuster.github@gmail.com>
2021-11-07 16:33:38 +02:00
OpenShift Merge Robot
abbd6c167e
Merge pull request #11890 from Luap99/ports
libpod: deduplicate ports in db
2021-11-06 10:39:16 +01:00
OpenShift Merge Robot
6805befec2
Merge pull request #12187 from Luap99/fix-swagger
[CI:DOCS] Fix swagger definition for the new mac address type
2021-11-05 22:14:01 +01:00
OpenShift Merge Robot
6c0690dbab
Merge pull request #12186 from jwhonce/issues/12181
Log Apache access_log-like entries at Info level
2021-11-05 20:53:54 +01:00
OpenShift Merge Robot
e04cb6c30c
Merge pull request #12178 from matejvasek/swagger-make-filed-optional
Update swagger doc make field optional
2021-11-05 19:43:55 +01:00
Paul Holzinger
02f67181a2
Fix swagger definition for the new mac address type
The new mac address type broke the api docs. While we could
successfully generate the swagger file it could not be viewed in a
browser.

The problem is that the swagger generation create two type definitions
with the name `HardwareAddr` and this pointed back to itself. Thus the
render process was stucked in an endless loop. To fix this manually
rename the new type to MacAddress and overwrite the types to string
because the json unmarshaller accepts the mac as string.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 19:25:40 +01:00
Jhon Honce
5c0351cc6f Log Apache access_log-like entries at Info level [NO NEW TESTS NEEDED]
Only log API access entries when --log-level set to Info or below.

Fixes #12181

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-05 10:34:02 -07:00
Adrian Reber
6b8fc3bd1d
Add 'stats-dump' file to exported checkpoint
There was the question about how long it takes to create a checkpoint.
CRIU already provides some statistics about how long it takes to create
a checkpoint and similar.

With this change the file 'stats-dump' is included in the checkpoint
archive and the tool checkpointctl can be used to display these
statistics:

./checkpointctl show -t /tmp/cp.tar --print-stats

Displaying container checkpoint data from /tmp/dump.tar

[...]
CRIU dump statistics
+---------------+-------------+--------------+---------------+---------------+---------------+
| FREEZING TIME | FROZEN TIME | MEMDUMP TIME | MEMWRITE TIME | PAGES SCANNED | PAGES WRITTEN |
+---------------+-------------+--------------+---------------+---------------+---------------+
| 105405 us     | 1376964 us  | 504399 us    | 446571 us     |        492153 |         88689 |
+---------------+-------------+--------------+---------------+---------------+---------------+

Signed-off-by: Adrian Reber <areber@redhat.com>
2021-11-05 16:15:00 +00:00
cdoern
ffa5ed0e0e Podman Image SCP rootful to rootless transfer
Added functionality for users to transfer images from root storage to rootless storage without using sshd. This is
done through rootful podman by running `sudo podman image scp root@localhost::image user@localhost:: the user is needed
in order to find and use their uid/gid to exec a new process.

added necessary tests, and functions for this implementation. Created new image function Transfer so that
the underlying code is majorly removed from CLI

Signed-off-by: cdoern <cdoern@redhat.com>
2021-11-05 12:04:20 -04:00
Paul Holzinger
7f433df7e7
rename rootless cni ns to rootless netns
Since we want to use the rootless cni ns also for netavark we should
pick a more generic name. The name is now "rootless network namespace"
or short "rootless netns".

The rename might cause some issues after the update but when the
all containers are restarted or the host is rebooted it should work
correctly.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-05 15:44:37 +01:00
Matej Vasek
ff92d73712 Update swagger doc make filed optional
[NO TESTS NEEDED]

Signed-off-by: Matej Vasek <mvasek@redhat.com>
2021-11-04 18:43:34 +01:00
Paul Holzinger
ba8eba83ef
Fix bindings container log test
The returned error was not checked, thus the test could hang forever
since it blocks on the log channel.

Also handle unexpectedEOF like EOF.

Fixes #12176

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-04 17:04:27 +01:00
OpenShift Merge Robot
4105b025d6
Merge pull request #12166 from Luap99/mac
MAC address json unmarshal should allow strings
2021-11-03 17:15:46 +01:00
Paul Holzinger
001d48929d
MAC address json unmarshal should allow strings
Create a new mac address type which supports json marshal/unmarshal from
and to string. This change is backwards compatible with the previous
versions as the unmarshal method still accepts the old byte array or
base64 encoded string.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-11-03 15:30:16 +01:00
OpenShift Merge Robot
a245da1e52
Merge pull request #12157 from afbjorklund/stop-message
Make stop message more similar to start
2021-11-03 01:35:54 +01:00
OpenShift Merge Robot
33643f4b09
Merge pull request #12159 from jwhonce/issues/12115
Implement top streaming for containers and pods
2021-11-02 22:28:54 +01:00
Anders F Björklund
6e6388eac4 Make stop message more similar to start
[NO TESTS NEEDED]

Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
2021-11-02 22:26:04 +01:00
OpenShift Merge Robot
e63e90999c
Merge pull request #12156 from matejvasek/docker-api-zero-value-fixes
Fix libpod API conformance to swagger
2021-11-02 18:19:24 +01:00
OpenShift Merge Robot
0d5aef47d3
Merge pull request #12051 from machacekondra/fix_http409_errorhandling
Handle HTTP 409 error messages properly
2021-11-02 16:11:28 +01:00
Jhon Honce
449cc7a5c2 Implement top streaming for containers and pods
* Implement API query parameter stream and delay for containers and
  pods top endpoints
* Update swagger with breaking changes
* Add python API tests for endpoints

Fixes #12115

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-11-02 08:11:14 -07:00
Ondra Machacek
f2115471dd Handle HTTP 409 error messages properly for Pod actions
This PR fixes the case when the API return HTTP 409 response. Where the
API return the body format different then for other HTTP error codes.

Signed-off-by: Ondra Machacek <omachace@redhat.com>
2021-11-02 12:28:52 +01:00