Commit graph

7499 commits

Author SHA1 Message Date
Ashley Cui
3f22e52964 Allow setting of machine stream and image path from containers.conf
Default is "testing"

Signed-off-by: Ashley Cui <acui@redhat.com>
2021-08-24 14:18:55 -04:00
OpenShift Merge Robot
23f9565547
Merge pull request #11263 from nalind/journal-read
libpod/Container.readFromJournal(): don't skip the first entry
2021-08-24 14:16:22 -04:00
OpenShift Merge Robot
24ee67bb09
Merge pull request #11315 from vrothberg/fix-11304
generate systemd: use --cidfile again
2021-08-24 13:10:45 -04:00
Valentin Rothberg
274d6fa19c generate systemd: use --cidfile again
Commit 9ac5267 changed the type of the generated systemd units from
`forking` to `notify`.  It further stopped using `--cidfile` and instead
intended systemd to take care of stopping the container, which turned
out to be a bad idea.

Systemd will send the stop/kill signals to conmon which in turn may exit
non-zero, depending on the signal, and ultimately breaking container
cleanup.

Hence, we need to use --cidfile again and let podman stop and remove the
container to make sure that everything's in order.

Fixes: #11304
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-24 15:46:00 +02:00
Paul Holzinger
4b2dc48d0b
podman inspect show exposed ports
Podman inspect has to show exposed ports to match docker. This requires
storing the exposed ports in the container config.
A exposed port is shown as `"80/tcp": null` while a forwarded port is
shown as `"80/tcp": [{"HostIp": "", "HostPort": "8080" }]`.

Also make sure to add the exposed ports to the new image when the
container is commited.

Fixes #10777

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-08-24 15:44:26 +02:00
Valentin Rothberg
74ab2aaf9f Revert "generate systemd: custom stop signal"
This reverts commit 70801b3d71.

It turns out that letting systemd handle stopping the container is not
working as I thought it will.  Conmon is receiving the stop/kill signals
and may exit non-zero, which in turn lets the systemd service transition
into the `failed` state.

We need to get back to letting Podman stop the containers and do a
partial revert of commit 9ac5267 which removed using --cidfile.

Happening in a following commit.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-24 13:33:06 +02:00
Paul Holzinger
c0b1edd6a4
Network interface
Implement a new network interface to abstract CNI from libpod. The
interface is implemented for the CNI backend but in the future we can
add more backends.

The code is structured in three new packages:
- `libpod/network/types`: contains the interface definition
  and the necessary types for it.
- `libpod/network/cni` contains the interface implementation for the CNI
  backend.
- `libpod/network/util` a set of utility functions related to
  networking.

The CNI package uses ginkgo style unit tests. To test Setup/Teardown the
test must be run as root. Each test will run in their own namespace to
make the test independent from the host environment.

New features with the CNI backend:
- The default network will be created in memory if it does not exists on
  disk.
- It can set more than one static IP per container network.
- Networks are loaded once from disk and only if this interface is
  used, e.g. for commands such as `podman info` networks are not loaded.
  This reduces unnecessary disk IO.

This commit only adds the interface it is not wired into libpod. This
requires a lot of breaking changes which will be done in a followup
commit.

Once this is integrated into libpod the current network code under
`libpod/network` should be removed. Also the dependency on OCICNI
should be dropped.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-08-24 12:28:03 +02:00
Valentin Rothberg
70801b3d71 generate systemd: custom stop signal
Commit 9ac5267598 changed the type of the generated systemd units from
forking to notify.  Parts of these changes was also removing the need to
pass any information via the file system (e.g., PIDFILE, container ID).
That in turn implies that systemd takes care of stopping the container.

By default, systemd first sends a SIGTERM and after a certain timeout,
it'll send a SIGKILL.  That's pretty much what Podman is doing, unless
the container was created with a custom stop signal which is the case
when the --stop-signal flag was used or systemd is mounted.

Account for that by using systemd's KillSignal option which allows for
changing SIGTERM to another signal.  Also make sure that we're using the
correct timeout for units generated with --new.

Fixes: #11304
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-24 10:50:16 +02:00
Nalin Dahyabhai
21f396de6f logs: adjust handling around partial log messages
In libpod/logs.LogLine.Write(), don't write a newline to stdout/stderr
when the log message is only part of a line.

In libpod.ConmonOCIRuntime.HTTPAttach(), don't send a newline over the
HTTP connection when the log message is only part of a line.

In pkg/api/handlers/compat.LogsFromContainer(), don't send a newline
over the HTTP connection when the log message is only part of a line,
and don't make doing so conditional on whether or not the client used
the docker or podman endpoint.

In pkg/domain/infra/tunnel.ContainerEngine.ContainerLogs(), don't add
our own newline to log messages, since they already come through from
the server when they need to.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-08-23 18:00:18 -04:00
OpenShift Merge Robot
90cf78b199
Merge pull request #11290 from flouthoc/volume-export
volumes: Add support for `volume export` which allows exporting content to  external path.
2021-08-23 14:01:20 -04:00
cdoern
d06d285e66 logFile until flag issue
we were adding a negative duration in podman events, causing inputs like
-5s to be correct and 5s to be incorrect.

fixes #11158

Signed-off-by: cdoern <cdoern@redhat.com>
2021-08-23 12:51:56 -04:00
flouthoc
edddfe8c4f volumes: Add support for exporting volumes to external tar
Adds support for transferring data between systems and backing up systems.
Use cases: recover from disasters or move data between machines.

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-23 20:42:41 +05:30
OpenShift Merge Robot
b644e37abc
Merge pull request #11296 from vrothberg/memory-profile
add flag to record memory profiles
2021-08-21 08:13:36 -04:00
OpenShift Merge Robot
2d43e685c3
Merge pull request #11286 from jwhonce/issues/11227
Update /version endpoint to add components
2021-08-20 10:55:00 -04:00
Valentin Rothberg
33fee83dea add flag to record memory profiles
Add a new flag `--memory-profile=$path` which creates a memory profile.
The generated profile can later be analyzed via `go tool pprof`.

[NO TESTS NEEDED] since it's a hidden flag, devs-only.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-20 15:35:58 +02:00
OpenShift Merge Robot
30b036c5d3
Merge pull request #11280 from Luap99/info-plugins
Podman info output plugin information
2021-08-19 17:28:22 -04:00
Jhon Honce
fd32c73e3a Update /version endpoint to add components
* Include OCI and conmon information as components

Fixes #11227

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-08-19 14:19:45 -07:00
OpenShift Merge Robot
f988cfe146
Merge pull request #11279 from guillaumerose/refactor1
machine: compute sha256 as we are reading the file
2021-08-19 14:50:26 -04:00
OpenShift Merge Robot
a3f4fbd176
Merge pull request #11271 from jwhonce/issues/8577
Clean up swagger
2021-08-19 14:49:22 -04:00
Paul Holzinger
16dfce486b
Podman info output plugin information
For docker compat include information about available volume, log and
network drivers which should be listed under the plugins key.

Fixes #11265

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-08-19 17:33:20 +02:00
Jhon Honce
0897946f60 [NO TESTS NEEDED] Clean up swagger
* Removed defined by unused responses
* Added missing body definitions
* Updated header input definitions

Outstanding issues:
* Supporting body ContainerConfig for /commit endpoint

Fixes #8577

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-08-19 07:57:54 -07:00
Guillaume Rose
b19812b5a2 machine: compute sha256 as we read the image file
It avoids to have the full file in memory.

[NO TESTS NEEDED]

Signed-off-by: Guillaume Rose <gurose@redhat.com>
2021-08-19 16:54:50 +02:00
Guillaume Rose
0434571920 machine: check for file exists instead of listing directory
[NO TESTS NEEDED]

Signed-off-by: Guillaume Rose <gurose@redhat.com>
2021-08-19 16:54:44 +02:00
OpenShift Merge Robot
8b848740cf
Merge pull request #11260 from nalind/remote-build-path
pkg/bindings/images.nTar(): slashify hdr.Name values
2021-08-19 05:57:02 -04:00
OpenShift Merge Robot
7a0d94837a
Merge pull request #11269 from Luap99/rootlessport-flake
fix rootlessport flake
2021-08-18 19:54:09 -04:00
Paul Holzinger
2d0a0c0d29
fix rootlessport flake
When the rootlessport process is started the stdout/stderr are attached
to the podman process. However once everything is setup podman exits and
when the rootlessport process tries to write to stdout it will fail with
SIGPIPE. The code handles this signal and puts /dev/null to stdout and
stderr but this is not robust. I do not understand the exact cause but
sometimes the process is still killed by SIGPIPE. Either go lost the
signal or the process got already killed before the goroutine could
handle it.

Instead of handling SIGPIPE just set /dev/null to stdout and stderr
before podman exits. With this there should be no race and no way to
run into SIGPIPE errors.

[NO TESTS NEEDED]

Fixes #11248

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2021-08-18 21:21:47 +02:00
Nalin Dahyabhai
aeffdb05d1 pkg/bindings/images.nTar(): slashify hdr.Name values
When setting path names in the build context archive, convert path names
to use forward slashes, as is normal for those archives, so that
directory hierarchies archived on Windows hosts extract correctly
everywhere.

Not really sure how to run the remote client in CI on a system that uses
`\` as a path separator, which is where this error crops up, so
[NO TESTS NEEDED]

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-08-18 11:03:28 -04:00
Brent Baude
cd40c875ac Add ability to build images in play kube
When playing a kube YAML file, it can be desirable to be able to build
an image on the fly.  This is good for development of an image and YAML
files and somewhat mocks what compose does.

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-18 09:55:24 -05:00
openshift-ci[bot]
a3d8b48fd5
Merge pull request #11212 from flouthoc/check-valid-systemd-session
cgroup-manager-systemd: Warn early if user is rootless and no relevent user session is present.
2021-08-17 21:12:51 +00:00
openshift-ci[bot]
856f1a2124
Merge pull request #11192 from ashley-cui/darwinwarn3
[NO TESTS NEEDED] Change connection error to be helpful for machine users
2021-08-17 19:46:04 +00:00
openshift-ci[bot]
a7a55ea7fb
Merge pull request #11231 from flouthoc/move-volume-dest-to-server
volume: move validating volume dest from client to server.
2021-08-17 19:05:42 +00:00
flouthoc
1f632f3572 fix: unifiedOverlays should be assigned if no conflicts found.
[NO TESTS NEEDED]

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-17 20:53:32 +05:30
openshift-ci[bot]
edf9ec38e4
Merge pull request #11224 from xatier/master
Add space trimming check in ValidateSysctls
2021-08-17 13:38:19 +00:00
flouthoc
e7ee15f8c3 cgroup-manager-systemd:Fail early if user:rootless and relevent session is not present.
[NO TESTS NEEDED]

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-17 16:18:54 +05:30
flouthoc
3cee85531c libpod/option.go remove error stutter from wrap/wraf
[NO TESTS NEEDED]
Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-17 16:14:02 +05:30
Valentin Rothberg
fe2be7f886 make sure that signal buffers are sufficiently big
Dealing with os.Signal channels seems more like an art than science
since signals may get lost.  os.Notify doesn't block on an unbuffered
channel, so users are expected to know what they're doing or hope for
the best.

In the recent past, I've seen a number of flakes and BZs on non-amd64
architectures where I was under the impression that signals may got
lost, for instance, during stop and exec.

[NO TESTS NEEDED] since this is art.

Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2021-08-17 12:10:20 +02:00
openshift-ci[bot]
8c228bdbd0
Merge pull request #11154 from cdoern/imagesPull
Libpod images pull changes
2021-08-16 15:46:33 +00:00
flouthoc
f985699460 volume: move validating volume dest from client to server.
[NO TESTS NEEDED]

Signed-off-by: flouthoc <flouthoc.git@gmail.com>
2021-08-16 16:36:39 +05:30
xatier
d997564342
Add space trimming check in ValidateSysctls
This is to catch invalid sysctl configs with extra spacing.

See
https://github.com/containers/common/issues/723#issuecomment-897395506

Signed-off-by: xatier <xatierlike@gmail.com>
2021-08-13 15:24:06 -05:00
Brent Baude
bef26f2582 rename oneshot initcontainers to once
after the init containers pr merged, it was suggested to use `once`
instead of `oneshot` containers as it is more aligned with other
terminiology used similarily.

[NO TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-12 12:57:15 -05:00
openshift-ci[bot]
94886d4aba
Merge pull request #11206 from baude/gvproxynewpath
Set gvproxy path to /usr/libexec/podman/gvproxy
2021-08-12 15:39:51 +00:00
Brent Baude
9fc9465630 Set gvproxy path to /usr/libexec/podman/gvproxy
We have reverted the previous patches to look for the gvproxy binary in
/usr/lib/podman and have again decided to use /usr/libexec/podman

[NO TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-12 09:11:09 -05:00
openshift-ci[bot]
de043a5a02
Merge pull request #11203 from rhatdan/codespell
Run codespell to fix spelling
2021-08-12 07:22:48 +00:00
Jhon Honce
5a32946d61 For compatibility, ignore Content-Type
Endpoint /build logs an info entry when a client uses the wrong
Content-Type for build payload. Given Content-Type is ignored and
assumed to be "application/x-tar".

Endpoint /libpod/build will fail unless "application/x-tar" or
"application/tar" is given for Content-Type. "application/tar" will
be logged as an info entry.

Fixes #11012

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2021-08-11 13:50:51 -07:00
Daniel J Walsh
404488a087
Run codespell to fix spelling
[NO TESTS NEEDED] Just fixing spelling.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-08-11 16:41:45 -04:00
Brent Baude
643178c1ec Revert "Use static path for gvproxy"
This reverts commit 4acc1d6850.

[NO TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2021-08-11 14:39:00 -05:00
Ashley Cui
2e3ba9cd99 Change connection error to be helpful for machine users
If a podman-remote connection fails, remind the user to check their
linux system and podman machine vm

Signed-off-by: Ashley Cui <acui@redhat.com>
2021-08-11 09:07:24 -04:00
openshift-ci[bot]
1968fdc874
Merge pull request #11173 from jmguzik/pod-ps-until-filter
Add until filter to podman pod ps
2021-08-11 08:38:20 +00:00
Jakub Guzik
ed30ae4a8a Add until filter to podman pod ps
This commit adds additional until filter to podman pod ps (ls/list).
Additionally, it also adds descriptions for podman pod ps filters available
via http api.

Signed-off-by: Jakub Guzik <jakubmguzik@gmail.com>
2021-08-10 22:10:40 +02:00
Daniel J Walsh
221b1add74 Add support for pod inside of user namespace.
Add the --userns flag to podman pod create and keep
track of the userns setting that pod was created with
so that all containers created within the pod will inherit
that userns setting.

Specifically we need to be able to launch a pod with
--userns=keep-id

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2021-08-09 15:17:22 -04:00