Commit graph

4427 commits

Author SHA1 Message Date
Miloslav Trmač
93e14b6198 Remove the forceSecure parameter on the pull call stack
DockerRegistryOptions.DockerInsecureSkipTLSVerify as an types.OptionalBool
can now represent that value, so forceSecure is redundant.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-12-06 23:34:59 +01:00
Miloslav Trmač
1b893be71c Remove manual handling of insecure registries in PushImageToReference
Instead, just set SystemRegistriesConfPath and let the transport do it.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-12-06 23:34:59 +01:00
Miloslav Trmač
7407d6621c Remove the forceSecure parameter of Image.PushImageTo*
DockerRegistryOptions.DockerInsecureSkipTLSVerify as an types.OptionalBool
can now represent that value, so forceSecure is redundant.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-12-06 23:34:59 +01:00
Miloslav Trmač
b134951d14 Minimally update for the DockerInsecureSkipTLSVerify type change
Following SystemContext.DockerInsecureSkipTLSVerify, make the
DockerRegistryOne also an OptionalBool, and update callers.

Explicitly document that --tls-verify=true and --tls-verify unset
have different behavior in those commands where the behavior changed
(or where it hasn't changed but the documentation needed updating).

Also make the --tls-verify man page sections a tiny bit more consistent
throughout.

This is a minimal fix, without changing the existing "--tls-verify=true"
paths nor existing manual insecure registry lookups.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-12-06 23:34:59 +01:00
Miloslav Trmač
b70f8b3884 Fix build on non-Linux
Unrelated to the rest of the PR.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-12-06 23:31:54 +01:00
Miloslav Trmač
6a0afce7c8 Remove some unused data structures and code
... instead of unnecessarily adapting it for the DockerInsecureSkipTLSVerify type change.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-12-06 23:31:54 +01:00
baude
39a036e24d bind mount /etc/resolv.conf|hosts in pods
containers inside pods need to make sure they get /etc/resolv.conf
and /etc/hosts bind mounted when network is expected

Signed-off-by: baude <bbaude@redhat.com>
2018-12-06 13:56:57 -06:00
OpenShift Merge Robot
5c6e02b55b
Merge pull request #1904 from umohnani8/volume
Add "podman volume" command
2018-12-06 08:59:13 -08:00
OpenShift Merge Robot
3e60de629d
Merge pull request #1912 from baude/prune
Add ability to prune containers and images
2018-12-06 08:31:20 -08:00
Matthew Heon
b945d9128a Add locking to Sync() on containers
Previously not needed as it only worked inside of Batch(), but
now that it can be called anywhere we need to add mutual
exclusion on its config changes.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-06 09:10:45 -05:00
Matthew Heon
a0c9be2061 Add --sync option to podman rm
With the changes made recently to ensure Podman does not hit the
OCI runtime as often to sync state, we can find ourselves in a
situation where the runtime's state does not match ours.

Add a --sync flag to podman rm to ensure we can still remove
containers when this happens.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-06 09:10:45 -05:00
Urvashi Mohnani
375831e976 Tests for podman volume commands
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
2018-12-06 10:17:23 +00:00
umohnani8
4c70b8a94b Add "podman volume" command
Add support for podman volume and its subcommands.
The commands supported are:
	podman volume create
	podman volume inspect
	podman volume ls
	podman volume rm
	podman volume prune

This is a tool to manage volumes used by podman. For now it only handle
named volumes, but eventually it will handle all volumes used by podman.

Signed-off-by: umohnani8 <umohnani@redhat.com>
2018-12-06 10:17:16 +00:00
baude
e037427035 Add ability to prune containers and images
Allow user to prune unused/unnamed images, the layer images from building,
via podman rmi --prune.

Allow user to prune stopped/exiuted containers via podman rm --prune.

This should resolve #1910

Signed-off-by: baude <bbaude@redhat.com>
2018-12-05 19:57:54 -06:00
OpenShift Merge Robot
be74acee1c
Merge pull request #1940 from wking/numeric-gid
libpod/container_internal_linux: Allow gids that aren't in the group file
2018-12-05 08:09:58 -08:00
OpenShift Merge Robot
50e754cd57
Merge pull request #1918 from mheon/use_db_paths
Use paths written in DB instead if they differ from our defaults
2018-12-05 00:55:48 -08:00
OpenShift Merge Robot
aba52cf588
Merge pull request #1939 from mheon/no_firewall_if_rootless
Don't initialize CNI when running as rootless
2018-12-04 12:55:40 -08:00
W. Trevor King
650f95cb06 libpod/container_internal_linux: Allow gids that aren't in the group file
When an image config sets config.User [1] to a numeric group (like
1000:1000), but those values do not exist in the container's
/etc/group, libpod is currently breaking:

  $ podman run --rm registry.svc.ci.openshift.org/ci-op-zvml7cd6/pipeline:installer --help
  error creating temporary passwd file for container 228f6e9943d6f18b93c19644e9b619ec4d459a3e0eb31680e064eeedf6473678: unable to get gid 1000 from group file: no matching entries in group file

However, the OCI spec requires converters to copy numeric uid and gid
to the runtime config verbatim [2].

With this commit, I'm frontloading the "is groupspec an integer?"
check and only bothering with lookup.GetGroup when it was not.

I've also removed a few .Mounted checks, which are originally from
00d38cb3 (podman create/run need to load information from the image,
2017-12-18, #110).  We don't need a mounted container filesystem to
translate integers.  And when the lookup code needs to fall back to
the mounted root to translate names, it can handle erroring out
internally (and looking it over, it seems to do that already).

[1]: https://github.com/opencontainers/image-spec/blame/v1.0.1/config.md#L118-L123
[2]: https://github.com/opencontainers/image-spec/blame/v1.0.1/conversion.md#L70

Signed-off-by: W. Trevor King <wking@tremily.us>
2018-12-04 12:00:42 -08:00
Matthew Heon
32aa45e344 Don't initialize CNI when running as rootless
We don't use CNI to configure networks for rootless containers,
so no need to set it up. It may also cause issues with inotify,
so disabling it resolves some potential problems.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-04 14:57:06 -05:00
Matthew Heon
e3882cfa2d Use runtime lockDir in BoltDB state
Instead of storing the runtime's file lock dir in the BoltDB
state, refer to the runtime inside the Bolt state instead, and
use the path stored in the runtime.

This is necessary since we moved DB initialization very far up in
runtime init, before the locks dir is properly initialized (and
it must happen before the locks dir can be created, as we use the
DB to retrieve the proper path for the locks dir now).

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-04 13:58:51 -05:00
baude
c8436b4912 generate kube
add the ability to generate kubernetes pod and service yaml representations
of libpod containers and pods.

Signed-off-by: baude <bbaude@redhat.com>
2018-12-04 08:03:49 -06:00
W. Trevor King
a4b483c848 libpod/container_internal: Deprecate implicit hook directories
Part of the motivation for 800eb863 (Hooks supports two directories,
process default and override, 2018-09-17, #1487) was [1]:

> We only use this for override. The reason this was caught is people
> are trying to get hooks to work with CoreOS. You are not allowed to
> write to /usr/share... on CoreOS, so they wanted podman to also look
> at /etc, where users and third parties can write.

But we'd also been disabling hooks completely for rootless users.  And
even for root users, the override logic was tricky when folks actually
had content in both directories.  For example, if you wanted to
disable a hook from the default directory, you'd have to add a no-op
hook to the override directory.

Also, the previous implementation failed to handle the case where
there hooks defined in the override directory but the default
directory did not exist:

  $ podman version
  Version:       0.11.2-dev
  Go Version:    go1.10.3
  Git Commit:    "6df7409cb5a41c710164c42ed35e33b28f3f7214"
  Built:         Sun Dec  2 21:30:06 2018
  OS/Arch:       linux/amd64
  $ ls -l /etc/containers/oci/hooks.d/test.json
  -rw-r--r--. 1 root root 184 Dec  2 16:27 /etc/containers/oci/hooks.d/test.json
  $ podman --log-level=debug run --rm docker.io/library/alpine echo 'successful container' 2>&1 | grep -i hook
  time="2018-12-02T21:31:19-08:00" level=debug msg="reading hooks from /usr/share/containers/oci/hooks.d"
  time="2018-12-02T21:31:19-08:00" level=warning msg="failed to load hooks: {}%!(EXTRA *os.PathError=open /usr/share/containers/oci/hooks.d: no such file or directory)"

With this commit:

  $ podman --log-level=debug run --rm docker.io/library/alpine echo 'successful container' 2>&1 | grep -i hook
  time="2018-12-02T21:33:07-08:00" level=debug msg="reading hooks from /usr/share/containers/oci/hooks.d"
  time="2018-12-02T21:33:07-08:00" level=debug msg="reading hooks from /etc/containers/oci/hooks.d"
  time="2018-12-02T21:33:07-08:00" level=debug msg="added hook /etc/containers/oci/hooks.d/test.json"
  time="2018-12-02T21:33:07-08:00" level=debug msg="hook test.json matched; adding to stages [prestart]"
  time="2018-12-02T21:33:07-08:00" level=warning msg="implicit hook directories are deprecated; set --hooks-dir="/etc/containers/oci/hooks.d" explicitly to continue to load hooks from this directory"
  time="2018-12-02T21:33:07-08:00" level=error msg="container create failed: container_linux.go:336: starting container process caused "process_linux.go:399: container init caused \"process_linux.go:382: running prestart hook 0 caused \\\"error running hook: exit status 1, stdout: , stderr: oh, noes!\\\\n\\\"\""

(I'd setup the hook to error out).  You can see that it's silenly
ignoring the ENOENT for /usr/share/containers/oci/hooks.d and
continuing on to load hooks from /etc/containers/oci/hooks.d.

When it loads the hook, it also logs a warning-level message
suggesting that callers explicitly configure their hook directories.
That will help consumers migrate, so we can drop the implicit hook
directories in some future release.  When folks *do* explicitly
configure hook directories (via the newly-public --hooks-dir and
hooks_dir options), we error out if they're missing:

  $ podman --hooks-dir /does/not/exist run --rm docker.io/library/alpine echo 'successful container'
  error setting up OCI Hooks: open /does/not/exist: no such file or directory

I've dropped the trailing "path" from the old, hidden --hooks-dir-path
and hooks_dir_path because I think "dir(ectory)" is already enough
context for "we expect a path argument".  I consider this name change
non-breaking because the old forms were undocumented.

Coming back to rootless users, I've enabled hooks now.  I expect they
were previously disabled because users had no way to avoid
/usr/share/containers/oci/hooks.d which might contain hooks that
required root permissions.  But now rootless users will have to
explicitly configure hook directories, and since their default config
is from ~/.config/containers/libpod.conf, it's a misconfiguration if
it contains hooks_dir entries which point at directories with hooks
that require root access.  We error out so they can fix their
libpod.conf.

[1]: https://github.com/containers/libpod/pull/1487#discussion_r218149355

Signed-off-by: W. Trevor King <wking@tremily.us>
2018-12-03 12:54:30 -08:00
Matthew Heon
795fbba769 Revert changes to GetDefaultStoreOptions
We don't need this for anything more than rootless work in Libpod
now, but Buildah still uses it as it was originally written, so
leave it intact as part of our API.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-03 15:48:20 -05:00
Matthew Heon
7c575bdce2 Fix libpod static dir selection when graphroot changed
When graphroot is set by the user, we should set libpod's static
directory to a subdirectory of that by default, to duplicate
previous behavior.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-03 15:13:07 -05:00
Matthew Heon
677c444463 Ensure directory where we will make database exists
Ensure that the directory where we will create the Podman db
exists prior to creating the database - otherwise creating the DB
will fail.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-03 11:10:02 -05:00
Matthew Heon
ea13264958 Fix typo
Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-03 10:48:33 -05:00
Matthew Heon
32fc865b6d Add better descriptions for validation errors in DB
When validating fields against the DB, report more verbosely the
name of the field being validated if it fails. Specifically, add
the name used in config files, so people will actually know what
to change it errors happen.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-03 10:38:32 -05:00
Matthew Heon
b104a45f35 Fix gofmt and lint
Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 16:53:15 -05:00
Matthew Heon
69ed2ccc54 Make locks dir in unit tests
Ensure we don't break the unit tests by creating a locks
directory (which, prior to the last commit, would be created by
BoltDB state init).

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 16:53:15 -05:00
Matthew Heon
03229239b0 Do not initialize locks dir in BoltDB
We already create the locks directory as part of the libpod
runtime's init - no need to do it again as part of BoltDB's init.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 15:32:06 -05:00
Matthew Heon
562fa57dc9 Move rootless storage config into libpod
Previous commits ensured that we would use database-configured
paths if not explicitly overridden.

However, our runtime generation did unconditionally override
storage config, which made this useless.

Move rootless storage configuration setup to libpod, and change
storage setup so we only override if a setting is explicitly
set, so we can still override what we want.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 15:21:35 -05:00
Matthew Heon
92ff83f5b9 Set default paths from DB if not explicitly overridden
If the DB contains default paths, and the user has not explicitly
overridden them, use the paths in the DB over our own defaults.

The DB validates these paths, so it would error and prevent
operation if they did not match. As such, instead of erroring, we
can use the DB's paths instead of our own.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 14:21:22 -05:00
Matthew Heon
aa7ce33b7a Add a struct indicating if some Runtime fields were set
To configure runtime fields from the database, we need to know
whether they were explicitly overwritten by the user (we don't
want to overwrite anything that was explicitly set). Store a
struct containing whether the variables we'll grab from the DB
were explicitly set by the user so we know what we can and can't
overwrite.

This determines whether libpod runtime and static dirs were set
via config file in a horribly hackish way (double TOML decode),
but I can't think of a better way, and it shouldn't be that
expensive as the libpod config is tiny.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 14:06:33 -05:00
Matthew Heon
137e0948ae Make DB config validation an explicit step
Previously, we implicitly validated runtime configuration against
what was stored in the database as part of database init. Make
this an explicit step, so we can call it after the database has
been initialized. This will allow us to retrieve paths from the
database and use them to overwrite our defaults if they differ.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 13:38:36 -05:00
Matthew Heon
b0f79ff4df Move DB configuration up in runtime setup
When we configure a runtime, we now will need to hit the DB early
on, so we can verify the paths we're going to use for c/storage are correct.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 13:38:36 -05:00
Matthew Heon
6e16702947 Add ability to retrieve runtime configuration from DB
When we create a Libpod database, we store a number of runtime
configuration fields in it. If we can retrieve those, we can use
them to configure the runtime to match the DB instead of inbuilt
defaults, helping to ensure that we don't error in cases where
our compiled-in defaults changed.

Signed-off-by: Matthew Heon <mheon@redhat.com>
2018-12-02 13:38:36 -05:00
Giuseppe Scrivano
1d2e1eece5
rootless: propagate XDG_RUNTIME_DIR to the OCI runtime
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2018-11-30 22:37:09 +01:00
OpenShift Merge Robot
b504623a11
Merge pull request #1317 from rhatdan/privileged
Disable mount options when running --privileged
2018-11-30 11:09:51 -08:00
Daniel J Walsh
a5be3ffa4d
/dev/shm should be mounted even in rootless mode.
Currently we are mounting /dev/shm from disk, it should be from a tmpfs.
User Namespace supports tmpfs mounts for nonroot users, so this section of
code should work fine in bother root and rootless mode.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-11-28 15:48:25 -05:00
OpenShift Merge Robot
22d53e2b54
Merge pull request #1798 from giuseppe/fix-notify-socket
systemd: fix NOTIFY_SOCKET with patched runc
2018-11-28 09:16:27 -08:00
OpenShift Merge Robot
f3289fed2e
Merge pull request #1880 from baude/f29fixes
Fix golang formatting issues
2018-11-28 08:18:24 -08:00
OpenShift Merge Robot
ade0b30844
Merge pull request #1846 from cgwalters/netns-dns-localhost
Use host's resolv.conf if no network namespace enabled
2018-11-28 07:58:55 -08:00
baude
61d4db4806 Fix golang formatting issues
Whe running unittests on newer golang versions, we observe failures with some
formatting types when no declared correctly.

Signed-off-by: baude <bbaude@redhat.com>
2018-11-28 09:26:24 -06:00
Giuseppe Scrivano
fe919e4914
oci: propagate NOTIFY_SOCKET on runtime start
with https://github.com/opencontainers/runc/pull/1807 we moved the
systemd notify initialization from "create" to "start", so that the
OCI runtime doesn't hang while waiting on reading from the notify
socket.  This means we also need to set the correct NOTIFY_SOCKET when
start'ing the container.

Closes: https://github.com/containers/libpod/issues/746

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2018-11-28 16:25:12 +01:00
OpenShift Merge Robot
effd63d6d5
Merge pull request #1848 from adrianreber/master
Add tcp-established to checkpoint/restore
2018-11-28 07:00:24 -08:00
OpenShift Merge Robot
d346996e15
Merge pull request #1849 from giuseppe/report-rootless-netmode
rootless: add new netmode "slirp4netns"
2018-11-28 06:18:28 -08:00
OpenShift Merge Robot
2a496aea30
Merge pull request #1833 from giuseppe/remove-exec-polling
exec: remove polling for PID file
2018-11-28 05:10:49 -08:00
Daniel J Walsh
3beacb73bc
Disable mount options when running --privileged
We now default to setting storage options to "nodev", when running
privileged containers, we need to turn this off so the processes can
manipulate the image.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-11-28 07:53:28 -05:00
Giuseppe Scrivano
fc3047322a
util: use fsnotify to wait for file
prefer a fsnotify watcher to polling the file, we take advantage of
inotify on Linux and react more promptly to the PID file being
created.

If the watcher cannot be created, then fallback to the old polling
mechanism.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2018-11-28 10:53:41 +01:00
Giuseppe Scrivano
0365f57371
rootless: fix cleanup
The conmon exit command is running inside of a namespace where the
process is running with uid=0.  When it launches again podman for the
cleanup, podman is not running in rootless mode as the uid=0.

Export some more env variables to tell podman we are in rootless
mode.

Closes: https://github.com/containers/libpod/issues/1859

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2018-11-28 10:19:13 +01:00