Commit graph

10739 commits

Author SHA1 Message Date
Matthew Heon
97681a5f2b Move lock init after tmp dir is populated properly
Don't initialize the lock manager until almost the end of libpod
init, so we can guarantee our tmp dir is properly set up and
exists. This wasn't an issue on systems that had previously run
Podman, but CI caught it.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-01-04 09:51:09 -05:00
Matthew Heon
01858c0eba DO NOT MERGE temporarily remove python tests
It's impossible to get good debug out of the python tests, so
nuke them for now so I can figure out what's wrong.

DO NOT MERGE THIS COMMIT

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-01-04 09:51:09 -05:00
Matthew Heon
16a5cbfac5 When refreshing libpod, if SHM locks exist, remove them
This will hopefully help cases where libpod is initialized
multiple times on the same system (as on our CI tests).

We still run into potential issues where multiple Podmans with
multiple tmp paths try to run on the same system - we could end
up thrashing the locks.

I think we need a file locks driver for situations like that. We
can also see about storing paths in the SHM segment, to make sure
multiple libpod instances aren't using the same one.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-01-04 09:51:09 -05:00
Matthew Heon
31df5e0e85 Ensure different error messages and creating/opening locks
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:51:09 -05:00
Matthew Heon
625c7e18ef Update unit tests to use in-memory lock manager
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:51:09 -05:00
Matthew Heon
35361595f3 Remove runtime lockDir and add in-memory lock manager
Remove runtime's lockDir as it is no longer needed after the lock
rework.

Add a trivial in-memory lock manager for unit testing

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:51:09 -05:00
Matthew Heon
d4b2f11601 Convert pods to SHM locks
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:51:09 -05:00
Matthew Heon
3de560053f Convert containers to SHM locking
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:51:09 -05:00
Matthew Heon
a364b656ea Add lock manager to libpod runtime
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:51:09 -05:00
Matthew Heon
e73484c176 Move to POSIX mutexes for SHM locks
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:51:09 -05:00
Matthew Heon
f38fccb48c Disable lint on SHMLock struct
Golint wants to rename the struct. I think the name is fine. I
can disable golint. Golint will no longer complain about the
name.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
Matthew Heon
a21f21efa1 Refactor locks package to build on non-Linux
Move SHM specific code into a subpackage. Within the main locks
package, move the manager to be linux-only and add a non-Linux
unsupported build file.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
Matthew Heon
3ed81051e8 Add an SHM-backed Lock Manager implementation
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
Matthew Heon
185136cf0e Add interface for libpod multiprocess locks
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
Matthew Heon
35cc71a9e8 Improve documentation and unit tests for SHM locks
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
Matthew Heon
52d95f5072 Propogate error codes from SHM lock creation and open
Also add a few more unit tests

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
Matthew Heon
b489feff71 Add mutex invariant to SHM semaphores.
Check value of semaphores when incrementing to ensure we never go
beyond 1, preserving mutex invariants.

Also, add cleanup code to the lock tests, ensuring that we never
leave the locks in a bad state after a test. We aren't destroying
and recreating the SHM every time, so we have to be careful not
to leak state between test runs.

Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
Matthew Heon
27cebb780b Initial skeleton of in-memory locks
Signed-off-by: Matthew Heon <matthew.heon@gmail.com>
2019-01-04 09:45:59 -05:00
OpenShift Merge Robot
6868b5aa14
Merge pull request #2045 from vrothberg/init
add init support
2019-01-04 06:41:07 -08:00
Valentin Rothberg
75578aad61 add container-init support
Add support for executing an init binary as PID 1 in a container to
forward signals and reap processes.  When the `--init` flag is set for
podman-create or podman-run, the init binary is bind-mounted to
`/dev/init` in the container and "/dev/init --" is prepended to the
container's command.

The default base path of the container-init binary is `/usr/libexec/podman`
while the default binary is catatonit [1].  This default can be changed
permanently via the `init_path` field in the `libpod.conf` configuration
file (which is recommended for packaging) or temporarily via the
`--init-path` flag of podman-create and podman-run.

[1] https://github.com/openSUSE/catatonit

Fixes: #1670
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-01-04 11:42:03 +01:00
Daniel J Walsh
b0fcd555db
If local storage file exists, then use it rather then defaults.
Currently we always force overlay if it exists even though a user might want
vfs.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2019-01-03 16:25:02 -05:00
OpenShift Merge Robot
9ffd480616
Merge pull request #2071 from baude/vendorcs
vendor in new containers/storage
2019-01-03 12:14:24 -08:00
baude
561e65969f vendor in new containers/storage
vendor in latest containers/storage which contains a fix for when
a filesystem that overlayfs is on is ENOSPC.

adding pgzip/compress as a new dep for c/s

Signed-off-by: baude <bbaude@redhat.com>
2019-01-03 12:49:30 -06:00
OpenShift Merge Robot
1334696814
Merge pull request #2072 from rhatdan/completions
Fix completions
2019-01-03 10:26:10 -08:00
Daniel J Walsh
0c26aec2ae
Fix completions
Currently completions do not work.  podman generate kube and podman play kube
completions broke this.

Also fixed podman import to work properly.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2019-01-03 11:57:55 -05:00
OpenShift Merge Robot
098c13418a
Merge pull request #2065 from TomSweeneyRedHat/dev/tsweeney/troublefix1
Touch up some troubleshooting nits
2019-01-02 12:02:43 -08:00
TomSweeneyRedHat
a25b707fdb Touch up some troubleshooting nits
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
2019-01-02 14:14:50 -05:00
OpenShift Merge Robot
a634d5559f
Merge pull request #2068 from mheon/log_container_command
Log container command before starting the container
2019-01-02 10:56:49 -08:00
Matthew Heon
a36006c2dc Warn on overriding user-specified storage driver w/ DB
Overriding storage.conf is not intuitive behavior, so pop up an
error message when it happens, so people know that bad things are
happening.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-01-02 12:54:54 -05:00
Matthew Heon
945d0e8700 Log container command before starting the container
Runc does not produce helpful error messages when the container's
command is not found, so print the command ourselves.

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-01-02 12:11:50 -05:00
OpenShift Merge Robot
e796f522f6
Merge pull request #2067 from mheon/fix_garbage_ports
Use sprintf to generate port numbers while committing
2019-01-02 07:55:24 -08:00
Matthew Heon
9c7d512136 Use sprintf to generate port numbers while committing
Casting integers to strings is definitely not correct, so let the
standard library handle matters.

Fixes #2066

Signed-off-by: Matthew Heon <matthew.heon@pm.me>
2019-01-02 10:26:44 -05:00
OpenShift Merge Robot
7438b7bd3a
Merge pull request #2052 from rhatdan/sparse
Add troubleshooting for sparse files
2018-12-31 08:23:29 -08:00
Daniel J Walsh
ad36345fde
Add troubleshooting for sparse files
A common failure with people building container images is the
creation of large sparse files, particularly useradd creating
the lastlog file.

This PR Documents the failures.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-12-30 08:08:13 -05:00
OpenShift Merge Robot
1aa55edda5
Merge pull request #2058 from adrianreber/enable-cr
Re-enable checkpoint/restore CI tests on Fedora
2018-12-28 17:54:35 -08:00
OpenShift Merge Robot
1438d5471f
Merge pull request #2059 from rhatdan/ociruntime
podman build is not using the default oci-runtime
2018-12-28 17:44:35 -08:00
OpenShift Merge Robot
0420215f65
Merge pull request #2060 from rhatdan/devices
Fix handling of symbolic links
2018-12-28 17:33:35 -08:00
Daniel J Walsh
c141c5cfd2
Fix handling of symbolic links
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-12-28 13:10:15 -05:00
Daniel J Walsh
652a985998
podman build is not using the default oci-runtime
Currently if the user installs runc in an alternative path
podman run uses it but podman build does not.

This patch will pass the default oci runtime to be used by podman
down to the image builder.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-12-28 12:36:25 -05:00
Adrian Reber
2b41cd5fd3
Re-enable checkpoint/restore CI tests on Fedora
Now that the correct distribution CRIU packages are installed the
checkpoint/restore tests should no longer fail. This re-enables the
disabled tests on Fedora.

Signed-off-by: Adrian Reber <areber@redhat.com>
2018-12-28 14:05:27 +01:00
OpenShift Merge Robot
c50332d057
Merge pull request #2051 from rhatdan/shm
Fixes to handle /dev/shm correctly.
2018-12-24 08:43:10 -08:00
Daniel J Walsh
df99522c67
Fixes to handle /dev/shm correctly.
We had two problems with /dev/shm, first, you mount the
container read/only then /dev/shm was mounted read/only.
This is a bug a tmpfs directory should be read/write within
a read-only container.

The second problem is we were ignoring users mounted /dev/shm
from the host.

If user specified

podman run -d -v /dev/shm:/dev/shm ...

We were dropping this mount and still using the internal mount.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-12-24 09:03:53 -05:00
OpenShift Merge Robot
bbfe7341b7
Merge pull request #2049 from baude/rootlessusestop
rootless tests using stop is more reliable
2018-12-24 02:53:45 -08:00
OpenShift Merge Robot
8fe30505ac
Merge pull request #2044 from rhatdan/list
Allow alias for list, ls, ps to work
2018-12-23 11:37:26 -08:00
baude
df923b2613 rootless tests using stop is more reliable
when testing rootless containers, it is more reliable to stop
a container with a zero timeout than kill a container.  We made
this change in non-rootless tests as well.  When IO or CPU are
taxed, it avoids a situation where the kill signal is sent but the
container has not been able to update its status when a subsequent
action occurs.

Signed-off-by: baude <bbaude@redhat.com>
2018-12-23 11:47:46 -06:00
OpenShift Merge Robot
eb982193c5
Merge pull request #2048 from isimluk/use-idtools-parser
Refactor: use idtools.ParseIDMap instead of bundling own version
2018-12-23 08:10:25 -08:00
Daniel J Walsh
a22f00d4c5
Allow alias for list, ls, ps to work
Allow multiple alias for listing containers and images.

Also fix documentation for umount and unmount

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-12-23 09:38:57 -05:00
Šimon Lukašík
4e85f468fc Refactor: use idtools.ParseIDMap instead of bundling own version
ParseIDMap function was extracted to idtools in

    https://github.com/containers/storage/pull/236

it is already used in containers/storage and buildah, it should be used in
libpod as well.

Signed-off-by: Šimon Lukašík <isimluk@fedoraproject.org>
2018-12-23 12:07:54 +01:00
OpenShift Merge Robot
792f10988e
Merge pull request #2023 from rhatdan/config
Switch all references to image.ContainerConfig to image.Config
2018-12-22 03:11:08 -08:00
OpenShift Merge Robot
7122b49464
Merge pull request #2042 from cevich/new_criu_images
Use new VM images with updated CRIU
2018-12-22 02:59:10 -08:00