Commit graph

6464 commits

Author SHA1 Message Date
Paul Holzinger
5952486df8
podman network inspect: include running containers
Like docker podman network inspect should output the information of
running container with their ip/mac address on this network.
However the output format is not docker compatible as this cannot
include all the info we have and the previous output was already not
compatible so this is not new.

New example output:
```
[
     {
          ...
          "containers": {
               "7c0d295779cee4a6db7adc07a99e635909413a390eeab9f951edbc4aac406bf1": {
                    "name": "c2",
                    "interfaces": {
                         "eth0": {
                              "subnets": [
                                   {
                                        "ipnet": "10.89.0.4/24",
                                        "gateway": "10.89.0.1"
                                   },
                                   {
                                        "ipnet": "fda3:b4da:da1e:7e9d::4/64",
                                        "gateway": "fda3:b4da:da1e:7e9d::1"
                                   }
                              ],
                              "mac_address": "1a:bd:ca:ea:4b:3a"
                         }
                    }
               },
               "b17c6651ae6d9cc7d5825968e01d6b1e67f44460bb0c140bcc32bd9d436ac11d": {
                    "name": "c1",
                    "interfaces": {
                         "eth0": {
                              "subnets": [
                                   {
                                        "ipnet": "10.89.0.3/24",
                                        "gateway": "10.89.0.1"
                                   },
                                   {
                                        "ipnet": "fda3:b4da:da1e:7e9d::3/64",
                                        "gateway": "fda3:b4da:da1e:7e9d::1"
                                   }
                              ],
                              "mac_address": "f6:50:e6:22:d9:55"
                         }
                    }
               }
          }
     }
]
```

Fixes #14126
Fixes https://issues.redhat.com/browse/RHEL-3153

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-28 16:33:26 +01:00
Mario Loriedo
8f02822c12 Remove xz unit tests as they are flaky
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-28 15:37:15 +01:00
openshift-merge-bot[bot]
c93ed35b09
Merge pull request #21848 from baude/machinestartcleanup
Clean up gvproxy if machine start fails
2024-02-28 12:08:21 +00:00
Valentin Rothberg
f8abd7ff75 machine config: make write atomic
As indicated in #21849, loading the machine config can flake/fail with
an EOF JSON error indicating an incomplete file.  Address the issue by
atomically writing the config.  This way, it is not possible to load an
incomplete or partially written file.  The lock can be acquired later on
to sync state.

[NO NEW TESTS NEEDED] as it's a hard-to-hit race.

Fixes: #21849
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
2024-02-28 09:44:41 +01:00
openshift-merge-bot[bot]
9d1704396a
Merge pull request #21572 from mheon/hyperv_9p_mkdir
Ensure HyperV 9p mounts work when a dir doesn't exist
2024-02-27 22:34:07 +00:00
Matt Heon
26ec570c65 Ensure HyperV 9p mounts work when a dir doesn't exist
Before, we required that the mount target exist and be a
directory for the 9p mount to successfully complete, which is not
how things are supposed to work - the user should be able to
mount anywhere. This should just be a simple mkdir, but with FCOS
the root directory is immutable so we need to undo that before we
can mkdir, and unfortunately we don't have a library that can do
chattr (and I didn't want to drag in a new dependency just for
that), so let's be gross and add it to the SSH command. I
aggressively dislike this but it does work.

[NO NEW TESTS NEEDED] Can worry about getting a more generic
mount test together for Machine later.

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-27 14:33:53 -05:00
Brent Baude
dfae51a4b8 Clean up gvproxy if machine start fails
Matt found a bug where if a machine start did not run to completion, a
gvproxy was left around running.  This gvproxy then subsequently stopped
the next attempt to start.

Signed-off-by: Brent Baude <bbaude@redhat.com>

[NO NEW TESTS NEEDED]
2024-02-27 13:33:43 -06:00
Mario Loriedo
7b6d9a586e Fixup: avoid overriding io.Copy error
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-27 16:48:47 +01:00
Mario Loriedo
59704665ae Avoid overriding io.Copy error
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-27 16:16:44 +01:00
Mario Loriedo
24a33a538c Move code for sparse optimized copy to a dedicated method
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-27 15:59:56 +01:00
Paul Holzinger
bed61806c8
pkg/machine/ocipull: add default policy.json file
So that this file can be inculded in our windows/macos packages and also
by other packagers.
Right now the default policy is allow everything but we plan to add
signing in the future.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-27 15:29:59 +01:00
Paul Holzinger
a02aa8f6a2
pkg/machine/ocipull: add custom policy.json location
The default policy file /etc/containers/policy.json location does not
work on windows and for packages that ship a default.

Now we search for the policy.json in the following overwrite locations:
macos and linux:
 - ~/.config/containers/policy.json
 - /etc/containers/policy.json
windows:
 - %APPDATA%\containers\policy.json

Also it offers an additional DefaultPolicyJSONPath var that should be
overwritten at built time with the path of the file that is shipped by
packagers. Thile file is used when none of the overwrite paths exist.

[NO NEW TESTS NEEDED]

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-27 15:29:58 +01:00
Daniel J Walsh
5b28a2b4a0
Fix podman stop -t -1 CID
Currently if a user specifies a negative time to stop a container the
code ends up specifying the negative time to time.Duration which treats
it as 0. By settine the default to max.Unint32 we end up with a positive
number which indicates > 68 years which is probably close enough to
infinity for our use case.

Fixes: https://github.com/containers/podman/issues/21811

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2024-02-26 16:15:10 -05:00
openshift-merge-bot[bot]
04f70320c1
Merge pull request #21594 from arixmkii/qemu-win-machine-5
Change QEMU netdev to Unix domain socket
2024-02-26 20:37:06 +00:00
Arthur Sengileyev
656bf7b764 Change QEMU netdev to Unix domain socket
This change migrates to new QEMU stream netdev added in 7.2.0.
It also unifies how gvproxy is used in QEMU and AppleHV.

Signed-off-by: Arthur Sengileyev <arthur.sengileyev@gmail.com>
2024-02-26 20:02:46 +02:00
Matt Heon
19b676f855 Enable lint for Darwin and fix identified issues
[NO NEW TESTS NEEDED] Just fixing lint issues

Signed-off-by: Matt Heon <mheon@redhat.com>
2024-02-26 11:01:54 -05:00
openshift-merge-bot[bot]
b8e0427a43
Merge pull request #21789 from Romain-Geissler-1A/ignore-docker-endpoint-config-in-non-bridge-mode
Ignore docker's end point config when the final network mode isn't bridge
2024-02-26 09:51:27 +00:00
Mario Loriedo
f099250beb Better file close and err handling
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:03 +01:00
Mario Loriedo
ea4553d590 integrating changes from #21768
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
fa99b9be93 renamed testfiles as testdata
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
0b3f3f0ef1 Use faster gzip reader
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
2245cf8dc4 Remove duplication and make consistent usage of the progress bar
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
c42d3a74ed Add a comment to explain why we look at file name for zip files
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
95aa2c4e68 Use github.com/stretchr/testify assert in compression_test.go
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
Mario Loriedo
88af8852db Refactor machine decompress.go
Added some tests to verify that files extractions works
with different compression format.

Created a decompressor interface with 2 main methods:
  reader(): returns an io.Reader for the specific compression algorithm
  copy(): extracts the compressed file into the file provided as param

Created 5 decompressor types:
- gzip: extract gzip files
- xz: extract xz files
- zip: extract zip files
- generic: extract any other file using github.com/containers/image/v5/pkg/compression
- uncompressed: only do a copy of the file

Minor fix to the progress bar instances: added a call to bar.Abort(false)
that happens before Progress.Wait() to avoid that it hangs when a bar is
not set as completed although extraction is done.

Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
2024-02-25 22:29:02 +01:00
openshift-merge-bot[bot]
1fc9d5ae6e
Merge pull request #21797 from mtrmac/sparse0
Reformulate sparseWriter
2024-02-25 14:36:08 +00:00
openshift-merge-bot[bot]
eabce71cd2
Merge pull request #21804 from n1hility/ready-race
Fix race conditions in hyperv backend
2024-02-24 14:07:38 +00:00
Jason T. Greene
b68d3c7a0e Fix race conditions in hyperv readiness checking
- Listen before starting the vm
- Fix a device race caused by lazy hv_vsock init by waiting on network manager

[NO NEW TESTS NEEDED]

Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
2024-02-24 01:43:28 -06:00
Romain Geissler
127a8060ab
Ignore docker's end point config when the final network mode isn't bridge.
Closes #21754

Signed-off-by: Romain Geissler <romain.geissler@amadeus.com>
2024-02-23 21:11:25 +00:00
Miloslav Trmač
5d303ca267 Reformulate sparseWriter to deal with starting/ending zeroes explicitly
... instead of using a multi-variable state machine.

The net effect of this code is exactly the same as the previous implementation,
except:
- the operation after Write() returns an error might differ
- If the file ends with zeroes, we don't Seek(-1), and
  we don't create a hole at all if it is too small, preferring
  to save a syscall.

But this formulation is hopefully easier to prove correct.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2024-02-23 02:09:39 +01:00
Miloslav Trmač
c5434bf711 Expand sparseWriter tests
- Add more test cases
- Test that we create the expected (large) holes;
  don't enforce anything for the <zerosThresholt ones.

O)nly changes test code, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2024-02-23 02:09:39 +01:00
openshift-merge-bot[bot]
36d8e27601
Merge pull request #21517 from jakecorrenti/fix-qemu-todos
machine: Address some QEMU TODOs
2024-02-22 21:21:50 +00:00
openshift-merge-bot[bot]
b338364fbf
Merge pull request #21786 from Luap99/machine-gvproxy-cleanup
pkg/machine: ignore gvproxy pidfile not exists error
2024-02-22 16:13:41 +00:00
openshift-merge-bot[bot]
5a844511c8
Merge pull request #21652 from Luap99/machine-http-proxy
machine: implement http proxy logic for all providers
2024-02-22 15:40:39 +00:00
Jake Correnti
d7f7f07c9b Stop using rm'd inspect format templates in tests
Fixes the "machine rm --save-ignition --save-image" test so that it no longer
uses the `{{ .Image }}` format string.

Fixes the "init should cleanup on failure" test so that it no longer
uses the `{{ .Image }}` and `{{ .ConfigPath }}` format strings.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-22 08:34:41 -05:00
Jake Correnti
be9aba7ccd Remove image provenance from machine inspect
We don't care about the provenance of the machine image, so this is no
longer applicable to have when displaying info.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-22 08:34:27 -05:00
Jake Correnti
d68240b150 Update machine files rm order and add QEMU rm
Changes the order in which the machine-specific files are removed in
`Remove()`. Removes the system connections first, then removes the
`configPath` last. `configPath` is removed last, because in the case of
an error with any of the previous files, the removal can be attempted
again since the machine still "exists".

Made the errors in `Remove` hard errors instead of soft errors.

Added the implementation for the QEMU-specific file removal.

[NO NEW TESTS NEEDED]

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-22 08:31:55 -05:00
Paul Holzinger
6f6925cca4
pkg/machine: ignore gvproxy pidfile not exists error
When gvproxy exits it will delete the pidfile itself so we need to
account for that and juts ignore the case, it just means gvproxy was
able to exit successfully on its own.

Also remove the useless defer and return the error so we can get an
error exit code not just a print on stderr.

Currently it shows this error which is not helpful to any user:
unable to clean up gvproxy: "unable to read gvproxy pid file /run/user/1000/podman/gvproxy.pid: open /run/user/1000/podman/gvproxy.pid: no such file or directory"

[NO NEW TESTS NEEDED] TODO: make machine tests check stderr for such
things.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-22 12:39:20 +01:00
openshift-merge-bot[bot]
669e718561
Merge pull request #21636 from jakecorrenti/inspectinfo-hostuser
machine: Remove unnecessary TODOs
2024-02-22 10:29:51 +00:00
Paul Holzinger
f218f8430a
machine: implement http proxy logic for all providers
Copy all proxy envs into the VM on each start, this allows for updates
without having to recrate the VM. This is implemented via shell script
that is passed via ssh to the VM after it is started.

With that we now use the same logic for all providers the old fw_cfg
logic for qemu has been removed and the WSL code as well which keeps the
behavior the same.

There is a small risk now because we only update the env via ssh that
processes started before will have the old incorrect env but it should
really only effect core system processes which likely do not need them
anyway. The podman system service should not be started at this point
so it should be good enough.

It also fixes the broken behavior with SSL_CERT_FILE/SSL_CERT_DIR which
were updated on each start which is not correct as the files are only
copied with ignition so these should not be updated and just set
statically when the VM was created.

e2e test has been added to ensure the behavior works as expected.

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
2024-02-22 11:23:45 +01:00
openshift-merge-bot[bot]
59b6f48d90
Merge pull request #21735 from jakecorrenti/inspect-conn-vals
machine: Add `ConnectionInfo` to inspect
2024-02-22 02:36:50 +00:00
openshift-merge-bot[bot]
70091d57e7
Merge pull request #21699 from vikas-goel/i21663
Comply to Kubernetes specifications for annotation size.
2024-02-21 15:00:25 +00:00
openshift-merge-bot[bot]
0d1a8c0f2b
Merge pull request #21768 from baude/zstd
zstd now default compression for podman machine
2024-02-21 14:46:38 +00:00
openshift-merge-bot[bot]
f756e5db68
Merge pull request #21747 from mheon/windows_lint
Fix Lint on Windows and enable the job
2024-02-21 00:47:04 +00:00
Vikas Goel
89b415ba37 Comply to Kubernetes specifications for annotation size.
An annotation is a pair of key-value. The key has two parts, viz. a name and an optional prefix in DNS format.

The limitations on name is 63, prefix 253 chars. The limitation on total size of all key+value pairs combined is 256KB.

https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set

Fixes: https://github.com/containers/podman/issues/21663

Signed-off-by: Vikas Goel <vikas.goel@gmail.com>
2024-02-20 14:58:35 -08:00
Brent Baude
a31e8d2a23 zstd now default compression for podman machine
given that we are moving to building our own machine images, we have
decided to use zstd compression as it is superior in speed to the
alternatives.  as such, this pr adds zstd to our machine code; and also
has to account for dealing with sparseness on darwin; which the default
zstd golang library does not.

[NO NEW TESTS NEEDED]

Signed-off-by: Brent Baude <bbaude@redhat.com>
2024-02-20 14:26:41 -06:00
openshift-merge-bot[bot]
1d4651bccc
Merge pull request #21763 from giuseppe/sparse-file
machine: add sparse file writer
2024-02-20 19:35:33 +00:00
openshift-merge-bot[bot]
d9c706ea95
Merge pull request #21694 from arixmkii/wait-for-gvproxy
Extract waitForGvProxy into shared utility function
2024-02-20 18:59:46 +00:00
Giuseppe Scrivano
0b861350c2
machine: add sparse file writer
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2024-02-20 15:57:24 +01:00
Jake Correnti
09095acf2b machine: Add ConnectionInfo to MachineConfig
Adds `ConnectionInfo()` to the `MachineConfig` and fills out
`InspectInfo` accordingly. Additionally fixes the "inspect with go format" test.

Changes `ConfigPath` to `ConfigDir` to better represent the
output.

Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
2024-02-20 09:08:11 -05:00