diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 7f21c0e397..a0893a22b9 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -11,6 +11,7 @@ - The `--mount` option to `podman create` and `podman run` now supports `dest=` as a valid alias for `destination=`. - The `podman kube play` command can now restrict container execution to specific CPU cores and specific memory nodes using the `io.podman.annotations.cpuset/$ctrname` and `io.podman.annotations.memory-nodes/$ctrname` annotations ([#26172](https://github.com/containers/podman/issues/26172)). - The `podman kube play` command now supports the `lifecycle.stopSignal` field in Pod YAML, allowing the signal used to stop containers to be specified ([#25389](https://github.com/containers/podman/issues/25389)). +- The `podman artifact` suite of commands for interacting with OCI artifacts is now available in the remote Podman client and the bindings for the REST API. - The `podman volume import` and `podman volume export` commands are now available in the remote Podman client ([#26049](https://github.com/containers/podman/issues/26409)). - The `--build-context` option to `podman build` is now supported by the remote Podman client ([#23433](https://github.com/containers/podman/issues/23433)). - The `podman volume create` command now accepts two new options, `--uid` and `--gid`, to set the UID and GID the volume will be created with. @@ -22,6 +23,7 @@ - A new command, `podman buildx inspect`, has been added to improve Docker compatibility ([#13014](https://github.com/containers/podman/issues/13014)). ### Changes +- The `podman artifact` suite of commands for interacting with OCI artifacts is now considered stable. - For users running `podman machine` VMs using the `libkrun` provider on an M3 or newer host running macOS 15+, nested virtualization is enabled by default. - When creating `podman machine` VMs on Windows using the WSL v2 provider, images are now pulled as artifacts from `quay.io/podman/machine-os`, matching the behavior of other VM providers. - Signal forwarding done by the `--sig-proxy` option to `podman run` and `podman attach` is now more robust to races and no longer forwards the `SIGSTOP` signal. @@ -58,7 +60,11 @@ - A full set of API endpoints for interacting with artifacts has been added, including inspecting artifacts (`GET /libpod/artifacts/{name}/json`), listing all artifacts (`GET /libpod/artifacts/json`), pulling an artifact (`POST /libpod/artifacts/pull`), removing an artifact (`DELETE /libpod/artifacts/{name}`), adding an artifact (or appending to an existing artifact) from a tar file in the request body (`POST /libpod/artifacts/add`), pushing an artifact to a registry (`/libpod/artifacts/{name}/push`), and retrieving the contents of an artifact (`GET /libpod/artifacts/{name}/extract`). - The Compat Create endpoint for Containers now accepts a new parameter, `HostConfig.CgroupnsMode`, to specify the cgroup namespace mode of the created container. - The Compat Create endpoint for Containers now respects the `base_hosts_file` option in `containers.conf`. -- The Compat Info endpoint now returns a new field, `DefaultAddressPools`. +- The Compat System Info endpoint now returns a new field, `DefaultAddressPools`. +- The Compat System DF endpoint has removed the deprecated `BuilderSize` field. +- The Compat Ping endpoint now sets `Builder-Version` to `1` to match Docker installs that do not include BuildKit. +- The Compat List endpoint for Images now returns the `shared-size` field unconditionally, even if the `shared-size` query parameter was not set to true. If not requested through query parameter, it is set to `-1`. This improves Docker API compatibility. +- The Compat Inspect endpoint for Images now no longer returns the deprecated `VirtualSize` field when Docker API version 1.44 and up is requested. - Fixed a bug where the Compat Delete API for Containers would remove running containers when the `FORCE` parameter was set to true; Docker only removes stopped containers ([#25871](https://github.com/containers/podman/issues/25871)). - Fixed a bug where the Compat List and Compat Inspect endpoints for Containers returned container status using Podman statuses instead of converting to Docker-compatible statuses ([#17728](https://github.com/containers/podman/issues/17728)). - Fixed a bug where healthchecks that exceeded their timeout were not properly terminated; they now receive SIGTERM, then SIGKILL after a delay, if their timeout is exceeded ([#26086](https://github.com/containers/podman/pull/26086)). @@ -67,10 +73,10 @@ ### Misc - Quadlet now no longer uses container/pod ID files when stopping containers, but instead passes the name of the container/pod directly to `podman stop`/`podman pod stop`. - When building Podman via Makefile, it will now attempt to dynamically link sqlite3 if the library and header are installed locally. This and other optimizations should result in a significant reduction in binary size relative to Podman 5.5.x. Packagers can use the `libsqlite3` build tag to force this behavior when not using the Makefile to build. -- Updated Buildah to v1.41.0 -- Updated the containers/common library to v0.64.0 -- Updated the containers/storage library to v1.59.0 -- Updated the containers/image library to v5.36.0 +- Updated Buildah to v1.41.1 +- Updated the containers/common library to v0.64.1 +- Updated the containers/storage library to v1.59.1 +- Updated the containers/image library to v5.36.1 ## 5.5.2 ### Security diff --git a/cmd/podman/artifact/add.go b/cmd/podman/artifact/add.go index 98211547ad..6049be02f5 100644 --- a/cmd/podman/artifact/add.go +++ b/cmd/podman/artifact/add.go @@ -23,20 +23,17 @@ var ( Example: `podman artifact add quay.io/myimage/myartifact:latest /tmp/foobar.txt podman artifact add --file-type text/yaml quay.io/myimage/myartifact:latest /tmp/foobar.yaml podman artifact add --append quay.io/myimage/myartifact:latest /tmp/foobar.tar.gz`, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, } ) -type artifactAddOptions struct { - ArtifactType string - Annotations []string - Append bool - FileType string +// AddOptionsWrapper wraps entities.ArtifactsAddOptions and prevents leaking +// CLI-only fields into the API types. +type AddOptionsWrapper struct { + entities.ArtifactAddOptions + AnnotationsCLI []string // CLI only } -var ( - addOpts artifactAddOptions -) +var addOpts AddOptionsWrapper func init() { registry.Commands = append(registry.Commands, registry.CliCommand{ @@ -46,34 +43,36 @@ func init() { flags := addCmd.Flags() annotationFlagName := "annotation" - flags.StringArrayVar(&addOpts.Annotations, annotationFlagName, nil, "set an `annotation` for the specified files of artifact") + flags.StringArrayVar(&addOpts.AnnotationsCLI, annotationFlagName, nil, "set an `annotation` for the specified files of artifact") _ = addCmd.RegisterFlagCompletionFunc(annotationFlagName, completion.AutocompleteNone) - addTypeFlagName := "type" - flags.StringVar(&addOpts.ArtifactType, addTypeFlagName, "", "Use type to describe an artifact") - _ = addCmd.RegisterFlagCompletionFunc(addTypeFlagName, completion.AutocompleteNone) + addMIMETypeFlagName := "type" + flags.StringVar(&addOpts.ArtifactMIMEType, addMIMETypeFlagName, "", "Use type to describe an artifact") + _ = addCmd.RegisterFlagCompletionFunc(addMIMETypeFlagName, completion.AutocompleteNone) appendFlagName := "append" flags.BoolVarP(&addOpts.Append, appendFlagName, "a", false, "Append files to an existing artifact") - fileTypeFlagName := "file-type" - flags.StringVarP(&addOpts.FileType, fileTypeFlagName, "", "", "Set file type to use for the artifact (layer)") - _ = addCmd.RegisterFlagCompletionFunc(fileTypeFlagName, completion.AutocompleteNone) + fileMIMETypeFlagName := "file-type" + flags.StringVarP(&addOpts.FileMIMEType, fileMIMETypeFlagName, "", "", "Set file type to use for the artifact (layer)") + _ = addCmd.RegisterFlagCompletionFunc(fileMIMETypeFlagName, completion.AutocompleteNone) } func add(cmd *cobra.Command, args []string) error { artifactName := args[0] blobs := args[1:] - opts := new(entities.ArtifactAddOptions) - annots, err := utils.ParseAnnotations(addOpts.Annotations) + annots, err := utils.ParseAnnotations(addOpts.AnnotationsCLI) if err != nil { return err } - opts.Annotations = annots - opts.ArtifactType = addOpts.ArtifactType - opts.Append = addOpts.Append - opts.FileType = addOpts.FileType + + opts := entities.ArtifactAddOptions{ + Annotations: annots, + ArtifactMIMEType: addOpts.ArtifactMIMEType, + Append: addOpts.Append, + FileMIMEType: addOpts.FileMIMEType, + } artifactBlobs := make([]entities.ArtifactBlob, 0, len(blobs)) diff --git a/cmd/podman/artifact/artifact.go b/cmd/podman/artifact/artifact.go index 2b05fd8f12..bae8187530 100644 --- a/cmd/podman/artifact/artifact.go +++ b/cmd/podman/artifact/artifact.go @@ -6,16 +6,13 @@ import ( "github.com/spf13/cobra" ) -var ( - // Command: podman _artifact_ - artifactCmd = &cobra.Command{ - Use: "artifact", - Short: "Manage OCI artifacts", - Long: "Manage OCI artifacts", - RunE: validate.SubCommandExists, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, - } -) +// Command: podman _artifact_ +var artifactCmd = &cobra.Command{ + Use: "artifact", + Short: "Manage OCI artifacts", + Long: "Manage OCI artifacts", + RunE: validate.SubCommandExists, +} func init() { registry.Commands = append(registry.Commands, registry.CliCommand{ diff --git a/cmd/podman/artifact/extract.go b/cmd/podman/artifact/extract.go index ed17e085b3..0e488aa67e 100644 --- a/cmd/podman/artifact/extract.go +++ b/cmd/podman/artifact/extract.go @@ -18,7 +18,6 @@ var ( ValidArgsFunction: common.AutocompleteArtifactAdd, Example: `podman artifact Extract quay.io/myimage/myartifact:latest /tmp/foobar.txt podman artifact Extract quay.io/myimage/myartifact:latest /home/paul/mydir`, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, } ) @@ -43,7 +42,7 @@ func init() { } func extract(cmd *cobra.Command, args []string) error { - err := registry.ImageEngine().ArtifactExtract(registry.Context(), args[0], args[1], &extractOpts) + err := registry.ImageEngine().ArtifactExtract(registry.Context(), args[0], args[1], extractOpts) if err != nil { return err } diff --git a/cmd/podman/artifact/inspect.go b/cmd/podman/artifact/inspect.go index 93866715fb..0573e149ee 100644 --- a/cmd/podman/artifact/inspect.go +++ b/cmd/podman/artifact/inspect.go @@ -17,7 +17,6 @@ var ( Args: cobra.MinimumNArgs(1), ValidArgsFunction: common.AutocompleteArtifacts, Example: `podman artifact inspect quay.io/myimage/myartifact:latest`, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, } ) diff --git a/cmd/podman/artifact/list.go b/cmd/podman/artifact/list.go index 69e41bb88d..fa07022816 100644 --- a/cmd/podman/artifact/list.go +++ b/cmd/podman/artifact/list.go @@ -25,7 +25,6 @@ var ( Args: validate.NoArgs, ValidArgsFunction: completion.AutocompleteNone, Example: `podman artifact ls`, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, } listFlag = listFlagType{} ) diff --git a/cmd/podman/artifact/pull.go b/cmd/podman/artifact/pull.go index 68db8773e2..2cb40e33d2 100644 --- a/cmd/podman/artifact/pull.go +++ b/cmd/podman/artifact/pull.go @@ -36,7 +36,6 @@ var ( Args: cobra.ExactArgs(1), ValidArgsFunction: common.AutocompleteArtifacts, Example: `podman artifact pull quay.io/myimage/myartifact:latest`, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, } ) diff --git a/cmd/podman/artifact/push.go b/cmd/podman/artifact/push.go index 4bcf50d2d0..ac76df463d 100644 --- a/cmd/podman/artifact/push.go +++ b/cmd/podman/artifact/push.go @@ -40,7 +40,6 @@ var ( Args: cobra.ExactArgs(1), ValidArgsFunction: common.AutocompleteArtifacts, Example: `podman artifact push quay.io/myimage/myartifact:latest`, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, } ) diff --git a/cmd/podman/artifact/rm.go b/cmd/podman/artifact/rm.go index 75fdc2f878..5d939af753 100644 --- a/cmd/podman/artifact/rm.go +++ b/cmd/podman/artifact/rm.go @@ -23,7 +23,6 @@ var ( ValidArgsFunction: common.AutocompleteArtifacts, Example: `podman artifact rm quay.io/myimage/myartifact:latest podman artifact rm -a`, - Annotations: map[string]string{registry.EngineMode: registry.ABIMode}, } rmOptions = entities.ArtifactRemoveOptions{} diff --git a/pkg/api/handlers/compat/images.go b/pkg/api/handlers/compat/images.go index 45b6c042c3..e9fdab590d 100644 --- a/pkg/api/handlers/compat/images.go +++ b/pkg/api/handlers/compat/images.go @@ -19,6 +19,7 @@ import ( "github.com/containers/podman/v5/libpod" "github.com/containers/podman/v5/pkg/api/handlers" "github.com/containers/podman/v5/pkg/api/handlers/utils" + "github.com/containers/podman/v5/pkg/api/handlers/utils/apiutil" api "github.com/containers/podman/v5/pkg/api/types" "github.com/containers/podman/v5/pkg/auth" "github.com/containers/podman/v5/pkg/domain/entities" @@ -340,7 +341,7 @@ func GetImage(w http.ResponseWriter, r *http.Request) { utils.Error(w, http.StatusNotFound, fmt.Errorf("failed to find image %s: %s", name, errMsg)) return } - inspect, err := imageDataToImageInspect(r.Context(), newImage) + inspect, err := imageDataToImageInspect(r.Context(), newImage, r) if err != nil { utils.Error(w, http.StatusInternalServerError, fmt.Errorf("failed to convert ImageData to ImageInspect '%s': %w", name, err)) return @@ -348,7 +349,7 @@ func GetImage(w http.ResponseWriter, r *http.Request) { utils.WriteResponse(w, http.StatusOK, inspect) } -func imageDataToImageInspect(ctx context.Context, l *libimage.Image) (*handlers.ImageInspect, error) { +func imageDataToImageInspect(ctx context.Context, l *libimage.Image, r *http.Request) (*handlers.ImageInspect, error) { options := &libimage.InspectOptions{WithParent: true, WithSize: true} info, err := l.Inspect(ctx, options) if err != nil { @@ -407,8 +408,13 @@ func imageDataToImageInspect(ctx context.Context, l *libimage.Image) (*handlers. RootFS: rootfs, Size: info.Size, Variant: "", - VirtualSize: info.VirtualSize, } + + if _, err := apiutil.SupportedVersion(r, "<1.44.0"); err == nil { + //nolint:staticcheck // Deprecated field + dockerImageInspect.VirtualSize = info.VirtualSize + } + return &handlers.ImageInspect{InspectResponse: dockerImageInspect}, nil } @@ -416,9 +422,10 @@ func GetImages(w http.ResponseWriter, r *http.Request) { decoder := utils.GetDecoder(r) runtime := r.Context().Value(api.RuntimeKey).(*libpod.Runtime) query := struct { - All bool - Digests bool - Filter string // Docker 1.24 compatibility + All bool + Digests bool + Filter string // Docker 1.24 compatibility + SharedSize bool `schema:"shared-size"` // Docker 1.42 compatibility }{ // This is where you can override the golang default value for one of fields } @@ -468,6 +475,26 @@ func GetImages(w http.ResponseWriter, r *http.Request) { // docker adds sha256: in front of the ID for _, s := range summaries { s.ID = "sha256:" + s.ID + // Ensure RepoTags and RepoDigests are empty arrays instead of null for Docker compatibility + // as per https://docs.docker.com/reference/api/engine/version-history/#v143-api-changes + // Relates to https://issues.redhat.com/browse/RUN-2699 + if s.RepoTags == nil { + s.RepoTags = []string{} + } + if s.RepoDigests == nil { + s.RepoDigests = []string{} + } + // Docker 1.42 sets SharedSize to -1 if ont passed explicitly + if !query.SharedSize { + s.SharedSize = -1 + } + // VirtualSize is deprecated in version 1.43 and removed in version 1.44 + // See https://docs.docker.com/reference/api/engine/version-history/#v143-api-changes + if _, err := apiutil.SupportedVersion(r, "<1.44.0"); err == nil { + s.VirtualSize = s.Size + } else { + s.VirtualSize = 0 + } } } utils.WriteResponse(w, http.StatusOK, summaries) diff --git a/pkg/api/handlers/compat/ping.go b/pkg/api/handlers/compat/ping.go index 2601788143..82b2fb2b5e 100644 --- a/pkg/api/handlers/compat/ping.go +++ b/pkg/api/handlers/compat/ping.go @@ -18,7 +18,8 @@ import ( func Ping(w http.ResponseWriter, r *http.Request) { // Note: API-Version and Libpod-API-Version are set in handler_api.go w.Header().Set("BuildKit-Version", "") - w.Header().Set("Builder-Version", "") + // Docker uses Builder-Version 1 for classic builder and 2 for BuildKit + w.Header().Set("Builder-Version", "1") w.Header().Set("Docker-Experimental", "true") w.Header().Set("Cache-Control", "no-cache") w.Header().Set("OSType", runtime.GOOS) diff --git a/pkg/api/handlers/compat/system.go b/pkg/api/handlers/compat/system.go index ac926d6a68..b8392725e6 100644 --- a/pkg/api/handlers/compat/system.go +++ b/pkg/api/handlers/compat/system.go @@ -9,6 +9,7 @@ import ( "github.com/containers/podman/v5/libpod" "github.com/containers/podman/v5/pkg/api/handlers" "github.com/containers/podman/v5/pkg/api/handlers/utils" + "github.com/containers/podman/v5/pkg/api/handlers/utils/apiutil" api "github.com/containers/podman/v5/pkg/api/types" "github.com/containers/podman/v5/pkg/domain/entities" "github.com/containers/podman/v5/pkg/domain/infra/abi" @@ -41,8 +42,12 @@ func GetDiskUsage(w http.ResponseWriter, r *http.Request) { RepoTags: []string{o.Tag}, SharedSize: o.SharedSize, Size: o.Size, - VirtualSize: o.Size - o.UniqueSize, } + + if _, err := apiutil.SupportedVersion(r, "<1.44.0"); err == nil { + t.VirtualSize = o.Size - o.UniqueSize //nolint:staticcheck // Deprecated field + } + imgs[i] = &t } @@ -91,11 +96,13 @@ func GetDiskUsage(w http.ResponseWriter, r *http.Request) { } utils.WriteResponse(w, http.StatusOK, handlers.DiskUsage{DiskUsage: docker.DiskUsage{ - LayersSize: df.ImagesSize, - Images: imgs, - Containers: ctnrs, - Volumes: vols, - BuildCache: []*build.CacheRecord{}, - BuilderSize: 0, + // BuilderSize was explicitly omitted since Docker deprecated its in ver 1.42 + // and suggests to use BuildCache. + // https://docs.docker.com/reference/api/engine/version-history/#v142-api-changes + LayersSize: df.ImagesSize, + Images: imgs, + Containers: ctnrs, + Volumes: vols, + BuildCache: []*build.CacheRecord{}, }}) } diff --git a/pkg/api/handlers/libpod/artifacts.go b/pkg/api/handlers/libpod/artifacts.go index 9d0d52cef1..efbdaac0d6 100644 --- a/pkg/api/handlers/libpod/artifacts.go +++ b/pkg/api/handlers/libpod/artifacts.go @@ -127,12 +127,12 @@ func PullArtifact(w http.ResponseWriter, r *http.Request) { rc := errcd.ErrorCode().Descriptor().HTTPStatusCode // Check if the returned error is 401 StatusUnauthorized indicating the request was unauthorized if rc == http.StatusUnauthorized { - utils.Error(w, http.StatusUnauthorized, errcd.ErrorCode()) + utils.Error(w, http.StatusUnauthorized, err) return } // Check if the returned error is 404 StatusNotFound indicating the artifact was not found if rc == http.StatusNotFound { - utils.Error(w, http.StatusNotFound, errcd.ErrorCode()) + utils.Error(w, http.StatusNotFound, err) return } } @@ -191,11 +191,11 @@ func AddArtifact(w http.ResponseWriter, r *http.Request) { return } - artifactAddOptions := &entities.ArtifactAddOptions{ - Append: query.Append, - Annotations: annotations, - ArtifactType: query.ArtifactMIMEType, - FileType: query.FileMIMEType, + artifactAddOptions := entities.ArtifactAddOptions{ + Append: query.Append, + Annotations: annotations, + ArtifactMIMEType: query.ArtifactMIMEType, + FileMIMEType: query.FileMIMEType, } artifactBlobs := []entities.ArtifactBlob{{ @@ -283,7 +283,7 @@ func PushArtifact(w http.ResponseWriter, r *http.Request) { rc := errcd.ErrorCode().Descriptor().HTTPStatusCode // Check if the returned error is 401 indicating the request was unauthorized if rc == 401 { - utils.Error(w, 401, errcd.ErrorCode()) + utils.Error(w, 401, err) return } } @@ -306,8 +306,9 @@ func ExtractArtifact(w http.ResponseWriter, r *http.Request) { decoder := r.Context().Value(api.DecoderKey).(*schema.Decoder) query := struct { - Digest string `schema:"digest"` - Title string `schema:"title"` + Digest string `schema:"digest"` + Title string `schema:"title"` + ExcludeTitle bool `schema:"excludetitle"` }{} if err := decoder.Decode(&query, r.URL.Query()); err != nil { @@ -316,15 +317,16 @@ func ExtractArtifact(w http.ResponseWriter, r *http.Request) { } extractOpts := entities.ArtifactExtractOptions{ - Title: query.Title, - Digest: query.Digest, + Title: query.Title, + Digest: query.Digest, + ExcludeTitle: query.ExcludeTitle, } name := utils.GetName(r) imageEngine := abi.ImageEngine{Libpod: runtime} - err := imageEngine.ArtifactExtractTarStream(r.Context(), w, name, &extractOpts) + err := imageEngine.ArtifactExtractTarStream(r.Context(), w, name, extractOpts) if err != nil { if errors.Is(err, libartifact_types.ErrArtifactNotExist) { utils.ArtifactNotFound(w, name, err) diff --git a/pkg/api/server/register_artifacts.go b/pkg/api/server/register_artifacts.go index c6d0ae2e5b..6bb4cf9154 100644 --- a/pkg/api/server/register_artifacts.go +++ b/pkg/api/server/register_artifacts.go @@ -242,9 +242,15 @@ func (s *APIServer) registerArtifactHandlers(r *mux.Router) error { // in: query // description: Only extract blob with the given digest // type: string + // - name: excludeTitle + // in: query + // description: When extracting a single Artifact blob, don't use the blob title as the filename in the tar + // type: boolean // responses: // 200: // description: Extract successful + // schema: + // type: file // 400: // $ref: "#/responses/badParamError" // 404: diff --git a/pkg/bindings/artifacts/add.go b/pkg/bindings/artifacts/add.go new file mode 100644 index 0000000000..aa497811eb --- /dev/null +++ b/pkg/bindings/artifacts/add.go @@ -0,0 +1,42 @@ +package artifacts + +import ( + "context" + "io" + "net/http" + + "github.com/containers/podman/v5/pkg/bindings" + entitiesTypes "github.com/containers/podman/v5/pkg/domain/entities/types" +) + +func Add(ctx context.Context, artifactName string, blobName string, artifactBlob io.Reader, options *AddOptions) (*entitiesTypes.ArtifactAddReport, error) { + conn, err := bindings.GetClient(ctx) + if err != nil { + return nil, err + } + + if options == nil { + options = new(AddOptions) + } + + params, err := options.ToParams() + if err != nil { + return nil, err + } + + params.Set("name", artifactName) + params.Set("fileName", blobName) + + response, err := conn.DoRequest(ctx, artifactBlob, http.MethodPost, "/artifacts/add", params, nil) + if err != nil { + return nil, err + } + defer response.Body.Close() + + var artifactAddReport entitiesTypes.ArtifactAddReport + if err := response.Process(&artifactAddReport); err != nil { + return nil, err + } + + return &artifactAddReport, nil +} diff --git a/pkg/bindings/artifacts/extract.go b/pkg/bindings/artifacts/extract.go new file mode 100644 index 0000000000..6e9e21a94c --- /dev/null +++ b/pkg/bindings/artifacts/extract.go @@ -0,0 +1,113 @@ +package artifacts + +import ( + "archive/tar" + "context" + "errors" + "fmt" + "io" + "io/fs" + "net/http" + "os" + "path/filepath" + + "github.com/containers/podman/v5/pkg/bindings" +) + +func Extract(ctx context.Context, artifactName string, target string, options *ExtractOptions) error { + conn, err := bindings.GetClient(ctx) + if err != nil { + return fmt.Errorf("getting client: %w", err) + } + + if options == nil { + options = new(ExtractOptions) + } + + // Check if target is a directory to know if we can copy more than one blob + targetIsDirectory := false + stat, err := os.Stat(target) + if err == nil { + targetIsDirectory = stat.IsDir() + } else if !errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf("stat target %q failed: %w", target, err) + } + + // If the target is not a directory, request API to return the blob without title. + // If a blob has a malicious title it will be returned from the API without it + // as the file will be written to the provided target + if !targetIsDirectory { + options.WithExcludeTitle(true) + } + + params, err := options.ToParams() + if err != nil { + return fmt.Errorf("converting options to params: %w", err) + } + + response, err := conn.DoRequest(ctx, nil, http.MethodGet, "/artifacts/%s/extract", params, nil, artifactName) + if err != nil { + return err + } + defer response.Body.Close() + + if !response.IsSuccess() { + return response.Process(nil) + } + + multipleBlobs := false + tr := tar.NewReader(response.Body) + for { + header, err := tr.Next() + if err == io.EOF { + break // End of archive + } + if err != nil { + return err + } + + if !targetIsDirectory && multipleBlobs { + return fmt.Errorf("the artifact consists of several blobs and the target %q is not a directory and neither digest or title was specified to only copy a single blob", target) + } + + // If destination isn't a file, extract to target/filename + fileTarget := target + if targetIsDirectory { + fileTarget = filepath.Join(target, header.Name) + } + + if header.Typeflag == tar.TypeReg { + err = extractFile(tr, fileTarget) + if err != nil { + return err + } + } + + // Signal that the first blob has been extracted so we can return an error if more + // than one blob are being extracted when target is not a directory. + multipleBlobs = true + } + return nil +} + +func extractFile(tr *tar.Reader, fileTarget string) (retErr error) { + outFile, err := os.Create(fileTarget) + if err != nil { + return err + } + + // Use an anonymous function to enable capturing the error from + // outFile.Close() upon returning. + defer func() { + cErr := outFile.Close() + if retErr == nil { + retErr = cErr + } + }() + + _, err = io.Copy(outFile, tr) + if err != nil { + return fmt.Errorf("failed to extract blob to %s: %w", fileTarget, err) + } + return nil +} diff --git a/pkg/bindings/artifacts/inspect.go b/pkg/bindings/artifacts/inspect.go new file mode 100644 index 0000000000..52017e02c1 --- /dev/null +++ b/pkg/bindings/artifacts/inspect.go @@ -0,0 +1,29 @@ +package artifacts + +import ( + "context" + "net/http" + + "github.com/containers/podman/v5/pkg/bindings" + "github.com/containers/podman/v5/pkg/domain/entities/types" +) + +func Inspect(ctx context.Context, nameOrID string, options *InspectOptions) (*types.ArtifactInspectReport, error) { + conn, err := bindings.GetClient(ctx) + if err != nil { + return nil, err + } + + response, err := conn.DoRequest(ctx, nil, http.MethodGet, "/artifacts/%s/json", nil, nil, nameOrID) + if err != nil { + return nil, err + } + defer response.Body.Close() + + var inspectedData types.ArtifactInspectReport + if err := response.Process(&inspectedData); err != nil { + return nil, err + } + + return &inspectedData, nil +} diff --git a/pkg/bindings/artifacts/list.go b/pkg/bindings/artifacts/list.go new file mode 100644 index 0000000000..260bfeed84 --- /dev/null +++ b/pkg/bindings/artifacts/list.go @@ -0,0 +1,30 @@ +package artifacts + +import ( + "context" + "net/http" + + "github.com/containers/podman/v5/pkg/bindings" + "github.com/containers/podman/v5/pkg/domain/entities" +) + +// List returns a list of artifacts in local storage. +func List(ctx context.Context, options *ListOptions) ([]*entities.ArtifactListReport, error) { + conn, err := bindings.GetClient(ctx) + if err != nil { + return nil, err + } + + response, err := conn.DoRequest(ctx, nil, http.MethodGet, "/artifacts/json", nil, nil) + if err != nil { + return nil, err + } + defer response.Body.Close() + + var artifactSummary []*entities.ArtifactListReport + if err := response.Process(&artifactSummary); err != nil { + return nil, err + } + + return artifactSummary, nil +} diff --git a/pkg/bindings/artifacts/pull.go b/pkg/bindings/artifacts/pull.go new file mode 100644 index 0000000000..1103e176ff --- /dev/null +++ b/pkg/bindings/artifacts/pull.go @@ -0,0 +1,52 @@ +package artifacts + +import ( + "context" + "net/http" + + imageTypes "github.com/containers/image/v5/types" + "github.com/containers/podman/v5/pkg/auth" + "github.com/containers/podman/v5/pkg/bindings" + "github.com/containers/podman/v5/pkg/domain/entities" +) + +func Pull(ctx context.Context, name string, options *PullOptions) (*entities.ArtifactPullReport, error) { + if options == nil { + options = new(PullOptions) + } + + conn, err := bindings.GetClient(ctx) + if err != nil { + return nil, err + } + + params, err := options.ToParams() + if err != nil { + return nil, err + } + params.Set("name", name) + + header, err := auth.MakeXRegistryAuthHeader( + &imageTypes.SystemContext{ + AuthFilePath: options.GetAuthfile(), + }, + options.GetUsername(), + options.GetPassword(), + ) + if err != nil { + return nil, err + } + + response, err := conn.DoRequest(ctx, nil, http.MethodPost, "/artifacts/pull", params, header) + if err != nil { + return nil, err + } + defer response.Body.Close() + + var report entities.ArtifactPullReport + if err := response.Process(&report); err != nil { + return nil, err + } + + return &report, nil +} diff --git a/pkg/bindings/artifacts/push.go b/pkg/bindings/artifacts/push.go new file mode 100644 index 0000000000..0a83f0d356 --- /dev/null +++ b/pkg/bindings/artifacts/push.go @@ -0,0 +1,45 @@ +package artifacts + +import ( + "context" + "net/http" + + imageTypes "github.com/containers/image/v5/types" + "github.com/containers/podman/v5/pkg/auth" + "github.com/containers/podman/v5/pkg/bindings" + "github.com/containers/podman/v5/pkg/domain/entities" +) + +func Push(ctx context.Context, name string, options *PushOptions) (*entities.ArtifactPushReport, error) { + if options == nil { + options = new(PushOptions) + } + + conn, err := bindings.GetClient(ctx) + if err != nil { + return nil, err + } + + params, err := options.ToParams() + if err != nil { + return nil, err + } + + header, err := auth.MakeXRegistryAuthHeader(&imageTypes.SystemContext{AuthFilePath: options.GetAuthfile()}, options.GetUsername(), options.GetPassword()) + if err != nil { + return nil, err + } + + response, err := conn.DoRequest(ctx, nil, http.MethodPost, "/artifacts/%s/push", params, header, name) + if err != nil { + return nil, err + } + defer response.Body.Close() + + var report entities.ArtifactPushReport + if err := response.Process(&report); err != nil { + return nil, err + } + + return &report, nil +} diff --git a/pkg/bindings/artifacts/remove.go b/pkg/bindings/artifacts/remove.go new file mode 100644 index 0000000000..d1eb4cc32c --- /dev/null +++ b/pkg/bindings/artifacts/remove.go @@ -0,0 +1,30 @@ +package artifacts + +import ( + "context" + "net/http" + + "github.com/containers/podman/v5/pkg/bindings" + "github.com/containers/podman/v5/pkg/domain/entities" +) + +// Remove removes an artifact from local storage. +func Remove(ctx context.Context, nameOrID string, options *RemoveOptions) (*entities.ArtifactRemoveReport, error) { + conn, err := bindings.GetClient(ctx) + if err != nil { + return nil, err + } + + response, err := conn.DoRequest(ctx, nil, http.MethodDelete, "/artifacts/%s", nil, nil, nameOrID) + if err != nil { + return nil, err + } + defer response.Body.Close() + + var artifactRemoveReport entities.ArtifactRemoveReport + if err := response.Process(&artifactRemoveReport); err != nil { + return nil, err + } + + return &artifactRemoveReport, nil +} diff --git a/pkg/bindings/artifacts/types.go b/pkg/bindings/artifacts/types.go new file mode 100644 index 0000000000..5151903b82 --- /dev/null +++ b/pkg/bindings/artifacts/types.go @@ -0,0 +1,89 @@ +package artifacts + +import "io" + +// PullOptions are optional options for pulling images +// +//go:generate go run ../generator/generator.go PullOptions +type PullOptions struct { + // Authfile is the path to the authentication file. + Authfile *string `schema:"-"` + // Password for authenticating against the registry. + Password *string `schema:"-"` + // ProgressWriter is a writer where pull progress are sent. + ProgressWriter *io.Writer `schema:"-"` + // Quiet can be specified to suppress pull progress when pulling. + Quiet *bool + // Retry number of times to retry pull in case of failure + Retry *uint + // RetryDelay between retries in case of pull failures + RetryDelay *string + // SkipTLSVerify to skip HTTPS and certificate verification. + TlsVerify *bool + // Username for authenticating against the registry. + Username *string `schema:"-"` +} + +// PushOptions are optional options for pushing images +// +//go:generate go run ../generator/generator.go PushOptions +type PushOptions struct { + // Authfile is the path to the authentication file. + Authfile *string `schema:"-"` + // Password for authenticating against the registry. + Password *string `schema:"-"` + // Quiet can be specified to suppress pull progress when pulling. + Quiet *bool + // Retry number of times to retry pull in case of failure + Retry *uint + // RetryDelay between retries in case of pull failures + RetryDelay *string + // SkipTLSVerify to skip HTTPS and certificate verification. + TlsVerify *bool + // Username for authenticating against the registry. + Username *string `schema:"-"` +} + +// RemoveOptions are optional options for removing images +// +//go:generate go run ../generator/generator.go RemoveOptions +type RemoveOptions struct { + // Remove all artifacts + All *bool +} + +// AddOptions are optional options for removing images +// +//go:generate go run ../generator/generator.go AddOptions +type AddOptions struct { + Annotations []string + ArtifactMIMEType *string + Append *bool + FileMIMEType *string +} + +// ExtractOptions +// +//go:generate go run ../generator/generator.go ExtractOptions +type ExtractOptions struct { + // Title annotation value to extract only a single blob matching that name. + // Conflicts with Digest. Optional. + Title *string + // Digest of the blob to extract. + // Conflicts with Title. Optional. + Digest *string + // ExcludeTitle option allows single blobs to be exported + // with their title/filename empty. Optional. + // Default: False + ExcludeTitle *bool +} + +// ListOptions +// +//go:generate go run ../generator/generator.go ListOptions +type ListOptions struct{} + +// InspectOptions +// +//go:generate go run ../generator/generator.go InspectOptions +type InspectOptions struct{} diff --git a/pkg/bindings/artifacts/types_add_options.go b/pkg/bindings/artifacts/types_add_options.go new file mode 100644 index 0000000000..f8088009ec --- /dev/null +++ b/pkg/bindings/artifacts/types_add_options.go @@ -0,0 +1,78 @@ +// Code generated by go generate; DO NOT EDIT. +package artifacts + +import ( + "net/url" + + "github.com/containers/podman/v5/pkg/bindings/internal/util" +) + +// Changed returns true if named field has been set +func (o *AddOptions) Changed(fieldName string) bool { + return util.Changed(o, fieldName) +} + +// ToParams formats struct fields to be passed to API service +func (o *AddOptions) ToParams() (url.Values, error) { + return util.ToParams(o) +} + +// WithAnnotations set field Annotations to given value +func (o *AddOptions) WithAnnotations(value []string) *AddOptions { + o.Annotations = value + return o +} + +// GetAnnotations returns value of field Annotations +func (o *AddOptions) GetAnnotations() []string { + if o.Annotations == nil { + var z []string + return z + } + return o.Annotations +} + +// WithArtifactMIMEType set field ArtifactMIMEType to given value +func (o *AddOptions) WithArtifactMIMEType(value string) *AddOptions { + o.ArtifactMIMEType = &value + return o +} + +// GetArtifactMIMEType returns value of field ArtifactMIMEType +func (o *AddOptions) GetArtifactMIMEType() string { + if o.ArtifactMIMEType == nil { + var z string + return z + } + return *o.ArtifactMIMEType +} + +// WithAppend set field Append to given value +func (o *AddOptions) WithAppend(value bool) *AddOptions { + o.Append = &value + return o +} + +// GetAppend returns value of field Append +func (o *AddOptions) GetAppend() bool { + if o.Append == nil { + var z bool + return z + } + return *o.Append +} + +// WithFileMIMEType set field FileMIMEType to given value +func (o *AddOptions) WithFileMIMEType(value string) *AddOptions { + o.FileMIMEType = &value + return o +} + +// GetFileMIMEType returns value of field FileMIMEType +func (o *AddOptions) GetFileMIMEType() string { + if o.FileMIMEType == nil { + var z string + return z + } + return *o.FileMIMEType +} diff --git a/pkg/bindings/artifacts/types_extract_options.go b/pkg/bindings/artifacts/types_extract_options.go new file mode 100644 index 0000000000..f8ce2ee0ee --- /dev/null +++ b/pkg/bindings/artifacts/types_extract_options.go @@ -0,0 +1,63 @@ +// Code generated by go generate; DO NOT EDIT. +package artifacts + +import ( + "net/url" + + "github.com/containers/podman/v5/pkg/bindings/internal/util" +) + +// Changed returns true if named field has been set +func (o *ExtractOptions) Changed(fieldName string) bool { + return util.Changed(o, fieldName) +} + +// ToParams formats struct fields to be passed to API service +func (o *ExtractOptions) ToParams() (url.Values, error) { + return util.ToParams(o) +} + +// WithTitle set field Title to given value +func (o *ExtractOptions) WithTitle(value string) *ExtractOptions { + o.Title = &value + return o +} + +// GetTitle returns value of field Title +func (o *ExtractOptions) GetTitle() string { + if o.Title == nil { + var z string + return z + } + return *o.Title +} + +// WithDigest set field Digest to given value +func (o *ExtractOptions) WithDigest(value string) *ExtractOptions { + o.Digest = &value + return o +} + +// GetDigest returns value of field Digest +func (o *ExtractOptions) GetDigest() string { + if o.Digest == nil { + var z string + return z + } + return *o.Digest +} + +// WithExcludeTitle set field ExcludeTitle to given value +func (o *ExtractOptions) WithExcludeTitle(value bool) *ExtractOptions { + o.ExcludeTitle = &value + return o +} + +// GetExcludeTitle returns value of field ExcludeTitle +func (o *ExtractOptions) GetExcludeTitle() bool { + if o.ExcludeTitle == nil { + var z bool + return z + } + return *o.ExcludeTitle +} diff --git a/pkg/bindings/artifacts/types_inspect_options.go b/pkg/bindings/artifacts/types_inspect_options.go new file mode 100644 index 0000000000..0589341a55 --- /dev/null +++ b/pkg/bindings/artifacts/types_inspect_options.go @@ -0,0 +1,18 @@ +// Code generated by go generate; DO NOT EDIT. +package artifacts + +import ( + "net/url" + + "github.com/containers/podman/v5/pkg/bindings/internal/util" +) + +// Changed returns true if named field has been set +func (o *InspectOptions) Changed(fieldName string) bool { + return util.Changed(o, fieldName) +} + +// ToParams formats struct fields to be passed to API service +func (o *InspectOptions) ToParams() (url.Values, error) { + return util.ToParams(o) +} diff --git a/pkg/bindings/artifacts/types_list_options.go b/pkg/bindings/artifacts/types_list_options.go new file mode 100644 index 0000000000..5881a052bd --- /dev/null +++ b/pkg/bindings/artifacts/types_list_options.go @@ -0,0 +1,18 @@ +// Code generated by go generate; DO NOT EDIT. +package artifacts + +import ( + "net/url" + + "github.com/containers/podman/v5/pkg/bindings/internal/util" +) + +// Changed returns true if named field has been set +func (o *ListOptions) Changed(fieldName string) bool { + return util.Changed(o, fieldName) +} + +// ToParams formats struct fields to be passed to API service +func (o *ListOptions) ToParams() (url.Values, error) { + return util.ToParams(o) +} diff --git a/pkg/bindings/artifacts/types_pull_options.go b/pkg/bindings/artifacts/types_pull_options.go new file mode 100644 index 0000000000..d62d84b18b --- /dev/null +++ b/pkg/bindings/artifacts/types_pull_options.go @@ -0,0 +1,139 @@ +// Code generated by go generate; DO NOT EDIT. +package artifacts + +import ( + "io" + "net/url" + + "github.com/containers/podman/v5/pkg/bindings/internal/util" +) + +// Changed returns true if named field has been set +func (o *PullOptions) Changed(fieldName string) bool { + return util.Changed(o, fieldName) +} + +// ToParams formats struct fields to be passed to API service +func (o *PullOptions) ToParams() (url.Values, error) { + return util.ToParams(o) +} + +// WithAuthfile set field Authfile to given value +func (o *PullOptions) WithAuthfile(value string) *PullOptions { + o.Authfile = &value + return o +} + +// GetAuthfile returns value of field Authfile +func (o *PullOptions) GetAuthfile() string { + if o.Authfile == nil { + var z string + return z + } + return *o.Authfile +} + +// WithPassword set field Password to given value +func (o *PullOptions) WithPassword(value string) *PullOptions { + o.Password = &value + return o +} + +// GetPassword returns value of field Password +func (o *PullOptions) GetPassword() string { + if o.Password == nil { + var z string + return z + } + return *o.Password +} + +// WithProgressWriter set field ProgressWriter to given value +func (o *PullOptions) WithProgressWriter(value io.Writer) *PullOptions { + o.ProgressWriter = &value + return o +} + +// GetProgressWriter returns value of field ProgressWriter +func (o *PullOptions) GetProgressWriter() io.Writer { + if o.ProgressWriter == nil { + var z io.Writer + return z + } + return *o.ProgressWriter +} + +// WithQuiet set field Quiet to given value +func (o *PullOptions) WithQuiet(value bool) *PullOptions { + o.Quiet = &value + return o +} + +// GetQuiet returns value of field Quiet +func (o *PullOptions) GetQuiet() bool { + if o.Quiet == nil { + var z bool + return z + } + return *o.Quiet +} + +// WithRetry set field Retry to given value +func (o *PullOptions) WithRetry(value uint) *PullOptions { + o.Retry = &value + return o +} + +// GetRetry returns value of field Retry +func (o *PullOptions) GetRetry() uint { + if o.Retry == nil { + var z uint + return z + } + return *o.Retry +} + +// WithRetryDelay set field RetryDelay to given value +func (o *PullOptions) WithRetryDelay(value string) *PullOptions { + o.RetryDelay = &value + return o +} + +// GetRetryDelay returns value of field RetryDelay +func (o *PullOptions) GetRetryDelay() string { + if o.RetryDelay == nil { + var z string + return z + } + return *o.RetryDelay +} + +// WithTlsVerify set field TlsVerify to given value +func (o *PullOptions) WithTlsVerify(value bool) *PullOptions { + o.TlsVerify = &value + return o +} + +// GetTlsVerify returns value of field TlsVerify +func (o *PullOptions) GetTlsVerify() bool { + if o.TlsVerify == nil { + var z bool + return z + } + return *o.TlsVerify +} + +// WithUsername set field Username to given value +func (o *PullOptions) WithUsername(value string) *PullOptions { + o.Username = &value + return o +} + +// GetUsername returns value of field Username +func (o *PullOptions) GetUsername() string { + if o.Username == nil { + var z string + return z + } + return *o.Username +} diff --git a/pkg/bindings/artifacts/types_push_options.go b/pkg/bindings/artifacts/types_push_options.go new file mode 100644 index 0000000000..345f6baeb7 --- /dev/null +++ b/pkg/bindings/artifacts/types_push_options.go @@ -0,0 +1,123 @@ +// Code generated by go generate; DO NOT EDIT. +package artifacts + +import ( + "net/url" + + "github.com/containers/podman/v5/pkg/bindings/internal/util" +) + +// Changed returns true if named field has been set +func (o *PushOptions) Changed(fieldName string) bool { + return util.Changed(o, fieldName) +} + +// ToParams formats struct fields to be passed to API service +func (o *PushOptions) ToParams() (url.Values, error) { + return util.ToParams(o) +} + +// WithAuthfile set field Authfile to given value +func (o *PushOptions) WithAuthfile(value string) *PushOptions { + o.Authfile = &value + return o +} + +// GetAuthfile returns value of field Authfile +func (o *PushOptions) GetAuthfile() string { + if o.Authfile == nil { + var z string + return z + } + return *o.Authfile +} + +// WithPassword set field Password to given value +func (o *PushOptions) WithPassword(value string) *PushOptions { + o.Password = &value + return o +} + +// GetPassword returns value of field Password +func (o *PushOptions) GetPassword() string { + if o.Password == nil { + var z string + return z + } + return *o.Password +} + +// WithQuiet set field Quiet to given value +func (o *PushOptions) WithQuiet(value bool) *PushOptions { + o.Quiet = &value + return o +} + +// GetQuiet returns value of field Quiet +func (o *PushOptions) GetQuiet() bool { + if o.Quiet == nil { + var z bool + return z + } + return *o.Quiet +} + +// WithRetry set field Retry to given value +func (o *PushOptions) WithRetry(value uint) *PushOptions { + o.Retry = &value + return o +} + +// GetRetry returns value of field Retry +func (o *PushOptions) GetRetry() uint { + if o.Retry == nil { + var z uint + return z + } + return *o.Retry +} + +// WithRetryDelay set field RetryDelay to given value +func (o *PushOptions) WithRetryDelay(value string) *PushOptions { + o.RetryDelay = &value + return o +} + +// GetRetryDelay returns value of field RetryDelay +func (o *PushOptions) GetRetryDelay() string { + if o.RetryDelay == nil { + var z string + return z + } + return *o.RetryDelay +} + +// WithTlsVerify set field TlsVerify to given value +func (o *PushOptions) WithTlsVerify(value bool) *PushOptions { + o.TlsVerify = &value + return o +} + +// GetTlsVerify returns value of field TlsVerify +func (o *PushOptions) GetTlsVerify() bool { + if o.TlsVerify == nil { + var z bool + return z + } + return *o.TlsVerify +} + +// WithUsername set field Username to given value +func (o *PushOptions) WithUsername(value string) *PushOptions { + o.Username = &value + return o +} + +// GetUsername returns value of field Username +func (o *PushOptions) GetUsername() string { + if o.Username == nil { + var z string + return z + } + return *o.Username +} diff --git a/pkg/bindings/artifacts/types_remove_options.go b/pkg/bindings/artifacts/types_remove_options.go new file mode 100644 index 0000000000..91e3142d13 --- /dev/null +++ b/pkg/bindings/artifacts/types_remove_options.go @@ -0,0 +1,33 @@ +// Code generated by go generate; DO NOT EDIT. +package artifacts + +import ( + "net/url" + + "github.com/containers/podman/v5/pkg/bindings/internal/util" +) + +// Changed returns true if named field has been set +func (o *RemoveOptions) Changed(fieldName string) bool { + return util.Changed(o, fieldName) +} + +// ToParams formats struct fields to be passed to API service +func (o *RemoveOptions) ToParams() (url.Values, error) { + return util.ToParams(o) +} + +// WithAll set field All to given value +func (o *RemoveOptions) WithAll(value bool) *RemoveOptions { + o.All = &value + return o +} + +// GetAll returns value of field All +func (o *RemoveOptions) GetAll() bool { + if o.All == nil { + var z bool + return z + } + return *o.All +} diff --git a/pkg/domain/entities/artifact.go b/pkg/domain/entities/artifact.go index bbf770f041..c8251cb70d 100644 --- a/pkg/domain/entities/artifact.go +++ b/pkg/domain/entities/artifact.go @@ -5,18 +5,18 @@ import ( "github.com/containers/image/v5/types" encconfig "github.com/containers/ocicrypt/config" - entityTypes "github.com/containers/podman/v5/pkg/domain/entities/types" - "github.com/containers/podman/v5/pkg/libartifact" - "github.com/opencontainers/go-digest" + entitiesTypes "github.com/containers/podman/v5/pkg/domain/entities/types" ) type ArtifactAddOptions struct { - Annotations map[string]string - ArtifactType string - Append bool - FileType string + Annotations map[string]string + ArtifactMIMEType string + Append bool + FileMIMEType string } +type ArtifactAddReport = entitiesTypes.ArtifactAddReport + type ArtifactExtractOptions struct { // Title annotation value to extract only a single blob matching that name. // Conflicts with Digest. Optional. @@ -24,13 +24,13 @@ type ArtifactExtractOptions struct { // Digest of the blob to extract. // Conflicts with Title. Optional. Digest string + // ExcludeTitle option allows single blobs to be exported + // with their title/filename empty. Optional. + // Default: False + ExcludeTitle bool } -type ArtifactBlob struct { - BlobReader io.Reader - BlobFilePath string - FileName string -} +type ArtifactBlob = entitiesTypes.ArtifactBlob type ArtifactInspectOptions struct { // Note: Remote is not currently implemented but will be used for @@ -38,8 +38,9 @@ type ArtifactInspectOptions struct { Remote bool } -type ArtifactListOptions struct { -} +type ArtifactListOptions struct{} + +type ArtifactListReport = entitiesTypes.ArtifactListReport type ArtifactPullOptions struct { // containers-auth.json(5) file to use when authenticating against @@ -79,6 +80,8 @@ type ArtifactPullOptions struct { IdentityToken string `json:"identitytoken,omitempty"` } +type ArtifactPullReport = entitiesTypes.ArtifactPullReport + type ArtifactPushOptions struct { ImagePushOptions CredentialsCLI string @@ -90,29 +93,13 @@ type ArtifactPushOptions struct { TLSVerifyCLI bool // CLI only } +type ArtifactPushReport = entitiesTypes.ArtifactPushReport + type ArtifactRemoveOptions struct { // Remove all artifacts All bool } -type ArtifactPullReport struct { - ArtifactDigest *digest.Digest -} +type ArtifactRemoveReport = entitiesTypes.ArtifactRemoveReport -type ArtifactPushReport struct { - ArtifactDigest *digest.Digest -} - -type ArtifactInspectReport = entityTypes.ArtifactInspectReport - -type ArtifactListReport struct { - *libartifact.Artifact -} - -type ArtifactAddReport struct { - ArtifactDigest *digest.Digest -} - -type ArtifactRemoveReport struct { - ArtifactDigests []*digest.Digest -} +type ArtifactInspectReport = entitiesTypes.ArtifactInspectReport diff --git a/pkg/domain/entities/engine_image.go b/pkg/domain/entities/engine_image.go index 366da29587..5ecd3fd39d 100644 --- a/pkg/domain/entities/engine_image.go +++ b/pkg/domain/entities/engine_image.go @@ -10,9 +10,9 @@ import ( ) type ImageEngine interface { //nolint:interfacebloat - ArtifactAdd(ctx context.Context, name string, artifactBlobs []ArtifactBlob, opts *ArtifactAddOptions) (*ArtifactAddReport, error) - ArtifactExtract(ctx context.Context, name string, target string, opts *ArtifactExtractOptions) error - ArtifactExtractTarStream(ctx context.Context, w io.Writer, name string, opts *ArtifactExtractOptions) error + ArtifactAdd(ctx context.Context, name string, artifactBlobs []ArtifactBlob, opts ArtifactAddOptions) (*ArtifactAddReport, error) + ArtifactExtract(ctx context.Context, name string, target string, opts ArtifactExtractOptions) error + ArtifactExtractTarStream(ctx context.Context, w io.Writer, name string, opts ArtifactExtractOptions) error ArtifactInspect(ctx context.Context, name string, opts ArtifactInspectOptions) (*ArtifactInspectReport, error) ArtifactList(ctx context.Context, opts ArtifactListOptions) ([]*ArtifactListReport, error) ArtifactPull(ctx context.Context, name string, opts ArtifactPullOptions) (*ArtifactPullReport, error) diff --git a/pkg/domain/entities/types/artifacts.go b/pkg/domain/entities/types/artifacts.go index 19e42f6d6f..37d1e038f5 100644 --- a/pkg/domain/entities/types/artifacts.go +++ b/pkg/domain/entities/types/artifacts.go @@ -1,8 +1,39 @@ package types -import "github.com/containers/podman/v5/pkg/libartifact" +import ( + "io" + + "github.com/containers/podman/v5/pkg/libartifact" + "github.com/opencontainers/go-digest" +) type ArtifactInspectReport struct { *libartifact.Artifact Digest string } + +type ArtifactBlob struct { + BlobReader io.Reader + BlobFilePath string + FileName string +} + +type ArtifactAddReport struct { + ArtifactDigest *digest.Digest +} + +type ArtifactRemoveReport struct { + ArtifactDigests []*digest.Digest +} + +type ArtifactListReport struct { + *libartifact.Artifact +} + +type ArtifactPushReport struct { + ArtifactDigest *digest.Digest +} + +type ArtifactPullReport struct { + ArtifactDigest *digest.Digest +} diff --git a/pkg/domain/entities/types/images.go b/pkg/domain/entities/types/images.go index 05db387bff..255a813fc9 100644 --- a/pkg/domain/entities/types/images.go +++ b/pkg/domain/entities/types/images.go @@ -16,7 +16,7 @@ type ImageSummary struct { Created int64 Size int64 SharedSize int - VirtualSize int64 + VirtualSize int64 `json:",omitempty"` Labels map[string]string Containers int ReadOnly bool `json:",omitempty"` diff --git a/pkg/domain/infra/abi/artifact.go b/pkg/domain/infra/abi/artifact.go index 366a896556..a32da1093e 100644 --- a/pkg/domain/infra/abi/artifact.go +++ b/pkg/domain/infra/abi/artifact.go @@ -197,21 +197,17 @@ func (ir *ImageEngine) ArtifactPush(ctx context.Context, name string, opts entit }, nil } -func (ir *ImageEngine) ArtifactAdd(ctx context.Context, name string, artifactBlobs []entities.ArtifactBlob, opts *entities.ArtifactAddOptions) (*entities.ArtifactAddReport, error) { +func (ir *ImageEngine) ArtifactAdd(ctx context.Context, name string, artifactBlobs []entities.ArtifactBlob, opts entities.ArtifactAddOptions) (*entities.ArtifactAddReport, error) { artStore, err := ir.Libpod.ArtifactStore() if err != nil { return nil, err } - if opts.Annotations == nil { - opts.Annotations = make(map[string]string) - } - addOptions := types.AddOptions{ - Annotations: opts.Annotations, - ArtifactType: opts.ArtifactType, - Append: opts.Append, - FileType: opts.FileType, + Annotations: opts.Annotations, + ArtifactMIMEType: opts.ArtifactMIMEType, + Append: opts.Append, + FileMIMEType: opts.FileMIMEType, } artifactDigest, err := artStore.Add(ctx, name, artifactBlobs, &addOptions) @@ -223,35 +219,33 @@ func (ir *ImageEngine) ArtifactAdd(ctx context.Context, name string, artifactBlo }, nil } -func (ir *ImageEngine) ArtifactExtract(ctx context.Context, name string, target string, opts *entities.ArtifactExtractOptions) error { +func (ir *ImageEngine) ArtifactExtract(ctx context.Context, name string, target string, opts entities.ArtifactExtractOptions) error { artStore, err := ir.Libpod.ArtifactStore() if err != nil { return err } - extractOpt := &types.ExtractOptions{ + extractOpt := types.ExtractOptions{ FilterBlobOptions: types.FilterBlobOptions{ Digest: opts.Digest, Title: opts.Title, }, } - return artStore.Extract(ctx, name, target, extractOpt) + return artStore.Extract(ctx, name, target, &extractOpt) } -func (ir *ImageEngine) ArtifactExtractTarStream(ctx context.Context, w io.Writer, name string, opts *entities.ArtifactExtractOptions) error { - if opts == nil { - opts = &entities.ArtifactExtractOptions{} - } +func (ir *ImageEngine) ArtifactExtractTarStream(ctx context.Context, w io.Writer, name string, opts entities.ArtifactExtractOptions) error { artStore, err := ir.Libpod.ArtifactStore() if err != nil { return err } - extractOpt := &types.ExtractOptions{ + extractOpt := types.ExtractOptions{ FilterBlobOptions: types.FilterBlobOptions{ Digest: opts.Digest, Title: opts.Title, }, + ExcludeTitle: opts.ExcludeTitle, } - return artStore.ExtractTarStream(ctx, w, name, extractOpt) + return artStore.ExtractTarStream(ctx, w, name, &extractOpt) } diff --git a/pkg/domain/infra/tunnel/artifact.go b/pkg/domain/infra/tunnel/artifact.go index 55af28fde0..da98f7f9e0 100644 --- a/pkg/domain/infra/tunnel/artifact.go +++ b/pkg/domain/infra/tunnel/artifact.go @@ -2,42 +2,119 @@ package tunnel import ( "context" + "errors" "fmt" "io" + "os" + "github.com/containers/image/v5/types" + "github.com/containers/podman/v5/pkg/bindings/artifacts" "github.com/containers/podman/v5/pkg/domain/entities" ) -// TODO For now, no remote support has been added. We need the API to firm up first. +func (ir *ImageEngine) ArtifactExtract(_ context.Context, name string, target string, opts entities.ArtifactExtractOptions) error { + options := artifacts.ExtractOptions{ + Digest: &opts.Digest, + Title: &opts.Title, + ExcludeTitle: &opts.ExcludeTitle, + } -func (ir *ImageEngine) ArtifactExtract(ctx context.Context, name string, target string, opts *entities.ArtifactExtractOptions) error { + return artifacts.Extract(ir.ClientCtx, name, target, &options) +} + +func (ir *ImageEngine) ArtifactExtractTarStream(_ context.Context, w io.Writer, name string, opts entities.ArtifactExtractOptions) error { return fmt.Errorf("not implemented") } -func (ir *ImageEngine) ArtifactExtractTarStream(ctx context.Context, w io.Writer, name string, opts *entities.ArtifactExtractOptions) error { - return fmt.Errorf("not implemented") +func (ir *ImageEngine) ArtifactInspect(_ context.Context, name string, opts entities.ArtifactInspectOptions) (*entities.ArtifactInspectReport, error) { + return artifacts.Inspect(ir.ClientCtx, name, &artifacts.InspectOptions{}) } -func (ir *ImageEngine) ArtifactInspect(ctx context.Context, name string, opts entities.ArtifactInspectOptions) (*entities.ArtifactInspectReport, error) { - return nil, fmt.Errorf("not implemented") +func (ir *ImageEngine) ArtifactList(_ context.Context, opts entities.ArtifactListOptions) ([]*entities.ArtifactListReport, error) { + return artifacts.List(ir.ClientCtx, &artifacts.ListOptions{}) } -func (ir *ImageEngine) ArtifactList(ctx context.Context, opts entities.ArtifactListOptions) ([]*entities.ArtifactListReport, error) { - return nil, fmt.Errorf("not implemented") +func (ir *ImageEngine) ArtifactPull(_ context.Context, name string, opts entities.ArtifactPullOptions) (*entities.ArtifactPullReport, error) { + options := artifacts.PullOptions{ + Username: &opts.Username, + Password: &opts.Password, + Quiet: &opts.Quiet, + RetryDelay: &opts.RetryDelay, + Retry: opts.MaxRetries, + } + + switch opts.InsecureSkipTLSVerify { + case types.OptionalBoolTrue: + options.WithTlsVerify(false) + case types.OptionalBoolFalse: + options.WithTlsVerify(true) + } + + return artifacts.Pull(ir.ClientCtx, name, &options) } -func (ir *ImageEngine) ArtifactPull(ctx context.Context, name string, opts entities.ArtifactPullOptions) (*entities.ArtifactPullReport, error) { - return nil, fmt.Errorf("not implemented") +func (ir *ImageEngine) ArtifactRm(_ context.Context, name string, opts entities.ArtifactRemoveOptions) (*entities.ArtifactRemoveReport, error) { + if opts.All { + // Note: This will be added when artifacts remove all endpoint is implemented + return nil, fmt.Errorf("not implemented") + } + return artifacts.Remove(ir.ClientCtx, name, &artifacts.RemoveOptions{}) } -func (ir *ImageEngine) ArtifactRm(ctx context.Context, name string, opts entities.ArtifactRemoveOptions) (*entities.ArtifactRemoveReport, error) { - return nil, fmt.Errorf("not implemented") +func (ir *ImageEngine) ArtifactPush(_ context.Context, name string, opts entities.ArtifactPushOptions) (*entities.ArtifactPushReport, error) { + options := artifacts.PushOptions{ + Username: &opts.Username, + Password: &opts.Password, + Quiet: &opts.Quiet, + RetryDelay: &opts.RetryDelay, + Retry: opts.Retry, + } + + switch opts.SkipTLSVerify { + case types.OptionalBoolTrue: + options.WithTlsVerify(false) + case types.OptionalBoolFalse: + options.WithTlsVerify(true) + } + + return artifacts.Push(ir.ClientCtx, name, &options) } -func (ir *ImageEngine) ArtifactPush(ctx context.Context, name string, opts entities.ArtifactPushOptions) (*entities.ArtifactPushReport, error) { - return nil, fmt.Errorf("not implemented") -} +func (ir *ImageEngine) ArtifactAdd(_ context.Context, name string, artifactBlob []entities.ArtifactBlob, opts entities.ArtifactAddOptions) (*entities.ArtifactAddReport, error) { + var artifactAddReport *entities.ArtifactAddReport -func (ir *ImageEngine) ArtifactAdd(ctx context.Context, name string, artifactBlob []entities.ArtifactBlob, opts *entities.ArtifactAddOptions) (*entities.ArtifactAddReport, error) { - return nil, fmt.Errorf("not implemented") + options := artifacts.AddOptions{ + Append: &opts.Append, + ArtifactMIMEType: &opts.ArtifactMIMEType, + FileMIMEType: &opts.FileMIMEType, + } + + for k, v := range opts.Annotations { + options.Annotations = append(options.Annotations, k+"="+v) + } + + for i, blob := range artifactBlob { + if i > 0 { + // When adding more than 1 blob, set append true after the first + options.WithAppend(true) + } + f, err := os.Open(blob.BlobFilePath) + if err != nil { + return nil, err + } + defer f.Close() + + artifactAddReport, err = artifacts.Add(ir.ClientCtx, name, blob.FileName, f, &options) + if err != nil && i > 0 { + _, recoverErr := artifacts.Remove(ir.ClientCtx, name, &artifacts.RemoveOptions{}) + if recoverErr != nil { + return nil, fmt.Errorf("failed to cleanup unfinished artifact add: %w", errors.Join(err, recoverErr)) + } + return nil, err + } + if err != nil { + return nil, err + } + } + return artifactAddReport, nil } diff --git a/pkg/libartifact/store/store.go b/pkg/libartifact/store/store.go index 545bc2f998..2f3b803d6b 100644 --- a/pkg/libartifact/store/store.go +++ b/pkg/libartifact/store/store.go @@ -217,8 +217,8 @@ func (as ArtifactStore) Add(ctx context.Context, dest string, artifactBlobs []en return nil, ErrEmptyArtifactName } - if options.Append && len(options.ArtifactType) > 0 { - return nil, errors.New("append option is not compatible with ArtifactType option") + if options.Append && len(options.ArtifactMIMEType) > 0 { + return nil, errors.New("append option is not compatible with type option") } // currently we don't allow override of the filename ; if a user requirement emerges, @@ -256,7 +256,7 @@ func (as ArtifactStore) Add(ctx context.Context, dest string, artifactBlobs []en artifactManifest = specV1.Manifest{ Versioned: specs.Versioned{SchemaVersion: ManifestSchemaVersion}, MediaType: specV1.MediaTypeImageManifest, - ArtifactType: options.ArtifactType, + ArtifactType: options.ArtifactMIMEType, // TODO This should probably be configurable once the CLI is capable Config: specV1.DescriptorEmptyJSON, Layers: make([]specV1.Descriptor, 0), @@ -314,13 +314,13 @@ func (as ArtifactStore) Add(ctx context.Context, dest string, artifactBlobs []en annotations[specV1.AnnotationTitle] = artifactBlob.FileName newLayer := specV1.Descriptor{ - MediaType: options.FileType, + MediaType: options.FileMIMEType, Annotations: annotations, } // If we did not receive an override for the layer's mediatype, use // detection to determine it. - if options.FileType == "" { + if options.FileMIMEType == "" { artifactBlob.BlobReader, newLayer.MediaType, err = determineBlobMIMEType(artifactBlob) if err != nil { return nil, err @@ -449,6 +449,7 @@ func (as ArtifactStore) BlobMountPaths(ctx context.Context, nameOrDigest string, if err != nil { return nil, err } + path, err := layout.GetLocalBlobPath(ctx, imgSrc, digest) if err != nil { return nil, err @@ -466,6 +467,7 @@ func (as ArtifactStore) BlobMountPaths(ctx context.Context, nameOrDigest string, if err != nil { return nil, err } + path, err := layout.GetLocalBlobPath(ctx, imgSrc, l.Digest) if err != nil { return nil, err @@ -525,6 +527,7 @@ func (as ArtifactStore) Extract(ctx context.Context, nameOrDigest string, target if err != nil { return err } + return copyTrustedImageBlobToFile(ctx, imgSrc, digest, filepath.Join(target, filename)) } @@ -534,6 +537,7 @@ func (as ArtifactStore) Extract(ctx context.Context, nameOrDigest string, target if err != nil { return err } + err = copyTrustedImageBlobToFile(ctx, imgSrc, l.Digest, filepath.Join(target, filename)) if err != nil { return err @@ -555,9 +559,6 @@ func (as ArtifactStore) ExtractTarStream(ctx context.Context, w io.Writer, nameO } defer imgSrc.Close() - tw := tar.NewWriter(w) - defer tw.Close() - // Return early if only a single blob is requested via title or digest if len(options.Digest) > 0 || len(options.Title) > 0 { digest, err := findDigest(arty, &options.FilterBlobOptions) @@ -569,11 +570,17 @@ func (as ArtifactStore) ExtractTarStream(ctx context.Context, w io.Writer, nameO // so we do not have to get the actual title annotation form the blob. // Passing options.Title is enough because we know it is empty when digest // is set as we only allow either one. - filename, err := generateArtifactBlobName(options.Title, digest) - if err != nil { - return err + var filename string + if !options.ExcludeTitle { + filename, err = generateArtifactBlobName(options.Title, digest) + if err != nil { + return err + } } + tw := tar.NewWriter(w) + defer tw.Close() + err = copyTrustedImageBlobToTarStream(ctx, imgSrc, digest, filename, tw) if err != nil { return err @@ -582,13 +589,40 @@ func (as ArtifactStore) ExtractTarStream(ctx context.Context, w io.Writer, nameO return nil } + artifactBlobCount := len(arty.Manifest.Layers) + + type blob struct { + name string + digest digest.Digest + } + blobs := make([]blob, 0, artifactBlobCount) + + // Gather blob details and return error on any illegal names for _, l := range arty.Manifest.Layers { title := l.Annotations[specV1.AnnotationTitle] - filename, err := generateArtifactBlobName(title, l.Digest) - if err != nil { - return err + digest := l.Digest + var name string + + if artifactBlobCount != 1 || !options.ExcludeTitle { + name, err = generateArtifactBlobName(title, digest) + if err != nil { + return err + } } - err = copyTrustedImageBlobToTarStream(ctx, imgSrc, l.Digest, filename, tw) + + blobs = append(blobs, blob{ + name: name, + digest: digest, + }) + } + + // Wrap io.Writer in a tar.Writer + tw := tar.NewWriter(w) + defer tw.Close() + + // Write each blob to tar.Writer then close + for _, b := range blobs { + err := copyTrustedImageBlobToTarStream(ctx, imgSrc, b.digest, b.name, tw) if err != nil { return err } @@ -612,7 +646,7 @@ func generateArtifactBlobName(title string, digest digest.Digest) (string, error // We must use os.IsPathSeparator() as on Windows it checks both "\\" and "/". for i := 0; i < len(filename); i++ { if os.IsPathSeparator(filename[i]) { - return "", fmt.Errorf("invalid name: %q cannot contain %c", filename, filename[i]) + return "", fmt.Errorf("invalid name: %q cannot contain %c: %w", filename, filename[i], libartTypes.ErrArtifactBlobTitleInvalid) } } return filename, nil @@ -733,9 +767,7 @@ func (as ArtifactStore) indexPath() string { // getArtifacts returns an ArtifactList based on the artifact's store. The return error and // unused opts is meant for future growth like filters, etc so the API does not change. func (as ArtifactStore) getArtifacts(ctx context.Context, _ *libartTypes.GetArtifactOptions) (libartifact.ArtifactList, error) { - var ( - al libartifact.ArtifactList - ) + var al libartifact.ArtifactList lrs, err := layout.List(as.storePath) if err != nil { diff --git a/pkg/libartifact/types/config.go b/pkg/libartifact/types/config.go index 7591ffdef9..19f516b4ec 100644 --- a/pkg/libartifact/types/config.go +++ b/pkg/libartifact/types/config.go @@ -6,13 +6,13 @@ type GetArtifactOptions struct{} // AddOptions are additional descriptors of an artifact file type AddOptions struct { - Annotations map[string]string `json:"annotations,omitempty"` - ArtifactType string `json:",omitempty"` - // append option is not compatible with ArtifactType option + Annotations map[string]string `json:"annotations,omitempty"` + ArtifactMIMEType string `json:",omitempty"` + // append option is not compatible with ArtifactMIMEType option Append bool `json:",omitempty"` // FileType describes the media type for the layer. It is an override // for the standard detection - FileType string `json:",omitempty"` + FileMIMEType string `json:",omitempty"` } // FilterBlobOptions options used to filter for a single blob in an artifact @@ -27,6 +27,10 @@ type FilterBlobOptions struct { type ExtractOptions struct { FilterBlobOptions + // ExcludeTitle option allows single blobs to be exported + // with their title/filename empty. Optional. + // Default: False + ExcludeTitle bool } type BlobMountPathOptions struct { diff --git a/pkg/libartifact/types/errors.go b/pkg/libartifact/types/errors.go index 8a9eedaa1d..681c9781e7 100644 --- a/pkg/libartifact/types/errors.go +++ b/pkg/libartifact/types/errors.go @@ -5,8 +5,9 @@ import ( ) var ( - ErrArtifactUnamed = errors.New("artifact is unnamed") - ErrArtifactNotExist = errors.New("artifact does not exist") - ErrArtifactAlreadyExists = errors.New("artifact already exists") - ErrArtifactFileExists = errors.New("file already exists in artifact") + ErrArtifactUnamed = errors.New("artifact is unnamed") + ErrArtifactNotExist = errors.New("artifact does not exist") + ErrArtifactAlreadyExists = errors.New("artifact already exists") + ErrArtifactFileExists = errors.New("file already exists in artifact") + ErrArtifactBlobTitleInvalid = errors.New("artifact blob title invalid") ) diff --git a/test/apiv2/10-images.at b/test/apiv2/10-images.at index 93492d1f14..6e25ec0287 100644 --- a/test/apiv2/10-images.at +++ b/test/apiv2/10-images.at @@ -42,6 +42,14 @@ t GET images/json?filter=$IMAGE 200 \ length=1 \ .[0].Names[0]=$IMAGE +# Test VirtualSize field is present in API v1.43 (backward compatibility) +t GET /v1.43/images/json 200 \ + .[0].VirtualSize~[0-9]\\+ + +# Test VirtualSize field is no longer present in API v1.44+ (deprecated since API v1.43) +t GET /v1.44/images/json 200 \ + .[0].VirtualSize=null + # Negative test case t GET images/json?filter=nonesuch 200 length=0 @@ -50,6 +58,14 @@ t GET images/$iid/json 200 \ .Id=sha256:$iid \ .RepoTags[0]=$IMAGE +# Test VirtualSize field is present in API v1.43 for single image inspect (backward compatibility) +t GET /v1.43/images/$iid/json 200 \ + .VirtualSize~[0-9]\\+ + +# Test VirtualSize field is no longer present in API v1.44+ for single image inspect (deprecated since API v1.43) +t GET /v1.44/images/$iid/json 200 \ + .VirtualSize=null + t POST "images/create?fromImage=alpine" 200 .error~null .status~".*Download complete.*" t POST "libpod/images/pull?reference=alpine&compatMode=true" 200 .error~null .status~".*Download complete.*" @@ -440,4 +456,26 @@ t GET "libpod/events?stream=false&since=$START" 200 \ .Actor.Attributes.name="localhost:5000/idonotexist" \ .Actor.Attributes.error~".*connection refused" +# test empty RepoTags and RepoDigests is an empty array +IIDFILE=$(mktemp) +podman image build --iidfile $IIDFILE -<