Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Produce OCI images by default #623

Merged
merged 5 commits into from
Mar 4, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/ko_apply.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ ko apply -f FILENAME [flags]
--password string Password for basic authentication to the API server (DEPRECATED)
--platform strings Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
-P, --preserve-import-paths Whether to preserve the full import path after KO_DOCKER_REPO.
--preserve-media-type If false, push images in OCI format regardless of base image format
--push Push images to KO_DOCKER_REPO (default true)
-R, --recursive Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.
--request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (DEPRECATED)
Expand Down
1 change: 1 addition & 0 deletions doc/ko_build.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ ko build IMPORTPATH... [flags]
--oci-layout-path string Path to save the OCI image layout of the built images
--platform strings Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
-P, --preserve-import-paths Whether to preserve the full import path after KO_DOCKER_REPO.
--preserve-media-type If false, push images in OCI format regardless of base image format
--push Push images to KO_DOCKER_REPO (default true)
--sbom string The SBOM media type to use (none will disable SBOM synthesis and upload, also supports: spdx, go.version-m). (default "spdx")
--tag-only Include tags but not digests in resolved image references. Useful when digests are not preserved when images are repopulated.
Expand Down
1 change: 1 addition & 0 deletions doc/ko_create.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ ko create -f FILENAME [flags]
--password string Password for basic authentication to the API server (DEPRECATED)
--platform strings Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
-P, --preserve-import-paths Whether to preserve the full import path after KO_DOCKER_REPO.
--preserve-media-type If false, push images in OCI format regardless of base image format
--push Push images to KO_DOCKER_REPO (default true)
-R, --recursive Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.
--request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (DEPRECATED)
Expand Down
1 change: 1 addition & 0 deletions doc/ko_resolve.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ ko resolve -f FILENAME [flags]
--oci-layout-path string Path to save the OCI image layout of the built images
--platform strings Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
-P, --preserve-import-paths Whether to preserve the full import path after KO_DOCKER_REPO.
--preserve-media-type If false, push images in OCI format regardless of base image format
--push Push images to KO_DOCKER_REPO (default true)
-R, --recursive Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.
--sbom string The SBOM media type to use (none will disable SBOM synthesis and upload, also supports: spdx, go.version-m). (default "spdx")
Expand Down
1 change: 1 addition & 0 deletions doc/ko_run.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ ko run IMPORTPATH [flags]
--oci-layout-path string Path to save the OCI image layout of the built images
--platform strings Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*
-P, --preserve-import-paths Whether to preserve the full import path after KO_DOCKER_REPO.
--preserve-media-type If false, push images in OCI format regardless of base image format
--push Push images to KO_DOCKER_REPO (default true)
--sbom string The SBOM media type to use (none will disable SBOM synthesis and upload, also supports: spdx, go.version-m). (default "spdx")
--tag-only Include tags but not digests in resolved image references. Useful when digests are not preserved when images are repopulated.
Expand Down
32 changes: 27 additions & 5 deletions pkg/build/gobuild.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ type gobuild struct {
sbom sbomber
disableOptimizations bool
trimpath bool
preserveMediaType bool
buildConfigs map[string]Config
platformMatcher *platformMatcher
dir string
Expand All @@ -99,6 +100,7 @@ type gobuildOpener struct {
sbom sbomber
disableOptimizations bool
trimpath bool
preserveMediaType bool
buildConfigs map[string]Config
platforms []string
labels map[string]string
Expand Down Expand Up @@ -126,6 +128,7 @@ func (gbo *gobuildOpener) Open() (Interface, error) {
sbom: gbo.sbom,
disableOptimizations: gbo.disableOptimizations,
trimpath: gbo.trimpath,
preserveMediaType: gbo.preserveMediaType,
buildConfigs: gbo.buildConfigs,
labels: gbo.labels,
dir: gbo.dir,
Expand Down Expand Up @@ -694,6 +697,16 @@ func (g *gobuild) buildOne(ctx context.Context, refStr string, base v1.Image, pl

ref := newRef(refStr)

baseType := types.OCIManifestSchema1
if g.preserveMediaType {
var err error
baseType, err = base.MediaType()
if err != nil {
return nil, err
}
}
base = mutate.MediaType(base, baseType)

cf, err := base.ConfigFile()
if err != nil {
return nil, err
Expand Down Expand Up @@ -884,7 +897,7 @@ func (g *gobuild) Build(ctx context.Context, s string) (Result, error) {
// Annotate the base image we pass to the build function with
// annotations indicating the digest (and possibly tag) of the
// base image. This will be inherited by the image produced.
if mt != types.DockerManifestList {
if mt != types.DockerManifestList && !g.preserveMediaType {
anns := map[string]string{
specsv1.AnnotationBaseImageDigest: baseDigest.String(),
}
Expand Down Expand Up @@ -963,11 +976,17 @@ func (g *gobuild) buildAll(ctx context.Context, ref string, baseIndex v1.ImageIn
if err != nil {
return err
}

mt := types.OCIManifestSchema1
if g.preserveMediaType {
mt = desc.MediaType
}

adds[i] = ocimutate.IndexAddendum{
Add: img,
Descriptor: v1.Descriptor{
URLs: desc.URLs,
MediaType: desc.MediaType,
MediaType: mt,
Annotations: desc.Annotations,
Platform: desc.Platform,
},
Expand All @@ -979,9 +998,12 @@ func (g *gobuild) buildAll(ctx context.Context, ref string, baseIndex v1.ImageIn
return nil, err
}

baseType, err := baseIndex.MediaType()
if err != nil {
return nil, err
baseType := types.OCIImageIndex
if g.preserveMediaType {
baseType, err = baseIndex.MediaType()
if err != nil {
return nil, err
}
}
idx := ocimutate.AppendManifests(mutate.IndexMediaType(empty.Index, baseType), adds...)

Expand Down
109 changes: 109 additions & 0 deletions pkg/build/gobuild_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -513,6 +513,17 @@ func TestGoBuildNoKoData(t *testing.T) {
t.Errorf("created = %v, want %v", actual, creationTime)
}
})

t.Run("check OCI media type", func(t *testing.T) {
mt, err := img.MediaType()
if err != nil {
t.Errorf("MediaType() = %v", err)
}

if got, want := mt, types.OCIManifestSchema1; got != want {
t.Errorf("mediaType = %v, want %v", got, want)
}
})
}

func validateImage(t *testing.T, img oci.SignedImage, baseLayers int64, creationTime v1.Time, checkAnnotations bool, expectSBOM bool) {
Expand Down Expand Up @@ -919,6 +930,104 @@ func TestGoBuildIndex(t *testing.T) {
t.Errorf("Digest mismatch: %s != %s", d1, d2)
}
})

t.Run("check OCI media type", func(t *testing.T) {
mt, err := idx.MediaType()
if err != nil {
t.Fatalf("MediaType() = %v", err)
}

if got, want := mt, types.OCIImageIndex; got != want {
t.Errorf("mediaType = %v, want %v", got, want)
}

for i, mf := range im.Manifests {
if got, want := mf.MediaType, types.OCIManifestSchema1; got != want {
t.Errorf("manifest[%d] mediaType = %s, want %s", i, got, want)
}
}
})
}

func TestPreserveMediaType(t *testing.T) {
mustRandomImage := func(t *testing.T) v1.Image {
img, err := random.Image(1, 1)
if err != nil {
t.Fatal(err)
}
return img
}
mustRandomIndex := func(t *testing.T) v1.ImageIndex {
idx, err := random.Index(1, 1, 3)
if err != nil {
t.Fatal(err)
}
return idx
}

for _, c := range []struct {
desc string
preserve bool
base Result
want types.MediaType
}{{
desc: "docker image -> oci image",
preserve: false,
base: mustRandomImage(t),
want: types.OCIManifestSchema1,
}, {
desc: "docker index -> oci index",
preserve: false,
base: mustRandomIndex(t),
want: types.OCIImageIndex,
}, {
desc: "docker image, preserved",
preserve: true,
base: mustRandomImage(t),
want: types.DockerManifestSchema2,
}, {
desc: "docker index, preserved",
preserve: true,
base: mutate.IndexMediaType(mustRandomIndex(t), types.DockerManifestList),
want: types.DockerManifestList,
}, {
desc: "oci image",
preserve: true,
base: mutate.MediaType(mustRandomImage(t), types.OCIManifestSchema1),
want: types.OCIManifestSchema1,
}, {
desc: "oci index",
preserve: true,
base: mutate.IndexMediaType(mustRandomIndex(t), types.OCIImageIndex),
want: types.OCIImageIndex,
}} {
t.Run(c.desc, func(t *testing.T) {
importpath := "github.com/google/ko"
ng, err := NewGo(
context.Background(),
"",
WithBaseImages(func(context.Context, string) (name.Reference, Result, error) { return baseRef, c.base, nil }),
WithPlatforms("all"),
WithPreserveMediaType(c.preserve),
withBuilder(writeTempFile),
)
if err != nil {
t.Fatalf("NewGo() = %v", err)
}

result, err := ng.Build(context.Background(), StrictScheme+filepath.Join(importpath, "test"))
if err != nil {
t.Fatalf("Build() = %v", err)
}

got, err := result.MediaType()
if err != nil {
t.Errorf("MediaType() = %v", err)
} else if got != c.want {
t.Errorf("Got %q, want %q", got, c.want)
}
})
}
}

func TestNestedIndex(t *testing.T) {
Expand Down
10 changes: 10 additions & 0 deletions pkg/build/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,16 @@ func WithTrimpath(v bool) Option {
}
}

// WithPreserveMediaType is a functional option that controls whether to
// preserve media types from base images. If false, images that are produced
// will use OCI media types instead.
func WithPreserveMediaType(v bool) Option {
return func(gbo *gobuildOpener) error {
gbo.preserveMediaType = v
return nil
}
}

// WithConfig is a functional option for providing GoReleaser Build influenced
// build settings for importpaths.
//
Expand Down
6 changes: 6 additions & 0 deletions pkg/commands/options/build.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,9 @@ type BuildOptions struct {

// BuildConfigs stores the per-image build config from `.ko.yaml`.
BuildConfigs map[string]build.Config

// If true, don't convert Docker-typed base images to OCI when building.
PreserveMediaType bool
}

func AddBuildOptions(cmd *cobra.Command, bo *BuildOptions) {
Expand All @@ -80,6 +83,9 @@ func AddBuildOptions(cmd *cobra.Command, bo *BuildOptions) {
"Which platform to use when pulling a multi-platform base. Format: all | <os>[/<arch>[/<variant>]][,platform]*")
cmd.Flags().StringSliceVar(&bo.Labels, "image-label", []string{},
"Which labels (key=value) to add to the image.")

cmd.Flags().BoolVar(&bo.PreserveMediaType, "preserve-media-type", false, "If false, push images in OCI format regardless of base image format")

bo.Trimpath = true
}

Expand Down
1 change: 1 addition & 0 deletions pkg/commands/resolver.go
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ func gobuildOptions(bo *options.BuildOptions) ([]build.Option, error) {
opts = append(opts, build.WithSPDX(version()))
}
opts = append(opts, build.WithTrimpath(bo.Trimpath))
opts = append(opts, build.WithPreserveMediaType(bo.PreserveMediaType))
for _, lf := range bo.Labels {
parts := strings.SplitN(lf, "=", 2)
if len(parts) != 2 {
Expand Down