Share kodata across multi-arch builds?

[mattmoor]

Talking to @jonjohnsonjr I realized that we might be able to save some cycles if we found a way to share the kodata layer across architectures in multi-arch builds. Right now we are deterministically producing the same layer N times, but it should be safe to share this if we can find a way to reasonably do so.

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

[imjasonh]

This might be a little more complicated now with Windows support, since kodata is laid out differently in Windows layers.

I wonder if we could just not include the layer if kodata isn't present, rather than include an empty layer.

[mattmoor]

I think we want the directory to exist, even if it is empty.

[imjasonh]

Sure. What if we put the empty /var/run/kodata in the same layer as the binary, and if there's anything in kodata, put the rest in the separate layer. That way if there isn't any kodata we can skip a layer.

(This is also complicated by Windows-vs-Linux containers, but maybe a bit less)

[mattmoor]

I don't have a problem with that, though you should confirm it WAI since I think the layers are in the wrong order.

[developer-guy]

Kindly ping here; maybe we cannot add the data layer that includes /var/run/ko if there is no <importpath>/kodata existing in the project. WDYT?

/cc @Dentrax

ko/pkg/build/gobuild.go

Lines 822 to 842 in 5e0452a

	// Create a layer from the kodata directory under this import path.
	dataLayerBuf, err := g.tarKoData(ref, platform)
	if err != nil {
	return nil, err
	}
	dataLayerBytes := dataLayerBuf.Bytes()
	dataLayer, err := tarball.LayerFromOpener(func() (io.ReadCloser, error) {
	return ioutil.NopCloser(bytes.NewBuffer(dataLayerBytes)), nil
	}, tarball.WithCompressedCaching, tarball.WithMediaType(layerMediaType))
	if err != nil {
	return nil, err
	}
	layers = append(layers, mutate.Addendum{
	Layer: dataLayer,
	History: v1.History{
	Author: "ko",
	CreatedBy: "ko build " + ref.String(),
	Created: g.kodataCreationTime,
	Comment: "kodata contents, at $KO_DATA_PATH",
	},
	})

[imjasonh]

I think we want to make sure there's an empty /var/run/ko directory even if there isn't a kodata directory to populate it.

The reason is that some code may depend on the var existing today even if it's empty, and changing that would break them.

The change is that /var/run/ko should be created in the binary layer instead of the data layer, so that the data layer can be omitted when there's no kodata.