Generate SBOM using buildkit's builtin mechanism #83

rucoder · 2024-02-27T23:55:42Z

linuxkit cache push is not implemented yet see Add 'cache push' command to push images by tag linuxkit/linuxkit#3990
tested on a local eve build
linuxkit cache push requires one tiny fix to avoid extra image tag creation when using cache push, make push of arch-specific tags optional linuxkit/linuxkit#3996

This PR should address #50

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>

eriknordmark · 2024-02-28T05:48:59Z

DCO signoff is missing.

rucoder · 2024-02-28T07:51:12Z

DCO signoff is missing.

@eriknordmark it is not missing for this PR. There are 2 old commits with wrong sign-off string and I'm not sure what to do about it.

Summary

Commit sha: 0c3194d, Author: Mikhail Malyshev, Committer: Mikhail Malyshev; Expected "Mikhail Malyshev mikem@zededa.com", but got "Nikolay Martyanov nikolay@zededa.com".
Commit sha: 0dc9b45, Author: Roman Shaposhnik, Committer: Mikhail Malyshev; The sign-off is missing.

eriknordmark · 2024-02-28T08:05:10Z

@eriknordmark it is not missing for this PR. There are 2 old commits with wrong sign-off string and I'm not sure what to do about it.

That's odd. So the DCO checker is broken?
We can ignore it for now, but I wonder if we'll see the same issue for the same old commits for the next PR.

deitch

LGTM, except that linuxkit cache push is not yet implemented. Working on it. Real Soon Now ™️

rucoder · 2024-02-28T13:53:41Z

tried linuxkit cache push from commit 54d9db8650400a9f6af72b176b64c6dcc30cba07. It works but one last tiny fix is required. Right now it creates 2 tags assuming that every image is multi-platform

- --sbom=true is not compatible with --load because docker doesn't support full OCI spec. Instead we export TAR file in OCI format and later load it linuxkit cache - since the image is now hosted by linuxkit cache we should use 'cache push' command to push it to dockerhub registry Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>

deitch

LGTM.

Did you build it and do you see the correct sbom info? You should be able to look into the cache directly and see it.

rucoder · 2024-03-04T12:22:22Z

@yash-zededa it seems we need to upgrade buildkit on github runner. https://github.com/lf-edge/eve-kernel/actions/runs/8128820721/job/22215156330?pr=83

docker buildx build \
--build-arg="SOURCE_DATE_EPOCH=1709322857" \
--build-arg="KBUILD_BUILD_TIMESTAMP=Fri Mar 1 19:54:17 2024 " \
--build-arg="LOCALVERSION=6c57bd876fea" \
--platform linux/amd64 -t lfedge/eve-kernel:merge-6c57bd876fea-gcc \
--sbom=true --output=type=oci,dest=/tmp/tmp.Fa2e4DNCy2-kernel.tar -f Dockerfile.gcc .
ERROR: attestations are not supported by the current buildkitd
make: *** [Makefile.eve:59: kernel-build-gcc] Error 1

rucoder · 2024-03-04T13:30:12Z

@yash-zededa it seems we need to upgrade buildkit on github runner. https://github.com/lf-edge/eve-kernel/actions/runs/8128820721/job/22215156330?pr=83

docker buildx build \
--build-arg="SOURCE_DATE_EPOCH=1709322857" \
--build-arg="KBUILD_BUILD_TIMESTAMP=Fri Mar 1 19:54:17 2024 " \
--build-arg="LOCALVERSION=6c57bd876fea" \
--platform linux/amd64 -t lfedge/eve-kernel:merge-6c57bd876fea-gcc \
--sbom=true --output=type=oci,dest=/tmp/tmp.Fa2e4DNCy2-kernel.tar -f Dockerfile.gcc .
ERROR: attestations are not supported by the current buildkitd
make: *** [Makefile.eve:59: kernel-build-gcc] Error 1

nvm, we are running the latest buildkit anyway.

deitch · 2024-03-04T13:29:51Z

Makefile.eve

@@ -47,16 +47,21 @@ help: Makefile
 	@echo "  clean: remove generated files"
 	@echo

+.PHONY: ensure-builder
+ensure-builder:
+	docker builder inspect eve-kernel-builder || docker builder create --name eve-kernel-builder --driver docker-container --bootstrap


I recommend saving the builder to a Makefile var and using that here and in the build target

I recommend redirecting the output of inspect to /dev/null, or it gets messy. You don't care about the output, only if it returns 0 or 1.

Maybe an @ at the beginning of the line?

I recommend saving the builder to a Makefile var and using that here and in the build target

this was just a POC :). Will fix

I recommend redirecting the output of inspect to /dev/null, or it gets messy. You don't care about the output, only if it returns 0 or 1.

it may be useful to troubleshoot issues with CI/CD in future

deitch · 2024-03-04T13:30:53Z

Makefile.eve

 	@echo "Building kernel version $(BRANCH):$(VERSION)-$* with compiler $*"
 	docker buildx build \
+	--builder=eve-kernel-builder \


As above, use a var

@deitch should we fix a build-kit version to something like 0.12.5 ? I could name --builder eve-kernel-builder-$(BK_VERSION)

Yeah, that is a good idea.

Our build may depend on buildx features e.g. --sbom so we need to make sure we have a correct builder version. This is not a big problem for local build but a big one for GH runners. Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>

rucoder · 2024-03-04T15:03:38Z

@deitch

Please approve to rerun actions
ocker buildx create vs docker builder create. and possible problems here?

rucoder · 2024-03-04T21:39:54Z

@deitch @rene @eriknordmark it seems all issues are now solved, we can merge this PR if there are no objections

Remove SBOM generation by external tools

9b109b5

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>

rucoder changed the base branch from main to eve-kernel-amd64-v6.1.38-generic February 27, 2024 23:56

rucoder mentioned this pull request Feb 28, 2024

Add native sboms with buildkit #50

Open

rucoder requested review from yash-zededa and deitch February 28, 2024 07:51

deitch approved these changes Feb 28, 2024

View reviewed changes

rucoder force-pushed the rucoder/new-sbom branch from b3f5f9b to 16ec9f3 Compare February 28, 2024 13:39

rucoder force-pushed the rucoder/new-sbom branch from 16ec9f3 to 6353f50 Compare March 1, 2024 19:54

rucoder changed the title ~~[WIP] Generate SBOM using buildkit's builtin mechanism~~ Generate SBOM using buildkit's builtin mechanism Mar 1, 2024

rucoder requested review from deitch, rene, rouming and eriknordmark March 1, 2024 19:54

deitch approved these changes Mar 3, 2024

View reviewed changes

rucoder requested a review from deitch March 4, 2024 13:29

rucoder closed this Mar 4, 2024

rucoder reopened this Mar 4, 2024

deitch requested changes Mar 4, 2024

View reviewed changes

Make sure we have correct buildx builder

732e980

Our build may depend on buildx features e.g. --sbom so we need to make sure we have a correct builder version. This is not a big problem for local build but a big one for GH runners. Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>

rucoder force-pushed the rucoder/new-sbom branch from 94aa2db to 732e980 Compare March 4, 2024 15:01

rucoder requested a review from deitch March 4, 2024 15:01

deitch approved these changes Mar 4, 2024

View reviewed changes

eriknordmark merged commit ae347d3 into eve-kernel-amd64-v6.1.38-generic Mar 4, 2024
2 checks passed

rucoder added the pr: eve-kernel-* PR has to be merged to other branches label Mar 5, 2024

rucoder added the pr-merged PR was merged to integration branches label Mar 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate SBOM using buildkit's builtin mechanism #83

Generate SBOM using buildkit's builtin mechanism #83

rucoder commented Feb 27, 2024 •

edited

Loading

eriknordmark commented Feb 28, 2024

rucoder commented Feb 28, 2024 •

edited

Loading

eriknordmark commented Feb 28, 2024

deitch left a comment

rucoder commented Feb 28, 2024 •

edited

Loading

deitch left a comment

rucoder commented Mar 4, 2024

rucoder commented Mar 4, 2024

deitch Mar 4, 2024

deitch Mar 4, 2024

deitch Mar 4, 2024

rucoder Mar 4, 2024

rucoder Mar 4, 2024

deitch Mar 4, 2024

rucoder Mar 4, 2024

deitch Mar 4, 2024

rucoder commented Mar 4, 2024

rucoder commented Mar 4, 2024

Generate SBOM using buildkit's builtin mechanism #83

Generate SBOM using buildkit's builtin mechanism #83

Conversation

rucoder commented Feb 27, 2024 • edited Loading

eriknordmark commented Feb 28, 2024

rucoder commented Feb 28, 2024 • edited Loading

eriknordmark commented Feb 28, 2024

deitch left a comment

Choose a reason for hiding this comment

rucoder commented Feb 28, 2024 • edited Loading

deitch left a comment

Choose a reason for hiding this comment

rucoder commented Mar 4, 2024

rucoder commented Mar 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rucoder commented Mar 4, 2024

rucoder commented Mar 4, 2024

rucoder commented Feb 27, 2024 •

edited

Loading

rucoder commented Feb 28, 2024 •

edited

Loading

rucoder commented Feb 28, 2024 •

edited

Loading