Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add a benchmark test for image pulls #2423

Closed
lucasrod16 opened this issue Apr 8, 2024 · 2 comments
Closed

test: add a benchmark test for image pulls #2423

lucasrod16 opened this issue Apr 8, 2024 · 2 comments
Labels
tech-debt 💳 Debt that the team has charged and needs to repay

Comments

@lucasrod16
Copy link
Contributor

lucasrod16 commented Apr 8, 2024

Describe what should be investigated or refactored

A benchmark test should be added for the PullAll() function

Links to any relevant code

https://github.com/defenseunicorns/zarf/blob/02389274c61b59385ac339d6d690a148878427f9/src/internal/packager/images/pull.go#L39

Additional context

This benchmark test will serve as a baseline for what we expect the performance of image pulls to be.

Example benchmark used to test #2409:

// SPDX-License-Identifier: Apache-2.0
// SPDX-FileCopyrightText: 2024-Present The Zarf Authors

// Package images provides functions for building and pushing images.
package images

import (
	"testing"

	"github.com/defenseunicorns/zarf/src/config"
	"github.com/defenseunicorns/zarf/src/pkg/transform"
)

// BenchmarkPullAll benchmarks the PullAll function.
func BenchmarkPullAll(b *testing.B) {
	config.CommonOptions.CachePath = b.TempDir()
	imageConfig := ImageConfig{
		ImagesPath: b.TempDir(),
		ImageList: []transform.Image{
			{
				Reference: "registry1.dso.mil/ironbank/fluxcd/source-controller:v1.2.4",
			},
			{
				Reference: "registry1.dso.mil/ironbank/fluxcd/kustomize-controller:v1.2.2",
			},
			{
				Reference: "registry1.dso.mil/ironbank/fluxcd/helm-controller:v0.37.4",
			},
			{
				Reference: "registry1.dso.mil/ironbank/fluxcd/notification-controller:v1.2.4",
			},
			{
				Reference: "registry1.dso.mil/ironbank/big-bang/base:2.1.0",
			},
			{
				Reference: "registry1.dso.mil/ironbank/twistlock/console/console:32.01.128",
			},
			{
				Reference: "registry1.dso.mil/ironbank/twistlock/defender/defender:32.01.128",
			},
		},
	}
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
		_, err := imageConfig.PullAll()
		if err != nil {
			b.Fatalf("Error in PullAll: %v", err)
		}
	}
}
@lucasrod16 lucasrod16 added the tech-debt 💳 Debt that the team has charged and needs to repay label Apr 8, 2024
@lucasrod16 lucasrod16 changed the title add a benchmark test for image pulls test: add a benchmark test for image pulls Apr 8, 2024
@Noxsios
Copy link
Contributor

Noxsios commented Apr 9, 2024

This benchmark should also cover cold vs warm caches

@lucasrod16
Copy link
Contributor Author

In hindsight, this is likely a premature optimization. If we feel the need to measure the performance of image pulls with a benchmark test in the future, we can reopen this issue or create a new one.

See rules 1 and 2 in Rob Pike's 5 Rules of Programming:

https://users.ece.utexas.edu/~adnan/pike.html

@lucasrod16 lucasrod16 closed this as not planned Won't fix, can't repro, duplicate, stale May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tech-debt 💳 Debt that the team has charged and needs to repay
Projects
None yet
Development

No branches or pull requests

2 participants