Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use local rekor and fulcio instances in e2e tests #3478

Merged
merged 3 commits into from
Jan 16, 2024

Conversation

cmurphy
Copy link
Contributor

@cmurphy cmurphy commented Jan 13, 2024

In 7068357 the e2e tests moved from running on the locally-spun-up
rekor instance to the public instance. This means test signatures are
piling up in the public instance, and the tests may be taking longer
than they need to since they are using an external service.

This change moves back to using the local rekor instance, which the e2e
has still been spinning up even though it has been going unused. Also
now do the same for fulcio.

This PR also includes minor cleanups in the e2e tests and test script.

Depends on sigstore/fulcio#1518
Relates to sigstore/sigstore-probers#105

Summary

Release Note

NONE

Documentation

Test changes only.

@@ -62,6 +79,8 @@ docker run -d -p 5000:5000 --restart always -e REGISTRY_STORAGE_DELETE_ENABLED=t
export COSIGN_TEST_REPO=localhost:5000
go test -tags=e2e -v ./test/... -run TestSignVerifyClean

# Use the public instance to verify existing images and manifests
unset SIGSTORE_REKOR_PUBLIC_KEY
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts on keeping this? Should we sign local images in a private registry in order to test the below verify commands?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this test would be entirely hermetic, would definitely improve performance, though we can investigate that as a follow up. The signature verification should be offline, though this test depends on ghcr and distroless maintaining signatures on its images.

Copy link

codecov bot commented Jan 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (76c1162) 39.57% compared to head (2fa0a82) 40.07%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3478      +/-   ##
==========================================
+ Coverage   39.57%   40.07%   +0.50%     
==========================================
  Files         155      155              
  Lines       10040    10040              
==========================================
+ Hits         3973     4024      +51     
+ Misses       5600     5530      -70     
- Partials      467      486      +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

- Use -v for clearer output
- All the tests tagged 'e2e' are in ./test, no need to search the whole
  tree
- There's no third_party directory, no need to grep it out
- Clean up services no matter when the script exits

Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
Clean up the key pair secret after the test so that the test suite can
be re-run locally if desired. Without this, the secret is set to
'immutable' and can't be overwritten by the next test run.

Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
In 7068357 the e2e tests moved from running on the locally-spun-up
rekor instance to the public instance. This means test signatures are
piling up in the public instance, and the tests may be taking longer
than they need to since they are using an external service.

This change moves back to using the local rekor instance, which the e2e
has still been spinning up even though it has been going unused. Also
now do the same for fulcio.

Signed-off-by: Colleen Murphy <colleenmurphy@google.com>
fi
done

echo "starting services"
export FULCIO_METRICS_PORT=2113
for repo in rekor fulcio; do
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a local instance of Fulcio is great - We don't have any tests currently using it, and we'll need to set up a mock oidc provider, but this will let us have much more rich e2e tests that exercise the certifciate issuance path

@@ -62,6 +79,8 @@ docker run -d -p 5000:5000 --restart always -e REGISTRY_STORAGE_DELETE_ENABLED=t
export COSIGN_TEST_REPO=localhost:5000
go test -tags=e2e -v ./test/... -run TestSignVerifyClean

# Use the public instance to verify existing images and manifests
unset SIGSTORE_REKOR_PUBLIC_KEY
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this test would be entirely hermetic, would definitely improve performance, though we can investigate that as a follow up. The signature verification should be offline, though this test depends on ghcr and distroless maintaining signatures on its images.

@haydentherapper haydentherapper merged commit 1ebb6d9 into sigstore:main Jan 16, 2024
28 checks passed
@github-actions github-actions bot added this to the v2.3.0 milestone Jan 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants