fix(docker): credentials issues around superset-cache in forks #26772
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mistercrunch
force-pushed
the
fix_docker
branch
from
5f22977 to
07398ed
Compare
craig-rueda
reviewed
Jan 24, 2024
scripts/docker_build_push.sh
Outdated
| @@ -130,14 +130,16 @@ if [[ -n "${BUILD_TARGET}" ]]; then | |||
| TARGET_ARGUMENT="--target ${BUILD_TARGET}" | |||
| fi | |||
|
|
|||
| CACHE_REF="${REPO_NAME}-cache:${TARGET}-${BUILD_ARG}" | |||
| CACHE_REF="${REPO_NAME}:cache-${TARGET}-${BUILD_ARG}" | |||
There was a problem hiding this comment.
We actually have a superset-cache registry setup that's intended to be used here: https://hub.docker.com/repository/docker/apache/superset-cache/general
The reason for this is so that we don't clutter up the "real" registry with a pile of cache images
There was a problem hiding this comment.
I tried to use this and got the exact same error messages on forks, I agree it's better to use this for caching
eschutho
approved these changes
Jan 24, 2024
mistercrunch
requested review from
villebro,
geido,
rusackas,
betodealmeida,
nytai,
john-bodley and
kgabryje
as code owners
|
Realizing this shouldn't even trigger on PRs of forks, somehow the conditional logic stopped working. Fixing it here. |
mistercrunch
added a commit
to preset-io/superset
that referenced
this pull request
Jan 24, 2024
While working on apache#26772, I realized that the has-secret logic was broken for unclear reasons. Now after doing this fix, I looked and realized that there's similar logic across about a dozen other gh actions, and thought it'd be a good thing to refactor/fix this with a reusable action. Now many of these workflows are set to trigger on push-on-master only meaning it's less of an issue since a false positive on has-secret dpesn't matter in that context since there's always a secret. I still thought I should refactor this to something we can trust and build upon. The solution introduces this new simple reusable action. One minor note is in cases where we need multiple secrets, as in say DOCKERHUB_TOKEN and DOCKERHUB_USER, we simply look at one and assume that's clear-enough of an indicator.
This was referenced Jan 24, 2024
mistercrunch
added a commit
that referenced
this pull request
Jan 25, 2024
While working on #26772, I realized that the has-secret logic was broken for unclear reasons. Now after doing this fix, I looked and realized that there's similar logic across about a dozen other gh actions, and thought it'd be a good thing to refactor/fix this with a reusable action. Now many of these workflows are set to trigger on push-on-master only meaning it's less of an issue since a false positive on has-secret dpesn't matter in that context since there's always a secret. I still thought I should refactor this to something we can trust and build upon. The solution introduces this new simple reusable action. One minor note is in cases where we need multiple secrets, as in say DOCKERHUB_TOKEN and DOCKERHUB_USER, we simply look at one and assume that's clear-enough of an indicator.
eschutho
pushed a commit
to preset-io/superset
that referenced
this pull request
Jan 31, 2024
DevNoctis1001
added a commit
to DevNoctis1001/Superset-data_visualization
that referenced
this pull request
Feb 11, 2024
While working on apache/superset#26772, I realized that the has-secret logic was broken for unclear reasons. Now after doing this fix, I looked and realized that there's similar logic across about a dozen other gh actions, and thought it'd be a good thing to refactor/fix this with a reusable action. Now many of these workflows are set to trigger on push-on-master only meaning it's less of an issue since a false positive on has-secret dpesn't matter in that context since there's always a secret. I still thought I should refactor this to something we can trust and build upon. The solution introduces this new simple reusable action. One minor note is in cases where we need multiple secrets, as in say DOCKERHUB_TOKEN and DOCKERHUB_USER, we simply look at one and assume that's clear-enough of an indicator.
sfirke
pushed a commit
to sfirke/superset
that referenced
this pull request
Mar 22, 2024
mistercrunch
added
🏷️ bot
vinothkumar66
pushed a commit
to vinothkumar66/superset
that referenced
this pull request
Nov 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixing the docker builds after merging a PR where I was getting a false positive since it was directly location on the main fork. Now when rebasing and executing docker from forks we get errors messages.
The root cause was this: https://github.com/apache/superset/pull/26766/files#diff-bcd99032795eb96dd42d38ceeeec262f6d4ab3952e3dbfb75905763c116cd0d7R133-R134
Where I used the
apache/superset-cacheDockerHub instead of the main one, as it was recommended for optimization purposes. In this PR, I'll point it back toapache/supersetafter confirming the issue, so I can confirm the fix.Error looked like this on cache-push:
This is confusing as we're able to push images and layers, but not use the hub as cache. Probably some permission setting of some kind. Though I think the security is a bit reversed, meaning people shouldn't be able to publish an actual image, say to
apache/superset:latestfrom a fork, but the "using the repo as a cache" is less of an issue as no one would pull from random cache. Probably one for the ASF infra folks