-
Notifications
You must be signed in to change notification settings - Fork 589
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Can't get .git directory to show up in remote builds #236
Comments
Indeed, if you look at the code for If you want it, you have several options:
See the gcloudignore documentation for further details. |
Thank you for the detailed reply, I got it working with your suggestions. 🎈 🎉 I agree it is rather unusual (I used to always add
|
Cloud this please be documented somewhere. I can't find any reference to default |
@bendory is there a different logic applied when using the GCB GitHub App? I'm trying to get private submodules working, but the extracted tarball doesn't contain a .git folder |
redirect to @wlynch who will know if there's a different logic in the app |
I think it’s worth reopening this issue. I appreciate the thorough answer, @bendory, but it seems the Or is there a separate issue tracking that in particular? |
I'm also running into this: with the |
If found it will work with GitHub this way:
In this way there is .git folder available in workdir during the build process (still need include it on .gcloudignore) |
The following build step fetches the - id: Get the .git directory
name: git
args:
- bash
- -c
- |-
# get the .git directory only
git clone --branch $BRANCH_NAME git@github.com:<USER_NAME>/$REPO_NAME /tmp/repo ;
# move it to the workspace
mv /tmp/repo/.git . |
@saminiemi Your workaround doesn't work if you want to take advantage of the "Build on PR" feature that's only available with the Cloud Build GitHub App repo :( |
We do this instead of moving the repo: # checkout git submodules
- name: 'gcr.io/cloud-builders/git'
entrypoint: 'bash'
args:
- '-c'
- |
git init
git remote add origin git@github.com:/orgname/$REPO_NAME.git
git fetch --depth=1 origin $COMMIT_SHA
git reset --hard FETCH_HEAD
git config -f .gitmodules submodule.core.url git@github.com:/orgname/$REPO_NAME.git
git submodule update --init And for authentication we use this project: |
Hi all, There's a much quicker (still hacky) way to get the .git folder in without having to download the entire world all over again: - id: get-git-history
name: gcr.io/cloud-builders/git
entrypoint: bash
args:
- '-c'
- |
git clone https://github.com/lookit/lookit-api.git --no-checkout
mv lookit-api/.git .
git fetch --unshallow
rm -rf lookit-api Basically, use |
Another alternative is using Google Cloud Secret Manager - put your GitHub OAuth token and retrieve in the Cloud Build image: Adding secret:
Retrieving secret in Build pipeline:
|
Here's a solution that you can use without having to authenticate with Github: First connect your GitHub repository to Cloud Source Repositories (https://https://source.cloud.google.com/) Then use the gcloud builder to auth against your cloud source repo:
Note that the value for SERVICE-ACCOUNT-EMAIL can be found under https://console.cloud.google.com/cloud-build/settings/service-account Also note that after you are no longer in the Good luck everyone! |
Really the solution here is just for the Cloud Build app to send Where would be the right place to file an issue about this? |
Heavily needed! Gitops without a rev history is like trying to eat soup with a fork... |
If you need access to the full repo, the simplest solution that I've found is the following:
No need to move the Credit: This guy's company blog |
@realpha I think that only works with GSC repos, no? |
I think for a private GitHub repo this will leave you with some extra hoops to jump through still, right? |
@Datamance @JohnPaton Sorry, I forgot to mention that you have to mirror your repository, otherwise it won't work. |
Why is this closed? Having a
That assumption is false. We need the Here's a
|
Just stumbled across this, it's beyond silly! Cloud Build is forcing a gitops model, and you don't have access to git 🤯 |
As noted above, the presence of the .git directory depends on how the build was created, and that directory is indeed not currently available when using the GitHub App. We're aware of the issue, please stay tuned. This issue remains closed as this isn't something that we plan to address within cloud-builders, even as there are various workarounds listed above which may work for some people/workflows. |
Interesting. That's not a problem using other GitHub apps like Travis or CircleCI. gcloud has a read-only SSH key to fetch the whole repo, so why doesn't gcloud check out the whole branch with Also, please don't forget to add git-lfs to fetch LFS files. |
Hi! Responding to the ping. I no longer work on the GCB GitHub App, but I can provide some context about why things are the way they are. The previous GitHub integration mirrors the repo to Cloud Source Repositories, and performs a git clone on the CSR copy. While this simplifies auth on the GCP side (e.g. you can clone using the gcloud helper), this is prone to sync issues. One of the goals of the GitHub App were to resolve some of these pain points by fetching repos directly from GitHub. However, when the GCB App was first launched there was no mechanism to scope installation tokens, so a GCP project could hypothetically affect any repo in the installation since you can only have 1 installation per owner (e.g. user or org). Since then, GitHub has added the ability to scope installation tokens by repo and permission, which looks promising to address the problems above. That said, there's probably more to think through and there's always priorities to juggle. ;) As @koolte mentioned, the team is aware of this. In general, https://issuetracker.google.com/issues/new?component=190802&template=1162663 tends to be the best mechanism to raise a bug or show interest in a particular feature. In the meantime, as others have mentioned, as a workaround you can perform git operations yourself in your builds. If you require auth, you can provide credentials to your worker via https://cloud.google.com/cloud-build/docs/securing-builds/use-encrypted-secrets-credentials. While this may not be the update you are hoping for, I hope this helps explain how things came to be! |
Thanks a lot for the detailed reply, @wlynch ! Hopefully this can be fixed now that installation tokens can be scoped. For now, I have a hack to check out the rest of my repo, submodules, and LFS files with my own token. It works. Thanks for the link to the cloudbuild build issue form for future issues |
I'm submitting a remote build with
gcloud container builds submit --config cloudbuild.yaml
and no matter what I try, the.git
directory is not showing up in the temporary tarballs. My Dockerfile needs to access the.git
directory for the build and obviously it can't find it and all my remote builds fail (local builds work fine).There is no
.dockerignore
, no.gitignore
, no.gcloudignore
, and no "Some files were not included in the source upload." message like I get in other projects where I use those ignore files.If I create a different hidden directory:
that directory does show up in the tarball, so I know it's not excluding all hidden directories.
Is there some hardcoding specifically to exclude
.git
when creating the tarball?The text was updated successfully, but these errors were encountered: