Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add repository file resource #318

Merged
merged 1 commit into from
Feb 11, 2020
Merged

Add repository file resource #318

merged 1 commit into from
Feb 11, 2020

Conversation

shoekstra
Copy link
Contributor

Hello,

This PR adds support to manage files in a git repo using the contents API. I think I've got most cases covered in tests and have updated the docs and README appropriately.

Note that certain CI tests will be skipped until the new GITHUB_TEST_USER_NAME and GITHUB_TEST_USER_EMAIL variables are exported.

Stephen

@ghost ghost added size/XL Type: Documentation Improvements or additions to documentation labels Jan 9, 2020
@shoekstra
Copy link
Contributor Author

shoekstra commented Jan 9, 2020

For some additional context regarding the "Fix go deps" commit: when using go 1.13 I received the following:

go: golang.org/x/oauth2@v0.0.0-20190604054615-0f29369cfe45: invalid pseudo-version: does not match version-control timestamp (2019-06-04T05:34:49Z)

I resolved this by following the steps I found at https://tip.golang.org/doc/go1.13#version-validation, if it's preferred to move this commit/change to a separate PR I'm happy to do so.

@kpfleming
Copy link
Contributor

That change to go.mod is already present in #310 (on its own) and some other PRs (bundled with other changes). I would remove that change from this PR so it can be handled on its own.

README.md Outdated Show resolved Hide resolved
Delete: resourceGithubRepositoryFileDelete,
Importer: &schema.ResourceImporter{
State: func(d *schema.ResourceData, meta interface{}) ([]*schema.ResourceData, error) {
parts := strings.Split(d.Id(), ":")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parseTwoPartID function can be used for this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered parseTwoPartID until I saw that the error returned is quite specific and doesn't apply to what I'm parsing. As parseTwoPartID is used in a couple of other funcs I decided if I was going to refactor the returned error it probably needs it's own PR/discussion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've got a PR at #313 to address that problem too, so if you switch to it now when that is merged your code will have to be updated to provide the proper labels for the left/right halves of the ID.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - what's the delay with merging #313? Feels a bit odd to amend the commit to include broken code, maybe better to wait until #313 is merged and then rebase this PR?

github/resource_github_repository_file.go Show resolved Hide resolved
website/docs/r/repository_file.html.markdown Outdated Show resolved Hide resolved
@shoekstra
Copy link
Contributor Author

That change to go.mod is already present in #310 (on its own) and some other PRs (bundled with other changes). I would remove that change from this PR so it can be handled on its own.

No problem, I've dropped that commit 👍

Copy link
Contributor

@jcudit jcudit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shoekstra thanks for this contribution. It lines up fairly well with functionality I was asking for over in this issue.

Pull Requests are a missing piece that I'd like to see added onto this either in a subsequent release or via a refactoring of this PR. Can you take a look at the linked issue and let me know if what was planned there aligns with your use cases that this patch solves? I am interested in your take on what next steps could be to involve the Pull Request functionality of the go-github client library.

Aside from that concern, here are a few questions I have around the implementation as-is:

  • I see the client.Repositories.CreateFile function was used in resourceGithubRepositoryFileUpdate instead of the available client.Repositories.UpdateFile function. What is the tradeoff around using one over the other?

  • I see that a branch needs to exist before it can receive commits. What are your thoughts around adding support for creating a branch if it does not exist (and throwing an error if a branch already exists)? Are you currently automating the creation of branches through some other means?

  • Are you open to generalizing the resource from _file to _content? This would allow for declaring multiple files within a resource block and iterating through the lifecycle management of each one. Have there been any pain points with using multiple repository_file resources with your use cases so far?

@shoekstra
Copy link
Contributor Author

Hey @jcudit,

Pull Requests are a missing piece that I'd like to see added onto this either in a subsequent release or via a refactoring of this PR. Can you take a look at the linked issue and let me know if what was planned there aligns with your use cases that this patch solves? I am interested in your take on what next steps could be to involve the Pull Request functionality of the go-github client library.

Skipping the pull request bit was a conscious decision I made; I wanted a way to manage a few files (GitHub Actions in my case) in a bunch of repos from a central Terraform repo. The review would already happen when updating the Terraform plan and I decided creating additional PRs created additional overhead for someone to review the changes a second time and, more importantly, this didn’t fit with Terraform’s job of ensuring “desired state” in that a file would be described in a plan with desired content but this could not be ensured unless committing and pushing to the specified branch.

Reading the issue you linked, the initial proposal is the better way to go should you want to add support for pull requests. With Terraform you want to manage the file and update it using a pull request, you don’t want to manage a pull request as a merged pull request would be void if the content is modified (I hope that makes sense).

I'm not convinced managing pull requests is something to do but am open to more discussion on how we could make it work if it's functionality you're after.

Aside from that concern, here are a few questions I have around the implementation as-is:

  • I see the client.Repositories.CreateFile function was used in resourceGithubRepositoryFileUpdate instead of the available client.Repositories.UpdateFile function. What is the tradeoff around using one over the other?

Looking now I see they’re the same, there is no difference between them. At the time I referred to the API docs and saw creating and updating files was done using the same method, so I didn’t look for an Update function in go-github.

  • I see that a branch needs to exist before it can receive commits. What are your thoughts around adding support for creating a branch if it does not exist (and throwing an error if a branch already exists)? Are you currently automating the creation of branches through some other means?

My use case is to create files in branches that already exist thus creating a branch is out of scope of this resource. If I wanted a branch to exist I would want to manage it via a github_branch resource and it would be a depended on by the github_repository_file resource. I would only create a branch via this resource if we implemented the pull request functionality.

  • Are you open to generalizing the resource from _file to _content? This would allow for declaring multiple files within a resource block and iterating through the lifecycle management of each one.

Sure, I'm happy to add a files field if it’ll be useful for others. I originally decided to keep to a single file per resource to ensure a file is only managed by one resource in a plan. I'm not sure how I would do this if I add multiple files in a single resource.

Have there been any pain points with using multiple repository_file resources with your use cases so far?

I haven’t had any issues with multiple resources yet; at the moment I’ve declared 3 files in a module and reference that module 6 times in a plan, so effectively managing 18 files across 6 repos in a single Terraform run.

@jcudit
Copy link
Contributor

jcudit commented Feb 3, 2020

The review would already happen when updating the Terraform plan

👍 this did not occur to me and removes my desire to have Terraform initiate PRs.


Thank you for addressing all other concerns raised as well. I think the workflow you are presenting here is valuable to the community and will work to get this merged and released as-is. My next steps are to:

@jcudit
Copy link
Contributor

jcudit commented Feb 4, 2020

@shoekstra I've merged #313. Can you rebase and consume the new parseTwoPartID function?

I will look into adding the new branch and environment variables to CI in the meantime.

Signed-off-by: Stephen Hoekstra <shoekstra@schubergphilis.com>
@shoekstra
Copy link
Contributor Author

I've rebased my branch. parseTwoPartID doesn't fit what I'm doing so no changes there.

I was thinking about your requests re UX. It's out of scope of me at the moment, but there should be enough of a base for someone to pick up adding the pull_request or files fields you mentioned.

Feel free to merge if you agree!

@jcudit jcudit merged commit b838aed into integrations:master Feb 11, 2020
@shoekstra shoekstra deleted the add_repository_file_resource branch February 14, 2020 10:19
@jcudit jcudit mentioned this pull request Feb 24, 2020
kfcampbell pushed a commit to kfcampbell/terraform-provider-github that referenced this pull request Jul 26, 2022
…e_resource

Add repository file resource
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size/XL Type: Documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants