Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

aws_codepipeline with Github OAuth causing persistent changes #2854

Closed
cornfeedhobo opened this issue Jan 3, 2018 · 21 comments
Closed

aws_codepipeline with Github OAuth causing persistent changes #2854

cornfeedhobo opened this issue Jan 3, 2018 · 21 comments
Assignees
Labels
bug Addresses a defect in current functionality. service/codepipeline Issues and PRs that pertain to the codepipeline service.
Milestone

Comments

@cornfeedhobo
Copy link

Terraform Version

Terraform v0.11.1
Terraform AWS Provider v1.6.0

Affected Resource(s)

Please list the resources as a list, for example:

  • aws_codepipeline

Terraform Configuration Files

resource "aws_codepipeline" "build" {
  name     = "pipeline-test"
  role_arn = "pipeline-test"

  artifact_store {
    type     = "S3"
    location = "pipeline-test-bucket"
  }

  stage {
    name = "Source"

    action {
      name     = "Source"
      category = "Source"
      owner    = "ThirdParty"
      provider = "GitHub"
      version  = "1"

      output_artifacts = ["code"]

      configuration {
        OAuthToken           = "${var.github_token}"
        Owner                = "${var.github_owner}"
        Repo                 = "${var.github_repo}"
        Branch               = "${var.github_branch}"
        PollForSourceChanges = "true"
      }
    }
  }

  stage {
    name = "Build"

    action {
      name     = "Build"
      category = "Build"
      owner    = "AWS"
      provider = "CodeBuild"
      version  = "1"

      input_artifacts  = ["code"]
      output_artifacts = ["package"]

      configuration {
        ProjectName = "${var.project_name}"
      }
    }
  }
}

Expected Behavior

Subsequent executions of terraform apply should not result in updates to the source attributes.

Actual Behavior

Running terraform plan/terraform apply always results in a change:

  ~ aws_codepipeline.build
      stage.0.action.0.configuration.%:          "4" => "5"
      stage.0.action.0.configuration.OAuthToken: "" => "REDACTED"

And AWS is incapable of accessing Github, even though the token is valid, tested, and with the correct scopes.

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. terraform plan
  2. terraform apply
  3. terraform apply
@cornfeedhobo cornfeedhobo changed the title aws_codepipeline failing to work with github oauth aws_codepipeline with Github OAuth causing persistent changes Jan 3, 2018
@cornfeedhobo
Copy link
Author

cornfeedhobo commented Jan 4, 2018

The solution is to use the environment variable GITHUB_TOKEN.
Edit: This is wrong, see below.

@ndench
Copy link
Contributor

ndench commented Mar 4, 2018

Even when I specify the GITHUB_TOKEN environment variable I still get the same issue as the OP. Is there something else you need to do as well?

@cornfeedhobo cornfeedhobo reopened this Mar 5, 2018
@catsby catsby added the bug Addresses a defect in current functionality. label Apr 5, 2018
@michaelmoussa
Copy link

michaelmoussa commented Apr 27, 2018

I'm experiencing the same issue, but managed to work around it by adding the following to my aws_codepipeline resource:

  # Workaround for Terraform insisting on "updating" OAuthToken every run. In
  # the event that the OAuthToken actually needs to be updated, comment out
  # the `lifecycle` block, run `terraform apply`, and then restore the block.
  #
  # https://github.com/terraform-providers/terraform-provider-aws/issues/2854
  lifecycle {
    ignore_changes = [
      "stage.0.action.0.configuration.OAuthToken",
      "stage.0.action.0.configuration.%",
    ]
  }

The GitHub token isn't likely to change often in my use case, so the inconvenience of having to remove & restore that lifecycle block is not a big deal compared to having to confirm that I want to "change" the token on every single run (and having it displayed on the screen in plaintext each time, too).

It doesn't address the root cause, but hopefully someone else will find this workaround useful.

EDIT (2019-05-09): See my updated workaround below if you're experiencing this problem with Terraform 0.12.0-rc1 or newer.

@huksley
Copy link

huksley commented Sep 17, 2018

The solution proposed by @michaelmoussa is good, but it is not applicable when you are using the module which, in turn, creates the aws_codepipeline resource.

It is very inconvenient to change the source code of that module to comment/uncomment lifecycle block all the time (if you have a group of infrastructure engineers). And downright impossible if you have it published in GitHub.

Another solution is to use conditional resources i.e. count={var.force_github_token ? 1 : 0} but the problem is I already have 6 different codepipeline resources and that, in turn, will lead to having 12 codepipeline resources.

@sunilkumarmohanty
Copy link
Contributor

sunilkumarmohanty commented Jan 7, 2019

I had a look at the provider code and it seems that the OAuthToken is getting deleted from the state file.

if ok && actionProvider == "GitHub" {
				delete(config, "OAuthToken")
			}

I suspect this has been done to not store secrets in state file. However, in other resources like aws_db_instance, we store the passwords in state file. The state file always has been the single source of truth. The issue pointed out here violates that principal and kind of degrades the developer experience.

I suggest we change this behaviour and store the token in the state file and keep the experience consistent across resource.

Moreover, the OAuthToken value is taken from an environment variable, which is again not consistent with other resources.

@sunilkumarmohanty
Copy link
Contributor

On further debugging, I found that the GetPipeline method of aws sdk for go returns **** instead of the actual OAuthToken, which means that the state file will always have **** in it instead of the actual OAuthToken. Hence, every time terraform plan is run, it will always state that the pipeline needs modification.

@cornfeedhobo
Copy link
Author

@sunilkumarmohanty if that is the case, then let's just store the asterisk and move on. Who cares if it's not an absolute truth, as long as it stops breaking expectations.

@michaelmoussa
Copy link

Update for 0.12.0-rc1:

This is still broken in 0.12.0-rc1, but the workaround I posted a year ago (hacky birthday! 🎂) doesn't work anymore.

You'll first see an error saying "Dot must be followed by attribute name", which can be fixed by using stage[0].action[0] instead of stage.0.action.0. That will fix the .OAuthToken portion, but the .configuration.% portion will not work. I also tried .configuration[%] and even tried incorporating the splat operator, but no dice there ("Splat expressions (.*) may not be used here.").

The following approach will work in 0.12:

  # Workaround for Terraform insisting on "updating" OAuthToken every run. This
  # will prevent *any* updates to this CodePipeline resource while in place. In
  # the event that any real updates are needed, comment out the `lifecycle`
  # block, run `terraform apply`, and then restore the block.
  #
  # https://github.com/terraform-providers/terraform-provider-aws/issues/2854
  # https://www.terraform.io/docs/configuration/resources.html#ignore_changes
  lifecycle {
    ignore_changes = all
  }

NOTE: You could technically use ignore_changes = [stage] as well, which will allow you to update the CodePipeline resource itself as long as you don't modify the stages. I prefer the all approach, because it will make it more obvious that something is wrong if I try to modify the resource itself and the stages. Using [stage] would allow top-level attribute changes to take place, while ignoring the changes to the stage block, which could lead to unpredictable results and an all-around bad time.

@cryptiklemur
Copy link

You can't do ignore_changes = ["stage[0]"] either

@aeschright aeschright added the service/codepipeline Issues and PRs that pertain to the codepipeline service. label Jun 21, 2019
@mattbryce93
Copy link

mattbryce93 commented Jul 16, 2019

You can't do ignore_changes = ["stage[0]"] either

ignore_changes = [stage[0]] works though

ignore_changes = [stage[0].action[0]] works also to get one layer lower but anything I've tried to get into the configuration section has thus far failed 😞

@eshiri
Copy link

eshiri commented Oct 6, 2019

You can't do ignore_changes = ["stage[0]"] either

ignore_changes = [stage[0]] works though

ignore_changes = [stage[0].action[0]] works also to get one layer lower but anything I've tried to get into the configuration section has thus far failed 😞

I am facing the same issue. Couldn't ignore just the OAuthToken. Does anyone know of a solution?

@kevinburke1
Copy link

This is ugly but adding this in the lifecycle section worked for me

ignore_changes = [stage[0].action[0]]

@hlarsen
Copy link

hlarsen commented Oct 6, 2019

looks like #2796 is related and #5764 would solve it - anyone have any thoughts?

@WGriffing
Copy link

WGriffing commented Oct 23, 2019

I was able to get as far as:
ignore_changes = [stage[0].action[0].configuration]

However, I couldn't figure out how to specifically ignore one attribute of configuration such as OAuthToken either.

Ignoring the entire configuration won't work for my use case.

@nik-shornikov
Copy link

nik-shornikov commented Mar 20, 2020

I was able to get as far as:
ignore_changes = [stage[0].action[0].configuration]

However, I couldn't figure out how to specifically ignore one attribute of configuration such as OAuthToken either.

Ignoring the entire configuration won't work for my use case.

i arrived at this too and its the superior workaround. however, like the doc says, maps cant be made to ignore newly added keys

so clearly the issue is that the read operation doesn't grab an empty or placeholder value for the token in the action configuration (which it obviously shouldnt be able to receive); instead it presents that key as missing

@cornfeedhobo
Copy link
Author

looks like #2796 is related and #5764 would solve it - anyone have any thoughts?

@hlarsen i don't use this right now. hopefully someone else can respond.

@nl-brett-stime
Copy link

nl-brett-stime commented Jun 19, 2020

I had a look at the provider code and it seems that the OAuthToken is getting deleted from the state file.

I suspect this has been done to not store secrets in state file.

I suggest we change this behaviour and store the token in the state file and keep the experience consistent across resource.

@bflad @gdavison (please forward if someone else should be looking at the CodePipeline provider).

In the worst case, a hash of the OAuthToken could be stored in the state file so that we can do change-detection without having to expose the actual secret.

@Stretch96
Copy link
Contributor

Stretch96 commented Jul 5, 2020

As @nl-brett-stime mentioned, if we could get the hashed password stored in the state file, it will allow to check for changes and also keep secrets secure(ish) - depends on the user to keep the state file private

We're experiencing this issue on the aws_codepipeline resource, OAuthToken in the source phase

Perhaps have it optional to store the hash

@bflad
Copy link
Contributor

bflad commented Jul 31, 2020

Hi folks 👋 This should be resolved, or at least now have different behavior with #14175 which was just merged and released with version 3.0.0 of the Terraform AWS Provider. If you are still having issues after upgrading to this release, please open a new issue and the maintainers will take a fresh look. 👍

@bflad bflad closed this as completed Jul 31, 2020
@ghost
Copy link

ghost commented Jul 31, 2020

This has been released in version 3.0.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost
Copy link

ghost commented Aug 30, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Aug 30, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Addresses a defect in current functionality. service/codepipeline Issues and PRs that pertain to the codepipeline service.
Projects
None yet
Development

No branches or pull requests