Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autoplan issue with terraform 0.12 and terragrunt 0.19 #728

Closed
milpog opened this issue Aug 6, 2019 · 17 comments
Closed

Autoplan issue with terraform 0.12 and terragrunt 0.19 #728

milpog opened this issue Aug 6, 2019 · 17 comments
Labels
question Further information is requested

Comments

@milpog
Copy link

milpog commented Aug 6, 2019

I am trying to use atlantis 0.8.3 with terraform 0.12.6 and terragrunt 0.19.16 and I am facing an issue with autoplan functionality.

Terragrunt from version 0.19.x is using terragrunt.hcl files instead of terraform.tfvars. After migration nothing is auto planned as it was previously.

I was even trying to use atlantis.yaml but apparently I cannot figure out how to set it up properly. I tried following configuration but I am still receiving information that no project was suitable for plan.

First approach

version: 3
projects:
  - dir: dev
    autoplan:
      when_modified: ["*.hcl"]

Logs

2019/08/06 19:17:35+0000 [INFO] middleware.go:42 server: POST /events – respond HTTP 200
2019/08/06 19:17:35+0000 [DBUG] project_command_builder.go:102 milpog/infra-live#129: Got workspace lock
2019/08/06 19:17:36+0000 [DBUG] project_command_builder.go:110 milpog/infra-live#129: 3 files were modified in this pull request
2019/08/06 19:17:36+0000 [DBUG] working_dir.go:79 milpog/infra-live#129: Clone directory "/home/atlantis/.atlantis/repos/milpog/infra-live/129/default" already exists, checking if it's at the right commit
2019/08/06 19:17:36+0000 [DBUG] working_dir.go:104 milpog/infra-live#129: Repo was already cloned but is not at correct commit, wanted "ca69b0872f42869f4283ae06a655ec1495de90f5" got "813615c07db85c2d38e6cd786b2d05efb8c98133"
2019/08/06 19:17:36+0000 [INFO] working_dir.go:123 milpog/infra-live#129: Creating dir "/home/atlantis/.atlantis/repos/milpog/infra-live/129/default"
2019/08/06 19:17:39+0000 [DBUG] working_dir.go:189 milpog/infra-live#129: Ran: git clone --branch Test-branch --depth=1 --single-branch https://milpog:<redacted>@github.com/milpog/infra-live.git /home/atlantis/.atlantis/repos/milpog/infra-live/129/default. Output: Cloning into '/home/atlantis/.atlantis/repos/milpog/infra-live/129/default'...
2019/08/06 19:17:39+0000 [INFO] project_command_builder.go:131 milpog/infra-live#129: Successfully parsed atlantis.yaml file
2019/08/06 19:17:39+0000 [DBUG] project_finder.go:84 milpog/infra-live#129: Checking if project at dir "dev" workspace "default" was modified
2019/08/06 19:17:39+0000 [INFO] project_command_builder.go:136 milpog/infra-live#129: 0 projects are to be planned based on their when_modified config
2019/08/06 19:17:39+0000 [INFO] command_runner.go:111 milpog/infra-live#129: Determined there was no project to run plan in

Second approach

version: 3
projects:
  - dir: dev
    autoplan:
      when_modified: ["**/*.hcl"]

Logs

2019/08/06 19:07:01+0000 [INFO] middleware.go:42 server: POST /events – respond HTTP 200
2019/08/06 19:07:03+0000 [DBUG] project_command_builder.go:102 milpog/infra-live#129: Got workspace lock
2019/08/06 19:07:03+0000 [DBUG] project_command_builder.go:110 milpog/infra-live#129: 4 files were modified in this pull request
2019/08/06 19:07:03+0000 [INFO] working_dir.go:123 milpog/infra-live#129: Creating dir "/home/atlantis/.atlantis/repos/milpog/infra-live/129/default"
2019/08/06 19:07:10+0000 [DBUG] working_dir.go:189 milpog/infra-live#129: Ran: git clone --branch Test-branch --depth=1 --single-branch https://finiata-machine-user:<redacted>@github.com/milpog/infra-live.git /home/atlantis/.atlantis/repos/milpog/infra-live/129/default. Output: Cloning into '/home/atlantis/.atlantis/repos/milpog/infra-live/129/default'...
2019/08/06 19:07:10+0000 [INFO] project_command_builder.go:131 milpog/infra-live#129: Successfully parsed atlantis.yaml file
2019/08/06 19:07:10+0000 [DBUG] project_finder.go:84 milpog/infra-live#129: Checking if project at dir "dev" workspace "default" was modified
2019/08/06 19:07:10+0000 [DBUG] project_finder.go:106 milpog/infra-live#129: File "dev/_global/aiven-project-secrets/terragrunt.hcl" matched pattern
2019/08/06 19:07:10+0000 [INFO] project_command_builder.go:136 milpog/infra-live#129: 1 projects are to be planned based on their when_modified config
2019/08/06 19:07:10+0000 [DBUG] project_command_builder.go:138 milpog/infra-live#129: Determining config for project at dir: "dev" workspace: "default"
2019/08/06 19:07:10+0000 [DBUG] global_cfg.go:319 milpog/infra-live#129: Setting apply_requirements: [approved,mergeable] from repos[1], id: /.*/
2019/08/06 19:07:10+0000 [DBUG] global_cfg.go:319 milpog/infra-live#129: Setting workflow: "default" from repos[1], id: /.*/
2019/08/06 19:07:10+0000 [DBUG] global_cfg.go:319 milpog/infra-live#129: Setting allowed_overrides: [] from repos[1], id: /.*/
2019/08/06 19:07:10+0000 [DBUG] global_cfg.go:319 milpog/infra-live#129: Setting allow_custom_workflows: false from repos[1], id: /.*/
2019/08/06 19:07:10+0000 [DBUG] global_cfg.go:171 milpog/infra-live#129: Final settings: apply_requirements: [approved,mergeable], workflow: default
2019/08/06 19:07:10+0000 [INFO] project_locker.go:74 milpog/infra-live#129: Acquired lock with id "milpog/infra-live/dev/default"
2019/08/06 19:07:10+0000 [DBUG] project_command_runner.go:136 milpog/infra-live#129: Acquired lock for project
2019/08/06 19:07:10+0000 [DBUG] working_dir.go:79 milpog/infra-live#129: Clone directory "/home/atlantis/.atlantis/repos/milpog/infra-live/129/default" already exists, checking if it's at the right commit
2019/08/06 19:07:10+0000 [DBUG] working_dir.go:101 milpog/infra-live#129: Repo is at correct commit "fe97a97f8fbf77d764a105e4aeb12b6d7ecf04bd" so will not re-clone
2019/08/06 19:07:11+0000 [DBUG] run_step_runner.go:54 milpog/infra-live#129: Error: exit status 1: running "terragrunt plan -no-color -out \"${PLANFILE}\"" in "/home/atlantis/.atlantis/repos/milpog/infra-live/129/default/dev":
[terragrunt] [/home/atlantis/.atlantis/repos/milpog/infra-live/129/default/dev] 2019/08/06 19:07:10 Running command: terraform --version
[terragrunt] 2019/08/06 19:07:10 Reading Terragrunt config file at /home/atlantis/.atlantis/repos/milpog/infra-live/129/default/dev/terragrunt.hcl
[terragrunt] 2019/08/06 19:07:10 Assuming IAM role arn:aws:iam::606443424440:role/atlantis-allow-full-access-from-other-accounts
[terragrunt] 2019/08/06 19:07:11 Did not find any Terraform files (*.tf) in /home/atlantis/.atlantis/repos/milpog/infra-live/129/default/dev
[terragrunt] 2019/08/06 19:07:11 Unable to determine underlying exit code, so Terragrunt will exit with error code 1
2019/08/06 19:07:11+0000 [INFO] command_runner.go:231 milpog/infra-live#129: Deleting plans because there were errors and automerge requires all plans succeed
2019/08/06 19:07:12+0000 [DBUG] command_runner.go:432 milpog/infra-live#129: Updating DB with pull results

Thanks for help!

@milpog
Copy link
Author

milpog commented Aug 6, 2019

My directory structure for files modified in this PR looks like this

dev
  ├── _global
  │   ├── aiven-project
  │   │   └── terragrunt.hcl
  │   └── aiven-project-secrets
  │       └── terragrunt.hcl
  └── eu-central-1
      └── aiven
          └── data-stores
              └── kafka
                  └── terragrunt.hcl

@ldormoy
Copy link

ldormoy commented Aug 7, 2019

I don't think you need a client-side config.

You should rather use a server-side repo config.

I use the same tools and versions as you, here is what works for me:

# repos.yaml
repos:
- id: github.com/my_org/my_repo
  apply_requirements: [approved]
  workflow: terragrunt

workflows:
  terragrunt:
    plan:
      steps:
      - run: terragrunt plan -no-color -out $PLANFILE
    apply:
      steps:
      - run: terragrunt apply -no-color $PLANFILE

With this config, autoplan works fine. The only bug is that it also detects the root terragrunt config file and tries to plan it.

@milpog
Copy link
Author

milpog commented Aug 7, 2019

Have you migrated all the files to new *.hcl format? As far as I remember when I still had some configuration files for terragrunt 0.18 it also worked.

My server-side repo config looks like this so it should not be a problem. But I will try again.

# repos lists the config for specific repos.
repos:
  # id can either be an exact repo ID or a regex.
  # If using a regex, it must start and end with a slash.
  # Repo ID's are of the form {VCS hostname}/{org}/{repo name}, ex.
  # github.com/runatlantis/atlantis.
  - id: /.*/

    # apply_requirements sets the Apply Requirements for all repos that match.
    apply_requirements: [approved, mergeable]

    # workflow sets the workflow for all repos that match.
    # This workflow must be defined in the workflows section.
    workflow: default

    # allowed_overrides specifies which keys can be overridden by this repo in
    # its atlantis.yaml file.
    allowed_overrides: []

    # allow_custom_workflows defines whether this repo can define its own
    # workflows. If false (default), the repo can only use server-side defined
    # workflows.
    allow_custom_workflows: false

# workflows lists server-side custom workflows
workflows:
  default:
    plan:
      steps:
        - run: terragrunt plan -no-color -out "${PLANFILE}"
    apply:
      steps:
        - run: terragrunt apply -no-color "${PLANFILE}"

@ldormoy
Copy link

ldormoy commented Aug 7, 2019

Have you migrated all the files to new *.hcl format? As far as I remember when I still had some configuration files for terragrunt 0.18 it also worked.

No I am still in the process, but the migrated files are properly discovered and planned.

My server-side repo config looks like this so it should not be a problem. But I will try again.

Your config looks fine indeed.

@milpog
Copy link
Author

milpog commented Aug 7, 2019

@ldormoy I used your configuration but still have the same issue. I tested all combinations with and without atlantis.yaml config.

@milpog
Copy link
Author

milpog commented Aug 7, 2019

I went trough atlantis code and found the part where files are filtered for auto plan functionality. It is a little strange that auto plan was run for *.hcl files for me and @ldormoy. I suspect it was because we were replacing our current terraform.tfvars files with terragrunt.hcl ones so atlantis found that given project was modified (because of removed tfvars files) and it run terragrunt plan -no-color -out "${PLANFILE}".

@ldormoy you can check by yourself whether auto plan will be run if you modify any part of the infrastructure where you already have terragrunt.hcl configuration in place. It should not work IMHO because of this part:

https://github.com/runatlantis/atlantis/blob/master/server/events/project_finder.go#L120-L131

// filterToTerraform filters non-terraform files from files.
func (p *DefaultProjectFinder) filterToTerraform(files []string) []string {
	var filtered []string
	for _, fileName := range files {
		// Filter out tfstate files since they usually checked in by accident
		// and regardless, they don't affect a plan.
		if !p.isStatefile(fileName) && strings.Contains(fileName, ".tf") {
			filtered = append(filtered, fileName)
		}
	}
	return filtered
}

It was only working with terragrunt because there is condition which is catching every file name matching *.tf* pattern and terragrunt 0.18.x followed this pattern with terraform.tfvars files for storing infrastructure configuration.

When I have changed following condition and run atlantis locally, auto plan started working also for terragrunt.hcl files.

- if !p.isStatefile(fileName) && strings.Contains(fileName, ".tf") {
+ if !p.isStatefile(fileName) && (strings.Contains(fileName, ".tf") || strings.Contains(fileName, ".hcl")) {
    filtered = append(filtered, fileName)
}

@ldormoy
Copy link

ldormoy commented Aug 7, 2019

oops you are right, I have the same problem.

@lkysow looks like @milpog change could be a solution?

If we go this way, could it be possible to use this opportunity to tackle the root terragrunt configuration issue?

repo
    |__terragrunt.hcl
    |__frontend
    |        |__terragrunt.hcl
    |
    |__backend
    |        |__terragrunt.hcl
    |
    |__modules

The frontend and backend terragrunt.hcl are legit projects, but the the root terragrunt.hcl is a config file that is not meant to be found by atlantis. To my knowledge, all terragrunt repositories work this way (see https://github.com/gruntwork-io/terragrunt#quick-start for details).

@lkysow
Copy link
Member

lkysow commented Aug 7, 2019

That function isn't used when autoplan.when_modified is set.

Also, it starts looking in the directory specified in your project setting. If you want it to find .hcl files that are further below that directory you need to specify that:

a/
  b/
    c/
      file.hcl
projects:
- dir: a
  autoplan:
    when_modified: [*.hcl] # won't match
    when_modified: [b/c/*.hcl] # will match
    when_modified: [**/*.hcl] # will match

@milpog
Copy link
Author

milpog commented Aug 7, 2019

@lkysow it looks like when_modified: ["**/*.hcl"] is not working as I stated in issue description. At least for me.

atlantis is recognizing that dev/_global/aiven-project-secrets/terragrunt.hcl was modified...

2019/08/06 19:07:10+0000 [DBUG] project_finder.go:106 milpog/infra-live#129: File "dev/_global/aiven-project-secrets/terragrunt.hcl" matched pattern

but instead of running atlantis plan -d dev/_global/aiven-project-secrets it is running atlantis plan -d dev.

2019/08/06 19:07:11+0000 [DBUG] run_step_runner.go:54 milpog/infra-live#129: Error: exit status 1: running "terragrunt plan -no-color -out \"${PLANFILE}\"" in "/home/atlantis/.atlantis/repos/milpog/infra-live/129/default/dev":

@lkysow
Copy link
Member

lkysow commented Aug 8, 2019

but instead of running atlantis plan -d dev/_global/aiven-project-secrets it is running atlantis plan -d dev.

That's expected behaviour. You need to configure the root of each Terraform project as a directory. It sounds like you need:

projects:
- dir: dev/_global/aiven-project-secrets/

Terraform projects are not multi-directory, that's why it's built this way.

@andrewmackett
Copy link

@lkysow Thanks for the clarification. I've been having the same issue when trying to setup Atlantis with Terragrunt for the first time.

Really wish we weren't using Terragrunt right now. I have a lot of dir to configure!

@milpog
Copy link
Author

milpog commented Aug 8, 2019

@lkysow would you consider implementing solution I proposed in #728 (comment) or something similar but in line with your vision of developing atlantis. I love atlantis but it would be huge pain for us to define each directory for our terragrunt setup.

We will probably fork atlantis and build our own binaries after each release to work with terragrunt seamlessly but I would like to avoid that if possible.

@lkysow lkysow added the question Further information is requested label Aug 9, 2019
@mmclane
Copy link

mmclane commented Aug 13, 2019

I am looking at the same issue. I would like it to pick up that .hcl files changed and not just tf files without have to specify per directory. I don't see a way to set this in the server side config

@kipkoan
Copy link
Contributor

kipkoan commented Aug 13, 2019

@milpog - maybe you guys can consider making this configurable? Have the default be how it is now, thus changing nothing, but allow you to configure it to pick up .hcl files (or any other type of files that someone may want to configure it to autoplan on).

@lkysow
Copy link
Member

lkysow commented Aug 14, 2019

@lkysow would you consider implementing solution I proposed in #728 (comment) or something similar but in line with your vision of developing atlantis. I love atlantis but it would be huge pain for us to define each directory for our terragrunt setup.

We will probably fork atlantis and build our own binaries after each release to work with terragrunt seamlessly but I would like to avoid that if possible.

I think a top-level autoplan.when_modified might make sense.

@kipkoan
Copy link
Contributor

kipkoan commented Aug 15, 2019

Just abstract the current autoplan logic. Maybe that's what the autoplan.when_modified could be.

@lkysow
Copy link
Member

lkysow commented Aug 26, 2019

This was closed by #748

@lkysow lkysow closed this as completed Aug 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

6 participants