File changes in modules are not detected

[ntietz]

Our project is set up in one of the ways listed in the README. Basically, we have this structure:

- infra
    - env
        - production.tfvars
        - staging.tfvars
    - modules
        - module1
            - main.tf
        - module2
            - main.tf
    - main.tf

When I make a change to infra/modules/module1/main.tf, the result of atlantis plan staging is always "Plan Failed: No Terraform files were modified."

When I make a trivial change to infra/main.tf (added a comment to trick Atlantis into realizing there is a change), the plan works as expected.

I can see two possible solutions:

1. Fix the file change detector to handle changes, even when nested very deep (this is preferable, and I am willing to contribute a fix if someone points me in the right direction).
2. Add ability to do a plan even if a change wasn't detected (Provide ability to run plan even if Atlantis doesn't think there was a change #144) which would also help for other corner cases (I'm also willing to contribute here, if someone wants; maybe add a "force" option).

[anubhavmishra]

@ntietz yes, that is currently the expected behaviour. We ignore files/directories that have modules, _modules, terraform.tfstate, terraform.tfstate.backup in the name. This is done on purpose to support smart terraform plan generation. If we weren't ignoring the above files then we would try running plan in the modules directory which would fail since those files are part of a terraform module that is being used somewhere else. In reality the plan should run on in the directory/project where the module is being included which currently atlantis doesn't know about. But all that being said we are looking to support this feature in the future! I hope that answered your question :)

[ntietz]

@anubhavmishra I'm not sure that I understand the rationale here for the expected behavior. Perhaps our use case is different or we are using Terraform differently from others. We have a number of modules we have defined ourselves, which we update fairly regularly as our infra needs evolve.

How do Atlantis users typically deal with changes in a module, and having that change get reflected in the plan? Does a lot of content get stuffed into main.tf, or is there another method of organizing the Terraform project that I'm not aware of? (I'm pretty new to Terraform, so apologies if this is a dumb question.)

As mentioned before, the fix suggested in issue #144 (providing the ability to trigger a plan even if no change is detected) would work for us here. I imagine it would be something like "atlantis plan staging force". I'm willing to work on a contribution for that, if someone points me in the right direction.

[anubhavmishra]

Hey @ntietz,
Yes, you can have local modules as part of the project. The project that is using the modules is the place where we want to be running terraform plan from. But currently Atlantis uses the files changed in the pull request to figure out where the plan should be run from. So, if you can changing the files in the directory that has your project(that uses the local modules), you will be able to generate a successful plan BUT if you are changing the local modules stored in modules/ or _modules/ directories Atlantis will ignore it because if Atlantis were to plan from those directories that contain your modules files the terraform plan will fail.

For now what you can do is that you can change files in the project that uses the modules. So, for example a hack would be to just add a whitespace in the main.tf file to force a change in the project that uses the local modules and then Atlantis will consider that a valid change and generate you a plan.

As I said earlier, we will support this use case soon!

Also at Hootsuite we had modules that were hosted on git and were pulled in when needed by main.tf. We didn't use local modules. That is a good practice that helps keep semantic versioning of the modules that are shared by projects.