sometimes I only want to run the roots that have changed

[spilliams]

What makes it so that a root is worth running?

1. the scope data changed
2. the root itself changed
3. the modules the root relied on changed? not sure how to do this quickly without either parsing the entire AST or assuming semver sources

How to get an exhaustive affected list?

1. if a root module's source code changes (the .tf files), the whole root is affected. But maybe only the scopes that adopt the latest version?
2. if a root module's config changes (terrascope.hcl), it depends
   1. if it's in a scope match, where some scopes are getting new attributes (which is something I haven't fully decided is a good idea yet), those scopes are affected.
3. y'know what, for now lets keep it a little broad and say, if any files in the root (but not in .terrascope/) have changed, the whole root is affected for all matching scopes.
4. scope data can change.
   1. If scope data changes, the whole scope could be affected. or maybe not, maybe only one sub-sub-sub scope uses a specific attribute.
   2. That, again, is oversolving. For now, we should hash each scope value, and save all the hashes in a file that's checked in. Then, when we're determining affected we compute a new hash, and figure out which scopes have changed.
   3. things to hash: all attributes in the scope
   4. also, detect if a scope is created or destroyed.
   5. if you don't want to check a hashfile into git, maybe the affected command clones the repo in a temp dir, runs a function to hash the clone, and runs the same function to hash the currect repo (or second clone).
   6. The other thing that can change (again, unimplemented) is wired outputs from root dependencies. Maybe what I need is a hash of the generated .tfvars file, and a matrix of scope and root. map[scope][root]hash turns into map[scope][root]affected? This would also solve the issue above, with parent scope attributes that are not used by all the child scopes.
   7. Yes, I'm more convinced now: the two things that can change in a root are the input values and the configuration itself. If the configuration changes (.tf files), the whole root is affected. If the input values change, certain scopes of the root are affected.

[spilliams]

Next idea: hashes.

First of all, the build step is required. There's no such thing as "only build the affected roots". We need to build in order to determine affected. At the end of the build phase, the builder should save a file in the root. This file will contain a history of hash sums for each of the build contexts in that root. This file probably should be checked in. As well, any CI/CD process that applies a root context should make sure its build sum is in the file, checked in, and pushed up.

When hashing, it may be important to know if there are any working or staged changes. I would call this a "dirty" hash, since it's impossible to replicate it without knowing those untracked changes.

The location of the file could be something like this:

terraform/roots/my-cool-bucket
├─ .terrascope/all/my/scopes/
│  ├─ .terrascope.context.hcl
│  ├─ main.tf
│  └─ terrascope.auto.tfvars
├─ .terrascope.sum.hcl
├─ main.tf
└─ terrascope.hcl

and the contents of the file might be like this:

"buildSums" : {
  "all/my/scopes" : {
    "GITSHA1" : { "date" : "2023-04-05T08:50:00-0700", "sum": "abcdef123456" }
  }
}

When the user says "apply this root in all affected scopes between GITSHA1 and GITSHA2", terrascope will:

1. check out GITSHA1
2. build the root for all scopes (applying a scope filter if specified). This saves a sum in .terrascope.sum.hcl
3. check out GITSHA2
4. build again
5. If the second sum differs from the first, apply. Perhaps applying a root scope should record in the sum file as well? Kind of a "history of applies"? (There might be some difficulty in making sure this record is 100% complete? I haven't finished thinking it through.)

I think I had other ideas while writing this, but I have to go now, so I'll have to remember them later.