[WIP] JSON plan format #4610
[WIP] JSON plan format #4610
Conversation
Making the Plan, Diff and various configuration structures JSON-serializable creates the possibility for machine-readable Terraform output of various kinds.
There are some steps that get done before writing a state to disk, and a converse set of steps that get done after reading a serialized state file back from disk. Here we factor out those operations so that they can be used when a state is included as part of another structure that is being serialized.
Previously Terraform used "gob" to create an opaque binary serialization of a plan. Now we instead generate a JSON serialization, which is more friendly to external tools doing tasks such as plan visualization or plan linting, allowing application-specific tools to apply extra checks on plans or present certain plans in a different way to human operators.
|
See also # #3170 |
|
So the difference between this and #3170 is that here the JSON plan format becomes the authoritative format. So this contains all the annotations for diffs, like the #3170, but also for resources, modules, &c. Still needs tests, plus update to resolve with master. @apparentlymart you still working on this? This would be spectacularly useful for testing. |
|
I do plan to pick it up again but I'm stalled right now due to other priorities limiting how much time I can spend on Terraform for the next few weeks. If someone else wanted to pick this up and run with it in the mean time I wouldn't object, since I'd love to have this capability whether or not I'm the one that implemented it 😀 . It would also be helpful if others could try this out against some of their (non-production!) use-cases to see if it does anything weird, since this is changing the serialization of a lot of things all at once and so it's unlikely that my testing would catch all possible scenarios. |
|
I'm not planning to continue down this path right now. There are some concerns that it might tempt people to try to hand-edit plans, which can cause weird effects when applying. All of the use-cases I had were for read-only consumption of plans, which #3170 would also address, so I'm closing this one in favor of that one. |
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
ghost
locked and limited conversation to collaborators
This change switches to a JSON serialization format for plans, rather than the
gob-based binary serialization.The motivation for this is to enable application-specific extra tools to do various sorts of checks or visualizations on the generated plan, to guide operators. For example:
Implementing this entails adding annotations to all of the structures included in a plan so that they can be JSON-serialized in a reasonable way. For just this change the diff feels rather disruptive, but it feels more reasonable if it's considered also a building block to enable machine readable progress output of various sorts as described in #2460: having a defined JSON serialization for Terraform's key structures makes Mitchell's suggestion there (having a new UI hook that produces JSON output) easier to achieve.
My intent is that at first this wouldn't really be documented as a stable format; similar to the JSON state files, it's an implementation detail that happens to be human- and machine-readable to enable expert users to do certain things that wouldn't otherwise be possible, as long as they're willing to maintain those things as the format evolves.
Whereas JSON state files are long-lived and so deserve their own version numbering, I figured that plan files are very short-lived and so it's reasonable to refuse to apply plan files generated by different versions of Terraform. It seems pretty unlikely that someone would upgrade Terraform between making a plan and applying that plan, and even if they do it's a trivial matter to produce an updated plan with
terraform plan -refresh=falseto update the format.This seems to basically work but I've not tested it extensively with various kinds of input and different sorts of plan. I intend to do some more thorough testing, but I figured I'd share this early to get feedback on the idea and in case others might want to try creating JSON plans against their own configurations and let me know if they run into any issues.