Why Job doesn't fail when terraform show errors

[francardoso93]

All jobs are completed and successful even though something went wrong with "init", "plan" or "apply".
Why? Wouldn't it be better to fail so we can collect metrics and do something about it without having to check pod logs?

[isaaguilar]

@francardoso93 I agree that it's odd that the pod completes and is marked as successful when something goes wrong. I personally haven't had an issue checking logs when I see something didn't occur that I expected to occur, but I think you're right that a failure metric is way more useful than waiting for other errors to happen if tf didn't run.

The script that powers the tfo runner (in v0.3.x) is https://github.com/isaaguilar/terraform-operator/blob/master/docker/terraform/run.sh. I think it'd be pretty easy to inject an exit flag that gets updated on init, plan, or apply events, then exit with that exit code. My problem with updating this script is actually a big flaw in TFO's design which is that the runners aren't version-able. (NOTE I'm fixing the runner-versioning flaw in v0.4.0+ releases).

So what I'll do is test out updating the script with an exit code flag and see if that works to get a pod to fail without any repercussions.

PS In v0.4.0+ versions, init, plan, and apply are all executed in different pods and do fail upon error. This stops the terraform workflow and even updates the terraform resource's Status. I'm still working on docs and testing for this design change.

[isaaguilar]

I'd also like to mention that users can create their own runner. It's very poorly documented, but hopefully this doc can help someone get started if they themselves wanted to changes the behaviour of init, plan, and apply: https://github.com/isaaguilar/terraform-operator/blob/master/docs/architecture.md#custom-terraform-runner