Terraform Crash

[ghost]

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Grepping ERRORs from the crash.log:

2015/12/14 15:41:39 [ERROR] root.svc_pmsadmin_lb_external: eval: *terraform.EvalConfigProvider, err: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.svc_pmsadmin_lb_external: eval: *terraform.EvalSequence, err: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.svc_pmsadmin_lb_external: eval: *terraform.EvalOpFilter, err: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.svc_pmsadmin_lb_external: eval: *terraform.EvalSequence, err: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.elasticsearch_lul: eval: *terraform.EvalRefresh, err: aws_elasticsearch_domain.lul: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.elasticsearch_lul: eval: *terraform.EvalSequence, err: aws_elasticsearch_domain.lul: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.rds: eval: *terraform.EvalRefresh, err: aws_db_parameter_group.default: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.elasticsearch_lul: eval: *terraform.EvalOpFilter, err: aws_elasticsearch_domain.lul: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalRefresh, err: aws_route53_zone.build: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalSequence, err: aws_route53_zone.build: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalOpFilter, err: aws_route53_zone.build: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalSequence, err: aws_route53_zone.build: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.elasticsearch_lul: eval: *terraform.EvalSequence, err: aws_elasticsearch_domain.lul: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalRefresh, err: aws_vpc.main: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalSequence, err: aws_vpc.main: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalOpFilter, err: aws_vpc.main: unexpected EOF
2015/12/14 15:41:39 [ERROR] root: eval: *terraform.EvalSequence, err: aws_vpc.main: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.rds: eval: *terraform.EvalSequence, err: aws_db_parameter_group.default: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.rds: eval: *terraform.EvalOpFilter, err: aws_db_parameter_group.default: unexpected EOF
2015/12/14 15:41:39 [ERROR] root.rds: eval: *terraform.EvalSequence, err: aws_db_parameter_group.default: unexpected EOF

The following have had no impact:

• Restart
• Deleting cached modules (.terraform)
• Resetting .tf files back to align with tfstate files (i.e. no changes)
• Upgrade to 0.6.8
• Upping aws max_retries.

Any ideas?

[jen20]

Hi @guyrichardson - thanks for opening an issue. Anything which results in a panic is definitely a bug which we need to rectify. Do you have the entire log file available to post instead of just the [ERROR] entries?

[ghost]

Thankyou, yes.
The crash.log is around 25Mb.

Not 100% comfortable with posting it publicly but open to ideas?

[phinze]

@guyrichardson totally understand the hesitation. The important bit is the line containing panic: and the stack trace directly following that line - usually the first 2-5 lines of that stack trace are sufficient for us to diagnose the cause of the crash

[ghost]

Thanks.
I've attached the full stack trace for completeness.

trace.txt

[catsby]

Hey @guyrichardson – thanks for reporting this. It's the same issue as #4294 . It is/was reportedly a bug in the RDS API. I think they've since patched it, but we've applied a patch to Terraform as well to guard against this.

The API itself should be resolved by now, or will be soon, and our patch will go out in the next release (should be in the next few days I think)

Thanks for opening this issue and reporting this! Sorry for the trouble

[ghost]

OK, that's great.
Much appreciated.

Guy

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.