Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error #131

leros321 · 2019-10-25T13:44:29Z

After destroying our openstack instance with terraform v0.12.10 then terraform-inventory v0.9 crash with following message and our pipeline cannot continue:
Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)
Of course, we understand terraform.tfstate is empty, but it did not happen with terraform v0.11.14 and terraform-inventory v0.8
Below is terraform.tfstate:
{ "version": 4, "terraform_version": "0.12.10", "serial": 12, "lineage": "24894aa7-228d-1dc3-0120-42a4be3cdc57", "outputs": {}, "resources": [] }

The text was updated successfully, but these errors were encountered:

paulmao1 · 2019-10-31T00:49:22Z

+1

qasimraz · 2019-11-05T00:48:41Z

+1

justinas · 2019-11-06T17:08:54Z

Looks like by default Terraform tries to directly read terraform.tfstate which matches the output of terraform state pull. However, terraform show -json produces a different JSON, where resources are nested under .values.root_module, like the program expects for a 0.12+ file.

I was able to make it work by setting a var:

export TF_STATE=.

By pointing terraform-inventory to a directory instead of a file, it takes a completely different code path that properly utilizes terraform show -json.

I am not sure what the best fix would be here. Maybe terraform-inventory should default to terraform show -json at all times?

screenman · 2019-11-07T07:28:33Z

+1

paulmao1 · 2019-11-08T05:23:27Z

thanks  your reply. it works now.

…

---Original--- From: "Justinas Stankevičius"<notifications@github.com> Date: Thu, Nov 7, 2019 01:09 AM To: "adammck/terraform-inventory"<terraform-inventory@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"paulmao1"<81876776@qq.com>; Subject: Re: [adammck/terraform-inventory] Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error (#131) Looks like by default Terraform tries to read terraform.tfstate which matches the output of terraform state pull. However, terraform show -json produces a different JSON, where resources are nested under .values.root_module, like the program expects for a 0.12+ file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

paulmao1 · 2019-11-08T05:25:51Z

And can you me a sample for playbook file. The following is my file and get an error: BR Paul ------------------ 原始邮件 ------------------ 发件人: "81876776"<81876776@qq.com>; 发送时间: 2019年11月8日(星期五) 中午1:23 收件人: "adammck/terraform-inventory"<reply@reply.github.com>;"adammck/terraform-inventory"<terraform-inventory@noreply.github.com>; 抄送: "Comment"<comment@noreply.github.com>; 主题: Re: [adammck/terraform-inventory] Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error (#131) thanks  your reply. it works now.

…

---Original--- From: "Justinas Stankevičius"<notifications@github.com> Date: Thu, Nov 7, 2019 01:09 AM To: "adammck/terraform-inventory"<terraform-inventory@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"paulmao1"<81876776@qq.com>; Subject: Re: [adammck/terraform-inventory] Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error (#131) Looks like by default Terraform tries to read terraform.tfstate which matches the output of terraform state pull. However, terraform show -json produces a different JSON, where resources are nested under .values.root_module, like the program expects for a 0.12+ file. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

sprnza · 2019-12-30T09:30:31Z

The proposed workaround doesn't work for me for some reason:
Terraform v0.12.18
terraform-ansible a408e89

justinas · 2019-12-30T09:56:14Z

Seems to work for me:

$ terraform version
Terraform v0.12.18
+ provider.digitalocean v1.12.0
$ terraform-inventory --list
Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)
$ TF_STATE=. terraform-inventory --list
{"all":{"hosts":["185.14.185.236"],"vars":{}},"test1":["185.14.185.236"],"test1_0":["185.14.185.236"],"type_digitalocean_droplet":["185.14.185.236"]}
$ TF_STATE=. ansible -i $(which terraform-inventory) -u root -m ping all
185.14.185.236 | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

terraform.tf

provider "digitalocean" {}

data "digitalocean_ssh_key" "j" {
  name = "j"
}

resource "digitalocean_droplet" "test1" {
    name = "test1"
    region = "ams2"
    size = "s-1vcpu-1gb"
    image = "ubuntu-18-04-x64"
    ssh_keys = [data.digitalocean_ssh_key.j.id]

    provisioner "remote-exec" {
        inline = [
            "apt update",
            "apt install -y python3",
        ]
    }
}

sprnza · 2019-12-30T10:37:53Z

Yep, sorry it appeared to be this issue #133

EugenDueck · 2020-02-29T22:24:45Z

I'm getting the same error. The TF_STATE=. workaround does not work for me. I tried terraform-inventory 0.9 and HEAD -> master as of Feb 29 (c7e468f).

$ terraform-inventory --inventory

Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)

$ TF_STATE=. terraform-inventory --inventory

Error reading Terraform state: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)

$ terraform-inventory --version

terraform-inventory version 0.9

$ terraform --version

Terraform v0.12.21
+ provider.aws v2.51.0
+ provider.random v2.2.1

$ cat terraform.tfstate

{
  "version": 4,
  "terraform_version": "0.12.21",
  "serial": 32,
  "lineage": "72c6e67a-e500-430a-341e-dab290bbf342",
  "outputs": {},
  "resources": []
}

justinas · 2020-04-02T14:20:13Z

Hi again everyone.

I have summarized the format differences below.

Terraform 0.11

A: `state pull` & `terraform.tfstate`

{
  "version": 3,
  "terraform_version": "0.11.14",
  "serial": 1,
  "lineage": "3c9951c1-3229-ca05-2cf3-25d8472a9d51",
  "modules": [
	{
	  "path": [
		"root"
	  ],
	  "outputs": {},
	  "resources": {
		"digitalocean_droplet.hanginx.0": {...}
	  }
	}
  ]
}

Terraform 0.12

B: `show -json`

{
  "format_version": "0.1",
  "terraform_version": "0.12.24",
  "values": {
	"root_module": {
	  "resources": [
		{...}
      ]
    }
  }
}

C: `state pull` & `terraform.tfstate`

{
  "version": 4,
  "terraform_version": "0.12.24",
  "serial": 6,
  "lineage": "1239f061-72f1-7e7b-161a-cb4bf72005f1",
  "outputs": {},
  "resources": [
	{...}
  ]
}

~~The problem right now is that TI tries to use terraform show -json but it can not actually parse that output (!!!)~~

Edit: the above statement is false. For 0.11 we support state pull or terraform.tfstate (the same format). For 0.12 we only support show -json, but not terraform.tfstate. Yet, we still try to read that file directly, because even if TF_STATE is not set, we default to terraform.tfstate.

I did not dig too deep here so I do not know if we actually have a hard requirement for something that is not present in state pull, but present in show -json or if it was just used as a new feature that could help us identify the version.

Anyway, I suggest to either drop support for 0.11 (0.12 has been out since May 2019 and people on 0.11 can keep using older versions of TI) OR detect version by parsing terraform version, terraform.tfstate, etc.

Having done that, stop calling show -json at all and use state pull that will match the format of terraform.tfstate.

I suggest this in order to avoid having to maintain 3 parsers: one for 0.11, one for 0.12 show -json, one for 0.12 state files.

@adammck we really need your thoughts here.

justinas · 2020-04-02T14:58:31Z

@AndiDog your input would also be appreciated.

It is not clear to me whether we should use show -json or state pull. The output of show -json is well documented, but it does not match the format of state files. The easiest thing here would be to simply drop supporting raw state files and only allow using show -json.

terraform state pull on the other hand matches the state file, but is not documented well. However, the website indicates:

This is useful for reading values out of state (potentially pairing this command with something like jq).

So it seems that this is a valid use-case.

AndiDog · 2020-04-21T18:44:01Z

The error is clear enough: the state is empty. And it doesn't make sense to use terraform-inventory on empty state, right? If people have a different opinion, we could change the error to a warning in case state is empty (and assume Terraform 0.12).

justinas · 2020-04-21T21:09:53Z

But the state file is not empty, it just contains a format we do not expect (B vs C in this comment). This is an obvious regression where previously pointing to either a directory or an individual state file worked, but now only a directory is processed correctly (and it is not even a default).

As I see it, out of the box (read: without pointing it to a directory via the env var) terraform-inventory 100% does not work with 0.12.

AndiDog · 2020-04-22T06:04:18Z

Understood now. Let me put this in a reproducible example:

$ rm -rf /tmp/example && mkdir /tmp/example && cd /tmp/example

$ cat >main.tf <<EOF
provider "null" {}

resource "null_resource" "example" {
  provisioner "local-exec" {
    command = "true"
  }
}
EOF

$ terraform init
[...]

$ terraform apply -auto-approve
[...]

$ jq -c . terraform.tfstate
{"version":4,"terraform_version":"0.12.24","serial":1,"lineage":"3e8f9aa4-9daf-322e-e68f-dcaebf391f0b","outputs":{},"resources":[{"mode":"managed","type":"null_resource","name":"example","provider":"provider.null","instances":[{"schema_version":0,"attributes":{"id":"3973960162525722052","triggers":null},"private":"bnVsbA=="}]}]}

$ terraform show -json
{"format_version":"0.1","terraform_version":"0.12.24","values":{"root_module":{"resources":[{"address":"null_resource.example","mode":"managed","type":"null_resource","name":"example","provider_name":"null","schema_version":0,"values":{"id":"3973960162525722052","triggers":null}}]}}}

$ terraform show -json .
[...]
Plan read error: read .: is a directory

$ terraform show -json terraform.tfstate
{"format_version":"0.1","terraform_version":"0.12.24","values":{"root_module":{"resources":[{"address":"null_resource.example","mode":"managed","type":"null_resource","name":"example","provider_name":"null","schema_version":0,"values":{"id":"3973960162525722052","triggers":null}}]}}}

$ terraform-inventory --list
Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)

$ terraform-inventory --list . ; echo
{"all":{"hosts":[],"vars":{}}}

$ TF_STATE=. terraform-inventory --list ; echo
{"all":{"hosts":[],"vars":{}}}

I suggest we drop 0.11 support and always call terraform show -json - no implementation effort needed apart from removing lots of old code. With the current implementation, 0.12 format parsing has been working without issues since I implemented it. terraform show -json gives the current state by default, and we should use that default as well. If TF_STATE/TI_STATE is defined explicitly, we can still pass it on to terraform show -json <argument> (changing TF_STATE=. to TF_STATE=terraform.tfstate with a warning, because only terraform show -json terraform.tfstate works fine).

ssarkarnetip · 2020-05-07T20:54:10Z

I am having same issue in AWS. Everything worked great in terraform 11.

alor · 2020-08-18T13:15:39Z

anyone already tested with terraform 0.13? asking before upgrading...

[UPDATE] I've upgraded to 0.13 and the issue is still the same and the workaround works even with 0.13. at least it's safe to upgrade.

patsevanton · 2020-10-30T13:08:27Z

I build terraform-inventory from source code

Same error:
terraform-inventory --list Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error: <nil> (nil error means no content/modules found in the respective format)

patsevanton · 2020-10-30T15:12:11Z

terraform-inventory build from source.

Try on repo https://github.com/yandex-cloud/examples/tree/master/active-directory

$ terraform -version
Terraform v0.11.14
+ provider.yandex v0.46.0

$ terraform show -json .
Usage: terraform show [options] [path]

  Reads and outputs a Terraform state or plan file in a human-readable
  form. If no path is specified, the current state will be shown.

Options:

  -module-depth=n     Specifies the depth of modules to show in the output.
                      By default this is -1, which will expand all.

  -no-color           If specified, output won't contain any color.

$ terraform show -json terraform.tfstate
Usage: terraform show [options] [path]

  Reads and outputs a Terraform state or plan file in a human-readable
  form. If no path is specified, the current state will be shown.

Options:

  -module-depth=n     Specifies the depth of modules to show in the output.
                      By default this is -1, which will expand all.

  -no-color           If specified, output won't contain any color.

$ terraform-inventory --list
Error running `terraform show -json` in directory /home/user/examples/active-directory, exit status 1, falling back to trying Terraform pre-0.12 command
{"ad":["10.0.0.26"],"ad_0":["10.0.0.26"],"all":{"hosts":["10.0.0.26"],"vars":{}},"type_yandex_compute_instance":["10.0.0.26"]}

$ terraform-inventory --list . ; echo
Error running `terraform show -json` in directory /home/user/examples/active-directory, exit status 1, falling back to trying Terraform pre-0.12 command
{"ad":["10.0.0.26"],"ad_0":["10.0.0.26"],"all":{"hosts":["10.0.0.26"],"vars":{}},"type_yandex_compute_instance":["10.0.0.26"]}

$ TF_STATE=. terraform-inventory --list ; echo
Error running `terraform show -json` in directory /home/user/examples/active-directory, exit status 1, falling back to trying Terraform pre-0.12 command
{"ad":["10.0.0.26"],"ad_0":["10.0.0.26"],"all":{"hosts":["10.0.0.26"],"vars":{}},"type_yandex_compute_instance":["10.0.0.26"]}

justinas mentioned this issue Apr 2, 2020

Error reading tfstate: 0.12 error #144

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error #131

Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error #131

leros321 commented Oct 25, 2019 •

edited

Loading

paulmao1 commented Oct 31, 2019

qasimraz commented Nov 5, 2019

justinas commented Nov 6, 2019 •

edited

Loading

screenman commented Nov 7, 2019

paulmao1 commented Nov 8, 2019 via email

paulmao1 commented Nov 8, 2019 via email

sprnza commented Dec 30, 2019

justinas commented Dec 30, 2019 •

edited

Loading

sprnza commented Dec 30, 2019

EugenDueck commented Feb 29, 2020 •

edited

Loading

justinas commented Apr 2, 2020 •

edited

Loading

justinas commented Apr 2, 2020 •

edited

Loading

AndiDog commented Apr 21, 2020

justinas commented Apr 21, 2020

AndiDog commented Apr 22, 2020

ssarkarnetip commented May 7, 2020

alor commented Aug 18, 2020 •

edited

Loading

patsevanton commented Oct 30, 2020 •

edited

Loading

patsevanton commented Oct 30, 2020

Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error #131

Error reading tfstate file: 0.12 format error: <nil>; pre-0.12 format error #131

Comments

leros321 commented Oct 25, 2019 • edited Loading

paulmao1 commented Oct 31, 2019

qasimraz commented Nov 5, 2019

justinas commented Nov 6, 2019 • edited Loading

screenman commented Nov 7, 2019

paulmao1 commented Nov 8, 2019 via email

paulmao1 commented Nov 8, 2019 via email

sprnza commented Dec 30, 2019

justinas commented Dec 30, 2019 • edited Loading

sprnza commented Dec 30, 2019

EugenDueck commented Feb 29, 2020 • edited Loading

justinas commented Apr 2, 2020 • edited Loading

Terraform 0.11

A: state pull & terraform.tfstate

Terraform 0.12

B: show -json

C: state pull & terraform.tfstate

justinas commented Apr 2, 2020 • edited Loading

AndiDog commented Apr 21, 2020

justinas commented Apr 21, 2020

AndiDog commented Apr 22, 2020

ssarkarnetip commented May 7, 2020

alor commented Aug 18, 2020 • edited Loading

patsevanton commented Oct 30, 2020 • edited Loading

patsevanton commented Oct 30, 2020

leros321 commented Oct 25, 2019 •

edited

Loading

justinas commented Nov 6, 2019 •

edited

Loading

justinas commented Dec 30, 2019 •

edited

Loading

EugenDueck commented Feb 29, 2020 •

edited

Loading

justinas commented Apr 2, 2020 •

edited

Loading

A: `state pull` & `terraform.tfstate`

B: `show -json`

C: `state pull` & `terraform.tfstate`

justinas commented Apr 2, 2020 •

edited

Loading

alor commented Aug 18, 2020 •

edited

Loading

patsevanton commented Oct 30, 2020 •

edited

Loading