Terraform (HCL) (*.tf): new parser #2952

antonysouthworth-halter · 2021-04-11T22:27:55Z

Add support for Terraform (HCL) files.

How to add support for Terraform (HCL) preservim/tagbar#757

TODO

add unit tests
update documentation
- manpages
- codebase

antonysouthworth-halter · 2021-04-11T22:29:05Z

Have opened this as a quick thing, please advise if there are other things I need to do before merging (I read through https://docs.ctags.io/en/latest/optlib.html but may have missed some things).

masatake · 2021-04-11T22:46:16Z

main/parsers_p.h

@@ -149,6 +149,7 @@
 	SystemTapParser, \
 	TclParser, \
 	TclOOParser, \
+  TfParser, \


TerraformParser or TerraFormParser is better.

masatake · 2021-04-11T22:49:08Z

optlib/terraform.ctags

+# GNU General Public License version 2 or (at your option) any later version.
+#
+# Derived from `vim-terraform-completion`:
+#   - https://github.com/juliosueiras/vim-terraform-completion/blob/master/ctags/terraform.ctags


If you reuse the regex patterns of @juliosueiras's work, ask @juliosueiras to allow to use one's .ctags in GPLv2 or later. Copy & paste the evidence of alowing to use here as comments.

what would I need to paste?

Could you fill and paste it at the head of your .ctags?

# # Copyright (c) 2021, <YOUR NAME HERE> # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; either version 2 # of the License, or (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, # USA. #

masatake · 2021-04-11T22:49:46Z

optlib/terraform.ctags

+#   - https://github.com/juliosueiras/vim-terraform-completion/blob/master/ctags/terraform.ctags
+#
+# Changed the name from `terraform` to `tf` so vim will recognise it properly based
+# on file extension (*.tf).


Do you mean the name of the parser must be "tf"?

masatake · 2021-04-11T22:52:51Z

optlib/terraform.ctags

+#   - https://www.terraform.io/docs/language/values/locals.html
+
+--langdef=tf
+--map-tf=+tf:.tf.tfvars


I guess this should be

--map-tf=+.tf --map-tf=+.tfvars

See https://docs.ctags.io/en/latest/man/ctags.1.html#options for more details.

masatake · 2021-04-11T22:54:13Z

optlib/terraform.ctags

+
+--langdef=tf
+--map-tf=+tf:.tf.tfvars
+--regex-tf=/^[[:space:]]*resource[[:space:]]*"([^"]*)"[[:space:]]*"([^"]*)"/\2/r,Resource/


Use --kinddef-=... option before using kinds.
https://docs.ctags.io/en/latest/man/ctags-optlib.7.html#options

masatake · 2021-04-11T22:54:44Z

win32/ctags_vs2013.vcxproj

Don't put a newline at the end of file.

masatake · 2021-04-11T22:55:14Z

win32/ctags_vs2013.vcxproj.filters

Don't put a newline at the end of file.

masatake · 2021-04-11T22:58:23Z

optlib/terraform.ctags

+# A notable abscence is `local`, because `locals` are defined inside a block and
+# it's way harder to write a good regex for that.
+#   - https://www.terraform.io/docs/language/values/locals.html
+


Could you put a URL for the language reference?

antonysouthworth-halter · 2021-04-11T23:10:55Z

@masatake thank you for fast and thorough review 😄, I will address comments as/when I can.

@juliosueiras apologies, perhaps I should have talked to you first? 😅

juliosueiras · 2021-04-11T23:11:52Z

@antonysouthworth-halter no, thats fine

masatake · 2021-05-03T06:28:41Z

optlib/terraform.ctags

+# it's way harder to write a good regex for that.
+#   - https://www.terraform.io/docs/language/values/locals.html
+
+--langdef=tf


Terraform or TerraformHCL is better. I'm not sure which one is better.
If you are not sure, too, use longer name as I did in "PupppetManifest".

masatake · 2021-05-03T06:29:39Z

win32/ctags_vs2013.vcxproj

About this file, no newline at the of file

masatake · 2021-05-03T06:29:41Z

win32/ctags_vs2013.vcxproj.filters

About this file, no newline at the end of file.

masatake · 2021-05-03T06:54:20Z

optlib/terraform.ctags

+--kinddef-terraform=o,Output,Terraform Output
+--regex-terraform=/^[[:space:]]*resource[[:space:]]*"([^"]*)"[[:space:]]*"([^"]*)"/\2/r,Resource/
+--regex-terraform=/^[[:space:]]*data[[:space:]]*"([^"]*)"[[:space:]]*"([^"]*)"/\2/d,Data/
+--regex-terraform=/^[[:space:]]*variable[[:space:]]*"([^"]*)"/\1/v,Variable/


v is already defined with --kinddef-.... So you don't have to specify v,Variable. Just v is enough:

--regex-Terraform=/^[[:space:]]*variable[[:space:]]*"([^"]*)"/\1/v/

I don't know well about the syntax of Terraforms. However, I guess a whitespace may be needed after variable keyword. We don't record a variable having an empty string as name. So + is better than *.

--regex-Terraform=/^[[:space:]]*variable[[:space:]]+"([^"]+)"/\1/v/

This is applicable to other lines of --regex-....

masatake · 2021-05-03T06:55:42Z

optlib/terraform.ctags

--regex-tf=/^[[:space:]]*module[[:space:]]*"([^"]*)"/\1/m,Module/
--regex-tf=/^[[:space:]]*output[[:space:]]*"([^"]*)"/\1/o,Output/
--regex-tf=/^([a-z0-9_]+)[[:space:]]*=/\1/f,TFVar/
+--langdef=terraform


Sorry, I forgot writing one thing. A parser that is part of ctags should have Capitalized name.
Please, do s/terraform/Terraform/g.

tonglil · 2021-08-10T21:55:08Z

I was able to add locals parsing if this PR still has any traction.

masatake · 2021-08-10T21:57:48Z

@tonglil, feel free to make a new pull request that includes the changes of this pull request.
(It means you take over this pull request.)

masatake · 2021-08-10T22:00:43Z

The biggest reason I didn't merge this is there is no test case.
@tonglil do you have an interest in polishing this pull request?

tonglil · 2022-01-26T21:12:49Z

Sorry, I will not be able to contribute to this PR.

Haggus · 2023-03-29T16:53:07Z

@masatake I'd like to take over and finish this PR

masatake · 2023-03-29T18:31:19Z

@Haggus You are welcome. optlib/terraform.c is automatically generated by misc/optlib2c.
So what you have to edit is optlib/terraform.ctags.
Test cases are needed.

antonysouthworth-halter · 2023-03-30T20:50:00Z

thanks @Haggus , unfortunately I never got time to come back to this. @masatake should we close this PR in favour of #3683 ?

masatake · 2023-03-31T03:34:55Z

I will close this via #3684.

masatake · 2023-04-01T19:17:06Z

@antonysouthworth-halter, thank you.

antonysouthworth-halter mentioned this pull request Apr 11, 2021

How to add support for Terraform (HCL) preservim/tagbar#757

Open

masatake reviewed Apr 11, 2021

View reviewed changes

antonysouthworth-halter mentioned this pull request Apr 11, 2021

Add Terraform (HCL) (*.tf) support preservim/tagbar#758

Open

masatake reviewed Apr 11, 2021

View reviewed changes

win32/ctags_vs2013.vcxproj Outdated

</Project>

Copy link

Member

masatake Apr 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't put a newline at the end of file.

masatake reviewed Apr 11, 2021

View reviewed changes

win32/ctags_vs2013.vcxproj.filters Outdated

</Project>

Copy link

Member

masatake Apr 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't put a newline at the end of file.

masatake reviewed Apr 11, 2021

View reviewed changes

antonysouthworth-halter added 4 commits May 3, 2021 18:14

add terraform support (preliminary)

616c2d1

add terraform c parser

c78ccef

add copyright notice and doc

c4d902e

fix build

ca9d8fe

antonysouthworth-halter force-pushed the master branch from 5247cf8 to ca9d8fe Compare May 3, 2021 06:14

fix newlines

489c04c

masatake reviewed May 3, 2021

View reviewed changes

pr fixes

7d58baf

masatake reviewed May 3, 2021

View reviewed changes

masatake changed the title ~~Add Terraform (HCL) (*.tf) support~~ Terraform (HCL) (*.tf): new parser May 23, 2021

Haggus mentioned this pull request Mar 29, 2023

Terraform (HCL) (*.tf): new parser #3683

Closed

masatake mentioned this pull request Mar 31, 2023

Terraform: new parser #3684

Merged

masatake closed this in #3684 Apr 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Terraform (HCL) (*.tf): new parser #2952

Terraform (HCL) (*.tf): new parser #2952

antonysouthworth-halter commented Apr 11, 2021 •

edited by masatake

Loading

antonysouthworth-halter commented Apr 11, 2021

masatake Apr 11, 2021

masatake Apr 11, 2021 •

edited

Loading

juliosueiras Apr 11, 2021

masatake Apr 11, 2021

juliosueiras Apr 11, 2021

masatake Apr 11, 2021

masatake Apr 11, 2021

masatake Apr 11, 2021

masatake Apr 11, 2021

masatake Apr 11, 2021

masatake Apr 11, 2021

masatake Apr 11, 2021

antonysouthworth-halter commented Apr 11, 2021

juliosueiras commented Apr 11, 2021

masatake May 3, 2021

masatake May 3, 2021

masatake May 3, 2021

masatake May 3, 2021

masatake May 3, 2021

masatake May 3, 2021

tonglil commented Aug 10, 2021

masatake commented Aug 10, 2021

masatake commented Aug 10, 2021

tonglil commented Jan 26, 2022

Haggus commented Mar 29, 2023

masatake commented Mar 29, 2023

antonysouthworth-halter commented Mar 30, 2023

masatake commented Mar 31, 2023

masatake commented Apr 1, 2023

Terraform (HCL) (*.tf): new parser #2952

Terraform (HCL) (*.tf): new parser #2952

Conversation

antonysouthworth-halter commented Apr 11, 2021 • edited by masatake Loading

TODO

antonysouthworth-halter commented Apr 11, 2021

Choose a reason for hiding this comment

masatake Apr 11, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antonysouthworth-halter commented Apr 11, 2021

juliosueiras commented Apr 11, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tonglil commented Aug 10, 2021

masatake commented Aug 10, 2021

masatake commented Aug 10, 2021

tonglil commented Jan 26, 2022

Haggus commented Mar 29, 2023

masatake commented Mar 29, 2023

antonysouthworth-halter commented Mar 30, 2023

masatake commented Mar 31, 2023

masatake commented Apr 1, 2023

antonysouthworth-halter commented Apr 11, 2021 •

edited by masatake

Loading

masatake Apr 11, 2021 •

edited

Loading