Help ! How to skip part(s) between markers from C++ source file

[guerinp38]

Hello,

I try to ignore extraction of function (or methods) between twomarkers, say
TAGS_OFF and TAGS_ON (TAGS_ON and TAGS_OFF are just macros defined somewhere with no "body")

...
TAGS_OFF
bool foo( int arg )
{
...
}
TAGS_ON
...

I've read the documentation and found "Advanced pattern matching with multiple regex tables" which seems
to act the way I want.
But I failed !

Here is the c.ctags file I've made and the ctags invocation on the first comments

#
# ./ctags.exe -n --kinds-C++=f --fields=zkZtse --options=c.ctags -u src.cpp
#
--langdef=X
--map-X=.cpp
#--kinddef-X=v,var,variables

--_tabledef-X=toplevel
--_tabledef-X=comment
# --_tabledef-X=vars

--_mtable-regex-X=toplevel/TAGS_OFF//{tenter=comment}
--_mtable-regex-X=toplevel/.//

--_mtable-regex-X=comment/TAGS_ON//{tleave}
--_mtable-regex-X=comment/.//

Any idea or advice ?

Thank you in advance.
Kind regards,
Patrice.

[masatake]

Thank you for reading the document. If you find a way to improve it, pull requests (and questions like this issue) are welcome.
In past years, I have been working hard to make ctags helpful for this kind of use case.

I think using 3 states will be more understandable.
The first step is implementing a parser X. The parser doesn't make a tag at all. Instead, it recognizes the areas of C++ code that should be tagged, and plans running C++ parser on the areas. And the second step is ... there is no second step. That's all.

I wrote two versions (toggle0.ctags and toggle1.ctags) that may fix this issue.

I changed the marker from "TAGS_ON" to "//TAGS_ON".
I changed the marker from "TAGS_OFF" to "//TAGS_OFF".

$ cat -n /tmp/src.cpp 
     1	bool extract_me_0( int arg )
     2	{
     3	  return false;
     4	}
     5	//TAGS_OFF
     6	bool dont_extract_me_0( int arg )
     7	{
     8	  return false;
     9	}
    10	//TAGS_ON
    11	bool extract_me_1( int arg )
    12	{
    13	  return false;
    14	}
    15	//TAGS_OFF
    16	bool dont_extract_me_1( int arg )
    17	{
    18	  return false;
    19	}
    20	//TAGS_ON
    21	
    22	int extract_me_3;
$ cat -n /tmp/toggle0.ctags
     1	--langdef=X
     2	
     3	--map-X=+.cpp
     4	--map-C++=-.cpp
     5	
     6	--extras=+{guest}
     7	
     8	--_tabledef-X=toplevel
     9	--_tabledef-X=comment
    10	--_tabledef-X=cpp
    11	
    12	--_mtable-regex-X=toplevel/\/\/TAGS_OFF\n//{tenter=comment}
    13	--_mtable-regex-X=toplevel/([^\n]*)\n//{_guest=C++,0start,}{tenter=cpp}
    14	
    15	# Go back to the comment table, and parse the line again.
    16	--_mtable-regex-X=cpp/\/\/TAGS_OFF\n//{_guest=,,0start}{_advanceTo=0start}{tleave}
    17	--_mtable-regex-X=cpp/([^\n]*)\n//
    18	# Now newline? we are at the end of input.
    19	--_mtable-regex-X=cpp/([^\n]*)//{_guest=,,0end}{tquit}
    20	
    21	
    22	--_mtable-regex-X=comment/\/\/TAGS_ON\n//{tleave}
    23	--_mtable-regex-X=comment/([^\n]*)\n//
$ ./ctags --sort=no --options=/tmp/toggle0.ctags -o - /tmp/src.cpp
extract_me_0	/tmp/src.cpp	/^bool extract_me_0( int arg )$/;"	f	typeref:typename:bool
extract_me_1	/tmp/src.cpp	/^bool extract_me_1( int arg )$/;"	f	typeref:typename:bool
extract_me_3	/tmp/src.cpp	/^int extract_me_3;$/;"	v	typeref:typename:int
$ cat -n /tmp/toggle1.ctags 
     1	--langdef=X
     2	
     3	--map-X=+.cpp
     4	--map-C++=-.cpp
     5	
     6	--extras=+{guest}
     7	
     8	--_tabledef-X=toplevel
     9	--_tabledef-X=comment
    10	--_tabledef-X=cpp
    11	
    12	--_mtable-regex-X=toplevel/\/\/TAGS_OFF\n//{tenter=comment}
    13	--_mtable-regex-X=toplevel/([^\n]*)\n//{_guest=C++,0start,}{tenter=cpp}
    14	
    15	# Switch to the comment table.
    16	--_mtable-regex-X=cpp/\/\/TAGS_OFF\n//{_guest=,,0start}{tjump=comment}
    17	--_mtable-regex-X=cpp/([^\n]*)\n//
    18	# Now newline? we are at the end of input.
    19	--_mtable-regex-X=cpp/([^\n]*)//{_guest=,,0end}{tquit}
    20	
    21	--_mtable-regex-X=comment/\/\/TAGS_ON\n//{tleave}
    22	--_mtable-regex-X=comment/([^\n]*)\n//
$ ./ctags --sort=no --options=/tmp/toggle1.ctags -o - /tmp/src.cpp
extract_me_0	/tmp/src.cpp	/^bool extract_me_0( int arg )$/;"	f	typeref:typename:bool
extract_me_1	/tmp/src.cpp	/^bool extract_me_1( int arg )$/;"	f	typeref:typename:bool
extract_me_3	/tmp/src.cpp	/^int extract_me_3;$/;"	v	typeref:typename:int

[masatake]

If your ctags is linked to pcre2, I guess we can make X much faster.

[guerinp38]

Hello Masatake,

Thank you for your answer. It works great !

I think my english is not so good and I have to read more than once the documentation ;)

• I've tried $ in place of \n : it doesn't work. I guess it's probably the same as for '^'...

• Maybe, you should explicit that the parser name can be empty.

I will try with pcre2,

Kind regards,
Patrice.

P.S.
-V option gives a lot of informations when parsing table. It helps to understand how that works.

[masatake]

Maybe, you should explicit that the parser name can be empty.

I don't understand this sentence. Is this suggestion to me?

-V option gives a lot of informations when parsing table. It helps to understand how that works.

Yes. Though I wrote, I also think so:-P

[masatake]

If your ctags supports pcre2, you can make .ctags shorter.
--list features can be used to know whether your ctags supports pcre2 or not.

--langdef=X

--map-X=+.cpp
--map-C++=-.cpp

--extras=+{guest}

--_tabledef-X=toplevel
--_tabledef-X=comment

--_mtable-regex-X=toplevel/(.*?\/\/TAGS_OFF\n)//{pcre2}{_guest=C++,1start,1end}{tenter=comment}
--_mtable-regex-X=toplevel/(.+)//{pcre2}{_guest=C++,1start,1end}
--_mtable-regex-X=comment/(?:.*?\/\/TAGS_ON\n)//{pcre2}{tleave}

You need only two tables.

[guerinp38]

Maybe, you should explicit that the parser name can be empty.

I don't understand this sentence. Is this suggestion to me?

Yes.
The documentation suggests in
{_guest=<PARSER>,<START>,<END>}
that is mandatory and is one of the known parsers (or user's one) by ctags.
Giving an empty name to is not mentioned.

[masatake]

Oh, I see. I will update the description. Thank you.

[guerinp38]

I've made some timings whithout and with toggle0 on

• windows 7 (Intel Q6600 2.4GHz)
• debian 9 (AMD Phenom II X4 945, 3 GHz)

I've build it myself for each OS from commit 05e6ab4

on 35 cpp files, (24 827 lines/832 432 bytes)
On windows ctags tooks 0.09s without and 0.8s with toogle0
On linux, 0.19s and 0.47s.

All files reside on the windows7 machine.
I think, it's not so bad...

To use pcre2 on windows, I will try to modify the makefiles.
For linux, it's easy, just install pcre2-dev and configure should do the job...

Kind regards
Patrice.