TypeScript: use CORK for scopes #2083

ksamborski · 2019-04-17T18:31:04Z

I've changed manually handling scopes as strings in favor of CORK indexes. Parser code is now shorter and more clear. I've also introduced myself in developers.rst

coveralls · 2019-04-17T18:38:06Z

Coverage increased (+0.009%) to 85.343% when pulling 4f105fb on ksamborski:typescript-cork into 46490ea on universal-ctags:master.

masatake

I wonder whether the condition if (scope) is needed or not in a if (scope) dest->scope = foo.
If it is redundant, we should remove the condition.
Could you look at lines setting the scope field?

masatake · 2019-04-18T00:34:24Z

parsers/typescript.c

-					vStringCopy (methodScope, scope);
-					vStringPut (methodScope, '.');
-					vStringCat (methodScope, member->string);
+					const int nscope = isGenerator ? emitTag (member, TSTAG_GENERATOR) :  emitTag (member, TSTAG_METHOD);


const int nscope = emitTag (member, (isGenerator? TSTAG_GENERATOR: TSTAG_METHOD));

reduces typings :-P

masatake · 2019-04-18T00:40:45Z

parsers/typescript.c

-		vStringCopy (token->scope, scope);
-		token->scopeParentKind = scopeParentKind;
-	}
+	if (scope != CORK_NIL) token->scope = scope;


Do we need the condtion?
I'm not sure but I think

token->scope = scope;

is o.k. even if scope == CORK_NIL.

masatake · 2019-04-18T00:41:35Z

parsers/typescript.c

-		vStringCopy (token->scope, scope);
-		token->scopeParentKind = scopeParentKind;
-	}
+	if (scope != CORK_NIL) token->scope = scope;


The last comment may be applicable to this line, too.

masatake · 2019-04-18T00:42:23Z

parsers/typescript.c

-		vStringPut (nscope, '.');
-	}
-	vStringCat (nscope, token->string);
+	if (scope != CORK_NIL) token->scope = scope;


The last comment is applicable to this line.

masatake · 2019-04-18T00:43:03Z

parsers/typescript.c

-							vStringCopy (member->scope, scope);
-							member->scopeParentKind = scopeParentKind;
-						}
+						if (scope != CORK_NIL) member->scope = scope;


The last comment is applicable to this line.

masatake · 2019-04-18T00:44:00Z

parsers/typescript.c

-		vStringCopy (token->scope, scope);
-		token->scopeParentKind = scopeParentKind;
-	}
+	if (scope != CORK_NIL) token->scope = scope;


The last comment is applicable to this line.

masatake · 2019-04-18T00:45:07Z

parsers/typescript.c

-		emitTag (token, TSTAG_GENERATOR);
-	else
-		emitTag (token, TSTAG_FUNCTION);
+	const int nscope = isGenerator ? emitTag (token, TSTAG_GENERATOR) : emitTag (token, TSTAG_FUNCTION);


This can be shorter with:

const int nscope = emitTag (token, isGenerator ? TSTAG_GENERATOR: TSTAG_FUNCTION);

.

masatake · 2019-04-18T00:48:31Z

parsers/typescript.c

-		vStringPut (nscope, '.');
-	}
-	vStringCat (nscope, token->string);
+	if (scope != CORK_NIL) token->scope = scope;


Do we need the condition?

masatake · 2019-04-18T01:15:03Z

I would like to write about a little bit away from the topic of this pull request.

Did you try --extras=+q option?
It emits full qualified tags.

[jet@living]~/var/ctags% cat /tmp/foo.c
cat /tmp/foo.c
struct ponit {
  double x, y;
};
[jet@living]~/var/ctags% ./ctags -o - /tmp/foo.c
ponit	/tmp/foo.c	/^struct ponit {$/;"	s	file:
x	/tmp/foo.c	/^  double x, y;$/;"	m	struct:ponit	typeref:typename:double	file:
y	/tmp/foo.c	/^  double x, y;$/;"	m	struct:ponit	typeref:typename:double	file:
[jet@living]~/var/ctags% ./ctags --extras=+q -o - /tmp/foo.c
ponit	/tmp/foo.c	/^struct ponit {$/;"	s	file:
ponit::x	/tmp/foo.c	/^  double x, y;$/;"	m	struct:ponit	typeref:typename:double	file:
ponit::y	/tmp/foo.c	/^  double x, y;$/;"	m	struct:ponit	typeref:typename:double	file:
x	/tmp/foo.c	/^  double x, y;$/;"	m	struct:ponit	typeref:typename:double	file:
y	/tmp/foo.c	/^  double x, y;$/;"	m	struct:ponit	typeref:typename:double	file:

--extras=+q (fq tag) is one of popular feature vim users want.
I have not tried yet, however, your parser has not supported the fq tag.

By setting def->requestAutomaticFQTag = true; in TypeScriptParser(), the TypeScript parser can emit fq tag. requestAutomaticFQTag emits fq tags from scope information.

parsers/tcl.c has a bit complicated code about requestAutomaticFQTag.
It tries avoiding automatic FQ emission.
It sets a default separator that combines two names.

parsers/php.c doesn't use requestAutomaticFQTag.
But it tells how to customize separators in the kind definition of the parser.

masatake · 2019-04-18T01:16:38Z

I made some minor comments. Other than them, It looks good to me.

ksamborski · 2019-04-18T04:37:58Z

Thanks @masatake! I'll fix them.

ksamborski · 2019-04-18T19:13:39Z

I think I've addressed all the issues as well as automatic fq tags. requestAutomaticFQTag is a fantastic option, thanks!

masatake · 2019-04-18T19:20:26Z

Could you make 2fb367d and 710bf3e one commit?
Then do git push --force.

ksamborski · 2019-04-18T19:26:31Z

@masatake Done. Is it now OK?

masatake · 2019-04-18T19:38:42Z

Sorry, I have one more comment.
I think adding a test case for --extras=+q is better.

ksamborski · 2019-04-18T19:40:29Z

I added ts-class-fq.d test case. Do you mean something else?

masatake · 2019-04-18T19:42:28Z

Wow, sorry. I missed the test case.
Please, merge this PR after appveyor says o.k.

I will take asleep:-).

ksamborski · 2019-04-18T19:43:18Z

OK, thanks!

masatake · 2019-04-19T07:01:17Z

Thank you.
You can find various interesting applications of CORK with grepping getEntryInCorkQueue ./parsers.

masatake · 2019-04-19T07:08:37Z

No big perfomance degrading with introducing CORK:

(before introducing CORK)
[yamato@slave]~/var/codebase%  ./codebase ctags TypeScript
version: 4aed4f00
features: +wildcards +regex +iconv +option-directory +xpath +json +interactive +sandbox +yaml +aspell +packcc
log: results/4aed4f00,TypeScript..........,..........,time......,default...,2019-04-19-16:04:04.log
tagsoutput: /dev/null
cmdline: + u-ctags --quiet --options=NONE --sort=no --options=profile.d/maps --totals=yes --languages=TypeScript -o - -R code/nest code/tslint code/typescript
15421 files, 1319786 lines (29953 kB) scanned in 11.4 seconds (2618 kB/s)
145494 tags added to tag file

real	0m11.465s
user	0m11.252s
sys	0m0.189s
+ set +x

(after introducing CORK)
[yamato@slave]~/var/codebase%  ./codebase ctags TypeScript
version: 54c1c2b8
features: +wildcards +regex +iconv +option-directory +xpath +json +interactive +sandbox +yaml +aspell +packcc
log: results/54c1c2b8,TypeScript..........,..........,time......,default...,2019-04-19-16:04:50.log
tagsoutput: /dev/null
cmdline: + u-ctags --quiet --options=NONE --sort=no --options=profile.d/maps --totals=yes --languages=TypeScript -o - -R code/nest code/tslint code/typescript
15421 files, 1319786 lines (29953 kB) scanned in 11.5 seconds (2613 kB/s)
145494 tags added to tag file

real	0m11.491s
user	0m11.273s
sys	0m0.189s
+ set +x

masatake requested changes Apr 18, 2019

View reviewed changes

Karol Samborski added 3 commits April 18, 2019 21:24

typescript: Use CORK index to store scopes

fd9643b

Docs: introduced new developer

03b7f57

typescript: support for --extras=+q (fully qualified tags)

4f105fb

ksamborski force-pushed the typescript-cork branch from a17c29f to 4f105fb Compare April 18, 2019 19:25

masatake approved these changes Apr 19, 2019

View reviewed changes

ksamborski merged commit 54c1c2b into universal-ctags:master Apr 19, 2019

ksamborski deleted the typescript-cork branch April 19, 2019 06:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeScript: use CORK for scopes #2083

TypeScript: use CORK for scopes #2083

ksamborski commented Apr 17, 2019

coveralls commented Apr 17, 2019 •

edited

Loading

masatake left a comment

masatake Apr 18, 2019

masatake Apr 18, 2019

masatake Apr 18, 2019

masatake Apr 18, 2019

masatake Apr 18, 2019

masatake Apr 18, 2019

masatake Apr 18, 2019

masatake Apr 18, 2019

masatake commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 19, 2019 •

edited

Loading

masatake commented Apr 19, 2019

TypeScript: use CORK for scopes #2083

TypeScript: use CORK for scopes #2083

Conversation

ksamborski commented Apr 17, 2019

coveralls commented Apr 17, 2019 • edited Loading

masatake left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

masatake commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 18, 2019

ksamborski commented Apr 18, 2019

masatake commented Apr 19, 2019 • edited Loading

masatake commented Apr 19, 2019

coveralls commented Apr 17, 2019 •

edited

Loading

masatake commented Apr 19, 2019 •

edited

Loading