Markdown: add hashtags functionality #3747

jiangyinzuo · 2023-06-22T07:52:31Z

Some markdown editors or plugins, such as Obsidian and VSCode markdown-hashtags, support parsing #hashtags and navigation. I have modified the markdown parser to support hashtag syntax parsing.

masatake · 2023-06-22T17:30:27Z

Looks nice. If you are good at git, polish the pull request with git rebase -i and git push --force.

e9d8451 comes first. Please, use "main,refactor: " as the prefix of the commit header.

8372df3 should comes second. So you can use functions declared in utf8_str.h in the code supporting the hashtag kind. Please, use "Markdown: " as the prefix of the commit header.

Put the sentences in the description of this pull request, "Some markdown editors or plugins, such as Obsidian and VSCode markdown-hashtags, support parsing #hashtags and navigation. I have modified the markdown parser to support hashtag syntax parsing." to the commit log of the second one. It is helpful to understand the commit.

a58be23 should be squashed into the first one.

@masatake's TODO:

parser versioning (a kind is added)

Thank you.

codecov · 2023-06-23T04:56:08Z

Codecov Report

Patch coverage: 95.38% and no project coverage change.

Comparison is base (47398af) 83.01% compared to head (9271582) 83.01%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #3747   +/-   ##
=======================================
  Coverage   83.01%   83.01%           
=======================================
  Files         226      227    +1     
  Lines       55086    55126   +40     
=======================================
+ Hits        45727    45764   +37     
- Misses       9359     9362    +3

Impacted Files	Coverage Δ
parsers/asciidoc.c	`95.45% <ø> (-0.36%)`	⬇️
parsers/rst.c	`92.55% <ø> (-0.43%)`	⬇️
main/utf8_str.c	`89.65% <89.65%> (ø)`
parsers/markdown.c	`99.59% <100.00%> (+0.07%)`	⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

jiangyinzuo · 2023-06-23T05:01:09Z

Thanks for your tips! Now the polished PR is ready.

parsers/markdown.c

.gitignore

parsers/markdown.c

main/utf8_str.c

parsers/markdown.c

masatake

Could you add the original comment of this pull request to the commit log? It helps a person understand this pull request.

Some markdown editors or plugins, such as [Obsidian](https://help.obsidian.md/Editing+and+formatting/Tags) and [VSCode markdown-hashtags](https://github.com/vanadium23/markdown-hashtags), support parsing #hashtags and navigation. I have modified the markdown parser to support hashtag syntax parsing.

Tmain/list-fields.d/stdout-expected.txt

Units/parser-markdown.r/hashtags-utf8.d/args.ctags

masatake · 2023-06-23T17:34:25Z

parsers/markdown.c

+	  .enabled     = false,
+	  .name        = "hashtag",
+	  .description = "#hashtag",
+	},


(review comment Group 1)

masatake · 2023-06-23T17:34:34Z

parsers/markdown.c

 };

 typedef enum {
 	F_MARKER,
+	F_HASHTAG,


(review comment Group 1)

parsers/markdown.c

Units/parser-markdown.r/hashtags-utf8.d/expected.tags

masatake · 2023-06-23T18:30:15Z

The first commit and the third commit are ready to merge. Thank you.
If you are comfortable, I will cherry-pick the first and third ones immediately.
In that case, let me know.

jiangyinzuo · 2023-06-24T01:07:09Z

The first commit and the third commit are ready to merge. Thank you. If you are comfortable, I will cherry-pick the first and third ones immediately. In that case, let me know.

It's OK

Units/parser-markdown.r/hashtags-utf8.d/input.md

masatake · 2023-06-24T09:11:10Z

This one is ready for merging.
Could you give me a time before merging?

I would like to try solving #3748 before merging this. If I find fixing #3748 may take longer time, I will merge this pull request first.

jiangyinzuo · 2023-06-24T09:18:40Z

This one is ready for merging. Could you give me a time before merging?

I would like to try solving #3748 before merging this. If I find fixing #3748 may take longer time, I will merge this pull request first.

It's OK, and it is better to cherry-pick the first and third commits. I plan to open a new PR (or continue in this one) for the qualified hashtag feature.

jiangyinzuo · 2023-06-24T15:40:55Z

I force-pushed again. Now the first commit is duplicated with #3749 and the third commit implements the features for qualified hashtags and subtags

@jiangyinzuo

Close universal-ctags#3748. A test input is derrived from Yinzuo Jiang (@jiangyinzuo)'s commit in universal-ctags#3747. Signed-off-by: Masatake YAMATO <yamato@redhat.com>

@jiangyinzuo

Close universal-ctags#3748. A test input is derrived from Yinzuo Jiang (@jiangyinzuo)'s commit in universal-ctags#3747. Signed-off-by: Masatake YAMATO <yamato@redhat.com>

masatake · 2023-06-25T02:04:47Z

Could you rebase this pull request on the latest master branch?
You must drop dc0c700 from this pull request.

…_raw_strlen function in main/utf8_str.c

jiangyinzuo · 2023-06-25T06:34:10Z

Could you rebase this pull request on the latest master branch? You must drop dc0c700 from this pull request.

Done. Could you please review the second commit again?

masatake · 2023-06-25T06:53:18Z

parsers/markdown.c

@@ -57,6 +60,8 @@ static kindDefinition MarkdownKinds[] = {
 	{ true, 'T', "l4subsection",  "level 4 sections" },
 	{ true, 'u', "l5subsection",  "level 5 sections" },
 	{ true, 'n', "footnote",      "footnotes" },
+	{ true, 'h', "hashtag",       "hashtags"},
+	{ true, 'H', "subtag",       "subtags in hashtags"},


Do you think we should have both hashtag and subtag?
Having only hashtag is not enough?

masatake · 2023-06-25T07:14:20Z

We are going the wrong way.

Could you push the original version to that the sub-hashtag was not introduced yet?
I will merge the original version.
We can work on the sub-hashtag after merging the original version.

masatake · 2023-06-25T07:26:34Z

BTW, I found you are using your master branch. I strongly recommend using a topic branch.
See https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows

So you can track the change on the master branch of the upstream and develop new features simultaneously.

Some markdown editors or plugins, such as [Obsidian](https://help.obsidian.md/Editing+and+formatting/Tags) and [VSCode markdown-hashtags](https://github.com/vanadium23/markdown-hashtags), support parsing #hashtags and navigation. I have modified the markdown parser to support hashtag syntax parsing.

masatake · 2023-06-25T11:09:41Z

Thank you!

masatake · 2023-06-25T11:22:33Z

I have some advises about tagging #a/b.

See php.c:

static scopeSeparator PhpGenericSeparators [] = {
        { K_NAMESPACE        , NAMESPACE_SEPARATOR },
        { KIND_WILDCARD_INDEX, "::" },
};

You can do the same as the PHP parser does:

static scopeSeparator MarkdownHashtagSeparators [] = {
        { K_HASHTAG        , "/" },  /* If a hashtag is the parent. */
        { KIND_WILDCARD_INDEX, "#" }, /* If something other is the parent. */
};
...
static kindDefinition MarkdownKinds[] = {
        { true, 'c', "chapter",       "chapters"},
        { true, 's', "section",       "sections" },
        { true, 'S', "subsection",    "level 2 sections" },
        { true, 't', "subsubsection", "level 3 sections" },
        { true, 'T', "l4subsection",  "level 4 sections" },
        { true, 'u', "l5subsection",  "level 5 sections" },
        { true, 'n', "footnote",      "footnotes" },
        { true, 'h', "hashtag",       "hashtags"
           ATTACH_SEPARATORS(MarkdownHashtagSeparators)}},

With this change, ctags generate full-qualified tags like #a/b with --extras=+qufalifed if you parser does:

making a tag for a with the hashtag kind, and
making a tag for b with the hashtag kind and with setting a to b's scope.

When making a tag for a, makeTagEntry or something returns an integer.
We call it a corkIndex for a.
Set the corkIndex for a to extensionFields.scopeIndex of b when making a tag for b.
See makeMarkdownTag.

masatake changed the title ~~add hashtags functionality to markdown parser~~ Markdown: add hashtags functionality Jun 22, 2023

jiangyinzuo force-pushed the master branch from a58be23 to 3224f98 Compare June 23, 2023 04:55

jiangyinzuo force-pushed the master branch from 3224f98 to f3705a9 Compare June 23, 2023 05:09

masatake requested changes Jun 23, 2023

View reviewed changes

parsers/markdown.c Show resolved Hide resolved

.gitignore Show resolved Hide resolved

masatake reviewed Jun 23, 2023

View reviewed changes

parsers/markdown.c Outdated Show resolved Hide resolved

masatake reviewed Jun 23, 2023

View reviewed changes

parsers/markdown.c Outdated Show resolved Hide resolved

masatake reviewed Jun 23, 2023

View reviewed changes

main/utf8_str.c Show resolved Hide resolved

jiangyinzuo force-pushed the master branch 2 times, most recently from 70ad291 to 8cb18cb Compare June 23, 2023 09:56

jiangyinzuo commented Jun 23, 2023

View reviewed changes

parsers/markdown.c Show resolved Hide resolved

masatake requested changes Jun 23, 2023

View reviewed changes

jiangyinzuo force-pushed the master branch from 8cb18cb to 45b97f1 Compare June 24, 2023 01:01

jiangyinzuo force-pushed the master branch from 45b97f1 to 728d733 Compare June 24, 2023 02:40

jiangyinzuo requested a review from masatake June 24, 2023 05:49

masatake reviewed Jun 24, 2023

View reviewed changes

Units/parser-markdown.r/hashtags-utf8.d/input.md Show resolved Hide resolved

masatake mentioned this pull request Jun 24, 2023

Markdown: tagging a chapter unexpectedly #3748

Closed

masatake approved these changes Jun 24, 2023

View reviewed changes

jiangyinzuo force-pushed the master branch from 728d733 to e5a5f9b Compare June 24, 2023 15:34

masatake mentioned this pull request Jun 24, 2023

Markdown: accept sections in the line started from spaces #3750

Merged

main,refactor: move utf8_strlen function to main/utf8_str.c, add utf8…

cd4b74f

…_raw_strlen function in main/utf8_str.c

jiangyinzuo force-pushed the master branch from e5a5f9b to 34ee315 Compare June 25, 2023 06:28

masatake reviewed Jun 25, 2023

View reviewed changes

jiangyinzuo added 2 commits June 25, 2023 16:41

gitignore: add .cache and compile_commands.json

9271582

jiangyinzuo force-pushed the master branch from 34ee315 to 9271582 Compare June 25, 2023 08:59

masatake approved these changes Jun 25, 2023

View reviewed changes

masatake merged commit e39a22b into universal-ctags:master Jun 25, 2023
38 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Markdown: add hashtags functionality #3747

Markdown: add hashtags functionality #3747

jiangyinzuo commented Jun 22, 2023

masatake commented Jun 22, 2023 •

edited

Loading

codecov bot commented Jun 23, 2023 •

edited

Loading

jiangyinzuo commented Jun 23, 2023

masatake left a comment

masatake Jun 23, 2023

masatake Jun 23, 2023

masatake commented Jun 23, 2023 •

edited

Loading

jiangyinzuo commented Jun 24, 2023

masatake commented Jun 24, 2023

jiangyinzuo commented Jun 24, 2023 •

edited

Loading

jiangyinzuo commented Jun 24, 2023

masatake commented Jun 25, 2023

jiangyinzuo commented Jun 25, 2023

masatake Jun 25, 2023

masatake commented Jun 25, 2023

masatake commented Jun 25, 2023

masatake commented Jun 25, 2023

masatake commented Jun 25, 2023

Markdown: add hashtags functionality #3747

Markdown: add hashtags functionality #3747

Conversation

jiangyinzuo commented Jun 22, 2023

masatake commented Jun 22, 2023 • edited Loading

codecov bot commented Jun 23, 2023 • edited Loading

Codecov Report

jiangyinzuo commented Jun 23, 2023

masatake left a comment

Choose a reason for hiding this comment

masatake Jun 23, 2023

Choose a reason for hiding this comment

masatake Jun 23, 2023

Choose a reason for hiding this comment

masatake commented Jun 23, 2023 • edited Loading

jiangyinzuo commented Jun 24, 2023

masatake commented Jun 24, 2023

jiangyinzuo commented Jun 24, 2023 • edited Loading

jiangyinzuo commented Jun 24, 2023

masatake commented Jun 25, 2023

jiangyinzuo commented Jun 25, 2023

masatake Jun 25, 2023

Choose a reason for hiding this comment

masatake commented Jun 25, 2023

masatake commented Jun 25, 2023

masatake commented Jun 25, 2023

masatake commented Jun 25, 2023

masatake commented Jun 22, 2023 •

edited

Loading

codecov bot commented Jun 23, 2023 •

edited

Loading

masatake commented Jun 23, 2023 •

edited

Loading

jiangyinzuo commented Jun 24, 2023 •

edited

Loading