Handle file extension conflicts in --list-languages #1135

Kienyew · 2020-08-08T04:45:38Z

Now the each outputted extension should be unique to only one language. Below is the diff output of the old bat --list-languages and new bat --list-languages.

13c13
< C:c,h
---
> C:c
49c49
< GLSL:vs,fs,gs,vsh,fsh,gsh,vshader,fshader,gshader,vert,frag,geom,tesc,tese,comp,glsl
---
> GLSL:vs,gs,vsh,fsh,gsh,vshader,fshader,gshader,vert,frag,geom,tesc,tese,comp,glsl
70,71c70,71
< JavaScript:js,htc
< JavaScript (Babel):js,mjs,jsx,babel,es6,cjs
---
> JavaScript:htc
> JavaScript (Babel):js,mjs,jsx,babel,es6,cjs,*.js
92,93c92,93
< Objective-C:m,h
< Objective-C++:mm,M,h
---
> Objective-C:m
> Objective-C++:mm,M
119c119
< Ruby Haml:haml,sass
---
> Ruby Haml:haml
123c123
< Sass:sass
---
> Sass:sass,*.sass

But since #1035, some of the output is appearing in the form Lang ext, *.ext.
For instance in the C++ entry of bat --list-languages, containing both h and *.h:

...
C++                                 cpp, cc, cp, cxx, c++, C, h, hh, hpp, hxx, h++, inl, ipp, *.h
...

and likewise

...
Sass                                 sass, *.sass
...
F#                                   fs, fsi, fsx, *.fs
...
JavaScript (Babel)                   js, mjs, jsx, babel, es6, cjs, *.js
...

Should this be the desired behavior? @sharkdp

sharkdp · 2020-08-16T10:59:50Z

src/syntax_mapping.rs

@@ -24,6 +24,12 @@ impl<'a> SyntaxMapping<'a> {
        let mut mapping = Self::empty();
        mapping.insert("*.h", MappingTarget::MapTo("C++")).unwrap();
        mapping.insert("*.fs", MappingTarget::MapTo("F#")).unwrap();
+        mapping
+            .insert("*.js", MappingTarget::MapTo("JavaScript (Babel)"))


Ideally, we should be able to implement this feature without adding theses new syntax mappings. Instead of just querying the syntax_mapping, we would need to run the full syntax detection on the test.xx filename.

Ops, I was just following the similar commits before and thought it would do 😅, should I revert it?
Also I doesn't quite understand what is the "full syntax detection" means, could you please do a little explanation for me?

Yes, please revert this.

By "full syntax detection", I mean: determine the syntax in the same way that bat usually does this for a normal file. This involves not just checking the syntax mapping, but also the full database of syntaxes. See src/assets.rs => HighlightingAssets::get_syntax.

sharkdp · 2020-08-16T11:00:09Z

src/bin/bat/main.rs

+            let test_file = Path::new("test").with_extension(extension);
+            match config.syntax_mapping.get_syntax_for(test_file) {
+                Some(MappingTarget::MapTo(primary_lang)) => lang_name == primary_lang,
+                Some(MappingTarget::MapToUnknown) => true,


Shouldn't this be false?

I think you're right, it should be false.

sharkdp · 2020-08-16T11:00:22Z

Thank you very much for your contribution!

Could you please add an entry to the "unreleased" section in CHANGELOG.md? The format for entries is:

- Description what has been changed, see #123 (@user)

where #123 links to the bug ticket and/or the PR and user is your username.

sharkdp · 2020-08-16T11:02:01Z

But since #1035, some of the output is appearing in the form Lang ext, *.ext.
For instance in the C++ entry of bat --list-languages, containing both h and *.h:
Should this be the desired behavior?

I think we can leave this for now. The fact that the list contains h (and *.h) really means that files named h would be highlighted with the C++ syntax. This is a limitation of sublime syntax patterns (that we are partially addressing in bat through syntax_mapping.rs).

sharkdp · 2020-08-30T17:39:12Z

src/syntax_mapping.rs

@@ -24,6 +24,12 @@ impl<'a> SyntaxMapping<'a> {
        let mut mapping = Self::empty();
        mapping.insert("*.h", MappingTarget::MapTo("C++")).unwrap();
        mapping.insert("*.fs", MappingTarget::MapTo("F#")).unwrap();
+        mapping
+            .insert("*.js", MappingTarget::MapTo("JavaScript (Babel)"))


Yes, please revert this.

By "full syntax detection", I mean: determine the syntax in the same way that bat usually does this for a normal file. This involves not just checking the syntax mapping, but also the full database of syntaxes. See src/assets.rs => HighlightingAssets::get_syntax.

sharkdp · 2020-09-20T10:05:48Z

Thank you very much for the updates. How does the diff for bat --list-languages look like now?

Kienyew · 2020-09-20T10:11:56Z

Thank you very much for the updates. How does the diff for bat --list-languages look like now?

This is the new diff for the new bat --list-languages by diff old.txt new.txt so far.

13c13
< C:c,h
---
> C:c
49c49
< GLSL:vs,fs,gs,vsh,fsh,gsh,vshader,fshader,gshader,vert,frag,geom,tesc,tese,comp,glsl
---
> GLSL:vs,gs,vsh,fsh,gsh,vshader,fshader,gshader,vert,frag,geom,tesc,tese,comp,glsl
70c70
< JavaScript:js,htc
---
> JavaScript:htc
92,93c92,93
< Objective-C:m,h
< Objective-C++:mm,M,h
---
> Objective-C:m
> Objective-C++:mm,M
119c119
< Ruby Haml:haml,sass
---
> Ruby Haml:haml

sharkdp · 2020-09-20T10:14:24Z

Thank you very much!

Handle file extension conflicts in --list-languages (sharkdp#1076)

2bad4df

sharkdp reviewed Aug 16, 2020

View reviewed changes

Kienyew added 2 commits August 16, 2020 19:39

Fix undesired behavior

91e9d2a

Update CHANGELOG.md

52ce5bd

sharkdp requested changes Aug 30, 2020

View reviewed changes

Kienyew added 3 commits September 6, 2020 17:29

Revert to previous commit

6246485

Handle file extension conflicts in --list-languages

7bf474c

Use unwrap

2e89b49

sharkdp approved these changes Sep 20, 2020

View reviewed changes

sharkdp merged commit 83c7750 into sharkdp:master Sep 20, 2020

Kienyew changed the title ~~Handle file extension conflicts in --list-languages (#1076)~~ Handle file extension conflicts in --list-languages Sep 20, 2020

sharkdp mentioned this pull request Sep 20, 2020

Improved handling of file-extension conflicts #1076

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle file extension conflicts in --list-languages #1135

Handle file extension conflicts in --list-languages #1135

Kienyew commented Aug 8, 2020 •

edited

Loading

sharkdp Aug 16, 2020

Kienyew Aug 16, 2020 •

edited

Loading

sharkdp Aug 30, 2020

sharkdp Aug 16, 2020

Kienyew Aug 16, 2020

sharkdp commented Aug 16, 2020 •

edited

Loading

sharkdp commented Aug 16, 2020

sharkdp Aug 30, 2020

sharkdp commented Sep 20, 2020

Kienyew commented Sep 20, 2020

sharkdp commented Sep 20, 2020

Handle file extension conflicts in --list-languages #1135

Handle file extension conflicts in --list-languages #1135

Conversation

Kienyew commented Aug 8, 2020 • edited Loading

sharkdp Aug 16, 2020

Choose a reason for hiding this comment

Kienyew Aug 16, 2020 • edited Loading

Choose a reason for hiding this comment

sharkdp Aug 30, 2020

Choose a reason for hiding this comment

sharkdp Aug 16, 2020

Choose a reason for hiding this comment

Kienyew Aug 16, 2020

Choose a reason for hiding this comment

sharkdp commented Aug 16, 2020 • edited Loading

sharkdp commented Aug 16, 2020

sharkdp Aug 30, 2020

Choose a reason for hiding this comment

sharkdp commented Sep 20, 2020

Kienyew commented Sep 20, 2020

sharkdp commented Sep 20, 2020

Kienyew commented Aug 8, 2020 •

edited

Loading

Kienyew Aug 16, 2020 •

edited

Loading

sharkdp commented Aug 16, 2020 •

edited

Loading