The highlight shortcode and template func unescapes the input

[pgundlach]

I use this code to try out chroma:

package main

import (
	"log"
	"os"

	"github.com/alecthomas/chroma/quick"
)

func main() {
	someSourceCode := `<Foo attr=" &lt; "></Foo>`
	err := quick.Highlight(os.Stdout, someSourceCode, "xml", "html", "monokai")
	if err != nil {
		log.Fatal(err)
	}
}

which results in <Foo attr=" &lt; "></Foo> whreas this code in Hugo (using chroma)

{{< highlight xml >}}
<Foo attr=" &lt; "></Foo>
{{< /highlight >}}

which renders as <Foo attr=" < "></Foo>. The difference is the interpretation of < into <, which would be fine if the contents is XML or HTML, but not verbose source code.

The same happens when using code fences.

Hugo generates attr=</span><span class="s">&#34; &lt; &#34;</span> whereas chroma (with the above code) generates attr=</span><span class="s">&#34; &amp;lt; &#34;</span>.

$ hugo version
Hugo Static Site Generator v0.31 darwin/amd64 BuildDate: 2017-11-27T11:14:58+01:00

Operating system: macOS 10.13.2

config.toml syntax highlighting setting:

pygmentsCodeFences = true
pygmentsUseClasses = true
pygmentsUseClassic = false

Chroma version (running the code above) from today:
alecthomas/chroma@c9f612c

[bep]

It doesn't make much sense to test with a different version of Chroma than used in your Hugo version.

[pgundlach]

I have now tried with alecthomas/chroma@9c81d25 which seems to be the version used in the 0.31 release (found in Gopkg.lock)

I have got the same result as above.

[kaushalmodi]

I just confirmed this issue on hugo 0.31.1:

• Sandbox site source
  • Test content file
  • Hugo rendered HTML
    

[kaushalmodi]

I have a sinking feeling that this issue is with blackfriday, which IMO is very poorly maintained.. no one responds to the Issues opened there, and I haven't seen a commit on that project even on that v2 branch since last Sept.

Github Insights/Pulse of russross/blackfriday for the past month

@bep I hope this is something that can be fixed on the hugo side. Or if you know folks on the blackfriday dev team, then inspire them to start responding to issues and fixing bugs.

[bep]

I have a sinking feeling that this issue is with blackfriday, which IMO is very poorly maintained..

Blackfriday isn't poorly maintained. And this isn't a BF issue.

[bep]

I would guess it is the unescape thing here that is the culprit:

https://github.com/gohugoio/hugo/blob/master/tpl/transform/transform.go#L58

This is not my code, so I have no recollection as to why it was put there.

Maybe @moorereason remembers?

[kaushalmodi]

That line has a loong history.. traced it back to template_funcs.go:

But looks like it lived in a template.go before that.

[kaushalmodi]

Alright, found the actual source.. db29f57c @spf13

commit db29f57cc485debcea76e48dcbb3a4a4f33a3f7f
Author: spf13 <steve.francia@gmail.com>
Date:   Thu Dec 5 09:43:49 2013 -0500

    Adding (source code) Highlight template helper

diff --git a/template/bundle/template.go b/template/bundle/template.go
--- a/template/bundle/template.go
+++ b/template/bundle/template.go
@@ -112,0 +114,14 @@
+	var str string
+	av := reflect.ValueOf(in)
+	switch av.Kind() {
+	case reflect.String:
+		str = av.String()
+	}
+
+	if strings.HasPrefix(strings.TrimSpace(str), "<pre><code>") {
+		str = str[strings.Index(str, "<pre><code>")+11:]
+	}
+	if strings.HasSuffix(strings.TrimSpace(str), "</code></pre>") {
+		str = str[:strings.LastIndex(str, "</code></pre>")]
+	}
+	return template.HTML(helpers.Highlight(html.UnescapeString(str), lang))

[bep]

That code is old and pre-Chroma -- and it doesn't make sense in my head as a general rule. People can easily use the htmlUnescape template func if they really need it.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.