hast-util-to-html href attribute handling

[Nickolai-developer]

I have a problem with toHtml. It stringifies href contents in a quite unexpected way for me. By stripping all the ampersands and replacing them with its hex equivalent (&) .

The issue

Basically, what I expect to see when converting a tree containing a link element is a <a href="//:example.com/?P1=a&P2=b"></a> (unchanged contents of href attribute), but toHtml returns <a href="//:example.com/?P1=a&#x26;P2=b"></a>.

It's not like the link doesn't work in browser anymore after such a metamorphosis. But a certain amazon service don't understand the joke and doesn't want to track modified link properly.

The "bug" and its location

Here's the problem place. The serializeAttribute function doesn't have check for links. It replaces characters defined in constants with hex equivalents for every attribute, including href. I can't tell if this is intended or not. In my perspective it would be more obvious to just call escape (sophisticated js method for escaping links) on href content.

Moreover, I cannot disable these substitutions by using input options.

The question

I want either:

1. Option to disable hex-substitutions in attributes
2. The use of escape on href's value in serializeAttribute
3. To clarify, if I'm missing something, and it is in fact possible to get links without & in output html

[wooorm]

it’s not a bug here. Rather, without it, this project would be completely broken.

It’s a bug with amazon. Raise it with them.

1. Option to disable hex-substitutions in attributes

That would be completely broken.

2. The use of escape on href's value in serializeAttribute

This is not a sentence that follows “I want either”? I don‘t understand what you want.

3. To clarify, if I'm missing something, and it is in fact possible to get links without &#x26; in output html

There are many options that affect character references, as explained in the readme: https://github.com/syntax-tree/hast-util-to-html. Perhaps the characterReferences option? Perhaps allowParseErrors might be of interest too?

[cheapsteak]

Hi there

I believe allowParseErrors doesn't seem to work for a hrefs, I added a failing test case here - https://github.com/syntax-tree/hast-util-to-html/pull/40/files

characterReferences also doesn't seem to be able to solve the issue, since the subset option is overriden

Would it make sense to special case how entities are stringified for anchor hrefs? happy to add a condition there if that's helpful

[wooorm]

Please take a minute to read https://github.com/syntax-tree/.github/blob/main/support.md#asking-quality-questions, and provide much more info on what your actual root problem is. What is the root of the root of what is going on here?

[cheapsteak]

The HTML spec says that that & should be encoded

hmm, I see from https://html.spec.whatwg.org/ explicit examples showing unescaped ampersands though?

<a href="?bill&ted">Bill and Ted</a> <!-- &ted is ok, since it's not a named character reference -->

I'm not sure if I'm interpreting "root problem" correctly, but I'm using this to generate html files, people inspecting the resulting html see that their hrefs have been messed with, try to copy and paste it into the url and see that it doesn't work, and ask me to fix it 😅 . i would rather not need to add html comments next to each generated anchor tag that link to this issue (would bloat generated html size). I guess I could monkey patch this, but I'm not sure I understand why we wouldn't want to enable an option to disable this? 🙏

[wooorm]

Yes, read the surrounding text. The spec both says what authors should do and how parsers should work. This section you quote is in the heading “Syntax errors”. You post something that is a syntax error. It is still handled gracefully. “To help reduce the number of users who accidentally run into such problems, they are made non-conforming.”
With that, you can also infer what tools like hast-util-to-html should do.

people inspecting the resulting html see that their hrefs have been messed with

Can you please explain more about these users who care very much about their HTML but not about encoding & correctly as &. They‘re wrong, and they should be taught how HTML works. There is nothing for you to fix other than their understanding.

[cheapsteak]

Users don't always write to me when it happens, some would just drop or switch to another tool that doesn't feel broken to them before I can explain to them that what they think is broken is actually fine 😅

I monkey patched this for now with patch-package, but was this not what allowParseErrors is meant for? ("Do not encode characters which cause parse errors (even though they work), to save bytes [..] works but isn’t pretty"?)

[wooorm]

some would just drop or switch to another tool that doesn't feel broken to them before I can explain to them that what they think is broken is actually fine

Sorry but this feels really weird. You have noobs as users who don‘t understand HTML and rather switch to something else? What type of tool is this?

I monkey patched this for now with patch-package

Then you are breaking other things.
I recommend to CMD+f “ambiguous ampersand” in the spec. Also look at the “13.2.5.73 Named character reference state” and related states of the parser.

parseError

That option does not suddenly turn on all possible parse errors. It instead looks allows several characters which are well-defined in how they are handled, but technically parse errors.

[adammcodes]

I have encountered this problem in Salesforce Marketing Cloud's Email Studio as well.

The sfmc email studio editor is a raw text editor. I paste HTML from my clipboard into that, which includes links with encoded &

e.g.

href="https://cloud.mail.company.ca/unsubscribe?imtoken=%%token%%&lang=fr"

When we click the test email from the inbox the resulting URL in the browser looks like it skips the parameter for lang entirely.

It's not clear whether it just took the first & of the entity and skipped the rest, or if it correctly decoded it to an "&" but skipped the key-value pair "lang=fr"

The link works but it defaults to the english page because there is no lang=fr parameter.

When I manually replace the entity for & in the source code href with & then I get the French unsubscribe page without any issue.

[adammcodes]

I think in my specific case the "root" of the issue could be that SFMC is escaping an already escaped URL?

Based on this info:
https://www.perplexity.ai/search/How-does-SFMC-m0Ig.U5.Qdu_rPoGk9Wx5w

This is just a guess. But I thought I would include this here in case it helps anyone else experiencing this problem.

[wooorm]

Perhaps try setting characterReferences.useNamedReferences = true. It may be that these tools do not support numeric character references. Or all character references. It’s still sounds like a bug they should fix, but it seems highly unlikely that they never got a report for & before?

%%token%%

Those percentages might also do something with percent encoding?

This is just a guess.

I would assume so too