Markdown rendering hides raw email content that resembles footnotes

[jmikola]

The raw text of https://externals.io/message/116044 is:

I just discovered that run-tests.php was changed to cache SKIPIF evaluation since 8.1.0beta3[1]. I believe ext-mongodb ran into the same issue as mysqli[2], as we use SKIPIF to check that database contents are clean going into a test. The present solution in core is to check SKIPIF output for "nocache" [3,4] and allow individual tests to opt out of caching; however, I'm worried that won't be portable for third-party extensions, where we test with run-tests.php for each version of PHP that we support.

Is there still time to consider an alternative solution? If "nocache" needs to remain in place for mysqli, perhaps 8.1.0's run-tests.php can be enhanced to consult an environment variable (for maximum BC and playing nice with `make test`) that disables caching entirely.

[1]: https://github.com/php/php-src/pull/6681
[2]: https://github.com/php/php-src/pull/6726
[3]: https://github.com/php/php-src/commit/4d43cbe333690171753e9b8663df93d3762e02a8
[4]: https://github.com/php/php-src/commit/0074a1d4e3a85d0d63118e7a30f4b7ed6da64695

I was not aware that externals.io rendered emails as Markdown until I happened to view the thread above and realized that the footnote links I added were stripped. I didn't have Markdown in mind when I wrote the email, and was just using that notation to avoid embedding the links in the paragraphs.

I'm not sure if this qualifies as a bug since the syntax I used is a bit different from actual Markdown footnotes (e.g. [^1]), but I'll defer to the authors to triage accordingly. In the future, I'll also try to keep this in mind when writing emails to internals.

[mnapoli]

Thanks for the report. Indeed, externals.io tries a Markdown approach which works well 90% of the time. The rest of the time, the result is slightly worse. Still overall it's an improvement ^^ so we have to bear with the limitations.

TBH I'm not sure there is much to do here, maybe someone will have an idea on adding a special case for footnotes? I'm open to ideas.

[jmikola]

Detecting if Markdown conversion loses content seems like an unsolved problem. https://stackoverflow.com/a/24690466 touches a bit on it and ends with a suggestion about training a model with sample data. I think that's beyond the scope of what you'd want to do here, though.

I suppose a naive approach in this case would be to compare the original content with the HTML produced by Markdown. If some URLs from the original content disappear entirely in the output, that might suggest that something of importance was lost and you could fall back to displaying the original content instead. Ideally, displaying the original content could still apply word wrap, to avoid what happened in the code block in the OP above.

[mnapoli]

Just for reference, I had a second look at the rendered version again and it seems to be interpreting the syntax as "links" (with deported values), not footnotes.

E.g. links 1 and 2 work correctly in the text.

Links 3 and 4 don't because the syntax isn't exactly right (for Markdown syntax).