Support MathJax for chat messages!

[ghost]

This is a feature request, which I'll admit is probably not too important for most users.

I think it would be really amazing if text in between $dollar signs$ could be rendered as inline math formulas using MathJax, similar to what you see on some of the stackexchange sites. It's also common to use $$double dollar signs$$ to denote full equation blocks.

This may obe an easy addition, seeing as markdown is already supported. With names like "matrix" and "vector", there's bound to be at least a small subset of users who actually use this chat client to talk about math! It's hopefully at least worth considering, anyway.

[ara4n]

Thanks for the feature request - agreed this would be very cool. What
typically goes between the dollar signs? MathML? TeX? ASCIImath?

On 10/08/2016 19:38, Jesse Maes wrote:

This is a feature request, which I'll admit is probably not too
important for most users.

I think it would be really amazing if text in between |$dollar
signs$| could be rendered as inline math formulas using MathJax
https://www.mathjax.org/, similar to what you see on some of the
stackexchange sites. It's also common to use |$$double dollar
signs$$| to denote full equation blocks.

This may obe an easy addition, seeing as markdown is already
supported. With names like "matrix" and "vector", there's bound to be
at least a small subset of users who actually use this chat client to
talk about math! It's hopefully at least worth considering, anyway.

— You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#1945, or mute the
thread
https://github.com/notifications/unsubscribe-auth/ABO_vZsg18vTcaqttEVn7_YKZBAeGrB0ks5qem7tgaJpZM4Jhta4.

[ghost]

I'm definitely most used to seeing TeX notation for inline math. As far as I know it's the most commonly used and versatile of those three - I've never found anything it couldn't write down. It's also the only one I've used though, so maybe I'm biased there.

[jansol]

As a CS student with a friend circle of massively nerdy people (math, physics, programming etc) I certainly would have some uses for this, although it's by no means critical.

+1 for TeX

[kykc]

As a DSP engineer I would much appreciate this feature, as sharing math with remote co-workers and friends has been rather painful for a long time.
+1 for TeX
markdown-preview-plus from Atom text editor supports something like this, btw.

[saad440]

So... When are we going to have it? :)
It will really come in handy in the physics chat.
+1 for TeX

[joelostblom]

It could be worthwhile looking into Khan Academy's KaTeX as a speedy, lightweight alternative to MathJax.

[uhoreg]

As far as encoding the math in the message event goes, I think that it makes sense to send it as MathML (obviously in a "format": "org.matrix.custom.html" message). MathJax can be used to convert various formats (including LaTeX) into MathML, and it can be used to display the MathML in browsers that don't support MathML. An alternative to sending MathML would be to stick LaTeX math inside a special element. For example, MathJax uses a <script type="math/tex"> element to indicate math text, or it could be a <span> with a special class..

Pros of sending MathML as opposed to sending the raw LaTeX wrapped in an element:

• MathML is already standardized, so no need to bikeshed about the container element for the math
• not stuck with LaTeX. e.g. math could be entered using AsciiMath (which MathJax also supports)

Cons of sending MathML:

• MathML is much, much, much more verbose than LaTeX
• MathJax would have to be invoked twice on the sending client (once to convert to MathML, and once to render the MathML), though I'm not sure how bad that actually would be
• MathML has very poor browser support (so MathJax would be needed pretty much all the time anyways)

This would all also depend on what can be used to do math on iOS and Android.

Note: MathML allows embedding the original LaTeX source via annotations

(BTW, a note about KaTeX: it is much faster than MathJax, but also doesn't support as much LaTeX as MathJax does. What we do on our site at work is that for each chunk of math, we first run it through KaTeX, and if it errors, then we run MathJax on it.)

[geez0x1]

Robotics engineer here, yes please!

LaTeX as default would definitely be most sensible I reckon; I don't think any of my colleagues know MathML or ASCIImath. Every single one knows LaTeX.

[quell-]

As a CS researcher, I would highly appreciate this feature. Including math formulas in our daily conversations is very common among my colleagues. It would be nice if we could do it instantly in chat flow.
+1 for TeX

[Cadair]

I would love to see this!

[ara4n]

I would too!!!! But we have no choice but throw ourselves on the generousity and creativity of the internetz for this one...

Patches welcome!!!

[MTRNord]

Just throwing a possible dep: https://www.npmjs.com/package/react-mathjax
Not sure yet how to use it in riot-web exactly but seems to be pretty simple.

[hgustafsson]

I made a proof of concept where the client renders math formulas between $ ... $ or $$ ... $$ using KaTeX on
https://github.com/hgustafsson/matrix-react-sdk/tree/hgustafsson/katex
https://github.com/hgustafsson/riot-web/tree/hgustafsson/katex

Ideally, there should be a room-specific option for turning this on and off (probably off by default), and for specifying different delimiters, but I don't know how to do that.

I was also playing with the idea that the sender generates an already formatted message (formatted_body) in html or mathml using KaTeX, but the riot-web receiver removed these tags in the html sanitizer of the function bodyToHtml.

[Cadair]

From an inter-client compatibility standpoint transforming to html on the way out makes sense to me? In that if (not riot) client B can render html then if riot with KaTeX sends html it will display as html and not latex source.

[t3chguy]

@Cadair for consistency the HTML transform is always done by the sender

[Cadair]

@t3chguy sure, that makes sense. @hgustafsson was saying that that approach isn't working.

[t3chguy]

yeah, currently the subset of allowed HTML is very strict, it may need to be loosened or another format being added to support it, currently the format is a custom html one, adding another would be pretty simple

[hgustafsson]

@Cadair @t3chguy I agree that this would be the best, but all markup was removed by the html sanitizer. I'm not confident enough with the security implications to change the allowed tags and attributes in sanitizeHtmlParams of HtmlUtils.js. (The same security implications would apply even if we make another message type/format for math which would allow more tags.)

The result of, for example, katex.renderToString("\cos(x)") looks like

<span class="katex-display">
    <span class="katex">
        <span class="katex-mathml">
            <math><semantics><mrow><mi>cos</mi><mo>(</mo><mi>x</mi><mo>)</mo></mrow><annotation encoding="application/x-tex">\cos(x)</annotation></semantics></math>
        </span>
        <span class="katex-html" aria-hidden="true">
            <span class="strut" style="height: 0.75em;"></span>
            <span class="strut bottom" style="height: 1em; vertical-align: -0.25em;"></span>
            <span class="base">
                <span class="mop">cos</span>
                <span class="mopen">(</span>
                <span class="mord mathit">x</span>
                <span class="mclose">)</span>
            </span>
        </span>
    </span>
</span>

Unfortunately, all the work I did in this direction was when marked was still used for Markdown for which I could use ViktorQvarfordt/marked and write a custom rendering engine for math using KaTeX. Hopefully something similar can be done with commonmark but I haven't looked at it.

[t3chguy]

then make an alternate format which is done client side, formatted_body being the LaTeX formatting and render clientside, assuming that the KaTeX implementation is sane and 100% consistent

[CsatiZoltan]

Any progress in it?

[RoyiAvital]

Please don't use KaTeX.
Use MathJaX as it is much more feature rich.

[uhoreg]

In my experience, MathJaX covers much more LaTeX than KaTeX does, but KaTeX still covers the vast majority of what most people need. However KaTeX is much faster than MathJaX (though I believe the MathJaX developers have been working on improving speed lately). Also, MathJaX would randomly fail some times when we had a lot of math. So what we did was we rendered math in KaTeX, and if KaTeX was unable to render it, then we fell back to MathJaX.

[jeanm]

My experience matches @uhoreg's, and I would urge whoever ends up developing this to consider using KaTeX by default.

KaTeX is much faster, and does not have the annoying reflowing issues. MathJax's performance made it almost unusable on my old Android phone. KaTeX also seems to cover the vast majority of what people normally need (at least in physics and compsci). New commands are constantly being added as the project is very active.

[uhoreg]

<braindump> (or maybe \begin{braindump})
In order for this to happen, several tasks need to be done:

1. decide on the message format for the math. The main options seem to be: LaTeX within some sort of container, MathML, or some pre-rendered format. Some considerations for this are:
   • how well the format is supported by different platforms (web, Android, iOS, GTK+, Qt, console, etc.), and how well the libraries work (in terms of bloat, speed, etc).
   • how well the format degrades for clients that don't support math
   • accessibility
   • how much of the format should be supported (e.g. for LaTeX, it probably makes sense to not support things like \newcomand. But should it support AMSMath? For MathML, it may make sense to just limit it to Presentation or Content MathML. etc.)
2. decide how to input math. This may involve both a Markdown and a non-Markdown method. For Markdown, the obvious solution is to enclose math (possibly LaTeX) within some delimiters (though using the traditional $...$ notation may cause non-math to be accidentally matched, so it may be better to use \(...\) instead). AsciiMath might be another option for hand-typed math. For a user-friendly, glossy math entry, something like MathQuill might be worth looking at.
3. decide which libraries to use. For web, the main contenders are MathJax and KaTeX. MathJax supports more of LaTeX, but KaTeX may be sufficient for most people and is more lightweight. MathJax also supports AsciiMath and MathML as input.
4. decide if the math libraries should be lazy loaded.

This issue needs someone to work through all of these things and write out a proposal.

\end{braindump} (or </braindump>)

[Evidlo]

I think Tex would ideally be transmitted in its raw format using extensible events.

[Cadair]

While I agree, there is no need to block this on that proposal becoming reality.

[hhassey]

Any updates on LaTeX being supported?

[Evidlo]

Here's a demo which compares Katex and Latex performance.

Also I think having a delimiter for inputting Latex outside of Markdown mode is unnecessary.

Mathquill looks really nice.

[uhoreg]

matrix-org/matrix-spec-proposals#2191 is a counter-proposal to matrix-org/matrix-spec-proposals#1722 that uses LaTeX instead of MathML.

[ara4n]

lots of experimentation going on currently on this between @thosgood, @Cadair, myself with input from @uhoreg. Approaches are split between experiments with render-at-receiver-client (matrix-org/matrix-react-sdk#3251), render-at-sender-client, and render-at-sender-mediarepo (or combinations of all 3).

Before I lose it, https://jsfiddle.net/8Logkhr5/35/ is a POC for render-at-sender-client via MathJax.

[acxz]

Sorry to bump this but I was wondering what the status of this feature is at and when we can expect it to land on develop?

[erdnaxe]

From the user perspective, render-at-sender-client will fill the media history of a room, and requires much more space server-side. I find KaTeX (or other equivalent) fast enough to be used on large rooms, so the render-at-receiver-client approach seems better.

I would really like to see this feature in develop in the following weeks (my university is using Matrix to chat during lessons). How can I help ? May I rebase matrix-org/matrix-react-sdk#3251 in matrix-react-sdk:develop as a new pull-request ?

Another way to render LaTeX is Matrix would be to have a bot uploading the rendered LaTeX but I find this approach non user-friendly.

[thosgood]

I've played around with a bot that responds to messages with the rendered image, but it really wasn't very pleasing to use, as you say. (There was another bot a while back that also did this (much better than mine), but I think the general consensus was that it was not really the best solution to this specific need).

If you wish to advance this project a bit more, then I suggest dropping by #pigeon.digital:matrix.org, where we've been having discussions about this pull request (and the issue in general). I really want to keep working on this, but sadly have a bunch of deadlines in these coming weeks that take priority. I'm still more than happy to discuss what we've done so far/ideas we had heading forwards though!

[correaswebert]

We must decide on a protocol first - #1945 (comment) seems to be nice

I agree on the fallback issue - matrix-org/matrix-spec-proposals#1722 (comment) - as for security, we may initially limit their usage to those having proper safeguards, and later roll in extras

#1945 (comment)
We can ask for permission to render the math henceforth if it is encountered in the message (and then add a toggle in the quick settings for those inclined toward math but also give them speed whenever needed; we may keep the toggle chat based...)

@uhoreg I want to participate in GSoC and would love to contribute towards this issue

[vale981]

Hi,
I did not know about this Issue here and created a Firefox extension to inject KaTeX into riot.
https://addons.mozilla.org/en-US/firefox/addon/riot-katex/ (Beware, it is quite a hack job :P)

This is of little relevance for an actual implementation in riot but may be of interest for some people who don't want to wait.

Cheers,
Valentin

[thosgood]

I've mentioned this a bunch of times in the other pull request, but have apparently forgotten to do so here: this test build is actually up and running at https://pigeon.digital , and I think it's currently being used quite a bit, and I haven't had any error complaints so far! Please feel free to use it, but just note that you have to enable KaTeX rendering in your user settings, and make sure that the 'maths' toggle is turned on in the composer. (There are some per-room settings, but these don't do anything).

![Uploading Screenshot 2020-04-12 at 16.22.13.png…]()

[vale981]

Great!

[a22sc]

wouldn't it be good to make a bot that creates png's to involve bridge users and users with other clients?
found this for telegram: https://github.com/vdrhtc/InLaTeXbot

[jtyard]

Great! I tried it out on https://pigeon.digital and it worked well, though I was unable to edit a post while keeping the katex formatting. Specifically,

• dollar signs were not reproduced in the edit box
• even if the dollar signs were put back in, the edited post was not processed as katex

[Esokrates]

@thosgood Any progress on this? pigeon.digital seems down to me.

[CsatiZoltan]

@Esokrates For me, this room works. This is where issues are discussed. The good news is that they are close to it!

[ArniDagur]

The current PR to watch seems to be matrix-org/matrix-react-sdk#5244

[stites]

bump, matrix-org/matrix-react-sdk#5244 has been merged!

[jryans]

As mentioned above, we now have experimental LaTeX support available via a labs flag on Develop / Nightly builds. 🎉

Please try it out and file new issues with any feedback, as that will help us understand what's left before making this generally available.

[tobiasBora]

So I just tried it out. For that, I went on https://riot.im/develop/ and I enabled the option by going in All settings (arrow on the right of your name on the upper left part of the screen), Labs, Render LaTeX maths in messages:

See some bugs here #15846 (comment)

[lon91ong]

So I just tried it out. For that, I went on https://riot.im/develop/ and I enabled the option by going in All settings (arrow on the right of your name on the upper left part of the screen), Labs, Render LaTeX maths in messages:
...
See some bugs here #15846 (comment)

@tobiasBora Element can't load
Something went wrong and Element was unable to load.

[t3chguy]

@lon91ong check the browser console, either an extension/firewall is blocking something or your browser is missing a required feature.

[jpmvferreira]

Is this not yet available on the desktop client? I'm running element-desktop 1.7.34-1 from the Manjaro repos and I am not able to see this feature anywhere.

On the general Matrix chat a user told me that I would be able to manually configure this somewhere, in a config file, but I can't seem to find where.

[SimonBrandner]

This is implemented but experimental therefore you need to enable it in a config file. Feel free to ask in the Element Web/Desktop room for help with that

[Pankratius]

It would also be very cool to have it for the ios-app.

[tobiasBora]

Any idea how to use this feature in 2022? The url https://riot.im/develop/ does not exist anymore and this feature does not appear in the experimental features (labs) in https://app.element.io/: I can only see "Threads" and "The new search".

[turt2live]

riot.im/develop is now develop.element.io

[t3chguy]

The old URL works just fine still anyway.