Dense pandoc-style TeX is not parsed

[jonathanjameswatson]

Describe the bug
Release 9.2.0 added support for Pandoc-style display math blocks.

Pandoc-style single-line display math blocks will not be parsed by LTeX, despite them being allowed in https://pandoc.org/MANUAL.html#math.

Even if LTex won't implement single-line display math block parsing, multi-line display math blocks that occurs after (an odd number of) single-line display math blocks should probably still be parsed.

Steps to reproduce

1. Open the below sample document in VSCode with LTeX
2. Line 6 should have a grammar warning, despite being in a math block. Line 3 should not have a grammar warning, despite not being in a math block.

Expected behavior
Lines 1 and 6 should not be checked, while line 3 should be.

Sample document

$$do not check me$$

do not check me

$$
do not check me
$$

LTeX configuration
Default configuration

Version information
List here the version information of the relevant software.

• Operating system: Windows 11
• ltex-ls: 15.2
• Java: 15

[valentjn]

LT_EX is not Pandoc, nor it is claiming to support all Pandoc extensions. Converting to feature request.

[valentjn]

It's quite complicated. LT_EX LS uses flexmark-java, which requires that all block nodes (such as source code with ``` delimiters, or the custom display math blocks with $$ delimiters) are on their own lines. This means that

$$do not check me$$

should not have been parsed as display math by LT_EX LS, as the $$ are not on their own lines. However, due to a bug, LT_EX LS ignored everything after the initial $$, so it parsed the example as follows:

[$$ | DISPLAY_MATH_START_DELIMITER][do not check me$$ | IGNORED]

[do not check me | DISPLAY_MATH_CONTENTS]

[$$ | DISPLAY_MATH_END_DELIMITER]
[do not check me | PARAGRAPH_TEXT]
[$$ | DISPLAY_MATH_START_DELIMITER]

Even when fixing this bug, it's not possible to parse the example like Pandoc does due to flexmark-java requiring the block delimiters are on their own lines. However, I implemented a workaround that parses the first display math block as inline math, which has a similar effect for our purposes:

[$ | INLINE_MATH_START_DELIMITER][$do not check me$ | INLINE_MATH_CONTENTS][$ | INLINE_MATH_END_DELIMITER]

[do not check me | PARAGRAPH_TEXT]

[$$ | DISPLAY_MATH_START_DELIMITER]
[do not check me | DISPLAY_MATH_CONTENTS]
[$$ | DISPLAY_MATH_END_DELIMITER]

[github-actions]

This issue is now fixed on develop. The fix will be included in the next release of LT_EX.

If you don't want to wait, you can try out the nightly pre-release tomorrow. Nightly pre-releases are published every morning at around 4am UTC.