Lua filter to preserve HTML comments

[jsx97]

I try to create a Lua filter to preserve HTML comments (but not any other HTML elements).

function RawInline(el)
  return nil
end

function RawBlock(el)
  if starts_with('<!--', el.text) then
    return el
  else
   return nil
  end
end

return {{Inline = RawInline, Block = RawBlock}}

It doesn't currently work. What might be the problem?

pandoc -f html+raw_html from.html -o to.md -t gfm --lua-filter preserve-comments.lua

6: attempt to call a nil value (global 'starts_with')

[kysko]

1. You seem to be using starts_with function from an example in the lua filter guide; paste it in your code.

2. If you test by exporting to native format (with raw_html extension), you'll see that html comments are stored in RawInline.

I think you take it from here...

[jsx97]

Thanks. But now it says 2: attempt to index a nil value (local 'str')...

[kysko]

That's what my point 2 is about. Make sure you're using the right filter.

For example, test with the following simple filter:

function RawInline(el)
  print(el.text)
end

It should print out all the comments it encounters.

/Edit:
As for the error you see, it's because you use Block = RawBlock. The Para or Plain blocks that contain the RawInline don't have a text field. You could try RawBlock = RawBlock (or just comment out your return line), but as said above, the comments are in RawInline... test it!

[jsx97]

Could you provide some futher advices?

I have tried

local function starts_with(start, str)
  return str:sub(1, #start) == start
end

function RawInline(el)
  if starts_with('<!--', el.text) then
    return el
  else
    return nil
  end
end

return {{Inline = RawInline}}

but there is still something wrong there.

[badumont]

return {{Inline = RawInline}} This statement is wrong: it means that every Inline will be passed through your function RawInline, which should trigger an error on elements that don't have a text field (such as Emph). If you want to pass every RawInline through the function RawInline, simply remove the return statement: this is the default behaviour. Returning nil from your function or returning the object without modifications does the same thing, i.e. doesn't change anything on the AST: see the page on Lua filters from “The return value of a filter function must be one of the following”. (By the way, if you want to delete an object, you must return an empty list {}.) If I understand right, what you want to do is to alter the behaviour of a writer. This can be done with a custom writer: https://pandoc.org/custom-writers.html

[jsx97]

If you want to pass every RawInline through the function RawInline, simply remove the return statement

But if I simply remove it, the filter doesn't seem to do anything at all... I mean, there must something else except functions to make the code work, right?

local function starts_with(start, str)
  return str:sub(1, #start) == start
end

function RawInline(el)
  if starts_with('<!--', el.text) then
    return el
  else
    return nil
  end
end

[badumont]

But if I simply remove it, the filter doesn't seem to do anything at all...

That's the point 2 of my answer!

[jsx97]

Cannot figure it out myself. Just in case, I have started bounty on Stack Overflow: https://stackoverflow.com/q/78063469

[badumont]

Once again, you must return something other than the unmodified element or nil, since in both case nothing will be changed in the AST. One solution would be to return the comment as a RawInline pretending to be a raw GFM fragment. (This is more or less equivalent with using a custom writer as I suggested above.) local function starts_with(start, str) return str:sub(1, #start) == start end function RawInline(el) if starts_with('<!--', el.text) then return pandoc.RawInline('gfm', el.text) end end

[jgm]

Note: getting rid of all raw HTML elements (besides comments) in the AST won't guarantee that you won't see HTML tags in the gfm output, because (given the expressive limitations of gfm) pandoc's markdown writer will resort to raw HTML to render some elements, e.g. Figure or Div.

[jsx97]

Hi, John. So in other words what I want to achieve, despite its apparent ease, is in a fact a top of the iceberg, and is not easy at all?

[jgm]

Well, there is a way, I think. Use -t gfm-raw_html-native_divs which will disable raw HTML and the HTML syntax for native divs. Since your filter reassigns comments to gfm instead of html, they will still be passed through.

[jsx97]

John, but when I try pandoc -f html+raw_html from.htm -o to.md -t gfm-raw_html-native_divs --lua-filter preserve-comments.lua, it says The extension native_divs is not supported for gfm. What might be a problem?

[jgm]

OK, leave that one out.

[badumont]

Now you know from the responses on Stackoverflow (and from a comment from me above if I remember well) how to suppress an element in the AST, so just do that in an else clause.

[tarleb]

Let me copy over my solution from StackOverflow.

local function starts_with(start, str)
  return str:sub(1, #start) == start
end

function RawInline(el)
  if not starts_with('<!--', el.text) then
    return {}
  end
end

As noted in the comments there, this is insufficient for your purpose. (I'd like to stress again that, in cases like this, a lot of everybody's time can be saved by including the example and expected output directly in the question). Since you don't want any HTML, it's best to use -t gfm-raw_html. This also suppresses all HTML snippets, so we need to pretend that our comments are actually Markdown. Modifying the filter like this will do the trick:

function RawInline (el)
  return starts_with('<!--', el.text)
    and pandoc.RawInline('markdown', el.text) -- pretend HTML comment is raw Markdown
    or {}  -- not an HTML comment, thus drop it
end

[jsx97]

Thank you so much, Albert! This is exactly what I have been trying to achieve for about a week without success. I think now we can copy this back to Stack Overflow for other people who might find it useful.