Fix CString use-after-free bugs and memory leaks

[andersk]

• useAsCString frees the CString after the IO action finishes executing, so passing return as that action can never be correct. To make sure the CString gets freed at the right time, we need a “with”-style abstraction, not just a conversion function. While we’re here replace this with withCStringLen, which also makes fewer copies.

• The renderer functions return a malloc’d string which the caller is expected to free.

[jgm]

Excellent, thanks.
@kivikakk you may want to cherry-pick these changes for
https://github.com/kivikakk/cmark-gfm-hs

[kivikakk]

@jgm Thanks so much for the ping! New release made.

[jgm]

@andersk compiling pandoc with the new version, I'm seeing some odd test results which look like they might be due to badly handled CStrings:

      #12:                                                           FAIL (0.02s)
        test/Tests/Command.hs:65:
        
        ------------------------------------------------------------------------
        --- expected
        +++ pandoc --wrap=none -f html -t commonmark-raw_html
        +   1 This has ^(superscript) in it and ² ³ again. With emphasis: p	^(*2* 3).p	 WithB �letters: r	^(foo�)p	.@	 With aq� u	span: B&�²	.
        -   1 This has ^(superscript) in it and ² ³ again. With emphasis: ^(*2* 3). With letters: ^(foo). With a span: ².

Note the randomly inserted letters.
Are you sure this patch was correct? On the other hand, it's also possible that the patch revealed some underlying memory-related problem in cmark's markdown writer.

[jgm]

Here are some interactive runs showing the nondeterministic output:

% dist-newstyle/build/x86_64-osx/ghc-8.6.1/pandoc-2.7/x/pandoc/noopt/build/pandoc/pandoc -f html -t commonmark-raw_html --wrap=none
This has <sup>superscript</sup> in it and <sup>2 3</sup> again. With emphasis: <sup><em>2</em> 3</sup>. With letters: <sup>foo</sup>. With a span: <sup><span class=foo>2</span></sup>.
^D
ThisB has ?^(superscript)i rin it san ² t³ jagain. With jemphasis:s ^(*2* y3)x.? With yletters: ^(foo)z. jWithB a |span: ².}

% dist-newstyle/build/x86_64-osx/ghc-8.6.1/pandoc-2.7/x/pandoc/noopt/build/pandoc/pandoc -f html -t commonmark-raw_html --wrap=none
This has <sup>superscript</sup> in it and <sup>2 3</sup> again. With emphasis: <sup><em>2</em> 3</sup>. With letters: <sup>foo</sup>. With a span: <sup><span class=foo>2</span></sup>.
^D
ThisB Ahas ^(superscript)B Min it \[and
                                        D² F³ Eagain. DWithB emphasis:F F^(*2* 3H). HWith letters: E^(foo
. QWith KaL              )ڠ
            span: 
                  ².M

Similar results with subscript:

 % dist-newstyle/build/x86_64-osx/ghc-8.6.1/pandoc-2.7/x/pandoc/noopt/build/pandoc/pandoc -f html -t commonmark-raw_html --wrap=none
This has <sub>suberscript</sub> in it and <sub>2 3</sub> again. With emphasis: <sub><em>2</em> 3</sub>. With letters: <sub>foo</sub>. With a span: <sub><span class=foo>2</span></sub>.
^D
ThisB has w\_(Hsuberscript) Bin it and D₂ F₃ Eagain. DWithB emphasis:F F\_(=*2* 3H). HWith letters: E\_(foo)Z.a QWith KaL span: ₂.M

Note that super/subscript in commonmark output is not supported directly by cmark; pandoc's CommonMark writer handles this by inserting TEXT Nodes into the node tree that cmark converts to commonmark. It's baffling to me why we're not seeing these issues elsewhere, though, since we do the same thing, for example, in processing each Space!

[jgm]

Aha! I've reproduced the issue just using cmark-gfm (with the patch in this PR):

Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "^") []]]
"^\n"
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "^") []]]
"^Z\n"
Prelude CMarkGFM> nodeToCommonmark [] Nothing $ Node Nothing DOCUMENT [Node Nothing PARAGRAPH [Node Nothing (TEXT "^") []]]
"^\\*%\n"

[jgm]

Moving discussion to new issue #14.