raw json in nbconvert output

[abchenson]

I converted the attached notebook as part of a sphinx build and got the attached html. As you can see there is a problem: the end of the html has a load of raw stuff from the nb printed out.

When I created the notebook, ran it and saved it I did not see this problem. Then I shut the notebook, reloaded it, ran it and saved again. At that point the error appeared. The part of the notebook at which the raw stuff begins starts after the string "Export to plot.ly" and around that point tere are a lot of escaped "s. But that is as far as I got diagnosing the problem. this issue is separate from the plot.ly output not appearing -- I'd consider it solved if there was no plot.ly output but the mess at the bottom of the page was gone.
error.zip

nbconvert (5.3.1)
Sphinx (1.7.0)

[takluyver]

@jasongrout I think this is related to the widgets state getting embedded at the bottom of the HTML page. Possibly something needs to be HTML escaped. Can you take a look?

[abchenson]

I don't understand the request well enough to give a good answer The first line of the bad output in the html is

", "application/vnd.plotly.v1+json": {"data": [{"name": "one", "x": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], "type": "bar", "marker": {"color": "rgb(0, 150, 0)"}, "y": [5, 6, 7, 8, 9, 10, 11, 12, 13, 14]}, {"name": "two", "x": [12, 13, 14, 15, 16, 17, 18, 19, 20, 21], "type": "bar", "marker": {"color": "rgb(150, 0, 0)"}, "y": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]}], "layout": {"title": "title"}}, "text/vnd.plotly.v1+html": "

This seems to correspond to line 230 in the notebook. Possibly the problem is in the text/html field in the same JSON object, which is

          "text/html": "<div id=\"af98c611-3bd0-400f-9bd4-d3ff67474876\" style=\"height: 525px; width: 100%;\" class=\"plotly-graph-div\"></div><script type=\"text/javascript\">require([\"plotly\"], function(Plotly) { window.PLOTLYENV=window.PLOTLYENV || {};window.PLOTLYENV.BASE_URL=\"https://plot.ly\";Plotly.newPlot(\"af98c611-3bd0-400f-9bd4-d3ff67474876\", [{\"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], \"name\": \"one\", \"y\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], \"type\": \"bar\", \"marker\": {\"color\": \"rgb(0, 150, 0)\"}}, {\"x\": [12, 13, 14, 15, 16, 17, 18, 19, 20, 21], \"name\": \"two\", \"y\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], \"type\": \"bar\", \"marker\": {\"color\": \"rgb(150, 0, 0)\"}}], {\"title\": \"title\"}, {\"showLink\": true, \"linkText\": \"Export to plot.ly\"})});</script>",
          "text/vnd.plotly.v1+html": "<div id=\"af98c611-3bd0-400f-9bd4-d3ff67474876\" style=\"height: 525px; width: 100%;\" class=\"plotly-graph-div\"></div><script type=\"text/javascript\">require([\"plotly\"], function(Plotly) { window.PLOTLYENV=window.PLOTLYENV || {};window.PLOTLYENV.BASE_URL=\"https://plot.ly\";Plotly.newPlot(\"af98c611-3bd0-400f-9bd4-d3ff67474876\", [{\"x\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], \"name\": \"one\", \"y\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], \"type\": \"bar\", \"marker\": {\"color\": \"rgb(0, 150, 0)\"}}, {\"x\": [12, 13, 14, 15, 16, 17, 18, 19, 20, 21], \"name\": \"two\", \"y\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], \"type\": \"bar\", \"marker\": {\"color\": \"rgb(150, 0, 0)\"}}], {\"title\": \"title\"}, {\"showLink\": true, \"linkText\": \"Export to plot.ly\"})});</script>"

I don't know if that helps. I should say that the notebook is not broken when run in Jupyter. It looks fine there but not in the html conversion.

[abchenson]

Maybe this is what you were asking for: on line 11895 of reproduce_plotly_err.html you can see the same string that comes at the top of the erroneous stuff at the bottom of the page.

", "application/vnd.plotly.v1+json": {"data": [{"name": "one", "x": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], "type": "bar", "marker": {"color": "rgb(0, 150, 0)"}, "y": [5, 6

right after </script> . It's hard to follow back by eye and read off what the context is, whether we are in a string, a tag, or what, but that is where it starts printing out the wrong content.

[hughchristensen]

@takluyver @jasongrout did you have any thoughts on this? Would be great to get a solution to it!

[mgeier]

See also spatialaudio/nbsphinx#183.

[takluyver]

So the JSON blob of widget state contains some HTML, which contains a </script> closing tag, so naturally the browser decides that this is the end of the <script> that's meant to contain that whole JSON blob.

I'm pretty sure that some kind of escaping is needed at some level, but I haven't figured out exactly what and where. I'm hoping that the widgets developers can investigate it.

[maartenbreddels]

import ipywidgets as widgets
widgets.Label("Let me bug you with a closing tag </script>")

Is the smallest example to make nbconvert+html-manager not work.

The state needs to be HTML escaped, and I think HTML manager needs to unescape.