No data is shown when using the widget renderer from an IDE to a browser

[T-Flet]

What is the intended/supported behaviour when running the widget renderer from a script in an IDE?

Minimal example: the widget renderer example in the documentation with an added .show() on the last line (since just chart does nothing in non-notebook context).

When trying this from an IDE (e.g. PyCharm), the plot is served on localhost and a browser tab is automatically opened, but no data is received by it (i.e. the 3 plot areas are there but empty).
A warning for the data not being found is issued (although the mentioned file IS there on inspection):

Displaying chart at http://localhost:59133/
WARNING:tornado.access:404 GET /_vegafusion_data/vegafusion-29a8824c1c6cebef5b2f03c6cc02b4539175a65d.feather (::1) 0.00ms

If a data folder is specified (i.e. vf.enable_widget(data_dir = ...) at the beginning of the example), then the no-data behaviour persists but no warning is issued.

Is this behaviour expected and/or correctable?

As a separate note, the example works "fine" (very slowly) with altair_data_server (i.e. replacing vf.enable_widget() with alt.data_transformers.enable('data_server')), thus not pointing to the typical causes of no-data being shown mentioned in the altair documentation.

Versions:

• VegaFusion 1.2.4
• Altair 4.2.2 (as 5.x currently has compatibility issues with altair_viewer Version 0.4.0 does not include updated listing.json altair-viz/altair_viewer#50 and altair_data_server Local data server breaks with transition to altair 5.0.0 altair#3048, as well as deprecation warnings and errors with this particular example)
• Python 3.10.4 (conda env on Windows 11)

[jonmmease]

Hi @T-Flet, thanks for the question.

tl;dr The widget renderer is not compatible with altair_viewer, so what you're seeing is expected unfortunately. One potential workaround would be to save the charts to a standalone HTML file with vf.save_html(chart, "chart.html") and then open the saved file in a browser (See https://vegafusion.io/save.html)

The widget renderer is based on a custom Jupyter Widget, so it will only work in contexts that are compatible with Jupyter Widgets. This includes the Jupyter Notebook, JupyterLab, Visual Studio Code notebook, Colab, and some other notebook environments. But AFAIK, there's not an easy solution for displaying Jupyter widgets in standalone browser windows from a Python script. What makes this tricky at a technical level is that Jupyter widgets require two-way communication between the Python kernel and the browser.

Could you add a little more detail on your desired workflow?

• Are you running a Python script with an if __name__ == "__main__": block, or using Pycharm's notebook interface?
• Are you hoping to run a script and have a single chart open in a browser at the end?
• Do the charts you're building generally have aggregations (e.g. histograms, heatmaps, bar charts, etc) or not (e.g. large scatter plots).

As a workaround for aggregated charts or moderate sized scatter charts (a few tens of thousands of rows) the vf.save_html function should do a good job of evaluating the chart's transforms, removing unused data columns, and writing the result to a standalone HTML file. Then to automatically open the file in a web browser, the Python webbrowser module could be used.

[T-Flet]

Thank you for the quick reply; perfectly reasonable and concise explanation.
Unfortunately, the two-way communication is precisely what I am looking for due to dataset sizes affecting plot responsiveness, therefore saving an html does not solve my issue.
My desired workflow is to have an interactive plot showing a slidable and x-y-resizable window of a small set of long (~500K+) time series (some overlaid, some in separate charts); ideally, multiple tabs containing separate such visualisations would be the goal.

Replying to your specific queries:

• I was indeed trying the documentation example in a __main__ block in a PyCharm script, this as a prelude to using this setup in a large program which produces multiple datasets which I would like to inspect with these interactive visualisations (i.e. the larger program would call a plotting function for each of the datasets and hang until all plot windows are closed).
• Running a script and opening one browser tab at the end is the starting point, but I would like to be able to open multiple ones.
• No, none of the datasets require aggregation in their visualisation, only a sliding window so that not all the data needs to be visualised at once.

Altair + VegaFusion's widget renderer seems to be a perfect match for my preferences and needs (grammar-of-graphics plotting + interactivity and avoiding sending all data at once); is there a way to make the above work? If not an exact way, is there something close I could do?

[jonmmease]

Thanks for the additional details @T-Flet,

The use case of running VegaFusion outside of Jupyter with two-way communication wired up crosses into the territory of a full Python dashboarding toolkit.

Presently, I'm aware of 3 Python dashboarding toolkits that have some support for Jupyter Widgets: Voila, Panel, and Solara.

Currently, Voila works well with VegaFusionWidget, but it requires writing your dashboard in a Jupyter notebook. Panel depends in ipywidget_bokeh, which unfortunately doesn't have support for the binary messages that VegaFusionWidget uses (See bokeh/ipywidgets_bokeh#46). I tried out Solara yesterday, and ran into an issue, but @maartenbreddels already fixed it in Solara 1.16.1.

From what I understand about your use case (and about Solara), I think Solara is currently the best solution for running VegaFusionWidget from a standalone Python script.

---

Here's an example:

pip install "vegafusion-jupyter[embed]" solara==1.16.0 vega-datasets altair==5.0.1

Create a file named try_solara.py that constructs a VegaFusionWidget assigned to a variable named chart_widget.

from vega_datasets import data
import altair as alt
import vegafusion as vf

source = alt.UrlData(
    data.flights_2k.url,
    format={'parse': {'date': 'date'}}
)

brush = alt.selection_interval(encodings=['x'])

# Define the base chart, with the common parts of the
# background and highlights
base = alt.Chart(width=160, height=130).mark_bar().encode(
    x=alt.X(alt.repeat('column')).bin(maxbins=20),
    y='count()'
)

# gray background with selection
background = base.encode(
    color=alt.value('#ddd')
).add_params(brush)

# blue highlights on the transformed data
highlight = base.transform_filter(brush)

# layer the two charts & repeat
chart = alt.layer(
    background,
    highlight,
    data=source
).transform_calculate(
    "time",
    "hours(datum.date)"
).repeat(column=["distance", "delay", "time"])

chart_widget = vf.jupyter.VegaFusionWidget(chart)

Then launch it from the command line with:

solara run try_solara.py:chart_widget

And the nice thing about this approach is that you can use all of the built-in ipywidgets as well (like dropdowns, layouts, tabs, etc. See https://ipywidgets.readthedocs.io/en/stable/examples/Widget%20List.html).

See what you think!

[T-Flet]

Thank you for going above and beyond the original tentative-bug-report / query.

Solara

I tried the Solara example both in the conda environment I need it in and in a fresh one (also Python 3.10), and in both cases I get the same error on the browser side:

RuntimeError: unreachable
    at http://localhost:8765/static/nbextensions/vegafusion-jupyter/2fe3e1b5de735a87f0db.module.wasm:wasm-function[3318]:0x4f3eb4
    at http://localhost:8765/static/nbextensions/vegafusion-jupyter/2fe3e1b5de735a87f0db.module.wasm:wasm-function[4544]:0x538cb0
    at http://localhost:8765/static/nbextensions/vegafusion-jupyter/2fe3e1b5de735a87f0db.module.wasm:wasm-function[3686]:0x51dae8
    at http://localhost:8765/static/nbextensions/vegafusion-jupyter/2fe3e1b5de735a87f0db.module.wasm:wasm-function[104]:0x1b90fc
    at http://localhost:8765/static/nbextensions/vegafusion-jupyter/2fe3e1b5de735a87f0db.module.wasm:wasm-function[3652]:0x51c08e
    at q (http://localhost:8765/static/nbextensions/vegafusion-jupyter/594.index.js:1:6539)
    at nt.p [as embedVegaFusion] (http://localhost:8765/static/nbextensions/vegafusion-jupyter/876.index.js:1:3878)
    at nt.value_changed (http://localhost:8765/static/nbextensions/vegafusion-jupyter/index.js:1:516831)
    at nt.render (http://localhost:8765/static/nbextensions/vegafusion-jupyter/index.js:1:515680)
    at async http://localhost:8765/_solara/cdn/@widgetti/solara-vuetify-app@4.0.0/dist/solara-vuetify-app8.min.js:2:4797881

Do you think this is a problem on my end, on VegaFusion's or should this be raised to a Solara issue referencing the one you raised and closed for that example?

Jupyter

Separately, it seems from the available options (please correct me if I am wrong) that I need to save the data to disk and run a separate visualisation in any case, as opposed to my original usecase intent of letting a program hang with it in RAM until plots are closed (I do realise that VegaFusion saves temporary feather files).
Therefore I thought that using a Jupyter environment reading from disk would be acceptable and perhaps automatable as far as opening and running after the main program generates the data.

I tried the widget renderer example in the documentation in VS Code, where no errors are raised (ignoring deprecations due to altair 5.x) but it is not clear whether the plot is generated since the widget area is too small:

Clicking on the VegaFusion logo and scrolling down suggests that the plot is not placed in the area at all (since the scroll distance is only for the opened menu):

Everything displays normally in a standard jupyter notebook (deprecations but present and working plot).
Any thoughts on this other front?

[jonmmease]

Have you updated to vegafusion/vegafusion-jupyter 1.3.0? I believe I saw a similar error to your solara error in 1.2.4 at one point, but that it was resolved in 1.3.0.

But now that I wrote this, I just checked and it looks like the conda-forge packages for vegafusion-jupyter 1.3.0 didn't go through. I'll have to look into that.

edit: Also, thanks for the reminder that this example needs to be updated to Altair 5!

[jonmmease]

Ok, vegafusion-jupyter 1.3.0 is now on conda-forge. I don't know that this will fix the solara issue you're seeing, but worth a try.

[T-Flet]

I am afraid both vegafusion and vegafusion-jupyter were/are 1.3.0, as they were pip-installed (though in a conda env).

[jonmmease]

Could you try adding the verbose=True flag to VegaFusionWidget and see if you get any additional logging in either the browser console or the terminal that solara was launched from?

e.g.

vf.jupyter.VegaFusionWidget(chart, verbose=True)

Also, do you get this same error for the exact example I posted above? I'm wondering if the error happens for every chart, or if it's only for particular chart types.

It's also worth force reinstalling vegafusion-jupyter

pip install --force-reinstall vegafusion-jupyter==1.3.0

[T-Flet]

I force-reinstalled vegafusion-jupyter and now it works on solara. I did re-check it was 1.3.0 before doing so, just as a sanity check, and it was. I do not know what changed, and I unfortunately did not run it today before the force-reinstall, therefore I do not even know whether that was indeed the solution.

(Not relevant now since it works, but adding verbose = True produced logs of requests sent and times and sizes of responses every time I interacted with the plots).

(Yes, the example was exactly the one you posted.)

The weird no-plot behaviour in VS Code persists, but I am happy to close this issue and open another for it if you prefer (or deem it worthy of one).

[jonmmease]

That's great to hear! Yeah, I'll close this and you can open another issue on the VSCode issue. Thanks