Implement new SolaraViz API

[Corvince]

This is the start of a new API for the Solara frontend. See #2236 for more background information. I will eventually update this intro to accommodate more information, but I wanted to start this draft PR now and not wait longer just because I am too lazy right now to write more sentences.

So far everything worked out nicely. The interesting changes are in the myviz.py file. This is where the new entrypoint can be found. The other components ModelController and UserParams only received very minor changes and were just moved to other files to not interefere with the current SolaraViz (during development).

MyViz also includes a minimal example, if someone wants to try this out just run solara run myviz.py

[github-actions]

Performance benchmarks:

Model	Size	Init time [95% CI]	Run time [95% CI]
BoltzmannWealth	small	🔵 -1.7% [-4.0%, +0.4%]	🔵 -0.9% [-1.0%, -0.7%]
BoltzmannWealth	large	🔵 -0.7% [-34.8%, +41.1%]	🔵 -0.9% [-3.7%, +2.1%]
Schelling	small	🔵 -0.8% [-1.1%, -0.5%]	🔵 -1.5% [-1.8%, -1.2%]
Schelling	large	🔵 -1.7% [-3.1%, -0.3%]	🔵 -4.2% [-9.5%, +1.0%]
WolfSheep	small	🔵 +2.3% [+0.7%, +3.9%]	🔵 +2.0% [+1.6%, +2.4%]
WolfSheep	large	🔵 +0.0% [-1.9%, +2.1%]	🔵 +1.6% [-1.7%, +5.8%]
BoidFlockers	small	🔵 -1.2% [-2.1%, -0.5%]	🔵 +0.4% [-0.2%, +1.0%]
BoidFlockers	large	🔵 -1.1% [-1.6%, -0.5%]	🔵 -0.1% [-0.6%, +0.4%]

[EwoutH]

Exciting!

I tried both

mesa>solara run mesa/visualization/myviz.py

and

mesa\mesa\visualization>solara run myviz.py

But for some reason I can't get the imports working. Will take a better look tomorrow.

[rht]

I tried it and it worked. Will add inline comments.

[rht]

New API looks more modular. This means user can add any existing Solara components from Solara library, including Plotly if they want to.

I think it would be convenient to have a string "default" that automatically expands into [ShowSteps, make_space_altair()]. Not sure about num agents being shown by default, but this is a minor point.

[Corvince]

Yes I think thats a good idea. Num agents are currently just shown for testing purpose - I don't think it should be default.

[wang-boyu]

Thanks for your work on this!

Am I right to say that, for Mesa-Geo, we would only need to implement a make_geospace_leaflet() method and a GeoSpaceLeaflet solara component?

[EwoutH]

@rht from where did you run it? I would also like to try it.

[rht]

Is this global variable a temporary artifact? Because we want to support showing multiple SolaraViz in one go: https://simrisk.pulcloud.io/.

[Corvince]

I know that has been an issue in older versionis of solara, but solara might handle this gracefully now. The docs now include this sentence

Reactive objects in Solara are also context-aware, meaning that they can maintain separate values for each browser tab or user session. This enables each user to have their own independent state, allowing them to interact with the web application without affecting the state of other users.
https://solara.dev/documentation/api/utilities/reactive

But I haven't tested his yet.

[rht]

What about the case when there are multiple page objects in a single Jupyter notebook? As can be seen in the visualization tutorial. They seem to share the same context.

[Corvince]

That shouldn't be a big problem luckily. It only forces a rerender, no data change. So at worst you have a few unnecessary rerenders in your notebook. Let's wait and see if this causes serious performance issues and if yes reconsider. But for now it should be good

[rht]

I also notice that all the components take model as an argument, and so, would it be possible to smuggle in the update counter as an attribute of the model?

[Corvince]

Interesting idea. Not sure this will work. I am rather sceptical, but will definitely try it out.

[rht]

I ran it In the visualization directory.

[EwoutH]

Maybe something with working directories? Hate this kind of stuff, and imports in general. So many inexplainable issues.

[EwoutH]

It was a namespace conflict between the installed Mesa package and the GitHub repo. Removing mesa and adding

import sys
import os

sys.path.insert(0, os.path.abspath('C:/Users/Ewout/Documents/GitHub/mesa'))

solved it. Let's take a look!

[EwoutH]

Looks clean, feels fast and responsive!

What's the difference between these two buttons? When running they both seem to pauze the simulation in the same way.

[rht]

The 2 buttons having the same functionality is out of scope for the PR. It is due to ipywidget displaying both and there is no way to turn either off.

[EwoutH]

I really love that you can just input an model instance, and if I'm correct, acces that model instance from the interpreter.

This would solve a huge part of #2176, right? And would this be possible?

[EwoutH]

So you can input a model instance, model class and params? I get either the first or the second two are required?

If you input an instance, will it be updated in place, so you can access it from your interpreter later?

[Corvince]

Actually it expects a reactive version of a model, so that the model from the outside still receives the recreated versions.

I now removed the model_cls, as it can be resolved from the model instance.

So now it receives an actual model (as a reactive variable) and user params. So I think ModelCreator isn't a good name

[Corvince]

I have updated the code so it now updates solara_viz.py instead of creating a new myviz. Also moved back the modified ModelController.
The "example" still lives in myviz, so you can still run it.

All in all the diff should be much more reflective of the actual code changes.

Things that are still missing (please add if you notice something):

• Layout stuff (draggable grid layout)
• reactive seed
• Easy way to plot simple measures just by their attribute name

[Corvince]

I really love that you can just input an model instance, and if I'm correct, acces that model instance from the interpreter.

This would solve a huge part of #2176, right? And would this be possible?

Yes, exactly! Although for it to fully work, even if you reset the model or change parameters, you should pass in the model as a reactive model, i.e. call solara.reactive(model) on it. Inside the SolaraViz it also attaches a call to force_update() after each step, which triggers a viz update.

[EwoutH]

If you need any input let me know!

[Corvince]

Now that you say it, what's your opinion on the layout? I thought at first to implement how it's currently done. But maybe we can be more unopinionated. In my implementation all components are just displayed with no surrounding layout. But the components themselves can of course include any layout. We could keep it that way and provide some layout components. So users could just pass solaraviz(model, components=GridDraggable(componentA, componentB) or something like that.

Also, regarding naming. Is Components a good enough name?

[EwoutH]

I think the current layout is quite good. I like controls on the left, visualisation and output in the center/right.

• Maybe it would be nice if you could specify is you want things below or right of previous components, but maybe keeping it adaptive is more flexible.
• It might be nice to be able to specify an aspect ratio per component. Some graphs you might want wide, some tall etc.
  • For the space_drawer this might be done automatically (by default) based on the grid width and height.

Also, regarding naming. Is Components a good enough name?

Many names could work here, but Components is quite good.

[rht]

I actually find the naming to be quite restrictive in order to encompass other Solara components, because it needs to be general, e.g. if you want to use the DataFrame/PivotTable component in https://solara.dev/documentation/components. "Measures" doesn't cover this. Meanwhile, "components" is what is already used by Solara.

[quaquel]

This is just a clarifying question from my end. In the various conversations on data collection, we converged on building this on top of some pub-sub or signal-based API. How might that choice affect this effort, or vice versa, what would be convenient to use inside MESA such that both data collection and model visualization can utilize it?

In #2176, @maartenbreddels suggested using either Solara reactive variables or psygnal. I have looked at the latter, and it is very elegant and does a lot of what we want for data collection. Importantly, it also allows for subscribing to updates in collections. However, it would add an additional dependency to MESA. I also started exploring Solara's reactive variables. It appears this covers a subset of what we would need for data collection, but I am not sure until I actually play around with it more.

In my vision, both the visualization side and the data collection side of MESA use the same mechanism for being informed/getting state information from a MESA model.

[maartenbreddels]

Hi Jan,

I also started exploring Solara's reactive variables. It appears this covers a subset of what we would need for data collection

Let me know what you think is missing (maybe after playing around).

Regards,

Maarten

[Corvince]

This is just a clarifying question from my end. In the various conversations on data collection, we converged on building this on top of some pub-sub or signal-based API. How might that choice affect this effort, or vice versa, what would be convenient to use inside MESA such that both data collection and model visualization can utilize it?

In #2176, @maartenbreddels suggested using either Solara reactive variables or psygnal. I have looked at the latter, and it is very elegant and does a lot of what we want for data collection. Importantly, it also allows for subscribing to updates in collections. However, it would add an additional dependency to MESA. I also started exploring Solara's reactive variables. It appears this covers a subset of what we would need for data collection, but I am not sure until I actually play around with it more.

In my vision, both the visualization side and the data collection side of MESA use the same mechanism for being informed/getting state information from a MESA model.

The top level view on how it currently works (in this PR) is that each component registers a "update_counter" as a render dependency. Whenever this counter increases the component rerenders. Currently the counter is increased after each mode..step.

Of course it would be much more elegant to auto-update whenever an attribute changes, for example not through a call to step, but maybe through some form of user interactions. If the variable would be a reactive variable this would already happen automatically.

So from a visualization point of view the use of solaras reactive values would be very charming. So I would appreciate it if you further investigate solaras reactive suitability.

But overall this PR is mostly neutral to how we handle this - it mostly deals with the surroundings and establishing a cleaner API. The use of signals/reactive values/pub-sub will mostly play some role for individual components (e.g. a plot of a specifc variable)

[github-actions]

Performance benchmarks:

Model	Size	Init time [95% CI]	Run time [95% CI]
BoltzmannWealth	small	🔵 +1.1% [-0.0%, +2.3%]	🔵 -0.1% [-0.2%, +0.0%]
BoltzmannWealth	large	🔵 -1.0% [-29.8%, +32.3%]	🔵 +0.1% [-2.3%, +2.2%]
Schelling	small	🔵 +0.9% [+0.7%, +1.2%]	🔵 -0.5% [-0.7%, -0.3%]
Schelling	large	🔵 +1.1% [+0.4%, +2.0%]	🔵 +3.7% [+1.3%, +6.7%]
WolfSheep	small	🔵 +3.5% [+1.8%, +5.1%]	🔵 +0.1% [-0.1%, +0.5%]
WolfSheep	large	🔵 +3.3% [+1.7%, +4.7%]	🔵 +3.1% [+1.2%, +5.3%]
BoidFlockers	small	🔵 +0.6% [-0.4%, +1.7%]	🔵 -0.3% [-1.1%, +0.6%]
BoidFlockers	large	🔵 +0.6% [-0.3%, +1.7%]	🔵 -0.5% [-1.3%, +0.4%]

[Corvince]

I just marked this as ready for review! Even though the tests are still failing, this should probably also be fixed as part of this PR?

Otherwise I think this is good to review as a first PR just for the API changes.

Next todos are:

• Update example models
• Write migration guide from current API to this PR. You should be able to have the same viz as before with small adjustemnts, but those adjustments need to be written out.
• Bring back GridDraggable layout. This was used before, but I found the code a bit confusing so I didn't manage to adapt just yet. Per default the layout now is unopinionated, but you can bring back the old style with SolaraViz(model, components=[GridDraggable(...components)]. But the GridDraggable component needs to be written in a further PR
• Document how to author custom components (esp. the use of update_counter)
• Test if we can pass update_counter as part of the model, so we can use a local reactive value instead of a global one
• Decide which subcomponents will be part of the public API for Mesa v3.

Not so immediate, but also a nice todo: Rewrite ModelController. There is already code for a much simpler threaded version and from a first test this finally seems to work. Maybe it doesn't even need to be threaded and just works. In any case it can be vastly simplified (and no more unnecessary pause and stop button)

But first lets get this PR through!

[EwoutH]

@Corvince looks awesome! Any specific things you would like me to test or review?

[Corvince]

Yes! You can try to use it 🙈

Heres a quick migration guide. Where previous you could have something like this for a minimal frontend

SolaraViz(model_cls, model_params, agent_portrayal=agent_portrayal)

you would now do

SolaraViz(model, components=[make_space_altair(agent_portrayal)])

you can test if that works for make_space_altair and make_space_matplotlib and then additionally add model_params to SolaraViz. And then test if everything works (start, stop, reset, reseed, change parameters).

That would be awesome!

[EwoutH]

Awesome, I'm going to

pip install -U -e git+https://github.com/projectmesa/mesa@solaraviz-api#egg=mesa

and test some examples.

I just talked to Jan, he is going to try to dust of hist pub-sub efforts to see if that will integrate nicely with this.

[EwoutH]

Playing a bit around! Got boltzmann_wealth_model_experimental working:

• Easy way to plot simple measures just by their attribute name

Yeah this would be really useful. We could think of another/more robust API to parse measures other than just a string name (@quaquel?), but I think this is important to get right.

I like that's way more flexible: You don't need to input model_params for values you don't need as sliders and such. You can just use the default values or override them.

Another this that's interesting is that the slider overrides default/other values. I think that's good to document.

It does show that something like

• A system for managing Model/Agent default values and ranges #2268

can be useful.

[EwoutH]

I also tried virus_on_network. However, it has a custom make_plot function, that seems difficult to adapt, and I didn't get it to work.

def make_plot(model):
    # This is for the case when we want to plot multiple measures in 1 figure.
    fig = Figure()
    ax = fig.subplots()
    measures = ["Infected", "Susceptible", "Resistant"]
    colors = ["tab:red", "tab:green", "tab:gray"]
    for i, m in enumerate(measures):
        color = colors[i]
        df = model.datacollector.get_model_vars_dataframe()
        ax.plot(df.loc[:, m], label=m, color=color)
    fig.legend()
    # Set integer x axis
    ax.xaxis.set_major_locator(MaxNLocator(integer=True))
    solara.FigureMatplotlib(fig)

page = SolaraViz(
    VirusOnNetwork,
    model_params,
    measures=[
        make_plot,
        make_text(get_resistant_susceptible_ratio),
    ],
    name="Virus Model",
    agent_portrayal=agent_portrayal,
)
page  # noqa

[Corvince]

I rebased this PR. Hopefully tonight or tomorrow at the latest I will update the tests, so ci will be happy again

[EwoutH]

Leftover from #2278, but could you fix this import?

mesa/tests/test_solara_viz.py

Line 8 in cfa2635

from mesa.visualization.solara_viz import Slider, SolaraViz, UserInputs

(I already approved)

[Corvince]

Yes, see my last comment

[EwoutH]

Perfect. Also feel free to remove any tests if those are blocking you.

A new name for the new viz would help a lot. Maybe just MesaViz?

[wang-boyu]

A new name for the new viz would help a lot. Maybe just MesaViz?

How about ModelViz or ModelView?

[Corvince]

I would like to extend the functionality to allow for more model interactions, adding plots interactively and having an "agent observer" for single agents. So not just a pure visualization. After a chat with chatgpt my best idea so far might be "ModelExplorer"

[EwoutH]

ModelExplorer sounds good. Also fresh, and new. You do set the expectations high ;).

I would everything that's strictly in Solara (like Solara-components) to keep a Solara-like name. Then if we add non-Solara stuff, we can give them non-Solara names.

[wang-boyu]

Hi @tpike3, we probably need to rethink the jupyter visualization for mesa-geo, considering the new API design from this PR.

[Corvince]

Can you point me to that? I can take a look tomorrow

[wang-boyu]

Thanks! The current implementation is here: https://github.com/projectmesa/mesa-geo/tree/main/mesa_geo/visualization, from PR projectmesa/mesa-geo#212.

There was some comments from @rht that needs to be addressed: projectmesa/mesa-geo#199 (comment) but we decided to merge and have something working first.

As I mentioned above in #2263 (comment), we probably don't need to copy over anything. Instead, we could try to refactor existing implementations into some solara components for geospace.

Let me know if you need anything else : ) also cc @tpike3

[Corvince]

Thanks for the extensive reply! Sorry, I didn't catch your previous comment, but you are spot on!

It will just require a function make_geospace_leaflet() that returns what I think is the "map" component currently. So something like this

from mesa.visualization.utils import update_counter

def make_geospace_leaflet(agent_portrayal, view, zoom, tiles,scroll_wheel_zoom):

    map_drawer = leaflet_viz.MapModule(
        portrayal_method=agent_portrayal,
        view=view,
        zoom=zoom,
        tiles=tiles,
        scroll_wheel_zoom=scroll_wheel_zoom,
    )

    center_default= ...


    @solara.component
    def map(model):
        update_counter.get() # <- Important line to notify the frontend whenever model updates (e..g step)
        # render map in browser
        zoom_map = solara.reactive(zoom)
        center = solara.reactive(center_default)

        base_map = map_drawer.tiles
        layers = map_drawer.render(model)

        ipyleaflet.Map.element(
            zoom=zoom_map.value,
            center=center.value,
            scroll_wheel_zoom=scroll_wheel_zoom,
            layers=[
                ipyleaflet.TileLayer.element(url=base_map["url"]),
                ipyleaflet.GeoJSON.element(data=layers["agents"][0]),
                *layers["agents"][1],
            ],
        )

    return map

Then in a notebook you should already be able to do

GeoSpaceLeaflet = make_geospace_leaflet(...)

GeoSpaceLeaflet(model)

And you should see your visualization! And then you can of course pass it to SolaraViz

Would you like to give it a shot? Unfortunately I don't have the mesa_geo stack currently installed and last time it was a bit of a hassle to get the geo tools to work under windows. Just ask if you need any help

[wang-boyu]

Thanks! This is precisely what I thought. I will find some time to try it out!

[EwoutH]

I would prefer to have a new name before a potential (alpha) release.

[EwoutH]

I would prefer to have a new name before a potential (alpha) release.

ModelExplorer sounds good. Also fresh, and new.

What do we think of this?

---

On another note, te Visualization tutorial, since it’s still based on the previous version of SolaraViz. I see two options:

1. Use the previous SolaraViz version for now, which we moved to experimental.
2. Update to the new API.

[Corvince]

On another note, te Visualization tutorial, since it’s still based on the previous version of SolaraViz. I see two options:

1. Use the previous SolaraViz version for now, which we moved to experimental.
2. Update to the new API.

I can quickly do a minimal update to the new API and then do a bit of an overhaul later on, if thats fine?

[EwoutH]

Sounds perfect!

[jackiekazil]

I am late, but I like ModelExplorer