Updated method for ternary contour figure factory

[emmanuelle]

Following the suggestions in #1413, and help from @empet, here is an alternative version for implementing contour plots in ternary diagrams.

Pros: this method uses a ScatterTernary trace, meaning that it is natively an object of a ternary plot, hence it uses the ternary pan/zoom.
Cons: the trace is not a Contour trace, hence quite a lot of boilerplate code will be needed for handling colorscale, colorbar, etc.

I post here a first proof of concept, the PR is far from being finished (TODO: fix tests, fix documentation, implement annotations better etc.).

@empet: I haven't had the time yet to compare this method with yours using ILR transform, but I'll surely do!

[emmanuelle]

[jackparmer]

Looks very promising!! Would definitely prefer this approach using the ternary axis system.

[jonmmease]

Thanks for giving this a go @emmanuelle! This looks really nice.

Here's an example of laying the grid of points over the contours (grabbed this example from one of the tests)

import numpy as np
from plotly.graph_objs import graph_objs as go
import plotly.figure_factory as ff
a, b = np.mgrid[0:1:20j, 0:1:20j]
mask = a + b < 1.
a = a[mask].ravel()
b = b[mask].ravel()
c = 1 - a - b
z = a * b * c
fig = go.FigureWidget(ff.create_ternarycontour(np.stack((a, b, c)), z, colorscale='Greens'))
fig

fig.add_scatterternary(a=a, b=b, c=c, mode='markers', marker={'color': z});
fig

This works great! Although it looks like the contours may be a little off center.

Here's what I'm thinking we would need for an initial release

1. Look into the centering issue I noted above.
2. Hide per-contour legend by setting showlegend = False for each contour trace.
3. Set line width to 1 and color to gray, then set the contour color using trace.fillcolor property. This will resemble the appearance of existing contour trace more closely (https://plot.ly/python/contour-plots/)
4. Add one extra dummy scatterternary trace to configure and display the colorscale. Something like

dict({
    'type': 'scatterternary',
    'a': [None],
    'b': [None],
    'c': [None],
    'marker': {'cmin': 0, 'cmax': 2, 'colorscale': 'Viridis', 'showscale': True},
    'mode': 'markers',
    'uid': 'ef0944fe-b4a8-4b82-b378-9e4a35228d2a'
})

If it makes things simpler to deal with for v1, we can just start with supporting only the named colorscales. We can add support for custom colorscales later on.

How does that sound?

Regarding the contour calculation approach, it looks like ggterm supports two methods (See "Use of Orthonormal Basis" in http://www.ggtern.com/2016/01/17/ggtern-2-0-now-available/). So it make sense to me to eventually support multiple here as well. But we could ship the first revision with whichever reasonable approach we're closest to having done.

[emmanuelle]

@jonmmease @jackparmer thanks for the comments and suggestions, I'll be working on it. What do you think about a scikit-image dependency? At the moment, plotly.py has very few dependencies and I don't know the policy about adding new dependencies. In the solution of @empet, which I still have to look into, it would be a matplotlib dependency.

[emmanuelle]

For the question of whether we also want this figure factory to support computing the kernel density directly from a collection of input points in Barycentric coordinates, we could keep the same API with z=None meaning that contours are computed for the kernel density. Or should we have a different function name?

[empet]

@emmanuelle, @jonmmease The orthonormal basis in ggtern means using the ilr-transformation. From references I read it follows that the ilr-transformation and its inverse are better than the transformation from barycentric to cartesian and back, because ilr preserves the distances, and angles. That's why the statistics of compositional data is now based on ilr-transformation. The barycentric-to-cartesian- and back to barycentric was the method used in ggtern in its first versions. It is still there as a second option. All plots on the ggtern site, for the last versions are performed with ilr-transformation.

[jonmmease]

Hi @emmanuelle ,

What do you think about a scikit-image dependency?

In terms of dependencies, we've tried to keep the core of plotly.py needing as few dependencies as possible, but the figure factories have lots of optional dependencies and there's no problem with adding scikit-image / matplotlib as additional ones. See plotly/figure_factory/_county_choropleth.py for an example of handling the optional dependencies on shapely, shapefile, and geopandas. The key parts are the use of optional_imports.get_module function, and the raising of a helpful error message if someone calls the figure factory without the required dependencies.

For the question of whether we also want this figure factory to support computing the kernel density directly from a collection of input points in Barycentric coordinates, we could keep the same API with z=None meaning that contours are computed for the kernel density. Or should we have a different function name?

I'm leaning towards a second function name because I can imagine wanting to add some additional arguments that are only relevant in the KDE case. For example weights and bandwidth. Maybe create_ternary_density_contour, and let's add another underscore here and name this version create_ternary_contour.

@empet, thanks for all of the background research on this. Yeah, it does look like the ilr transform approach is the right way to do this in the long run.

[jonmmease]

Here's what I'm thinking. Since we probably need a few more days on this, I'll go ahead and merge #1413 and call it a preview of ternary contour support in the release notes.

Then we keep working on this PR for the next release.

[emmanuelle]

Still WIP, with some additions:

• ILR transform based on @empet 's implementation
• colorscale
• possibility to add markers superimposed on contours with a keyword argument

Remains to be done:

• set same min and max values for markers and contours for nicer display, since they share the same colorscale
• see if the filling of contours can be done in a better way. At the moment fill is set to 'toself', and the area between the ternary triangle and the outer contour is not filled. Also, if contours enclose local maxima it's fine but local minima are filled with the color of larger contours which are filled later on (I checked that using 'tonext' did not improve it). I first have to understand exactly how the filling of lines works.
• understand why are small oscillations of contours close to the triangle boundary for ilr transform

Stylistic question: what is the policy of plotly regarding names of keyword arguments, whether to use underscores or not? (eg show_markers or showmarkers) I have the impression that underscores are not used frequently.

And a maths question for @empet: in ILR mode, is it ok when no data are available for the poles of the triangle? I had the impression that the function behaved better when the poles of the triangle belong to the set of data points.

I'll post a notebook later on to show some examples.

[emmanuelle]

The "deep" colormap used by @empet to visualize filled contours looks really good (https://nbviewer.jupyter.org/github/empet/Ternary-contour-plot/blob/master/Plotly-ternary-contour-plot.ipynb), is it ok if I add it to Plotly colorscales in colors.py ?

[empet]

@emmanuelle

The ilr transformation maps the open triangle (i.e. without its boundaries) to the 2d-space. It is an isomorphism. The closed simplex cannot be mapped continuously on all 2d-space, by a one-to-one map. The ilr trasnformation was introduced because the classical statistics works
in the euclidean space, referenced to an orthonormal system. Ternary data live in a triangle, i.e. in a reference system with non-orthogonal axes.

As I stressed before, the ilr preserves distances and angles and so the statistical info associated to the ilr-transformed data is mapped back to the simplex, via the inverse ilr-transformation, without deformation.
When working with data for contour plots the points on the simplex sides are slightly perturbed, because otherwise the ternary coordinates that contain at least one zero coordinate
cannot be transformed: either log(a/b) or log(ab/c^2) is undefined.

On the other hand, if we have a pdf concentrated on the simplex, the probability to select/generate data on the simplex sides is zero, i.e. the integral of that density on the sides is 0.

All drawbacks you outlined above, when working with ilr-transformation, manifest only because you don't define a real plotly contour.

The pl_deep colorscale is the Plotly version of the the cmocean.cm.deep https://matplotlib.org/cmocean/.

[emmanuelle]

Making some good progress! The code is very ugly and there are still some bugs, but now we can have the whole triangle colored!

[jonmmease]

Wow, this is awesome @emmanuelle! Thanks for sticking with it 🙂 Just let me know when you're ready for me to take another look.

[jackparmer]

Looks amazing!!

…

On Wed, Feb 6, 2019 at 6:37 PM Jon Mease ***@***.***> wrote: Wow, this is awesome @emmanuelle <https://github.com/emmanuelle>! Thanks for sticking with it 🙂 Just let me know when you're ready for me to take another look. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1418 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABx4ashn0VCf1PxFpQOtoaOZXCIsrbqhks5vK2cugaJpZM4adQf4> .

[emmanuelle]

OK, so I think I made it work finally. It's been quite hard to find a way to handle boundaries correctly (setting values outside the valid range of ternary values before computing contours and setting correctly a background color), but now I'm happy with all the examples I tried. The notebook which I have been using for testing is this one https://plot.ly/~emmanuelle/54

@jonmmease ready for a review!

[jackparmer]

The plots aren't display for me on plot.ly - looks like an issue on plotly's end. Maybe upload to nbviewer if possible? I'd love to share this with some folks.

[emmanuelle]

Hi @jackparmer it's here https://nbviewer.jupyter.org/gist/emmanuelle/28d0e1fe7a662d3a413e3be3626847fd

[jackparmer]

Thank you so much - looks really, really great.

The only major issue I saw is the hover. One hack around this is to turn off hover for all traces, then add a transparent trace (opacity = 0) of scatter points where hover is turned on.

[emmanuelle]

Yes, I did not understand why the hover is different when the trace is displayed as lines or as filled contours...

[jonmmease]

These examples look awesome @emmanuelle! I'll take another look over the code soon, and we can get this in for 3.7.0.

[jonmmease]

Finally had a chance to dig in here, and I'm really excited to have this in plotly.py! Here are some notes and suggestions.

1) plot_bgcolor

the plot_bgcolor option seems to have no effect. I'd be in favor of dropping it for uniformity with other figure factories, and so that it doesn't override the any plot_bgcolor specified in a theme.

2) font family

Similarly, I'd prefer to not change the default font family in the figure factory as is done in _ternary_layout.

3) hover / tooltips

As @jackparmer noted, the hover behavior is a bit odd on the contours. I think I would prefer to disable hover on the contour scatter traces and only display hover info on the markers. To handle the showmarkers=False case, we could add the same markers trace as when showmarkers=True but set the trace opacity property to zero. I think this hover behavior would be consistent with the plotly.js contour trace.

Rather than construct the hover text using strings, could you try using the new plotly.js hovertemplate property? The idea is to leave the scatterternary.text property empty, and then set the scatterternary.hovertemplate property to something like:

        hovertemplate = ("a: %{a:.3f}<br>"
                         "b: %{b:.3f}<br>"
                         "c: %{c:.3f}<br>"
                         "z: %{marker.color:.3f}")

And it would also be nice to use the pole_labels instead of a/b/c if they are specified.

Hovertemplate docstring

Template string used for rendering the information that appear
on hover box. Note that this will override `hoverinfo`.
Variables are inserted using %{variable}, for example "y:
%{y}". Numbers are formatted using d3-format's syntax
%{variable:d3-format}, for example "Price: %{y:$.2f}". See http
s://github.com/d3/d3-format/blob/master/README.md#locale_format
for details on the formatting syntax. The variables available
in `hovertemplate` are the ones emitted as event data described
at this link https://plot.ly/javascript/plotlyjs-events/#event-
data. Additionally, every attributes that can be specified per-
point (the ones that are `arrayOk: true`) are available.
Anything contained in tag `<extra>` is displayed in the
secondary box, for example "<extra>{fullData.name}</extra>".

We might need to set the name property of the scatterternary trace to '' in order to hide it from the tooltip.

4) marker outline

When we have both filled contours and markers shown, I think it would be nice to add a thin outline to the markers to keep them from blending in to the contours. Here's marker.line.width=1 and marker.line.color='rgb(120, 120, 120)'

How does that look to you?

5) Discrete colorscale

When we are showing markers I think the current continuous colorscale makes sense since the markers are colored continuously. But when we are plotting only the contours (filled or not), I think it would be clearer if the colorscale showed descrete bands the way the plotly.js contour traced does (See https://plot.ly/python/contour-plots/#basic-contour-plot). This is helpful for understanding whether the contour lines are colored by the min or max of the contour region.

In case you haven't come across it before, you can create a colorscale with discrete bands by repeating a color scale value in the definition. like:

colorscale=[[0, 'red'], [0.5, 'red'], [0.5, 'blue'], [1.0, 'blue]]

Thanks again for working on this!

[emmanuelle]

Thank you @jonmmease for the detailed review. I'm quite busy at the moment with other projects (webinar coming next Wednesday), can this wait a few days or is 3.7.0 due soon?

[jonmmease]

Thanks for the update @emmanuelle, no problem. plotly.js 1.45.0 came out yesterday so I'd like wait and see if there will be a 1.45.1 soon, and release 3.7.0 in a week or so. Does that sound workable for you?

[emmanuelle]

yes!

[emmanuelle]

@jonmmease there you go! I addressed all points, hope it's OK!

[jonmmease]

Looks great! Thanks for making all of these updates @emmanuelle. I made a few more small updates and I'm really happy with how everything turned out. I'll merge this in when the tests are green.

[jonmmease]

💃 🎉
cc @jackparmer

[emmanuelle]

Awesome! Thanks a lot @jonmmease I learnt a lot of things while working on this PR!