-
-
Notifications
You must be signed in to change notification settings - Fork 402
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement cnorm option #4567
Implement cnorm option #4567
Conversation
Summarizing a discussion between @jbednar @philippjfr and myself:
Open issue:
Other than option naming and the issue with the 'fire' colormap, I think this sounds like a good plan! |
Sounds good. Yes, I think there's already an open issue for dealing with |
Looking for alternatives, the only other one I'll throw out there is |
One issue where colormaps is discussed is #2487. Here is the relevant bit: |
Sure, I have no preference between chigher/clower and cabove/cbelow. Yes, #2487 (comment) is the prior issue about colormaps, which remains unaddressed. |
You have the strongest opinions here so would you mind coming up with a proposal for the colormap changes? Then we can veto if we hate it 😆 |
No, I just want to complain! :-) Sigh. I'll try to think of something suitable, but please ping me if I haven't done so in a reasonable time. |
I thought a bit about this issue and here is the best suggestion I have right now:
Don't love any of this but that seems the best approach to me right now... Edit: Pinging @jbednar as requested! |
After discussion with @jbednar, @philippjfr we came to an agreement that The new suggestion is to remove setting a dimension range at all and to be explicit and clear about what we want to express: a special value that indicates missing/no data. This isn't needed for floats which naturally have a no data value (NaN) but is needed for integer data which has no such special value. Zero counts mean no data for the datashader count aggregator but there is plenty of data in the wild that use special 'nodata' values such as -32767 or -9999. Therefore, if you need transparency for a count aggregate after using
I think this is clear, explicit with no more mysterious implicit behavior or abuse of clipping options. |
Sounds good! We'll have to decide what to about integers vs. floats for I can see arguments for either approach. For (1), if it can be achieved, only applying the For (2), I can easily imagine data originally integer showing up in workflows as floats, and given that short integers work fine as floats, it may be more convenient to simply ignore whether it's float or not and accept e.g. I don't think I have much preference, except to not prefer option (3) of not allowing floats while also not selectively applying |
I think there is an option (that makes sense to me) that isn't 1) or 2):
I have no opinion on that but |
My only opinion about option 4 is that it doesn't let people declare the obvious |
That is right - I'm of the opinion that as float have a dedicated NaN value, there is no need for |
@jbednar @philippjfr Regarding the default datashader colormap issue, why not just use the blueish colormap that is already the default for Unless the default for hvplot is also problematic? |
What about some raster with transparency overlaid on a tile source? |
Jean-Luc and I discussed this issue today and came to the conclusion that any changes to the colormap are independent of this PR in its current state. Now that we have decided that rasterize won't generate transparent Images for the default count aggregation, then most of the colormap problems go away, because fire is already a suitable colormap for cases with no transparency. With the default To summarize, with the defaults (nodata=None, cmap=Fire) rasterize should work well (no ambiguity of peak color vs. background showing through, no sharp edge between white background and weak data points). The defaults will not allow overlaying the results on a web tile source or other data, but the user must explicitly enable |
Note that there will still be issues for non-uint aggregations (not Count or Any), where NaN will automatically be transparent and a fire colormap may be inappropriate on a white background. Even so, in that case users will very often want to select their own colormap anyway, e.g. a diverging one if values can be both positive and negative, as they can be for many non-Count aggregations. So I'm not particularly concerned about what may need to be adjusted once people are using a non-uint aggregation; once they mess with agg then they will also often need to set cmap appropriately. |
One thing to point out about Ideally, the hover would show the original value (i.e 0) but Bokeh doesn't support transparency masks afaik. This also means in the float case, the |
I suppose for the integer case, you could smooth over the hover issue with a custom hover tool (i.e formatter) that knows that Deciding to implement this (if we do) is orthogonal to the rest of the PR though even if it is a nice extra to consider. |
You could use a custom hover formatter here. |
That is exactly what I was suggesting in my previous comment. Do you think we should? |
Ah sorry, missed that. It's not a huge amount of effort so seems worth doing. |
50ac0c0
to
6b8e2c2
Compare
@jbednar When you pull you will want to change Edit: With the new cmap, the gif needs updating too: |
Remaining to-do:
Remaining issues not expected to be addressed in this PR, but worth noting for later:
|
Tests are passing. Time to hit merge on this rather large and long running PR! |
Implements
cnorm
option to switch between 'linear', 'log' and 'eqhist'.logz
option