Speedy image rendering

[zbarry]

Hi all,

I work mostly on computer vision projects involving quite a lot of video data, whether from webcam or microscopy sources. As you can imagine, these datasets can be quite large and often of high resolution. Right now, I'm using DynamicMaps to render each frame, along with DataShader to handle resolution-reduction when zoomed out on the current image enough so as to be not necessary to have pixel-level detail.

Was wondering the most efficient / speedy way to render the frames such that users can use a slider to go through them smoothly. Currently, it seems like the rendering times of the images, even with DataShader, are just on the cusp of usability. There is some precedent for very smooth navigation of super-detailed images being possible in other frameworks, e.g., http://openseadragon.github.io/. Not sure if this is possible with HoloViews at the moment (or even can be possible if there are design limitations). Would appreciate your thoughts!

Also, wondering if the initialization time improvements in #2646 might also bring frame-rendering time improvements, but these seem to be completely independent.

Thanks!

[jlstevens]

I think we would need to benchmark to figure out how fast we can make large image updates.

One suggestion I made that might have helped at the bokeh level was bokeh/bokeh#6896 to use PNG to reduce the size of the data transferred and maybe speed up rendering (HTML5 canvas is pretty quick with PNG data). That issue was closed but it did sound like such a PR would be accepted if someone implemented such a scheme and showed it to have clear benefits.

[VolkerH]

I'm interested in similar things. One thing I stumbled across was ipyleaflet, which lets you use leaflet.js in jupyter. If you can arrange your images in tiles with different resolutions this should work. However I would also like to be able to change intensity ranges, select Z layers, time points etc.

[jlstevens]

@VolkerH For maps, you can use tile sources in geoviews although it might not support all the features you described. Just mentioning this in case you weren't aware of it...

[zbarry]

Hmm, thanks. I'll give them a ping and see what they think about rendering efficiency, at least. Not sure what the rate-limiting step on framerate is.

Should have mentioned that I've played around with just using straight-up Bokeh for this kind of rendering with fast redraws, and while it seems like there is indeed some performance improvement, it's still not nearly as usable as, for example, that framework I referenced earlier designed specifically for large image viewing. This wasn't exactly a scientifically rigorous test, though, so no guarantees that just Bokeh alone is significantly faster, haha.