Datashader in 15 Minutes | Machine Learning & Data Science Open-source Spotlight #6

Real-time interactive(!) big data visualizations with Datashader + Bokeh 👁️
The main challenge of visualizing huge datasets is NOT computing power or memory! It's actually making meaningful plots which are able to highlight the dense sections in your data together with the outliers.
Datashader pre-renders even the largest datasets into a fixed-size raster image that faithfully represents the data’s distribution. Together with Bokeh, we can build interactive plots that dynamically re-render these images when zooming and panning, allowing you to always get the best data representation out of an area-of-interest.
If you're interested in the platform I was working on for hosting my notebook on the cloud - It's called Saturn Cloud. It supports parallel computing with Dask, and I've found it very convenient to use! (especially compared to trying to set-up a notebook on AWS on your own)
Link: www.saturncloud.io/s/
Linked to Notebook:
nbviewer.jupyter.org/github/d...

[@anantharamaniyer9135]

Watching this again reveals even more ideas! Many thanks for this Dan. Also do you have similar ones using Datashader and hvplot using pandas / polars /dask for plotting line,bar charts etc?

[@NitinPasumarthy]

Thank you Dan for enlightening everyone about these wonderful & very powerful open source projects. Enjoyed watching the problem statement, incrementally complex solution and the final blast - super analysis on 11M in a browser without any manual sampling!

[@JimmyGelhaar]

Great job on this video, Dan! Datashader is pretty sweet!

[@alhdlakhfdqw]

thank you very much for your great video Dan :)

[@r.m10234]

Thanks Man!

[@motbus3]

really nice video

[@aydoganavcoglu2675]

Many thanks your very sophisticated steps by steps instructions! I would like to ask you how we can reach these kind of data set like more than a millition being able to use datashader?

[@anantharamaniyer9135]

Thanks for this great video explaining Datashader step by step. James Bednar and team from PyViz have really done a wonderful job to keep it Open Source. Do you know or have you tried integrating this with Plotly and Dash? For example, if you have many components of a dashboard made using Plotly Dash, can the map element be a Datashader app, while others are from other libraries..

[@danbochman]

Hey!
Sorry for the delay, for some reason UA-cam doesn't show me notifications on comments on my videos (although marked as ok in settings).
Regarding your question if it's still relevant, there is a project in Plotly Dash to incorporate datashader visualizations in the dashboards, you can take a look here:
github.com/plotly/dash-datashader
However, personally I didn't have much success building such a dashboard with Dash (but that could be my fault).
Usually in cases such as these, I build a dashboard with Tableau, and then add a Python visualization embedded manually in HTML.