SHAP Violin and Heatmap Plots | Interpretations and New Insights

The latest version of the SHAP Python package has gifted us with new plots - the violin and heatmap. We explore how to interpret these plots and what new insights into our machine-learning models they bring. Join me as we discuss this new addition to the explainable AI (XAI) ecosystem.
🚀 Free Course 🚀
*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course)
SHAP course: adataodyssey.com/courses/shap...
XAI course: adataodyssey.com/courses/xai-...
Newsletter signup: mailchi.mp/40909011987b/signup
🚀 Companion Article (no-paywall link): 🚀
towardsdatascience.com/new-sh...
🚀 Previous tutorial and other useful articles: 🚀
Intro to SHAP: towardsdatascience.com/introd...
Maths behind Shapley Values: towardsdatascience.com/from-s...
Limitations of SHAP: towardsdatascience.com/the-li...
🚀 Get in touch 🚀
Medium: / conorosullyds
Twitter: / conorosullyds
Mastodon: sigmoid.social/@conorosully
Website: adataodyssey.com/

[@adataodyssey]

*NOTE*: You will now get the XAI course for free if you sign up (not the SHAP course)
SHAP course: adataodyssey.com/courses/shap-with-python/
XAI course: adataodyssey.com/courses/xai-with-python/
Newsletter signup: mailchi.mp/40909011987b/signup

[@alexanderberns1234]

Hi Connor, thank you so much for the videos, absolutelly love SHAP. Helped me a lot when comparing why my deployed models were performing differently with production data compared to the training phase. It just sometimes bugs me that the scaling and representation, slightly changes when switching between Algorithms (RandomForest, XGBoost, LightGBM, CatBoost), which makes it difficult to directly compare the SHAP Analysis. Still I'm barelly scratching the surface, but hope to dive a little deeper with the next project. The Videos were a huge help. Might look into your course soon. Regards from Rhineland Palatinate

[@adataodyssey]

Thanks Alexander! You are right, there are some subtle differences between the modelling packages. I plan to do tutorials that focus on each of these. So, I may do another that compares the differences.

[@ms.mousoomibora9526]

Highly helped by your video lectures..SHAP, LIME these tools are model agnostic. Can we use the same in our own model that may contain different deep learning architectures like CNN-ANN or CNN-LSTM etc...

[@adataodyssey]

Hi Mousoomi! Good question. These methods are all model agnostic in theory... but necessarily in practice. They may not be implemented for these deep learning models. I have had trouble getting SHAP to work for computer vision problems with deep learning models.

[@rajeshkalakoti2434]

can you give an example of how to plot heatmaps for a PyTorch model?

[@adataodyssey]

I will keep this in mind. I am planning to do a few tutorials using different packages --- Scikit learn, catboost, pytorch etc...