#85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]

Dr. Petar Veličković is a Staff Research Scientist at DeepMind, he has firmly established himself as one of the most significant up and coming researchers in the deep learning space. He invented Graph Attention Networks in 2017 and has been a leading light in the field ever since pioneering research in Graph Neural Networks, Geometric Deep Learning and also Neural Algorithmic reasoning. If you haven’t already, you should check out our video on the Geometric Deep learning blueprint, featuring Petar. I caught up with him last week at NeurIPS. In this show, from NeurIPS 2022 we discussed his recent work on category theory and graph neural networks.
petar-v.com/
/ petarv_93
TOC:
Categories (Cats for AI) [00:00:00]
Reasoning [00:14:44]
Extrapolation [00:19:09]
Ishan Misra Skit [00:27:50]
Graphs (Expander Graph Propagation) [00:29:18]
Pod: anchor.fm/machinelearningstre...
MLST Discord: / discord
Support us! / mlst
Host: Dr. Tim Scarfe
References
MLST#60 Geometric Deep Learning Blueprint (Special Edition)
• GEOMETRIC DEEP LEARNIN...
Categories for AI
cats.for.ai/
Organised by:
Andrew Dudzik - DeepMind
Bruno Gavranović - University of Strathclyde
João Guilherme Araújo - Cohere / Universidade de São Paulo
Petar Veličković - DeepMind / University of Cambridge
Pim de Haan - University of Amsterdam / Qualcomm AI Research
[Petar Veličković] Graph Attention Networks
arxiv.org/abs/1710.10903
Learning to Configure Computer Networks with Neural Algorithmic Reasoning [NeurIPS 2022] [Luca Beurer-Kellner, Martin Vechev, Laurent Vanbever, Petar Veličković]
openreview.net/forum?id=AiY6X...
Graph Neural Networks are Dynamic Programmers [Andrew Joseph Dudzik, Petar Veličković]
arxiv.org/abs/2203.15544
Expander Graph Propagation [Andreea Deac, Marc Lackenby, Petar Veličković]
openreview.net/forum?id=IKevT...
[Pim de Haan, Taco Cohen, Max Welling] Natural Graph Networks
papers.nips.cc/paper/2020/fil...
[Uri Alon, Eran Yahav] On the Bottleneck of Graph Neural Networks and its Practical Implications (they discovered oversquashing)
arxiv.org/abs/2006.05205
[Topping,...,Bronstein] Understanding over-squashing and bottlenecks on graphs via curvature
arxiv.org/abs/2111.14522
[Andreea Deac, Petar Velickovic, Ognjen Milinkovic et al] XLVIN: eXecuted Latent Value Iteration Nets
arxiv.org/abs/2010.13146
[Petar Veličković et al] Reasoning-Modulated Representations (RMR)
openreview.net/forum?id=cggph...
Dual Algorithmic Reasoning [PetarV, under review ICLR]
openreview.net/pdf?id=hhvkdRd...
[Petar Veličković, Charles Blundell] Neural Algorithmic Reasoning
arxiv.org/abs/2105.02761
[Andreea Deac, Petar Veličković, ...] Neural Algorithmic Reasoners are Implicit Planners (which got a NeurIPS spotlight in 2021!)
arxiv.org/abs/2110.05442
A Generalist Neural Algorithmic Learner
arxiv.org/abs/2209.11142
ETA Prediction with Graph Neural Networks in Google Maps
arxiv.org/abs/2108.11482
[Randall Balestriero] A Spline Theory of Deep Networks
proceedings.mlr.press/v80/bal...
[Ahmed Imtiaz Humayun ] Exact Visualization of Deep Neural Network Geometry and Decision Boundary
arxiv.org/pdf/2009.11848.pdf
[Ahmed Imtiaz Humayun ] MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining | ICLR 2022
• MaGNET: Uniform Sampli...
[Randall Balestriero, Jerome Pesenti, Yann LeCun] Learning in High Dimension Always Amounts to Extrapolation
arxiv.org/abs/2110.09485
[Hattie Zhou] Teaching Algorithmic Reasoning via In-context Learning
arxiv.org/abs/2211.09066
[Ahmed Imtiaz] Exact Visualization of Deep Neural Network Geometry and Decision Boundary
openreview.net/pdf?id=VSLbmso...
[Beatrice Bevilacqua] Size-Invariant Graph Representations for Graph Classification Extrapolations
arxiv.org/pdf/2103.05045.pdf
[Brendan Fong David I. Spivak] Seven Sketches in Compositionality: An Invitation to Applied Category Theory
math.mit.edu/~dspivak/teachin...
Tim’s examples of applied Category theory cited:
[Lennox] Robert Rosen and Relational System Theory: An Overview
academicworks.cuny.edu/cgi/vi...
[Bob Coecke] Introducing categories to the practicing physicist
www.cs.ox.ac.uk/bob.coecke/Ca...
[Bob Coecke] Categorical Quantum Mechanics I: Causal Quantum Processes
www.researchgate.net/publicat...
[Bob Coecke] Quantum Natural Language Processing
www.cs.ox.ac.uk/people/bob.co...

[@phamtheanhphu1139]

Really admire for your contributions on GNN❤

[@ZachDoty0]

This interview was fabulous! Thank you!

[@NoNTr1v1aL]

Absolutely brilliant video!

[@DavenH]

This man does not need any time to think!

[@Godfather-qr6ej]

he has explained it to so many people. so he has become really good at explaining what he is doing.

[@JTMoustache]

This was gold.

[@bluewhale37]

So much great ideas.I want new sorting algorithm.Amazing!

[@asdf_600]

Let me tell you, i love this Guy but he just scooped m’y master thesis topic i was working on yesterday 😢

[@miscibi]

Thank you for making these videos ❤️

[@kubricksghost6058]

This video is definitely fast forwarded

[@olegt3978]

Petar is amazing. How can ai reasoning help to solve complex social problems like: degrowth society, thesimplerway, lonelyness, poverty, self sufficient society, depressions, family breakup?

[@johntanchongmin]

Interesting idea to use an expander graph to alleviate information bottleneck. Would the expander graph cause an exponential increase in the number of nodes required though?

[@shiva_kondapalli]

What's your take on Algebraic geometry and categories for Deep learning, especially since the fact that categories were developed quite a bit for AG.

[@mikenashtech]

👏👏

[@vivekpetrolhead]

Ishan Mishra.... This was a crossover episode

[@pengcheng8299]

I absolutely endorse cats for AI :)

[@rjScubaSki]

Last 5 mins is like doing a balloon

[@rjScubaSki]

Rest is good though!

[@MachineLearningStreetTalk]

Lol! Not sure what happened there, we snipped it out. It was a Mosem Dabhi asking Petar some more questions, we interviewed him too, so will add it back in on that one.