Somehow, during my sleep, it ultimately reached this channel. And in my waking time, listened to this for a while.. and felt like I needed to go back to school. I felt like I had zero knowledge about what is happening on earth 🙈🫣😝😝 my day job is nothing and being a self taught ML dude felt like I am wasting my time doing useless things to put food on the table😅
I love Taco's point about the community having too heavy of a cognitive reliance on benchmark datasets and ground truth labels. I have learned this lesson to my bones recently when I developed a "best ever" model in an industrial setting, but all of the metrics showed it to be WORSE because the model was actually better than the ground truth data, and e.g., true positives were counted as false positives. When your data/labels are biased from the data collection process, you can come up with all sorts of counter-intuitive conclusions when we don't keep those biases in mind during interpretation.
I couldn't agree more, this is why in an industrial setting I always take new SOTA approaches with a grain of salt. It is not to say that the results are not convincing, but there's danger in having benchmarks that have been static for years. Benchmarks have generally been collected on a single temporal (and, at times, spatial) scale using a uniform collection method. What I found is that once you start using data that has been evolving over years and years, it becomes non-trivial to deal with the non-uniformity of the data or biases that have been induced throughout the years. I remember finding this "great" pattern in our data and only later finding out that this was simply due to one of the systems assigning a certain property to data points if the system was running out of memory #rip.
During my study of Statistics, ML and DL I never understood the connection between different NN architectures - they simply pop up without any history or proofs. Shut up, learn and memorize. Therefore my professors in ML do not earn my respects, because they do not real understand what they are teaching. I like thank you for bringing of the fundamental understanding of the zoo of DNN models. I lost the intuition for long time about DNN models. Again thank you for the clarity.
Honestly, this channel has such a great format. It's the perfect mix between a podcast and a documentary. It reminds me a lot of Sixty Symbols or Numberphile but for ML instead of physics and maths
As a lot of this is quite over my head. BUT, I'm using OpenAI's playground to have the concepts I don't understand explained to me. And it works extremely well. Ideas like gage symmetry and such now make sense to me. On a very broad level of course, but still. An AI explaining tome the concepts that went into it's own creation! Truly amazing!
Again, thanks so much to the MLST team and you Tim for bringing this to us! Nothing like it, truly special stuff! Will be very intrigued to hear the next few months of conversations while y’all chew on Hawkins vs graphs. 😆
Great episode! Learned a lot, really appreciate all the work that makes these possible. The videos are incredibly well done! Great guest speakers too of course.
Thanks! I don't really want the audience to pay for the content. Most of them are poor Ph.D students! Also -- even very large channels make nothing on patreon so it's pretty pointless adding it, monetization is off too. Perhaps one day we will get a viable sponsorship offer. Right now I would rather keep the channel pure i.e. almost any amount of sponsorship money would pale into insignificance next to the effort we put into making the content and just seems out of place.
@@MachineLearningStreetTalk I understand what you're saying but I still feel like I owe it to you to tell you that I'd definitely sign up for your Patreon if you ever decided to make one
Why not just privately message the person responsible and make a massive donation- privately - if that’s the way you know to encourage someone’s work. All grateful helpful comments are welcome, This is on you tube. I hope it continues to be be free for me and students forever
excellent episode 👌👌 the geometric dl viewpoint is truly fascinating. More than the obvious repercussions on the ML/DL community, I also hope that it will have an impact on the theoretical statistics community's research interests as well. I'm eager to see what lies ahead 😉
Love this topic, looking forward to hear this. I think this episode is great and land at the right technical level, not as low-level as "In these paper we did these augmentation and it works well", but not too philosophically useless as the "knowledge is universal..." argument.
At 2:10:43 the speaker describes how the methods used to analyze an N based model breaks down at 2N. The Theory of Complex Dynamic Systems includes the concepts “self similarity independent of scale”, fractals to relate scaled features, and non-integer dimensionality. Something worth considering …
I really enjoyed this one the second time around. One thing, the transformer is mostly a pyramidal graph network -- just FYI.It's a fully-connected Feed Forward NN A pyramidal graph is a type of graph that has a hierarchical structure where nodes are organized into layers. The nodes in each layer are connected to all nodes in the layer above it. Pyramidal graphs have been used in various applications such as object detection1, EEG classification2, and spatial significance exploration3
omg!! what a great generalization effort and applauses for the metanalogy with geometry Erlangen program.. how many problems are in some way only really the same problem!! congratulations and thank you.
Thanks for podcasts. It helps me with other stuff i found to be in touch with ML, Ai, and neuroscience. Before this new hype around ML and transformers i didnt know that i will found that i will be in love with neuroscience
Absolutely amazing episode! When you eavesdrop on a topical conversation of such detail you are almost bound to pick up some wisdom and sync in with the speakers at least for some time (prerequisite you have some background)
I can never keep up with these long episodes. It means I miss an awful lot of MLST if the guest or topic title doesn't really jump out at me. Once in a blue moon, the stars align to when the YT notification pops up and I can envisage fitting in an episode over the next couple of days in chunks. The promise of the Netflix-style Part 1 got me intrigued and it's one of those stars-aligned moments today (probably just because it's Sunday though). I think it's time to crack open the notepad and follow along. I can tell it's going to be good. Most of your stuff is just top content for us researchers in the ML domain (whether partially or fully). I really would consider trying to make smaller videos that are less intimidating length-wise, as I can more easily just ignore this entire episode, but would be much more likely to watch a relevantly-titled section I found interesting that was more around the 20-40 min mark, and thereby find myself enjoying more of your content.
That's a fair point. I sometimes have a similar issue where even if the title attracts my attention, I wont come to finish episodes, just because theyre so long. Also given the often challenging content it is hard to just resume where you stopped. I really appreciate MLSTs effort to make it easier to absorb the content. Perhaps, shorter videos that summarize a certain topic using scenes from different epsiodes would help in that aspect. On the other hand, I assume this is a lot of work.
I hear where you all are coming from but I want to voice that the beauty of these conversations is there fullness, and distillation or bite sizing seems a big ask given the discussion is already at the edge of current understanding. I love the long form. If it could also be snackable that’d be overpowered 😅
A thing to keep in mind is you don't want to use neural networks for things that other computer methods do much better at like sorting, computing, mapping, and information lookup. A good example is Apple's Seri.
Hi, i learned that I can use the depth of my brain itself. Perhaps there are people who can use this idea. As far as I know, this hasn't been done. I myself try to realize whilst typing. Which might be too unclear a way of getting inside. This is THE legacy ever. Namely our brains.
If you attack a certain kind of complexity, you need not do that on the same level. I.e. there are cheap ways of working through certain problems. Example: to apply all the necessary facts, after you have collected all the related phenomena and then choose how to work with those first. First the easy way, and when you are done, you apply 'all'. No need to do too expensive computations. To some degree.
While the focus is on approximation error, I think understanding will be limited. The most interesting behaviour is when useful novel outputs occur for new inputs, based on assimilation of abstract patterns.
it would be extremely helpful for phd students who are very busy if you can make a highlight video of this that contains only the most eseential academical discussions done in this video.
If the problem of creating intelligence didn't end up becoming conquered by human ingenuity - then we weren't truly intelligent in the first place. So if we really are supposedely intelligent, we should be able to mechanize it. Thus, machine intelligence is an extension of the fact of human intelligence. If we can't do it then our intelligence should be able to be surpassed given that the speed of human learning can be beat by a machine. Either way, in the final analysis we arent the end all be all of intelligence in the universe. Nature always transcends itself
A good attempt to geometrization of machine learning by extending the Erlang program using group theory to found networks that capture symmetries. But, unfortunately, although valid for artificial CNN this is only valid to simple neuron equations that only capture very superficially the signal processing of real biological neurons.
I still think that coming from an understanding of a thing ,it is much easier to determine the way to think about it. BIOS Which are rules...I'm getting interested in computer science.
The content was great! The editing was a bit disjointed. I love long form videos like this, but a touch more planning may have made for a smoother experience.
Our sincere thanks to these 4 brilliant researchers: Professor Michael Bronstein www.imperial.ac.uk/people/m.bronstein twitter.com/mmbronstein Dr. Petar Veličković twitter.com/PetarV_93 petar-v.com/ Dr. Taco Cohen twitter.com/TacoCohen tacocohen.wordpress.com/ Prof. Joan Bruna twitter.com/joanbruna cims.nyu.edu/~bruna/ References: ICLR 2021 Keynote - "Geometric Deep Learning: The Erlangen Programme of ML" ua-cam.com/video/w6Pw4MOzMuo/v-deo.html (Note we used some clips from this, and the graphics designer was Jakub Kuba Makowski www.linkedin.com/in/jakub-kuba-makowski-19b17143/) Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković] geometricdeeplearning.com/ arxiv.org/abs/2104.13478 Review: Deep Learning on Sets fabianfuchsml.github.io/learningonsets/ arxiv.org/pdf/2107.01959.pdf AMMI Course "Geometric Deep Learning" (12 lectures) ua-cam.com/video/PtA0lg_e5nA/v-deo.html Beyond the Patterns 28 - Petar Veličković - Geometric Deep Learning ua-cam.com/video/9cxhvQK9ALQ/v-deo.html Neural Algorithmic Reasoning [Petar Veličković, Charles Blundell] arxiv.org/abs/2105.02761 Equivariant convolutional networks [Taco Cohen] pure.uva.nl/ws/files/60770359/Thesis.pdf Solving Mixed Integer Programs Using Neural Networks arxiv.org/abs/2012.13349 Project CETI: meaningful communication with another species audaciousproject.org/ideas/2020/project-ceti Discovering Symbolic Models from Deep Learning with Inductive Biases [Cranmer] arxiv.org/pdf/2006.11287.pdf Node2Vec arxiv.org/pdf/1607.00653.pdf Deepwalk arxiv.org/pdf/1403.6652.pdf AMMI Course "Geometric Deep Learning" - Lecture 12 (Applications & Conclusions) - Michael Bronstein ua-cam.com/video/caQV-Vb9TBw/v-deo.html Graph Attentional Networks [PetarV] arxiv.org/pdf/1710.10903.pdf A Generalization of Transformer Networks to Graphs arxiv.org/pdf/2012.09699.pdf The Hardware Lottery [Sara Hooker] arxiv.org/pdf/2009.06489.pdf Developments in fractal geometry [Barnsley] link.springer.com/content/pdf/10.1007/s13373-013-0041-3.pdf Super-Resolution from a Single Image [Fractals] www.wisdom.weizmann.ac.il/~vision/single_image_SR/files/single_image_SR.pdf XLVIN: eXecuted Latent Value Iteration Nets arxiv.org/pdf/2010.13146.pdf Analogy as the Core of Cognition [Hofstadter] worrydream.com/refs/Hofstadter%20-%20Analogy%20as%20the%20Core%20of%20Cognition.pdf The CLRS Algorithmic Reasoning Benchmark github.com/deepmind/clrs WHAT CAN NEURAL NETWORKS REASON ABOUT? [Xu] openreview.net/forum?id=rJxbJeHFPS Graph Representation Learning [Hamilton] www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf
Great content! It may be better to put the references in the description so they don't get pushed too far down in the comment section. I initially thought there were no references and just happened to scroll down the list of comments.
2:19:17 and the next 15 seconds contains all the dreaded words of my undergraduate education in a single sentence. Anyone dare to explain what it means in ELI5 fashion.
1:59:35 Is the growth really EXPONENTIAL in a hyperbolic space ALSO? Or did Bernstein misspeak? I'm no expert here sure, just trying to understand. - EDIT: No! Of course, if the embedding space grows exponentially, this also captures better ..what ever represented.
Jeff Hawkins is asking questions about locations of mobiles here ua-cam.com/video/p_KRsF-ncJQ/v-deo.html that I think could eventually be answered by the future GPT3… Here is my hypothesis: I was thinking of this last week : Brain makes maps of the environment. But I was wondering how it managed mobile objects. How it worked to know where it is. It's location. And then I remembered that when I was a kid, my mom would always ask me this question when I had lost something: When did you use it the last time? The question when. And I would try to remember what happened the last days, all the chronological events with a context linked to the object. And then I would remember where it could be. I'm amazed to consider that finally, the brain has a chronological map of events and their context, remembering every sequence and is able to navigate in this chronological map of events. And every object is there, the brain has all these objects identified and painted in every events, and keep all these pictures (I say picture, but it can be linked to any senses) chronologically and can navigate in this chronological mapping, like if the events were layers it can access. Soooooo if the events are like layers that can be accessed chronologically (or you can make jumps, but knowing where it stands in the chronology), could a future GPT3 do pattern recognitions on sequences. Could it learn sequences of patterns and then make predictions?
Hey can you educate me on why this is being ridiculed so much by the comment section😂im new to both religions and it would be helpful for me in my studies
Okay Tim... I have a question worthy of your intellect and imagination. Granting arguendo the proposition that we operate within a carefully designed simulation, is it conceivable that the manifold hypothesis illustrates an intentionally implemented efficiency in our computational regime? Otherwise put, is the low dimensionality of crucial correlations and symmetries in source data an artificially induced property of our technical methodology in some sense? If proper pedagogy compels the sim architects to deploy logic that is never compromised by later learning--but the sim requires _some_ mechanism for invisibly adjusting the computational reach of our algorithms--then would it not be logical to establish that control inside the _dimensionality_ of our data?
Please, someone tell these guys about Numenta. It was heartbreaking to hear “that no one knows how to define general intelligence”. Numenta has layed out the basic principles of AGI and all they lack is mathematical foundation. These guys could move the needle if they look into Numenta‘s work.
![The moment we stopped understanding AI [AlexNet]](http://i.ytimg.com/vi/UZDiGooFs54/mqdefault.jpg)
I cannot believe it! I am so happy that you got the whole GDL crew on ML street talk. This episode is great; thanks for your awesome content!
This episode is beyond good... Finally, some visionaries turning ML from its alchemy stage into proper science!
You don't know about alchemy obviously. You should be ashamed
The amount of detail here is astounding
This video really needs 100k views. Everyone working in RL research needs to view this
Somehow, during my sleep, it ultimately reached this channel. And in my waking time, listened to this for a while.. and felt like I needed to go back to school. I felt like I had zero knowledge about what is happening on earth 🙈🫣😝😝 my day job is nothing and being a self taught ML dude felt like I am wasting my time doing useless things to put food on the table😅
Yesss normalize awkward people in cinematic shots! I love it
I love Taco's point about the community having too heavy of a cognitive reliance on benchmark datasets and ground truth labels. I have learned this lesson to my bones recently when I developed a "best ever" model in an industrial setting, but all of the metrics showed it to be WORSE because the model was actually better than the ground truth data, and e.g., true positives were counted as false positives. When your data/labels are biased from the data collection process, you can come up with all sorts of counter-intuitive conclusions when we don't keep those biases in mind during interpretation.
+Prof. Bronsteins’ 26:03 mic drop
I couldn't agree more, this is why in an industrial setting I always take new SOTA approaches with a grain of salt. It is not to say that the results are not convincing, but there's danger in having benchmarks that have been static for years.
Benchmarks have generally been collected on a single temporal (and, at times, spatial) scale using a uniform collection method. What I found is that once you start using data that has been evolving over years and years, it becomes non-trivial to deal with the non-uniformity of the data or biases that have been induced throughout the years. I remember finding this "great" pattern in our data and only later finding out that this was simply due to one of the systems assigning a certain property to data points if the system was running out of memory #rip.
During my study of Statistics, ML and DL I never understood the connection between different NN architectures - they simply pop up without any history or proofs. Shut up, learn and memorize. Therefore my professors in ML do not earn my respects, because they do not real understand what they are teaching.
I like thank you for bringing of the fundamental understanding of the zoo of DNN models. I lost the intuition for long time about DNN models. Again thank you for the clarity.
This has taken epic proportions, wow! 💪
So glad to see more attention here on geometric deep learning. Thanks for sharing your chats with these fantastic thinkers!
The way this video builds up over the fundamentals totally blew my mind.
Honestly, this channel has such a great format. It's the perfect mix between a podcast and a documentary. It reminds me a lot of Sixty Symbols or Numberphile but for ML instead of physics and maths
One of the best UA-cam videos I’ve ever watched. Videos, not only ML videos.
Agreed
Bloody hell your introductions are brilliant, they are mini documentaries.
Wow, three and a half hours of amazing content. Thank you so much for making it. It's like a documentary.
May this podcast last for another 10 years !
why not forever?
As a lot of this is quite over my head. BUT, I'm using OpenAI's playground to have the concepts I don't understand explained to me. And it works extremely well. Ideas like gage symmetry and such now make sense to me. On a very broad level of course, but still. An AI explaining tome the concepts that went into it's own creation! Truly amazing!
Paradigm shift
Remarkable channel. Inspiring, challenging, eye opening... I have been following it for months and I love it. Thank you
This is the longest UA-cam video I’ve ever watched.
A fantastically detailed video, I'm starting a PhD project on this topic now so this is perfect to watch! Thanks.
I've never known anyone more prepared than Tim.
😎😃
Now getting close to the end off your video , I realize that the" architecture" is the grande unified theory. Got excited , jumped the gun...
Again, thanks so much to the MLST team and you Tim for bringing this to us! Nothing like it, truly special stuff!
Will be very intrigued to hear the next few months of conversations while y’all chew on Hawkins vs graphs. 😆
Still trying to figure out how this guy stays so buff despite being five magnitudes more nerd than me
He's probably straight
Aren't they all AI generated?
Nome of them are really that buff. I think you're referring to general healthy eating
The buff helps make the nerd
Great episode! Learned a lot, really appreciate all the work that makes these possible. The videos are incredibly well done! Great guest speakers too of course.
This is great! Thank you for making this episode :)
Heard rumors about this MLST - and missed it when it came out. Looking forward to the talk!
I enjoyed watching it very much!!!. Thanks.🙂
Sensational, thanks for the content coverage, all in one video for GDL lovers!
I really enjoyed your introduction. This is starting to be my favorite science podcast besides Lex Fridman's!
You really need a patreon. Would love to give you money to help make more of these documentary style parts. Excellent podcast
Thanks! I don't really want the audience to pay for the content. Most of them are poor Ph.D students! Also -- even very large channels make nothing on patreon so it's pretty pointless adding it, monetization is off too. Perhaps one day we will get a viable sponsorship offer. Right now I would rather keep the channel pure i.e. almost any amount of sponsorship money would pale into insignificance next to the effort we put into making the content and just seems out of place.
@@MachineLearningStreetTalk I understand what you're saying but I still feel like I owe it to you to tell you that I'd definitely sign up for your Patreon if you ever decided to make one
Why not just privately message the person responsible and make a massive donation- privately - if that’s the way you know to encourage someone’s work.
All grateful helpful comments are welcome,
This is on you tube.
I hope it continues to be be free for me and students forever
💰
Excellent !
thank you for all your hard work putting this together, been waiting for this to drop for a long time
excellent episode 👌👌 the geometric dl viewpoint is truly fascinating. More than the obvious repercussions on the ML/DL community, I also hope that it will have an impact on the theoretical statistics community's research interests as well. I'm eager to see what lies ahead 😉
Love this topic, looking forward to hear this. I think this episode is great and land at the right technical level, not as low-level as "In these paper we did these augmentation and it works well", but not too philosophically useless as the "knowledge is universal..." argument.
I am enjoying it thoroughly. Its fascinating to see different perspectives from all GDL experts!
At 2:10:43 the speaker describes how the methods used to analyze an N based model breaks down at 2N. The Theory of Complex Dynamic Systems includes the concepts “self similarity independent of scale”, fractals to relate scaled features, and non-integer dimensionality. Something worth considering …
One year after, still one of my favorite episode, with the Chomski one
I really enjoyed this one the second time around. One thing, the transformer is mostly a pyramidal graph network -- just FYI.It's a fully-connected Feed Forward NN A pyramidal graph is a type of graph that has a hierarchical structure where nodes are organized into layers. The nodes in each layer are connected to all nodes in the layer above it. Pyramidal graphs have been used in various applications such as object detection1, EEG classification2, and spatial significance exploration3
Epic...3 hours of pure bliss
omg!! what a great generalization effort and applauses for the metanalogy with geometry Erlangen program.. how many problems are in some way only really the same problem!! congratulations and thank you.
i love the rotational intro sequence around michael i n the tradition of a science documentary
Tim, your channel is the best of its kind. Kudos man, much love 🤘🏻
Excellent content again. Damn another book to read !
Thanks Eric!
I somehow fell asleep watching UA-cam, and I woke up to this being 2 hours in!
Glad to be of service!
The most epic video you have ever made!
Thanks for podcasts. It helps me with other stuff i found to be in touch with ML, Ai, and neuroscience. Before this new hype around ML and transformers i didnt know that i will found that i will be in love with neuroscience
thanks so much for making this! amazing video
Nice gem of a channel.
It's my 1st time here. Your show already brings me wow!!! Thank you!!
hey Street Talk crew. Watching out for the next one ✌️
Something very cool on its way ;)
Graph permutation invariance is trivial. It's like saying that if you renumber the nodes it's the same graph.
Graphcore should hire Tim
Absolutely amazing episode! When you eavesdrop on a topical conversation of such detail you are almost bound to pick up some wisdom and sync in with the speakers at least for some time (prerequisite you have some background)
Love the format! Personally I'd prefer more content than more production value. If you can do both though that would be phenomenal lol
People on the right are saying Creationism. That is the answer. We can blame this all on Darwinism.
(Read Project 2026)
I can never keep up with these long episodes. It means I miss an awful lot of MLST if the guest or topic title doesn't really jump out at me. Once in a blue moon, the stars align to when the YT notification pops up and I can envisage fitting in an episode over the next couple of days in chunks. The promise of the Netflix-style Part 1 got me intrigued and it's one of those stars-aligned moments today (probably just because it's Sunday though). I think it's time to crack open the notepad and follow along. I can tell it's going to be good. Most of your stuff is just top content for us researchers in the ML domain (whether partially or fully). I really would consider trying to make smaller videos that are less intimidating length-wise, as I can more easily just ignore this entire episode, but would be much more likely to watch a relevantly-titled section I found interesting that was more around the 20-40 min mark, and thereby find myself enjoying more of your content.
That's a fair point. I sometimes have a similar issue where even if the title attracts my attention, I wont come to finish episodes, just because theyre so long. Also given the often challenging content it is hard to just resume where you stopped.
I really appreciate MLSTs effort to make it easier to absorb the content. Perhaps, shorter videos that summarize a certain topic using scenes from different epsiodes would help in that aspect. On the other hand, I assume this is a lot of work.
I hear where you all are coming from but I want to voice that the beauty of these conversations is there fullness, and distillation or bite sizing seems a big ask given the discussion is already at the edge of current understanding. I love the long form. If it could also be snackable that’d be overpowered 😅
amazing stuff. thank you for the time stamps. thus i can only listen to questions relevant for my domain:=)
A thing to keep in mind is you don't want to use neural networks for things that other computer methods do much better at like sorting, computing, mapping, and information lookup. A good example is Apple's Seri.
Hi, i learned that I can use the depth of my brain itself. Perhaps there are people who can use this idea. As far as I know, this hasn't been done. I myself try to realize whilst typing. Which might be too unclear a way of getting inside. This is THE legacy ever. Namely our brains.
Hello sir
can you group all these talks in a single youtube playlist please
thank you for the tremendous effort you are doing
Wow, this chair looks comfortable.
What a great episode, thank you guys so much for this!
A grande unified theory is needed.
If you attack a certain kind of complexity, you need not do that on the same level.
I.e. there are cheap ways of working through certain problems. Example: to apply all the necessary facts, after you have collected all the related phenomena and then choose how to work with those first.
First the easy way, and when you are done, you apply 'all'.
No need to do too expensive computations.
To some degree.
Best episode ever
Fantastic, thank you. (Minor improvement suggestion - might just be me, but I would prefer if the music was not constantly playing.)
While the focus is on approximation error, I think understanding will be limited. The most interesting behaviour is when useful novel outputs occur for new inputs, based on assimilation of abstract patterns.
it would be extremely helpful for phd students who are very busy if you can make a highlight video of this that contains only the most eseential academical discussions done in this video.
Stunning material
AI is an advanced knowledge based system with varied human mimicry potential and its use cases are the different forms of human kinetic mimicry
If the problem of creating intelligence didn't end up becoming conquered by human ingenuity - then we weren't truly intelligent in the first place. So if we really are supposedely intelligent, we should be able to mechanize it. Thus, machine intelligence is an extension of the fact of human intelligence. If we can't do it then our intelligence should be able to be surpassed given that the speed of human learning can be beat by a machine. Either way, in the final analysis we arent the end all be all of intelligence in the universe. Nature always transcends itself
Best episode so far :) !!!!
A good attempt to geometrization of machine learning by extending the Erlang program using group theory to found networks that capture symmetries. But, unfortunately, although valid for artificial CNN this is only valid to simple neuron equations that only capture very superficially the signal processing of real biological neurons.
This is amazing! Thanks!
Teaching a computer to recognize natural laws as it computes and extrapolate relevance.
I still think that coming from an understanding of a thing ,it is much easier to determine the way to think about it. BIOS
Which are rules...I'm getting interested in computer science.
The content was great! The editing was a bit disjointed. I love long form videos like this, but a touch more planning may have made for a smoother experience.
Amazing!
Love it
thank you for this
wow....some episode !
Our sincere thanks to these 4 brilliant researchers:
Professor Michael Bronstein
www.imperial.ac.uk/people/m.bronstein
twitter.com/mmbronstein
Dr. Petar Veličković
twitter.com/PetarV_93
petar-v.com/
Dr. Taco Cohen
twitter.com/TacoCohen
tacocohen.wordpress.com/
Prof. Joan Bruna
twitter.com/joanbruna
cims.nyu.edu/~bruna/
References:
ICLR 2021 Keynote - "Geometric Deep Learning: The Erlangen Programme of ML"
ua-cam.com/video/w6Pw4MOzMuo/v-deo.html
(Note we used some clips from this, and the graphics designer was Jakub Kuba Makowski www.linkedin.com/in/jakub-kuba-makowski-19b17143/)
Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
[Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković]
geometricdeeplearning.com/
arxiv.org/abs/2104.13478
Review: Deep Learning on Sets
fabianfuchsml.github.io/learningonsets/
arxiv.org/pdf/2107.01959.pdf
AMMI Course "Geometric Deep Learning" (12 lectures)
ua-cam.com/video/PtA0lg_e5nA/v-deo.html
Beyond the Patterns 28 - Petar Veličković - Geometric Deep Learning
ua-cam.com/video/9cxhvQK9ALQ/v-deo.html
Neural Algorithmic Reasoning [Petar Veličković, Charles Blundell]
arxiv.org/abs/2105.02761
Equivariant convolutional networks [Taco Cohen]
pure.uva.nl/ws/files/60770359/Thesis.pdf
Solving Mixed Integer Programs Using Neural Networks
arxiv.org/abs/2012.13349
Project CETI: meaningful communication with another species
audaciousproject.org/ideas/2020/project-ceti
Discovering Symbolic Models from Deep Learning with Inductive Biases [Cranmer]
arxiv.org/pdf/2006.11287.pdf
Node2Vec
arxiv.org/pdf/1607.00653.pdf
Deepwalk
arxiv.org/pdf/1403.6652.pdf
AMMI Course "Geometric Deep Learning" - Lecture 12 (Applications & Conclusions) - Michael Bronstein
ua-cam.com/video/caQV-Vb9TBw/v-deo.html
Graph Attentional Networks [PetarV]
arxiv.org/pdf/1710.10903.pdf
A Generalization of Transformer Networks to Graphs
arxiv.org/pdf/2012.09699.pdf
The Hardware Lottery [Sara Hooker]
arxiv.org/pdf/2009.06489.pdf
Developments in fractal geometry [Barnsley]
link.springer.com/content/pdf/10.1007/s13373-013-0041-3.pdf
Super-Resolution from a Single Image [Fractals]
www.wisdom.weizmann.ac.il/~vision/single_image_SR/files/single_image_SR.pdf
XLVIN: eXecuted Latent Value Iteration Nets
arxiv.org/pdf/2010.13146.pdf
Analogy as the Core of Cognition [Hofstadter]
worrydream.com/refs/Hofstadter%20-%20Analogy%20as%20the%20Core%20of%20Cognition.pdf
The CLRS Algorithmic Reasoning Benchmark
github.com/deepmind/clrs
WHAT CAN NEURAL NETWORKS REASON ABOUT? [Xu]
openreview.net/forum?id=rJxbJeHFPS
Graph Representation Learning [Hamilton]
www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf
Great content! It may be better to put the references in the description so they don't get pushed too far down in the comment section. I initially thought there were no references and just happened to scroll down the list of comments.
pin this comment
Digital beings are pretty amazing :3
That was really interesting.
Symmetry is all you need
2:19:17 and the next 15 seconds contains all the dreaded words of my undergraduate education in a single sentence. Anyone dare to explain what it means in ELI5 fashion.
I was listening to this and I really thought the interviewer was Brian Cox.
I am so happy!!!!!!!!!!! thank tou so much
1:59:35 Is the growth really EXPONENTIAL in a hyperbolic space ALSO? Or did Bernstein misspeak? I'm no expert here sure, just trying to understand.
- EDIT: No! Of course, if the embedding space grows exponentially, this also captures better ..what ever represented.
Jeff Hawkins is asking questions about locations of mobiles here ua-cam.com/video/p_KRsF-ncJQ/v-deo.html that I think could eventually be answered by the future GPT3… Here is my hypothesis: I was thinking of this last week : Brain makes maps of the environment. But I was wondering how it managed mobile objects. How it worked to know where it is. It's location. And then I remembered that when I was a kid, my mom would always ask me this question when I had lost something: When did you use it the last time? The question when. And I would try to remember what happened the last days, all the chronological events with a context linked to the object. And then I would remember where it could be. I'm amazed to consider that finally, the brain has a chronological map of events and their context, remembering every sequence and is able to navigate in this chronological map of events. And every object is there, the brain has all these objects identified and painted in every events, and keep all these pictures (I say picture, but it can be linked to any senses) chronologically and can navigate in this chronological mapping, like if the events were layers it can access. Soooooo if the events are like layers that can be accessed chronologically (or you can make jumps, but knowing where it stands in the chronology), could a future GPT3 do pattern recognitions on sequences. Could it learn sequences of patterns and then make predictions?
Hey can you educate me on why this is being ridiculed so much by the comment section😂im new to both religions and it would be helpful for me in my studies
Okay Tim... I have a question worthy of your intellect and imagination.
Granting arguendo the proposition that we operate within a carefully designed simulation, is it conceivable that the manifold hypothesis illustrates an intentionally implemented efficiency in our computational regime?
Otherwise put, is the low dimensionality of crucial correlations and symmetries in source data an artificially induced property of our technical methodology in some sense?
If proper pedagogy compels the sim architects to deploy logic that is never compromised by later learning--but the sim requires _some_ mechanism for invisibly adjusting the computational reach of our algorithms--then would it not be logical to establish that control inside the _dimensionality_ of our data?
🎨🤓🖌️
1:24:44
1:28:00 those nerdchills love it 👌👌
5:55 you have written 'the WORD is full of simulac..' don't you mean the "worLd"?
btw love it so far, thank you so much for your hard work!
Whoops!!
Cellular automata. Stephen wolfram ,I think , is working on that.
Oh dear a contributor from Imperial College, the most dysfunctional college
0:17 cool Tim became Morpheus :) is Yannic Neo?
Please, someone tell these guys about Numenta.
It was heartbreaking to hear “that no one knows how to define general intelligence”.
Numenta has layed out the basic principles of AGI and all they lack is mathematical foundation.
These guys could move the needle if they look into Numenta‘s work.
No they haven't really.
This is really good but I am totally stressed out from the sound effects.