Вставка
- Опубліковано 16 тра 2024
- Patreon: / mlst
Discord: / discord
Pod version: anchor.fm/machinelearningstre...
An emergent behaviour or emergent property can appear when a number of simple entities operate in an environment, forming more complex behaviours as a collective. If emergence happens over disparate size scales, then the reason is usually a causal relation across different scales. Weak emergence describes new properties arising in systems as a result of the low-level interactions, these might be interactions between components of the system or the components and their environment.
In our epic introduction we focus a lot on the concept of self-organisation, complex systems, cellular automata and strong vs weak emergence. In the main show we discuss this in detail with Dr. Daniele Grattarola and cover his recent NeurIPS paper on learning graph cellular automata.
Featuring;
Dr. Daniele Grattarola
Dr. Tim Scarfe
Dr. Keith Duggar
Prof. David Chalmers
Prof. Ken Stanley
Prof. Julian Togelius
Dr. Joscha Bach
David Ha
Dr. Pei Wang
[00:00:00] Special Edition Intro: Emergence and Cellula Automata
[00:49:02] Intro to Daniele and CAs
[00:57:23] Numerical analysis link with CA (PDEs)
[00:59:50] The representational dichotomy of discrete and continous at different scales
[01:05:21] Universal computation in CAs
[01:10:27] Computational irreduciblity
[01:16:33] Is the universe discrete?
[01:20:49] Emergence but with the same computational principle
[01:23:10] How do you formalise the emergent phenomenon
[01:25:44] Growing cellula automata
[01:33:53] Openeded and unbounded computation is required for this kind of behaviour
[01:37:31] Graph cellula automata
[01:43:40] Connection to protein folding
[01:46:24] Are CAs the best tool for the job?
[01:49:37] Where to go to find more information
Thanks to our patrons!
VIP patrons;
Alex McNamara
Ebonia Elliott-Lewis
All access patrons;
John Mitchell
Konstantinos Barmpas
And also;
Federico Rios
Francesco Puddu
Ian Finley
John
Lukas Segelmark
MarkMc
Paul Oreto
Philip Leggier
Timothy O’Hear
Łukasz Stafiniak
References;
(intro)
Strong and Weak Emergence [Chalmers]
consc.net/papers/emergence.pdf
Weak Emergence [Bedau]
people.reed.edu/~mab/papers/we...
Complexity - a guided tour [Melanie Mitchell]
www.amazon.co.uk/Complexity-G...
en.wikipedia.org/wiki/Emergen...
en.wikipedia.org/wiki/Reducti...
The Case for Strong Emergence [Sabine Hossenfelder]
philpapers.org/rec/HOSTCF-3
The Future of Artificial Intelligence is Self-Organizing and Self-Assembling [Sebastian Risi]
sebastianrisi.com/self_assemb...
Thanks to marknzed from Discord
(main show)
Learning Graph Cellular Automata [Grattarola]
arxiv.org/pdf/2110.14237.pdf
Grattarola article on GCA
danielegrattarola.github.io/p...
Dwarf Fortress
procedural-generation.tumblr....
Growing Neural Cellular Automata [Alexander Mordvintsev]
distill.pub/2020/growing-ca/
AlphaFold [Deepmind]
www.deepmind.com/research/hig...
Mandlebrot set
en.wikipedia.org/wiki/Mandelb...
FDM/PDE
en.wikipedia.org/wiki/Finite_...
en.wikipedia.org/wiki/Partial...
A new kind of science [Wolfram]
www.wolframscience.com/nks/
Wofram physics project
www.wolframphysics.org/
writings.stephenwolfram.com/2...
Game of life clock
codegolf.stackexchange.com/qu...
AlphaFold
www.nature.com/articles/s4158...
Michael Levin
as.tufts.edu/biology/people/f...
ase.tufts.edu/biology/labs/le...
www.growbyginkgo.com/2022/03/...
www.growbyginkgo.com/2020/06/...
Emergent garden
/ emergentgarden
/ max_romana
The edge of chaos
en.wikipedia.org/wiki/Edge_of...
Computation at the edge of chaos: Phase transitions and emergent computation [Langton]
www.sciencedirect.com/science...
Lenia
chakazul.github.io/lenia.html
Slackermanz
slackermanz.com/understanding...
Daniele doing a talk on his work
• Learning Graph Cellula...
Smarter podcast (Daniele’s pod!)
/ @smarterpodcast6806
Thanks for having me on, I had a really great time chatting with you guys. The work you do with the channel is absolutely insane, such a high production value!!!
Your introductions are so spectacular. Really sets MLST apart. Thank you!
One of my favorite topics as a junior neuroscientist. Finding simplicity in the unexplainable
Wolfram has been on Lex Friedman's podcast 3 times in which they go into depth on CA and Wolfram's theory of everything, all of which are good to listen to to try and understand Wolfram's grasp of languages and various stuff he's into! 👍🏽
Heading off to the Santa Fe Institute in June to study and can't think of a better talk to listen to prior to going. Thank you.
Hearing about Sebastian Risi, I'm immediately reminded of spooky action at a distance. Contrary to the idea of localization where self-assembly is based solely on local interactions. How might a system with an object that has a property read by two separate functions operate? Now you have cellular automata but only when not in opposition to a map like DNA. Moreover, the map is evolved over generations.
Very interesting! Need to come back to rewatch properly. Emergence can be boiled down to the very basics of even the mathematics set theory: if a set has no elements, will also have no properties at all; if you add 1 element, now the set has more properties; if you add 1 more element, new properties emerge; 3,4,5 elements... properties increase exponentially. 'Merely quantitative differences, beyond a certain point, pass into qualitative changes'...
Love this! I was into Complex Systems before ML started booming and took over. Talks from the Santa Fe Institute are amazing.
Man what an incredible episode! I'm going to have to watch it a few more times because it is packed to the brim with information.
Thanks Ian! 😍
Absolutely fascinating! I'd wondered what happened to CAs since Life in the 1980s. So excited to see the new developments.
David Bohm's implicate and explicate orders keep popping into mind, I wonder why? Thank you....
Top notch content!
Loved the attitude of Dr. Grattarola: modesty, curiosity and striving for semplicity over this-is-how-it-works and my-method-is-key.
Exactly!
Awesome, thanks!
So dope as always. Thank you!
Great video guys! Initially I found the idea of Chalmers viewing consciousness as strongly emergent to be counterintuitive at first, but when you parse in the role of supervenience it becomes clear his proposed dualism and emergence from the physical are not contradictory. I thought this distinction was well handled here.
Thanks for a great interview
Watch out, Dalle2 becomes imaginative and more human like. We are close to reach general intelligent. It is great work to understand human intelligence.
Great video! This feels a bit uncanny, especially the work on neural cellular automata, since, through sheer coincidence, I was talking with someone yesterday about how the nodes in Hopfield networks can be thought about _like_ cellular automata.
Tons and tons of stuff in this amazing episode, thanks!! One thing I kept thinking of as it applies to all the topics in some sense, but did not quite come up- concept of Boltzmann brains and ergodicity. General observation and the random brain idea seem to point that real world systems are not truly ergodic and this is another mark against infinity in reality, more in the time dimension I suppose? Anyway the Boltzmann brain question really makes you wonder though if a lesser form of proto life would (must?) arise on the basis of large scale state space exploration via correct seeding somewhere in the system. This gets to guest’s comments on CA rules vs state encoding the behavior.
Great stuff!!
Deep respect!
Is Sabine implying that strong emergence exists? From I gathered, she is saying that strong emergence cannot be ruled out due to our current fundamental theories being more effective than fundamental.
Yes that's right
Are there any analogues to the continuum hypothesis when it comes to the informal notion of emergence “levels”?
Fantastic! Another episode that is really inspiring for digging deeper. Thank you!
There is a typo at 28:42 I think.
Yes sorry about the editing mistake there
If I build a radio receiver, but there are no radio waves, there is no emergent behavior. If there are radio waves, then sound emerges. It seems to me that if consciousness requires a metaphysical signal, we have strong emergence. If consciousness requires that we have some connection to an unknowable source, that's in every practical sense strong emergence. My intuition tells me that the unknowable is a vastly larger set than the knowable; so strong emergence seems entirely possible, but this is based on the limitations of our sensors and ability to understand, not on any shortcomings of the practices like logic and reasoning. Moreover, the phenomenon of being unknowable is functional not intrinsic; so what is unknowable to one species may be knowable to another. The question that stands like a 500 pound gorilla in the room is what good does this understanding provide? What questions do we need to ask when attempting the analysis of some phenomenon or another? How do we keep from going down unproductive rabbit holes? Otherwise, what is the prize at the end of this speculative quest?
Hayek describes the program of the Scottish Enlightenment - which founded the modern disciplines of economics, legal and linguistic studies and evolutionary biology - as the study of emergent phenomena grounded in the basic units of analysis. In the humanities the point of view was summarized by Adam Ferguson’s phrase “ the result of human action but not of human design.”
#ManyToManyMapping is the key.
God I love this channel
At 12:25 the discussion cut short, but there is a connection between organisations and brains none-the-less.. The rules at the lower level in a company are a combination of fixed and adaptive, the employees have autonomy and will evolve a response, and it will involve middle management regulation plus employee initiative. By analogy the brain injury also has a response, not so much to top-down objectives but the analogy would be cognitive demand, where neighboring brain areas will pick up the demand, something we'll known as "plasticity". I can well imagine that this feature would evolve under selection pressure. If neural networks are to become biologically inspired then the language of description will need to evolve to embrace more organisational concepts.
I think this is being done. But deep in the "Big Labs", not the pre-trained LLMs released to/on the public.
We hear rumours of AI systems that contain specialized "subsystem" AIs in some form of network coms architecture.
This would be a fair simulacrum of vertebrate brains.
The topic of complex system theory is still unresearched area. Because complex systems are depending on initial condition, highly non-linear and stochastic. Therefore those system can not be simulated - the each outcome of the simulations will be different, if the search spaces are not restricted.
What would you say stellar nuclear fusion is, vis a vis its reliance on quantum tunneling? I want to say weak, but our description of it is inexact. I don't think we can avoid having to use our ability to describe it as a factor somehow
If the universe is essentially computational, what does it mean to be infinite? Does space extend to point b somewhere out in the tremendum? Perhaps not until it is computed. Go there, and it exists.
Need Stuart Kauffmann on show to talk on emergence and non-algorithmic evolution and the “adjacent possible”
I think you all would really enjoy having Michael Levin on the show (one of the co-authors of the "Growing Neural Cellular Automata" paper). He has plenty of talks on youtube, and also recently gave a talk at ICLR in the workshop "From Cells to Societies: Collective Learning Across Scales" if you want to check out some of his work.
We will, thanks!
It's interesting to see that Joscha is a logical positivist.
Are you able to share when the discussion with Joscha Bach is scheduled to be released? Thank you.
realistically not for another couple of months
@@MachineLearningStreetTalk Thank you.
Thoughts are CA. Conversations are CA. CA is cellular automata, whereby one level of complexity is defined by a discretely related similar level of complexity. DNA is a CA. The turning of food into energy is a CA. The big bang was the instantiation of the biggest CA. Need I say more?
You shouldn’t say more, because everything you just said is total nonsense. Congrats.
What's interesting is not that there is emergence in each 'layer', it is that we percieve 'layers' in the emergence at all...
No, it’s definitely interesting that there is emergence in each layer. Try again.
I'm surprised Stephen Wolfram wasn't mentioned at all in this video... He coined the term: computational irreducibility
He was mentioned my friend! Check 33:10 Hopefully we will get him on the show soon to discuss his new book / work
He was mentioned. Try again.
If the ants programming only makes sense in the context of environment and colony, is the ants programming strongly emergent from the colony?
It is also not true that we are not capable to change structure of atoms - we as human need to build tools. The tools we built reflect our brain and thoughts - we are the tools what we built.
If you look at self-organizing systems, they do use a plan, but it has been coded in the most compressed format, and therefore, it is hidden. Keep in mind that an "understanding" of the physics of the entire system are a part of the encoding. That is to say, the encoding would be meaningless without the laws of physics and the attributes of the various substrates.
We know.
💓
Please explain why does the hyper-specialization lead to generalization?
Counterintuitive at first, but idea is that generalization (if you unpack what it really means) requires you handle novelty, and you cannot train for novelty, you need a toolbox that may let you handle aspects of the novel state. Implication of modular specialized functions, not optimized for specific synergy or task necessarily, so that you have open ended correspondence with novel states and hence they are tractable, hence you are “generalized”.
An expert system can only do one thing. I. e. a chess computer can only play chess - those systems are hyper-specialized in specific tasks. How can those systems generalize, if they are trained for deterministic tasks.
How can a hyper-specialized system generalize where the search space is infinite? How can those systems identify the stepping stone lead to generalization.
Coherence is a fractal.
Nonsense.
I am still starved for that Joscha Bach episode XD
Sorry for the delay, me and Keith are building a startup company and it's sapping all our time. I hope we can focus more on MLST soon
The background music is unnerving
The OBSERVABLE universe is OBVIOUSLY not continuous, as nobody has ever seen the back of their own head.
Wow. You have no idea what these terms mean or what you’re talking about. What a hilarious comment 😂
@@therainman7777 Yeh, right, next thing you'll try and tell me is that the OBSERVABLE universe is ISOTROPIC.
Can't help but feel deeply flattered that, not only did I arrive at the stance that infinity is cognitively meaningless at the age of 18 or so, but there are professors in their 50's that still don't even understand the claim
One of the most masturbatory comments I’ve ever seen, congrats 😂
Pei next :)
Pei is the one after next :) It will be worth it I promise!
were you having a giggle at the accent? lol
Reductionism fails whenever the whole influences the parts.
No it doesn’t.
@@therainman7777 Have you studied complexity science?
@@netscrooge Yes
stop wobbling around the papers you're citing, it's giving me a nausea!
LOL! We will turn the wobble down!