Your brain is a simulation machine.
Вставка
- Опубліковано 6 лют 2024
- I recently read "A Brief History of Intelligence" by Max Bennett - and I highly recommend you check it out - it's an absolutely brilliant book!
I will be having a few conversations with Max but for now here is a taster from our first chat about how our brain works like a simulation machine.
You can buy his book from Amazon - amzn.to/3weN3uS
Watch behind the scenes, get early access and join the private Discord by supporting us on Patreon. We have some amazing content going up there with Kenneth Stanley this week!
Listen to this entire conversation now (Simulations chapter) on Patreon: / max-bennett-of-1-97975425
Watch part 2 (Mentalizing chapter) on Patreon - / max-bennett-of-2-98181787
Watch part 3 (Language) here / max-bennett-3-0-100006225
/ mlst (public discord)
/ discord
/ mlstreettalk - Наука та технологія
I think there are compelling reasons to dispense with the "statistical parrot" hypothesis for LLMs:
1. As LeCun et al. showed --- and discussed on MLST -- LLMs exclusively extrapolate, never interpolate. Even in compressed latent spaces, the curse of dimensionality makes interpolation virtually impossible.
2. LLMs have remarkable zero-shot & few-shot capabilities. They can learn new skills on the fly, often faster and better than humans.
3. LLMs can solve a unique problem that you construct from a combination of many skills. The combinatorial math here pretty quickly requires that folks who believe LLMs are hash maps to also believe in miracles. (A more formal sketch: make a massive, arbitrary set of know LLM skills S = { bunch of ~disjoint skills }, e.g. S = { "rapping", "C# unit testing", "US tax law recall", "solving 1D kinematics problems", etc }. Compose a question using a handful of such skills X ⊂ S. For set sizes s = |S| and x = |X|, the chance of an LLM seeing/solving a similar question before is ~ 1 / C(s, x). That is, it's inversely proportional to s choose x. And those initial astronomical odds are just for starters.)
4. There _are_ truly open-source models with corresponding data; OLMo being a recent example. Conversely, researchers & companies must still carefully attend to training splits, contamination, etc. in their private data. There's little reason to expect some smoking gun to reveal that model reasoning is merely 3 interpolation algorithms in a trench coat or whatever.
Taking a broader perspective, dismissive skeptics often seem tunnel visioned on _rationalizing_ their position rather than _informing_ it. Their confidence feels misplaced beside our modest scientific reality. We don't understand how _our own_ minds work, let alone how _other_ minds could. We can't agree on _the definition_ of intelligence, let alone its objective measure or mechanisms or detection. Ditto for consciousness, reasoning, agency, etc.
I reflect on the transformations of this past year. The hundreds of millions of people helped, and billions in market value. The immense help *I've* received. The crisis in universities & schools. The untold thousands of people now saved from the trauma of moderating content of the most unspeakably heinous acts of (in)humanity. The ingenuity of the impaired who are taking their lives back. The productivity multiplier now augmenting engineers, lawyers, healthcare workers, data analysts, teachers, etc. The nuance and intentions unlocked in communications the world over. The expressions of artistic creativity. The deep societal anxiety. The leading machine learning experts, academics, and millions of people globally who believe that these systems exhibit genuine intelligence or reasoning...
Those who look at _all of this_ and (often instantly) categorically dismiss these LLMs as basically handy autocorrect that has fooled the world...it's difficult to see that as anything but naked bias. A "real" vs simulacrum distinction usually asserted without testable, falsifiable predictions of real-world consequence, i.e. pseudoscience. Metaphysical hairsplitting. And hey, maybe these systems _are_ convincing impostors. Or maybe the origin of intelligence is disappointingly prosaic; a pedestrian byproduct of unceremoniously throwing scale at any sufficiently general problem. And expecting otherwise was just anthropocentrism, human presumption that led to Copernican humblings -- learning that humans occupy no special place at the center of the universe, the solar system, the tree of life, or (finally) on the spectrum of intelligence.
I don't know. I guess what bothers me is that nobody else does, either. So anyone pronouncing otherwise ironically sounds less authentic to me than anything said by GPT-4.
Wonderful comment. Yes, we don’t know what intelligence is in terms of a scientific or mathematical theory; and the engineering and experimental results are way ahead of the science.
I was about to comment on the use of the word “pastiche” in the video because the LLM isn’t imitating since that would actually require intelligence. Taking “pastiche” more literally, an LLM doesn’t store any text therefore cannot paste it together.
You said it so much better!
Please invite me to any groups or sites where this level of discussion is held. I need to talk to smarter people more frequently.
Hear hear
Great teaser! When will full episode be released?
Already looking forward to the full episode!
IMO we just get really good at constraining hallucinations as we age. This is useful as it allows our simulations to be more realistic, but if you've ever talked to a 4yo it's amazing how much more vivid and creative their simulations/hallucinations are without all those constraints.
I'm a sucker for your book recommendations. I just purchased it this morning.
love these short form videos
Less is more with editing
5:40 a) sensory experience bombards the brain with a shipload of data, it's only logical that the phenomenology is more intense if compared to a local to the brain, internally induced simulation
b) it makes no evolutionary sense to waste a lot of energy to generate all the finest details when reproducing a situation in our imagination, when a sketch is more than enough to evaluate pros and cons
There you have it Max!
What's interesting to me is that I've met multiple people who claim their internal visualizations are just as vivid and clear as the real thing, down to the tiniest identifiable detail.
@@10ahm01 possibile, happened to me while dreaming under medication
The end was epic 😂
I've been on a Donald Hoffman binge lately so this is perfect.
I guess Science is finally understanding & trying to peek behind the circus 🎪 of Physicalism. That it's nothing but an illusion of consciousness..what the mystics have been saying since centuries.
Book is EXCELLENT
Wonder if Patreon chat says what Max thinks In Breakthrough #6
It does, there is a 90 minute interview on it 😁
A social calculator i called it.
"from a phenomenology point of view"
The intelligence I am interested in is the kind that are trained while deployed without catastrophic forgetting.
Adaptive Resonance Theory offers that
Is this ai biased interpretation of brain ?
Talk to Bernardo Kastrup!
Tim, since last almost 3 years you try to figure out the diffdrences between human and machine intelligence. Therefore you started your channel mlst to have the opportunities to interview brilliant researcher in ML, CS and neuroscience. End the day we can conclude that the machine is machine and the flows in system can not be fixed. Now, the question what is the human intellegence in era of ai? Is iq tests, universities exams, degrees or number of scientific publications? What is exactly and the human intellegence should defined? Every expert and researchers have different answers.
Here is simple definition of human intelligence in era of ai: if you can tasks and reasoning the ai-system can not, you are intelligent. The reason: if the ai system can solve 95% of problems where the system trained on, because the correlations are easy to detect. If you are capable to solve problem in rest of 5%, you are smarter than most people on this planet.
The universities do not prepare the students the ai-culture schock, after they are joining the workforce - next generation will be hopeless lost and no chances to win tge competition against ai-system.
Well this all sounds fine and dandy but how do blind people experience a simulation ?
Words of the week:
diodastic
corpus
dispersed
Introspections
recapitulating
inductive biases
stochastic
infosphere
I'm just a simple guy living my simple life maybe I should incorporate some of these words into my everyday vocabulary to understand simulation 😁
Embrace the simplicity, friend! Using complex words requires education. Avoiding them requires wisdom. As Feynman famously said, “if you can’t explain something in simple terms, you don’t understand it.” Convoluted language is a sin. You risk sounding pretentious, or insecure, or (worst of all) inconsiderate about complexity's burden on your audience. Those "words of the week" have (several) simpler, popular alternatives, e.g:
Educational
Data
Spread
Reflections
Repeat
Prior beliefs
Random
Cyberspace
Expanding your vocabulary is great! Just use jargon sparingly, where it adds genuine value. Cheers!
@@Dan-hw9iu Hey I haven't seen Neil Degrasse Tyson jump through those hoops at Isaac Asimov Memorial Debate: Is the Universe a Simulation?
Four physicists: Zohreh Davoudi, Max Tegmark, James Gates, and Lisa Randall
One philosopher: David Chalmers
😁
Universe loves novelty
Internal and an external simulation
Corpus colossal was working
Controlled hallucinations
Holographic universe, so intelligence is going to be holographic at many levels
Human augmented functional capacity that’s all it is. You must have a baseline of enormous amount of data and vocabulary to be AI work.
Holographic human brain for gathering more data and decoding its environment will be coding how it works itself
are you okay? you left like 10 comments without context. If you're trying to react to the video like a live stream, you can add time stamps by typing the time so you don't just look like a fucking crazy person
2:00 3:00 4:26