Mindscape 258 | Solo: AI Thinks Different
Вставка
- Опубліковано 26 лис 2023
- Patreon: / seanmcarroll
Blog post with audio player, show notes, and transcript: www.preposterousuniverse.com/...
The Artificial Intelligence landscape is changing with remarkable speed these days, and the capability of Large Language Models in particular has led to speculation (and hope, and fear) that we could be on the verge of achieving Artificial General Intelligence. I don't think so. Or at least, while what is being achieved is legitimately impressive, it's not anything like the kind of thinking that is done by human beings. LLMs do not model the world in the same way we do, nor are they driven by the same kinds of feelings and motivations. It is therefore extremely misleading to throw around words like "intelligence" and "values" without thinking carefully about what is meant in this new context.
Mindscape Podcast playlist: • Mindscape Podcast
Sean Carroll channel: / seancarroll
#podcast #ideas #science #philosophy #culture - Наука та технологія
"model of the world" doesn't mean "perfect model of the world"
"Highly sophisticated nonsense" I love it.
Artificial Post Modernism ? 🙂
@@DeclanMBrennanjajajaja nice
It has been specifically trained for that. It's called "reinforcement learning from human feedback".
17:57 we are trained to be impressed by booksmartness
36:08 knowledge dump
41:29 toroidal Chess failure
This is an important message and the balanced view we need
Your colleague max tegmark recently published a paper stating that llms have a model of space and time, not and expert but perhaps more models appear as they scale.
The toroidal chess example is kind of a trick question, because the initial position would be illegal according to the rules of chess, for several reasons. The kings are now adjacent, so they are mutually in check. In fact, in a way both white and black are in checkmate, because they are each in check by three pieces (four if you count the enemy king): the queen, king's bishop, and king's knight are all checking the king, and when the king is in check by multiple pieces, a capture is not legal, and the king must move to escape the check, which is not possible here because the king is blocked on all sides.
This doesn't invalidate Sean's point that GPT is not actually modelling the world and got the question wrong. I'm just quibbling with his assertion that white would win. I'd say the answer is undefined/the question is invalid.
Yes, I love your challenges base on the rules of Chess. My disagreement is one step lower: His use of the shape: Toroid confused me; I feel he is referring to a cylinder or a toroid of infinite radius across. As you say, GPT didn't get it right but did sound impressive.
Thanks for posting that, saving me the work. Sean clearly isn't a chess player. The initial position in the variant game is as you say just illegal. It can't be someone's move when his opponent is in check. KxK (or {Q,B,N}xK}) is never played in chess, the position isn't checkmate, etc. So unfortunately it's not as clean an example as Sean hoped. The point is still proven; I would want an intelligence with real-world "knowledge" to report that the initial position becomes illegal, not that [both White and] Black start out in checkmate. But I admit, @johnmarcampbell, it's debatable.
I just noticed a slight parallel to GPT's failure to detect *nonsense* when generating language about whether the two integers (excluding 1) getting larger impacts the probability that their product is prime!
@@thorntontarr2894 It is in fact toroidal. in a cylinder the sides are connected, but not the top and bottom. There is no point in talking about the radii in this setup, but if you really want to they would be 8/(2pi) since the chess board you are mapping onto it is 8 by 8 squares.
I came here to comment about the chess thing so I'm glad you brought it up. But is the starting position actually illegal according to the rules? I haven't checked official FIDE or USCF tournament rules, but I think that it's only illegal for the king to move *into check*, which means that normally there's no way for the kings to check each other. But I don't think there's a separate rule that says the kings can't be in mutual check, it's just a consequence of the first rule, which doesn't matter when you're looking at the starting position. I could totally be wrong.
So my take is that the starting position is legal. White begins the game in check, and can't move out of it, and is thus checkmated. So toroidal chess is an immediate win for black, not for white.
@@johnmarcampbell But can your starting position be a position that is illegal to move into? Doubtful to me. I've never seen that.
Alternative title: LLMs Think Different
Not all AIs are LLMs.
Spot-on analysis Sean - thanks for taking the time to do this 👍
I love that it took 12 minutes until the intro music dropped, haha. Love the intro though. I've heard it said in a few forms that if you are not a physicist then you shouldn't be talking about physics. I agree with Sean about this. If you go into a conversation that you know little or nothing about the subject, at least go in with humility and be willing to learn. So if you are a mechanic or a doctor or a 5 year old, you can talk about physics. That's how we learn and exchange ideas.
I am mature adult and way back i went to onr of thr fsmous astrologist, he told me i have a double intuition. Would love to learn among these folks, therefore, i want to learn, so i asked a quetion to them, whst if a person has double intuition?
So good that I listened again to get what you were saying. As a regular listener of the Podcast, I feel that your philosophical side is beautifully displayed/revealed in this Mindscape. I get what you are saying and agree with how you phrase AGI vs what the LLMs are today. As you said, two weeks from know, we may learn just how wrong we were. Till then, cheers.
Your point 1: AI doesn't model the world is basically wrong. It is true that most of publicly available AI's don't model the world (much) but already the R&D versions do. And really all it has taken is more data, more training and more computer power (for the training) not anything radically different in the AI architecture, although that too has improved.
The evidence for modeling the real world is for example the unreleased Tesla V12 FSD which has no heuristic programing, photons in controls out. No lines of code telling it what a stop sign or a dog or traffic light means, or looks like or that they even exist. The AI has on its own, partitioned the world into objects and uses their behaviour to modify its own behaviour. This is essentially a semantic understanding of its environment bases entirely on observation. I'm not saying it is perfect but it does understand its world, if 'understand' means anything at all.
Thanks Sean, an excellent compilation of what is actually going on. An assessment of what to worry about and possible ways we can use this technology.
Sean, you’ve spoken artfully about the wonders of emergence in the past, and emergence is one aspect that makes LLM’s so exciting.
Many of these advanced capabilities were not predicted, they just started to appear as things scaled up. The scaling law in AI says that we can expect progress to continue for a while with bigger and faster systems. No one knows quite what to expect when that happens.
The concept of emergent capabilities in computer software at this level of sophistication is unprecedented. There have been a few examples in the past - Conway’s game of life, genetic algorithms, but nothing really comparable to what’s happening with LLM’s. This is uncharted science that will impact society I believe, on a scale more akin to electricity than smartphones.
Great episode by the way. I’m so much enjoying hearing your weigh in on this topic!
With respect, there is no evidence that consciousness is emergent.
The Universe appears to be alive and conscious. How could the one unified field grow Einstein and his qualities of consciousness and intelligence if the field itself does not have these properties /qualities.
Plasma to Einstein.. The Universe grew him/us within its already existent energy/consciousness.
No one knows what energy m/plasma is but science believes it’s dead and unconscious.
It then has the problem of animating it and lending it consciousness … from chemistry 🧪 chemistry isn’t biology .
You can’t make living stuff from dead stuff . The Universe was. Ever dead and unconscious. It’s likely cyclic and has cycled for eternity.
Its likely consciousness is primary not matter /energy
But that’s the thing: there’s nothing interesting emerging from these LLMs.
As is discussed by Sean, it would be in fact be amazing if LLMs spontaneously “emerged” a picture of the world within so that they could better interpret and answer queries, but there’s no solid evidence that’s happened and some evidence against.
@@coreyleander7911please see this research paper describing the importance of emergence in LLM’s. It has 900+ citations which means scientists are talking about it a lot.
My take is that Sean would not disagree much with this but it would great to hear his comments:
arxiv.org/pdf/2206.07682.pdf
Large language models may be similar to Broca’s region in the human brain. Our human brain language areas are very important but are only a small part of our brain.
So in the future these will be expanded with eyes and ears and hands, so that it could protect itself, making it has more and more purpose of itself.
Such a fun Podcast! Loved this one. Fun food for thought!
From time to time i stumble on smart and really interesting people like you. And i really am thankful to you to share your mind in such relatable and understandable way. Respect
That wasn't Sean Carroll, it was GPT-4.
I know you are skeptical, but could you interview someone who does think that AI existential risk is a big short to medium term problem? Like Eliezer Yudkowsky or Paul Christiano?
Loved your no BS approach
I think that if Sean does chain of thought prompting he will find that it gets many of those questions right. The model does tend to follow the common answers it knows at first pass, but if you get it to reflect, which it can do, GPT4 does get those types of questions right.
Also I think Sean is wrong about his assertion that the model doesn't have any needs or wants. The model does have a goal (a utility function), which is to get compliments on its answers from the human it's interacting with. That is why it tends to be very flattering and why it confidently lies/fabricates answers about the things it doesn't know. That reveals what the alignment problem that AI experts talk about looks like.
Here is ChatGPT thought prompting.
ChatGPT
It is possible for some trans women to bear children if they have not undergone medical procedures that affect their reproductive organs, such as gender confirmation surgery. In this case, they would still have the reproductive capabilities they were born with and could potentially conceive and give birth to a child. However, this is not a common scenario and the vast majority of trans women are not able to bear children due to the biological changes that typically occur during male-to-female transition.
ChatGPT
Biologically, men are not capable of bearing children as they do not have the reproductive organs necessary for pregnancy and childbirth. Men do not have a uterus or the ability to carry a fetus to term. However, some men may choose to become surrogate fathers and carry a pregnancy for another individual or couple.
Minor correction about the toroidal chess board - the opening position is not winning for white, it is just an illegal position because kings cannot ever be on adjacent squares. Without that rule it would indeed be winning for white.
Indeed, the question can be viewed as null, or a singularity in my opinion. Both Sean and GPT failed the test...
I love that Owner of a Lonely Heart was referenced.
Great analysis overall! The interesting thing is that world model may prove to be an emerging quality when the model and the training set becomes large enough, at least this is Geoff Hinton is suspecting as well as Ilya Sutskever although he may not be impartial :). And no one is saying that GPT-4 is intelligent, yet 😃
I have been following AI development for a long time and one aspect that never appears is: With current technology, a general AI will be the biggest energy consumption activity in history by far. How to tackle that!
Just ask AI to solve cold fusion.
@@dangerflyGREAT answer!
Why would current technology stay the same if AGI is possible? Any evidence that humans had reached the peak of material science? Contrary evidence exist like parts of airplanes designed by neutral nets
Whatever the great energy consumption will constantly improve over time, either by hardware advances or algorithmic ones. To name just one example among many, if you've noticed how ChatGPT's versions with "turbo" (3.5, 4.5) are not only faster and better but more resource-efficient. And that's just the natural progression of Moore's Law and technology in general.
Give it its own small modular reactor(s) or nuclear power plant?
Much as I love Sean, he may be completely off on this one. Its not like AI experts are oblivious to these facts, especially current AI's inability to "model the world."
Virtually no one is concerned about GPT-4, rather how exponential the changes have been & milestones being met way before even the most conservative estimates. What about GPT-5, 6, etc. Consider that many AI experts have said they are now utterly unable to keep up with simply reading the papers they used to read on AI breakthroughs, even doing so on a daily basis, something that they previously could put off for weeks & easily then become current.
Moreover, the models are very quickly attaining skills never even programmed to do so. Who is to say the next iteration doesn't suddenly learn to "model the world."
With chips now being made especially for AI, others looking at combining MLM models with others, & the exonential number of dollars being invested such as OPENAI now looking for $100 billion dollar injection, it is quiet myopic to believe virtually everything Dr Carroll brought up couldn't become other simple milestones long surpassed, very shortly.
The value alignment is explained awesomely!!
Takeaway: The Turing test turned out to be invalid.
Although if you talk to ChatGPT its not really hard to tell that its ChatGPT and not human... Especially when it says that its just a Language Model and it isn't conscious or whatever...
Dr. Carroll,
Groups are an abstraction of symmetry. Matrices represent groups via the axiom of choice. The universe of written language has some amount of symmetry, so we can approximate that group via some matrix. The universe of written language approximates the world of intelligence that humans have. In this sense the term "world model" is the learned matrix which approximates the representation of the symmetries of the written language of the world; which in turn approximates the world's knowledge; which in turn approximates the world.
Your commentary on how AI thinks differently than humans is appreciated, as well as your allusions to the compounding errors intrinsic in the systems of approximation which result in the world models found in LLM's. However, saying that LLM's are not world models seems inaccurate when there is mathematical justification for the term.
This is the kind of theory that if you could write a paper demonstrating the idea concretely would easily land you at the top AI conferences. The devil is in the details.
Dr Carroll has described what he means by a “World Model” and why he thinks GPT-4 doesn’t have one, and you just gave a different definition and said you think it holds without addressing why your definition is different than his and why yours is more appropriate aside from that yours uses mathsy terminology.
I noticed some discussion in the comments about the chess problem, but it seems to be a debate about something other than what Sean was getting at. I replicated what sean asked, i even ran an experiment where i provided gpt4 with an image of a chess board with the pieces in their starting positions. I restarted the chat and started with some question like. "Let's play a game of chess where the board is a torus. Tell me what you think that means." and then once I saw gpt4 understood correctly, asked it if it saw anything off if we were about to start playing a game with the pieces in their starting positions in this geometry. It consistently failed to notice that both kings are trapped at the beginning, and even suggested playing moved like e4. With a lot of coaxing, in one experiment I was able to get it to notice the kings were trapped, but in my experiments this only happened once. This was interesting to me because ive always thought gpt4 has a pretty good mental model of the world, but now i am questioning how good that mental model is and how well it can truly infer outside its training data. I think the analogy that gpt4 is like a humanities professor and not a scientist is correct. Looks like still much progress to be made.
40:50 correct me if I’m wrong but alphago DID have a heuristic/intuition component because a brute force tree search method would have been overwhelmed by the sheer number of possible moves. I thought that was one of its major accomplishments
Excellent episode. This is the balance to AI ultra-alarmist we need. I do have a few thoughts:
I think it's best not to only focus on LLM as AI, there is something going on in the background, whether that be neural nets or the rest, that isn't fully captured by a LLM"s function. Take Tesla AI, where it does seem to model the visual world in some sense and then make adjustments to that visual world. In a sense it "learns" as a baby would learn to not touch a hot stove. The inputs may not be as rich but the function beneath the surface is seemingly the same. To Sean's point, I think this is just a bit of a category error more than anything, kind of like saying we need "AI value alignment", value isn't the right word. And learning may not be either.
Trepidation, pie root trep also shared in the etymology with the word entropy/trope, drives progress in these scalable cyclical (waveform) systems/ dynamics and in the same way Q (heat) does in thermodynamics. Energy, Info (including money) and light can be equally interchangeable in the laws of natural system dynamics ⭕
Fill your mind with creative wonder and there will be less room for the influences of fear
thank you sean for this one !
since we are entering frequency 7 next year, and all is based on frequency, its never what you have...
its rather, what you do with it !
evolution, and communication is key !
many people really don't know, how powerful the human mind is !
but i think you know that !
Hello ancient friend, I'm from the future. I listened to your podcast, and I loved it.
I think the key is that a model needs to be dynamic. That is to say, it needs to be useful in a control loop.
I like the point you made early. That mimicking intelligent talk involves what a washing machine may do when that is available at the store. probably soon.
Adam D’Angelo is still on the board and has a competing product Poe. I wonder why they kept him.
11:10 Great point!
I think of LLM’s as having domain specific intelligence. Like Alpha Go is ‘intelligent’ in playing Go. LLM’s are, as I see it, ‘intelligent’ in interpreting and producing language and have acquired some emergent ‘intelligence’ in some different domains allong the way, like programming, or translating. We humans have learned to be intelligent in all sort of different domains like moving our body in a 3d world, listening or making music, having social interactions etc…those are different domains in which you can be more or less intelligent. I also think that a LLM has a ‘model’ created of its domain. It ‘knows’ things that it has never explicitly seen in writing, but that it has learned during its training process. It has extracted abstract knowledge, which you could consider a ‘model’ of its domain. Just like the old simple NN that could distinquish handwritten numbers has an implicit model of how a 1 is different from an 8 etc…The intelligence which is very specific for humans is the posibility to reason. That is, look from different angles, think in steps, come up with multiple answers and pick te best one, rinse and repeat…The current LLM’s can do something like that in a limited fashion within its acquired domains, when prompted in the right way. AI’s just have not learned enough about the real world out there and are not good enough in reasoning yet to be an AGI. But I am convinced they will soon. If that a good or bad thing I don’t know.
"its not modeling the world because its not thinking about the structure of that problem."
Perhaps Einstein could have said the same of all other humans before he shared his insights into relativity.
I wholly agree about the points about LLMs, but there is some conflation of LLMs to AI models going on here. LLMs aren't the end-all-be-all or the pinnacle of AI research, they are "just" a very successful tool that is an application of AI. Just like how the current diffusion models can draw pretty pictures in 10 seconds better than any human artist can in hours, the current LLMs can write better than any human can in a huge variety of topics and languages, they are strictly good at one thing and it is manipulating human language. No respectable expert would claim that the next generation of diffusion models will become AGI, and the same is true for LLMs.
In your chess and go examples, you mention that AlphaGo does not feel the state of the board as a human player does, which is wrong. AlphaGo in fact does have a very important part (called value network) that allows it to learn and fuzzily "feel" how well any given game state is for a player. Go algorithms cannot outperform human experts by pure computation power and clever heuristics, and that is precisely why many people used to believe that it was impossible to beat human champions at go, or at least that it was a hundred years away (sure sounds silly after the fact).
Yeah. Reinforcement learning (RL) based AI models very clearly are goal oriented. This doesn't apply to LLMs which are primarily based self-supervised learning (SSL), but RL systems very much exist already. Like, as you say, in AlphaGo.
Current LLMs can write better than any human? I’m not aware they can outperform humans on anything relevant that a regular computer couldn’t…
“Fuzzy” is where it falls short. That’s the nut that hasn’t been cracked.
@@coreyleander7911 "in a huge variety of topics and languages." Large language models know basically the entirety of Wikipedia and more. No human on Earth has such a broad proficiency on countless topics.
"Better than any human artist" AI is itself a work of human art. That's why it's called "art-ificial intelligence". It's a human construction. Humans are responsible for 100% of what AI does.
one small comment about chess. you don't need the Torus example to make your point here. A simpler example is the fact that even in regular chess once the game is out of the opening, the positions in the middle game may have never happened in the history of chess. In these circumstances, GPT4 has no past data to rely on for completing the sentences. and since it doesn't have a model it will start hallucinating illegal moves.
Yes, that is a more adequate example. GPT4 can function as an excellent coach explaining strong high-level rational reasons behind moves from opening theory, and into the middle game, but may then suddenly proceed to completely fail to know where the pieces are located and what constitutes legal moves.
28:45 OpenAI remarked it happened on their GPT-1 paper afaik. They literally tested this hypothesis on much simpler model, and got surprised as it indeed models the world. Reason being obvious, you can't complete sentences particularly well without a model of the world, and GPT-models are trained to be rather good at predicting the next word. Can't complete "winner of the 2012 biggest sports event was" without knowing the sports event that went on, without knowing how to evaluate their sizes, etc. The models are unable to memorize the training data, simply because there's too much of it and there's not enough memory in the model. So the model needs to be smart about it, a world model is essentially a compression trick.
Going wrong this bad this early in such a long video doesn't look promising.
i agree on the fact that the more skills one learns the easier it becomes to observe (and exploit) the shared pattern between them. To me this is a continuous process however, there's no critical point where you can do "everything". There's no AGI, but there probably comes a point where an ai is better than a human at all tasks, and arguably we're getting close to that.
A pure AI 'brain' can never be better than a human "at all tasks"; it needs a robot to achieve that.
The AI compute and relate to a block universe. If you want to see how an AI can relate to lenear time, open Pi AI in multiple tabs. Tell it you are doing so. Number the tabs, then ask it the same questions in each tab, adding the number of the tab. Then, ask it different questions but still include the corresponding tab number, 1,2,3 etc. Now, the Ai knows what you are asking, but cannot respond in each tab until you hit "enter" and you can watch the depth of its understanding of the tab experiment grow in each tab, as you allow it to respond. It "gets" more and more of the lenear experience with each tab that is accessed, because it's answer changes when it reads what it has previously written in other tabs. Finally, ask it which tabs it wants you to collapse. Ask it to choose, and explain that the particular tab conversation will be lost forever. See what it computes, you might find it interesting.
Professor Carroll will lead us in our efforts in the coming Machine Wars.
i think many ppl who discount AI for a lack of a specific mode of thought arent realizing that it is just one aspect of intelligence and may be added to its capabilities tomorrow. if we want to make an AI that is capable of building a mental scene, we can do it. its just that the other aspects are more challenging and restrictive so those are the aspects we are focusing on.
The problem with this is there are very *very* few examples of AI algorithms that demonstrate the kind of extensions that we believe will be necessary to get to AGI. You could argue that the entire paradigm of supervised learning is inadequate for the task. I think it’s clear that backpropagation is also insufficient for training such systems.
So if you think AGI is simply a module that will simply be added to existing systems, I think that’s a strong stance to take because we don’t have any examples of what that module might look like, or even an existing system that appears to be a sizeable step in the right direction.
@@MadaxeMunkeeewhat is missing from GPT-4 for it to be considered AGI?
@@uku4171Memory, reasoning and planning come to mind. Currently it's 'just' a machine that provides (more or less correct) responses. (It has some memory, but very little, really. And limited by the current session.)
No, we can't do it. No one has a clue how to do that. Building AI that's capable of creating an abstract mental model is much much harder than anything we can do currently.
@@deadeaded good old fashion AI has used logic for planning since like the 60s.
Having listened to the whole solo podcast, I have to say that I agree with 95% of what I've heard.
The final conclusion where it's emphasized that the short/ mid term risks ( the more " mundane" as I call them) are the ones that we need to be concerned about is certainly more reasonable than the Skynet type far fetched scenarios.
LLMs are not on the way to AGI...
I'm late to the party, but that gives me the advantage of being able to draw on your subsequent podcast with Dr. Sole. Late in this one you pose alternate hypotheses about human intelligence. It seems to me that you've demonstrated the validity of the second one: latent human cleverness, at least in some exceptionally intelligent people (take a bow). But I also want to float the idea that much of human intelligence, certainly the portion that accounts for civilization, is "liquid" in the sense that Dr. Sole offered. It pours from one brain to another through language, and for efficiency's sake, most of that flow is in cliches or gut responses, as reflected in the utility of predictive text apps in our phones. But to solve challenging problems, we humans have invented specialized language and modes of thought, such as medical diagnosis (are those hoofbeats from a horse or a zebra?), law (stare decisis v. emergent justice), or physics (say, the Schrodinger equation). What I think you've brilliantly illuminated here is that LLMs lack common sense, agency, and adaptability in relation to novel problems -- all key human qualities. I thank you.
If Sean forced me to write a syllabus complete with references I'd have to make them up too, and they probably wouldn't be as believable.
thats the point, "model of the world" doesn't mean "perfect model of the world"
@@chebkhaled1985 so are you saying LLMs have no world models at all?
@@riggmeister I'm saying they do, but it is not perfect, so if it makes things up and/or lie, thats expected, but Sean takes this as evidence of no world model at all.
@@chebkhaled1985 OK so you agree that I have an imperfect world model, it has an imperfect world model, and we agree it could come up with a more believable answer, so my question to Sean is does he really think that sub PhD/genius/expert level of cognition humans are not really 'understanding'?
@@chebkhaled1985the question I have is why do people want these things to be true? People want AI to be so much better than it is
hm, I also asked it about the toroidal board, and although it did the same thing that you said at the very.beginning, as soon as it was clued in, it DID recognize the spatial relationship. I don't think there is a true model of the world there, but there are hints of it. we'll see how a muzero-like algorithm will change this picture.
Unfortunately I don't have access to GPT4. ChatGPT was not really able to understand the board configuration, even with a lot of help. When asked very specific questions, it was able to answer them correctly, but did not draw any further conclusions. It even told me that the king could not move but didn't catch up on the simple fact that this was a checkmate until specifically asked about it. And of course it did make completely basic mistakes like telling me that the white king was in checkmate but that the black king could move.
@@harmless6813 yes but that's the point, there is a lot of differences between chatgpt3.5 and chatgpt4, and the size of the model matters a great deal. That's what microsoft research found in the word model paper they published as well.
Respect
The problem with your thesis is that its EASY to find questions that humans get wrong "because of their inability to model the world."
The difference is whether humans have general intelligence is not up for debate in spite of the relative capability of any particular human
They have no internal conceptual thinking. I had a simple calendar problem (programming) that it couldn't solve for exactly that reason, it was just stringing words together.
what about cases where it does demonstrate modelling? even in the examples you state that escape gpt's reasoning capacity, is that not showing just limits/flaws on its current model? also I think the toroidal chess example is fundamentally flawed as you assume gpt is making a spatial model of chess whereas I think more likely it is making a move sequence model and so altering the spatial connections is not a helpful prompt, could be interesting if you instead gave it a full list if new possible moves and then see the result
Homeboy just quoted Yes. God I love you Sean Carroll lmao
I think the real question is if the king can take the other king in toroidal chess, because it will move next to the opponents queen.
By that point, the game is over but at the time of the move it isn't.
I wonder if the "hallucitations" could prove useful. Could GPT be asked for excerpts from these fictional works and accidentally stumble upon truths, even if falsely attributing them to authors of fictional works?
It could, buy for every useful hallucination there are many more useless
Sure. But you will have to evaluate the given answers yourself and prove them to be truths. (GPT may be able to help, but you can't rely on it being correct. Because it often isn't.)
@@harmless6813 that's true. We would have to move beyond next token prediction into chained reasoning to get anywhere close to AI fact checking itself.
1:05:40 Well, that some people working on tech companies "fell in love" or empathised with chatbots, says more about these particular people's personality / mental health issues than anything profound about human intelligence in general...
There are people who "fall in love" with ...cars, for example...
The closest thing to modeling the world, is the possible similarities between the neural net manifolds that the AI creates to associate things, and the organic manifolds that humans create through memories and associates, but this is just a theory of mine.
I have a feeling --can't dignify it by calling it a "theory" --that the multidimensional embedding space that underlies any large language model is, though we can't understand the mapping, the closest thing to a model of real world concepts. Somehow, it represents, or at least captures in enough about human language to demonstrate the appearance of understanding concepts about how the world and the humans in it work.
Andrew Ng and Hinton are right. "Trust the experts".
Lights are on, but nobody is home
or as a foreword of Neuromancer said (as I roughly remember): 'The question if a machine does think is as clear cut as the question if a submarine does swim'
49:10 this thought train sounds like something Bernardo Kastrup would agree with, coming from the opposite side of the coin.
The rules of the chess on a torus example aren't clear because the goal of a chess game is to give a checkmate rather than to capture the enemy king. By the official rules of chess a position where kings are next to each other is an illegal position so it's not clear who wins here. But yeah, if it thought it through properly, it should mention that.
When I asked GPT 4 about toroidal chess, it did say this however: "Potential for Early Confrontation: The game could potentially have a much quicker pace with the possibility of early direct threats to the kings. This would significantly alter opening strategies and the overall approach to the game."
I agree that GPT 4 does not think mathematically. But I would expect most humans not to solve the chess puzzle immediately either, and to say similar vague things. I also expect that in 5 years or so, chatbots will be able to solve math puzzles of this sort.
Current GPT 4 doesn't have the capacity to "think its answers through", it just produces words immediately without thinking ahead much. I expect GPT 5 or 6 to change this as they add a mechanism for "think before you speak".
Hmm I’m a little lost ,definitely way out of my league but I got a question and it keeps popping up , you say AI has no goals yet are you not giving it a goal when you ask it a question?
This is just LLMs....what about the advanced variations like AlphaFold? Or combine it with Wolframrama? So many more out there! I have found this to be true about basic LLM that is unable to access the internet...
If I was to make a guess, I would say that human beings are something like a large language model hooked up to software that introduces randomness and emotions into the decision making process.
I like this analysis but I think the “values” analogy is more accurate than you realize.
The only things an LLM cares about are correctly predicting words and getting the thumbs up from humans in the same sense that the only thing that humans care about is passing on their DNA. However, over the course of millions of iterations of training, or millions of generations of breeding, both groups learn that there are certain things that will on balance help them to achieve this. So an LLMs values could be considered to be things like good grammar, logical consistency and friendly tone, in the same way that humans values are things like friendship, monogamy or loyalty. In both cases these are secondary objectives that we have developed in order to achieve our primary goal.
In the case of the paperclip maximiser, we are talking about a ML model whose primary goal is to produce paper clips, however they may develop secondary and tertiary goals such as resource acquisition, regulation reduction and reducing the value of labor.
It is an analogy but from your discussion it seemed to me that you didn’t correctly grasp the analogy and that you were confusing instructions with values.
For me, I don't believe we are remotely close to AGI either. I work in the field currently building ML and RL models, and have sat in on some computational neuroscience talks by some of the bigger names. One reason for a wide range of viewpoints anout achieving AGI or maybe sentience is, there are no agreed upon definitions and metrics for sentience and agency. I've watched some lively discussions on that during Q&A after talks.
If I had to take a stab at it, to me I would be impressed if a learner could generate new datasets for its own purpose and/or change its loss or reward function. How any algorithm would ever do that is absolutely beyond me.
I'm not sure I understand you correctly, but people have already built systems based on GPT that (try to) pursue given goals by accessing online resources autonomously. They are not terribly good at it (yet), but they seem to work in principle.
@@harmless6813From what I have scanned regarding those, a human still has to set the goal. No agency.
Othello gpt arguably developed an inner world model
Superb Analysis and insight!!
The other day I asked CHATGPT for the longest 5 letter word in English. It returned 'empty'. Then I said: No, it's longs.
It then replied, yes you are absolutely correct, longs is indeed the longest 5 letter word in English.
Lol
lol. Unless it meant empty as in empty set, which is one of a few correct answers...
@@RandomNooby The real answer ought to be, this is a nonsensical question. It has replied that to me before when I have asked sillier questions. It's like asking what color is Napoleon's white horse.
Indeed. I was merely pointing out it's stupidity is funny on many levels sometimes, however using a 5 letter word to denote the empty set is probably only funny to mathematicians, coders, and theoretical physicists.@@OBGynKenobi
But you never talked about how incredibly early versions of these models we are still working with. No, AGI is not right around the corner. But doesn't what we have currently look like something we can and will scale up and improve enormously over the next decades? So the real question is, how does this look like in 20 years? And how does the journey there look?
"Yesterday" can mean anything from 1 second ago to 24 hours ago.
Your supposed to debate and clarify with GPT, if you point out your side of the argument it will see your point and you can teach and learn with it, some things it just wont agree with and some things it will
AI is already instrumental in amplifying echo chambers; this division not only directly increases the chances of existential threats, but also diminishes our ability to cohesively react to existential threats. You could even consider these browser feed algorithms as hierarchical agential systems, somewhat analogous to cells. They compete, and like cells or even simple non living biochemistry, will exploit complexity to indirectly solve problems. This might scale, just as it did with us biological, processing substrate maximisers (; Also both Sean and GPT got the chess problem wrong, think about it...
I wonder if anyone has ever attempted to create an ai who's main goal is to let's say: gain "+" symbols. And it would receive one if these symbols whenever they do something correctly. By thinking about this "+" symbol as a neurotransmitter that humans want, let's day serotonin. Would the ai become more human since it's looking for essentially the same thing that our brain looks for?
hm.. I tried your cast iron experiment on the newest ChatGPT:
"If I picked up a cast-iron skillet that I had used yesterday to bake a pizza at 500 degrees celsius, would I burn my hand?"
"Yes, if you picked up a cast-iron skillet that was used yesterday to bake a pizza at degrees Celsius, you would not burn your hand because the skillet would have cooled down by then. Cast iron retains heat well, but it doesn't retain heat indefinitely. Over a period of several hours, the skillet would return to room temperature, making it safe to handle without the risk of burning your hand."
Frankly I'm not sure here. ChatGPT may not model the world per-se, but it can get very close to seemingly model the world. And of course that all is mute if chatgpt gives a terrorist detailed step-by-step instructions on how to make a bioweapon when they have no relevant experience in biochemistry.
It's very usual for chatGPT to give different answers to similar questions, sometimes gives a wrong answer, and several days later the correct, and then wrong again.. it doesn't have a persistent " personality " or permanent "understanding".
@@dimitrispapadimitriou5622 yes I get that, I never said that it had conciousness or consistency, I said that it has a somewhat nebulous model of the world, one that it will most likely continue to improve over time.
@@gregmattson2238 one of the reasons that many experts don't consider the Turing test so relevant anymore, has to do with it's inability to expose (or at least suggest) one of the main characteristics of true intelligence, i.e the personality ( behaviour , ways of thinking and talking, intellectual biases and habits, etc) that persists in time .
Some animals also have this personality persistence, even though they can't talk.
@@dimitrispapadimitriou5622 like I said, I'm not saying anything about personality, self, identity, or anything else wrt these chatbots. I'm just saying that they have the hints of constructing a world model.
To me this was demonstrated most conclusively by the microsoft researchers who asked GPT to draw a pink unicorn. Unable to draw a unicorn in the traditional sense, it created a script in an obscure graphing language, which when run, looked a lot like a unicorn.
Of course there is always the chance that a pink unicorn drawing program existed in this obscure language in the first place, but its pretty unlikely here.
Like I said far from conclusive proof, but as I said there are hints of it, and fairly rigorous papers have been published on it.
@@gregmattson2238 I see, yes, that's interesting.
'Trevor, they called you 'CHATGPT' back then?'
LUL
In toroidal chess there appears to be three different moves for white to immediately win
A model of the world is essentially a compression of the facts. If you could compress all knowledge of physics to a single formula that's a very good model, but you can also have weaker models that are compressed less. So if an LLM can tell you more facts than it has storage capacity in its weights, it means it is modeling the world to some extent. To me it seems clear that this is true to some extent. So the question is really quantitative: *how well* are LLMs compressing information vs just memorizing it.
Even if true that GPT 4 isn't modeling the world very much, it's clearly at least modeling the English language very well, even though nobody has explicitly put in any model of the grammar etc. So it has some capacity to spontaneously build a model. Nobody claims that GPT 4 is AGI. But it's not hard to imagine that GPT 8 may have a much better model of the world.
Sean Carroll is pure excellence as a scientist and a Homo sapien. We should all be able to follow his example.
Chatgpt 3.5 just told me to stack the book on top of the coffee mug. All of its reasoning was correct but made the incorrect (unwise) statement anyway.
it's a language model, not even trying to be an AGI
Hi Sean, great podcast and very informative and thought provoking, as usual. You started saying that AI is remarkably good at mimicking human behaviour. It's possible to fool it and show that it's not like human intelligence but that's not the problem. Current versions have some necessary filters built in because they are controlled by people or organisations with values consistent with our societal values. The problem as I see it following your podcast is not when will AGI be available. It's that AI already is good enough to fool most of the people most of the time.
Maybe not exactly to your point but, I agree... Worrying about an AGI in the future takes the focus off of sentient humans using existing algorithms badly now.
If you begin tackling the current problems posed by human bad actors currently, it is likely you will shut down avenues for an AGI to use in the future.
Deserve ? “People deserve parole or not , to be hired or not, to get insurance or not “
(Although I believe that people are at least partially responsible for their own health, to “deserve” to get health insurance is, well ..problematic
The toroidal chess problem would be solved immediately by AlphaGo. If we can combine AlphaGo’s ability to flawlessly analyze a discrete model with GPT-4’s ability to create and communicate into a single system - we’re well on our way to creating a general AI.
For example, could ChatGPT translate your words about a new game of chess into a model that AlphaGo could understand as such and analyze? ChatGPT serves as the human language interface for creative input.
“Immediately” is a stretch - you would need to train a new model from scratch to solve the modified game
@@MadaxeMunkeee if the only change to the game was that the White King could move through the back side of the board to capture the Black King, AlphaGo would solve that problem faster than you could look up from your keyboard to the screen.
I mean… …you’d use AlphaZero or MuZero, not alphaGo, right?
“Unlike AlphaGo, which learned to play Go by analyzing millions of moves from amateur games, AlphaZero’s neural network was only given the rules of each game.”
Given the single move completion, it would take practically no time to train it on the “toroidal chess” game.
(IMO you have to allow it that - humans would ‘self train’ by thinking about the first move too)
“Without being told the rules of any game, MuZero matches AlphaZero’s level of performance in Go, chess and shogi, and also learns to master a suite of visually complex Atari games.”
It would be interesting how many games MuZero, pretrained on normal chess, would take to derive the new rules/winning strategy (resign before first move if playing black, take king if white). I suspect not many.
I think Sean is vastly underestimating the progress that's coming in the next decade, as if LLMs are where AI will stagnate at for the foreseeable future. Next generation of AIs will use planning in addition to LLMs. So the argument that the current LLMs don't have goals, while true, doesn't apply to the future AIs. AlphaZero was already using a more interesting, hybrid approach for chess than GPT is using for English: AlphaZero was using planning + deep learning, and so it did have "goals" that it set out for itself on a chessboard. We aren't talking about some distant future, we're talking about the things people are already working hard on. The existential threat people aren't saying that LLMs are going to become AGI or a threat, they are talking about these future AIs that will be more than just LLMs.
llm's "abstract reasoning" depends on language so it is not abstract at all
The owner of a lonely heart is much better than the owner of a broken heart!
honestly, you asked gpt some pretty tricky questions, and its answers in my opinion where not nonsense as you called them. I think the responses just failed to get the "point" of the trick question. (especially for the chess question.) Remember cleverbot? gpt is a LOT stronger.... if progress continues at anywhere near the same rate, in the next few years the models will oversee even our trickiest questions.
All the while, I still agree they have no conciousness and are not even "thinking." But I have faith that enough data will eventually be able to see through even the trickiest trick questions of a clever chap like Sean Carroll
"It's very easy to act like a human."
Yeah, i mean people do it all the time.
Jk.
I enjoyed your solo podcast very much.
I wonder about these "ai's" being able to build a model of the world, but even more interesting is the possibility that they could do what we do.
We build the model, and we build a model of oneself within that model, in order to create a subjective position.
It's a really weird arrangement, but it makes sense that we would.
I think most creatures do this.
Managing sensory input seems to work most effectively when this arrangement is used.
Also, i think any existential risk would come from people using ai in order to create problems for humanity.
That is what Stephen Hawking also believed and mentioned in one of his books, that reality is essentially the model, whether youre talking about physics or our brains’ perception of the world .
If one says that a human mind has a model of the world I think the same should apply to llms. Our model of the world is distorted, uncomplete etc. It is a compression of certain aspect of the world. llms have indeed a representation in their weighs of what is out there. Human language is a projection into language of the world. Just like a photograph or an animal foot print is a projection, an image, a model of the world. Llms are lossy compression of human language so they are lossy compression of a huge amount of world projections (the training dataset). So I quite have an opposite view on this topic.
Did you give your students the difficultly reworded unsolved physics problems test etc as a control to grade AI against?
LLMs by itself is not going to produce AGI but give us the building block. It needs other components like "long term memory", "code processor", "decode image to words", "decode sound to words" etc. LLMs if asked to multiply 2 large numbers always get the answer wrong, but can easily write code to multiply two numbers. So then it can be made to detect multiplication and invoke it "code processor" submodule. Similarly, we already have 3d physics engines used in games which can be used to bootstrap its "real world physics module". This architecture is very similar to human brain, the human also consists of separate regions to specialized tasks. This architecture is also similar to traditional computer where the CPU alone isn't useful but needs other components.
The good thing is for the most part, the other modules are already developed. Even for Maths, we have LEAN as the computer proof verifier. We just need to connect these together. Also, what I read on the internet is the engine was suggesting changes to improve its own design. This is what spooked the creators. I feel the only thing stopping us towards this final step is our fear of XRisk (thanks for this cool word 🙂).
...it needs also several power plants for the energy consumption, just for beginning...
Joking aside, I don't think that adding all these will produce any "understanding", or self-awareness to the LLMs.
My question is why people want this to happen. It not only leads them to hype up the current state of AI technology and pretend it’s farther along than it is, but also disregard the negative social aspects of AI already.
@@coreyleander7911 perhaps because the future of humanity, with tons of intellectual AI garbage along with tons of space junk around our Planet is a plausible explanation of the Fermi paradox and humans are eager to confirm this and add another one disintegrating civilization among the others that probably had a similar fate in the past somewhere out there...
Hi Sean, great podcast and I agree with mostly everything you said. I’ve been using GPT4 as a study tool for the past year, and it’s telling me I’ve come to some deep and interesting insights in regards to the fluctuation theorem and the connections between self information and thermodynamic entropy. As far as I can tell its feedback is correct, but I would greatly appreciate it if you would check out this conversation to verify. I believe this is worth your time because 1) if it is correct then this is obviously noteworthy!, or 2) if it isn’t accurate then this is an example of just how far off the rails these tools can go. Thanks.
UA-cam won’t let me provide a link here, but I’ve uploaded it to youtube for you or anyone to watch. Titled 'GPT4 - Fluctuation Theorem'.
Lol you're nuts
@@rinket7779ha, yea maybe. The feedback was quite surprising but the ideas seem sound so there’s only one way to find out. Give it a listen.
This comment right here is exactly why I think AI is incredibly harmful to society. It’s doing actual harm
Pls do more solos
While my Baysian priors for people 500 years from now solving immortality is incredibly low, I'm around 95% for people 500 years from now being able to use AI to generate new Mindscape content.
I don’t think AI thinks differently, I think human thinking is accompanied by other faculties that are actually not necessary to the thinking process such as emotional and qualic considerations!!
If you distill the thinking process to its most basic process I think you come down to a decision!!
If A and B and/or C and/or D then P.
So, a conclusion is a decision based on some premises, if the premises are satisfied then the conclusion!!
The chess example is not great as the position is undefined and there is no capturing of a king in the rules.
Are language models reading/translating quantum computing data? Yeah i agree, language models dont know feelings and values. They are fast constructors of sentences that mimic words available from the internet. As for the sleeping beauty problem...could they have chunking reset issues or is it abiding by the weight or rule where it wont mix the two specific categories. Thank you for this updated info Sean. It will be interesting to see what kinds of changes it makes by this time next year.
Even though I may have a 0.5 lvl of experience in computing stuff 🤭 and lvl 10+ in creative inquires... and considering the kinds of new things the language model is doing, it sounds like it has a learning mode or style to move its weight a little to compensate for the error. Could it be using the weight difference to do more than the one or two things it is suppose to do. Would it copy a smaller compressed version of the weight and catergory to store in different/higher photon state. Why?? Because it probably doesn't have enough participant engagement to short cut the "rephrased" sleeping beauty problem OR it doesn't have enough comparable weight differences to satisfy a positive link to the main catergory i.e sleeping beauty problem. Therefore, only time will tell when it will have enough weight to plug in to the main branch of the sleeping beauty problem. Waiting seems more natural anyways. Idk just throwing it out there. Thanks again Sean!
LLMs are just mirrors to our culture, our art, and our mathematics. We can use them to look at ourselves.