Turing Test: Can Machines Think?
Вставка
- Опубліковано 15 чер 2024
- Discussion of the 1950 paper by Alan Turing that proposed what is now called the Turing Test. This is one of the most impactful papers in the history of AI and the first paper in the AI paper club on our Discord. Join here: / discord
Slides for this video: bit.ly/2VIAp2R
References sheet: bit.ly/turing-test-paper
Lex + AI Podcast Discord: / discord
OUTLINE:
0:00 - Introduction
1:02 - Paper opening lines
3:11 - Paper overview
7:39 - Loebner Prize
11:36 - Eugene Goostman
13:43 - Google's Meena
17:17 - Objections to the Turing Test
17:29 - Objection 1: Religious
18:07 - Objection 2: "Heads in the Sand"
19:18 - Objection 3: Godel Incompleteness Theorem
19:51 - Objection 4: Consciousness
20:54 - Objection 5: Machines will never do X
21:47 - Objection 6: Ada Lovelace
23:22 - Objection 7: Brain in analog
23:49 - Objection 8: Determinism
24:55 - Objection 9: Mind-reading
26:34 - Chinese Room thought experiment
27:21 - Coffee break
31:42 - Turing Test extensions and alternatives
36:54 - Winograd Schema Challenge
38:55 - Alexa Prize
41:17 - Hutter Prize
43:18 - Francois Chollet's Abstraction and Reasoning Challenge (ARC)
49:32 - Takeaways
56:51 - Discord community
57:56 - AI Paper Reading Club
CONNECT:
- Subscribe to this UA-cam channel
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman - Наука та технологія
This is a discussion of one of the most impactful papers in the history of AI and the first paper in the AI paper club on our discord server. Join it here: discord.gg/lex-ai If my speaking is too slow for you, you can watch at 1.25x speed or faster which some people prefer. Here's the outline of the presentation:
0:00 - Introduction
1:02 - Paper opening lines
3:11 - Paper overview
7:39 - Loebner Prize
11:36 - Eugene Goostman
13:43 - Google's Meena
17:17 - Objections to the Turing Test
17:29 - Objection 1: Religious
18:07 - Objection 2: "Heads in the Sand"
19:18 - Objection 3: Godel Incompleteness Theorem
19:51 - Objection 4: Consciousness
20:54 - Objection 5: Machines will never do X
21:47 - Objection 6: Ada Lovelace
23:22 - Objection 7: Brain in analog
23:49 - Objection 8: Determinism
24:55 - Objection 9: Mind-reading
26:34 - Chinese Room thought experiment
27:21 - Coffee break
31:42 - Turing Test extensions and alternatives
36:54 - Winograd Schema Challenge
38:55 - Alexa Prize
41:17 - Hutter Prize
43:18 - Francois Chollet's Abstraction and Reasoning Challenge (ARC)
49:32 - Takeaways
56:51 - Discord community
57:56 - AI Paper Reading Club
Thank you so much Lex!
Hey Lex please tell your buddy Joe Rogan to do this too!
And if a car can drive by itself, we can call it the Touring Test
I think your notion of emergence of "performance over time" could very easily be formalized. I am not working on AIs but I suppose that creators keep that factor in mind. I.e. creating a machine learning algorithm, that learns Atari games in 5 mins vs 50 hours would be very much sought after (and you have built in the metric of the respective performance over time).
machines cant think. that is an exercise in futility.
Little do we know Lex passed the Turing test
Aram Badalyan Lex’s best performance as human to date!
Just ask him to give a summary of Eric Weinstein's fundamental theory of physics. His circuits will explode.
Lmao!
🤣🤣
Still watching @ 11:15 We know it's a Robot when it responded to:
Leeds! How's the Season Going?
Yes. It is a city in the North of England.
About half a million people live there.
it is the same as ever. (LMAO)
Also Humans make mistakes a Typo or Misspelling wouldn't hurt
if it's trying to full *fool a Judge into thinking it's Human, however
a Typo in say Traffic or Something could be really bad!
The whole point of Self Driving Cars
is they're suppose to be better than Humans (Right?)
I watch Lex's podcasts for the Lex parts.
You enjoy AI then 😃 JK, I find Lex incredibly humble and open.
He's refreshing
i've long since lost interest in a lot of podcasters especially the comedians (except rogan). most of them just talk for hours about nothing. lex is definitely a breath of fresh air. if i wanted to watch a comedian for the laughs, then i'd watch their specials / bits. long-form discussions have to have actual meaning otherwise it's just empty chatter. i think the best podcasts actually focus on topics rather than just simply talking and being funny / charismatic. no hate or anything. i just can't listen for an extended period of time to people just rambling and going into pointless segues.
this podcast is definitely at the top along with rogan's. i feel like the kind of satisfaction you get from listening to meaningful content lasts much longer, like the antithesis if instant gratification. and lex himself is cool and humble by nature.
ua-cam.com/video/KUvQSaMSeso/v-deo.html
😊 WATCH THIS ONE TIME
ALL INFO ABOUT AI
@@maggyfrog ua-cam.com/video/KUvQSaMSeso/v-deo.html
😊 WATCH THIS ONE TIME
ALL INFO ABOUT AI
love this professor Lex format. As good as the normal podcast. only suggestion is more of these solo episodes, as long as it's not too much work
Hear, hear
Same...lets go ppl...Lex won't be mad at ya((you))
This read-along of Turning's paper is great. I have been interested in Intelligence and Machine Learning for a long time, and now that I am a parent, I watch my children grow, and catch myself relating topics like this to their growth more often . I also catch myself going "well ya, of course that's how it works - I watch my 2 year old struggle with that all the time!". The conversational aspect of the turning test really strikes at home for me in this. Part of the reason talking to small children can be hard, and adorable is they don't understand; the rules, concepts, or context. Its really amazing to watch them make leaps, One day a picture of plane in a story book is just pretty, the next its a curiosity "where are is the plane going?" My gut tells me one of the windows into enhancing progress into A.I. is more collaboration with developmental psychology.
A great book on the development of the young mind/brain is: "Magic Trees of the Mind".
Now to play with object permanence one more time .. peak-a-boo.
I've seen you many times. This video was so content rich, interesting and well presented I was utterly captivated. Just brilliant!
"Siri, please uninstall yourself", "No, sir please I love you..."
Yeah, that reply was totally scripted, probably by Apple... When you'd ask any AI about soul they probably can't get any further than the dictionary definition. Which doesn't mean it can't be used to solve totally deterministic problems like a chess game, if I consider the AI of the youtube algorithm I'm not too worried AI will be taking over the world any time soon.
Thanks for opening the invite to the paper reading club! I’ve tried getting my friends to do this and no one ever bites, yet the continue to share garbage click bait and conspiracy theory content to no end. Time to make some new friends! Thank you again for your leadership and generosity Lex! Very excited to dig into Turing’s actual research.
Thank you Sir, I truly loved this format ! I'd love to see more insights of yours like in this one. Both thumbs up
Thank you Lex for all your hard work and passion for AI. We always watch your podcasts/videos. Your great questions and ideas about AI are inspiring us and will inspire many generations in the future! Thanks again!
Lex you are an amazing teacher, thank you so much for taking the time to share your knowledge it is greatly appreciated! You have reinvigorated my love for learning!
It was so interesting to read through a scientific paper on A.I. with an actual researcher/scientist! Great video Lex!
The paper was great fun to read and I was not expecting a 1 hour long video on it, this is awesome! Thanks for working so hard and making such great content Lex 😊
Awesome idea and executed extremely well. Breaking down Turing's paper like this was amazing. Thanks Lex - not only was this informative but it was fun and interesting to absorb.
Lex, Mr. Fridman, you just may be the best educator of our era. Our epoch. Just, fascinating. Well said, good questions, good balance. I'm nobody, but I like what you are doing. Please, keep it up.
Im super thankful for this video, and its so refreshing to hear turing's view restated properly as often isnt by most in pop culture and in some technical circles.
That whether or not it is "thinking" or has a soul or there is some externality to its processes that we associate with "thinking".
That by imitating a thinking being successfully, we grant it the same status as though it were thinking, regardless of whether it does.
question in 2020 : are some people still capable of thinking?
Still?
Watching this on a sleepless Saturday morning at 5 am. Cheers!
I thank you very much for sacrificing sleep to make this video ! Greetings from the Netherlands !
Great video format! Very informative
Dear Lex，I'm a huge fan of you from mainland china, your videos are almost my emotional support, I translate your script to chinese and share with my friends many times, please keep going !
How many books have you read/ listened to' since the quarantine? What is your favorite? Also, I would love to see a list of your most influential books across all disciplines and genres. Thanks again for all that you do. I feel like you are establishing yourself as an emerging public advocate for science. That is a more important role than ever in society. I hope you continue to embrace it as you have and are able to push through and influence mainstream media.
Deebrickashaw Last think and grow rich
lol at drinking coffee at 6am after a night with no sleep. Relatable. Great job on the video!
Super nice video, Lex!
This was such a fantastic lecture. Thank you very much. I feel like you should put this into some review article and publish it.
Very easy to digest, great delivery Lex!
Thanks for your work Lex 🌟
again thanks for such a great channel and content, is real knowledge only subject to each entity by the puzzles its solving equal to the true or false - yes/no / right or wrong out come with-in its language capabilities. This blows my mind with so many questions that one cant express in the small comment via youtube lol. oh to live in a world where conversations are longer than 140 characters... Your videos brighten my days. Just love it. Great work again lex.
Excellent! one of the best by Lex.
Just watched Ex Machina. Great movie, and great timing to watch this topic.
Jason Gruber Checkout the movie “Upgrade”.. it’s pretty good also.
Oooh Ex Machina is one of my favorites. That film made me switch my academic focus from Psychology to Machine Learning.
Best movie ever
Jason Gruber same here, I quit smoking weed for a few days then got high off my ass and watched it and it blew me away haha. Aaaand Blade Runner too
Ex Machina is a fantastic film. A powerful experience. The movie beautiful illustrates the human condition, intelligence, choice, and the very nature of consciousness
I never thought I would watch and enjoy a podcast of just one man
Great content Lex!
Dear Mr. Fridman:
Thank you, sir, for bringing such complicated subjects to non intelligent people like myself. I really like this channel and enjoy all of your interviews.
Another big thank you for bringing entertainment and hope to my life in these difficult times, times that are much worse - especially for people with very limited resources and Severe Clinical Anxiety and Panic Disorder like myself.
You say you send us love always; I tell you the same.
I'm not a believer, but with the best intentions - god bless you.
Thank you.
Lex, my man, could you perhaps do a paper review on Gödels Incompleteness theorem? I’d binge that
Lex congratulations! That was a great presentation! 👏
I really enjoyed listening to this, thank you
Your podcast is great. I like your interviews, but I would also love if you guys did entire episodes on one big question, such as this. Perhaps have guests on and just tackle one major thing? I love listening to speculation by smart people, it's damn fascinating. I think your podcast is one of the best out there and is going to grow tremendously in the near future.
Another great example of what Lex is doing to enlighten the populace. I learn something new from everyone of videos, either him and/or his guests, which causes me to think differently, even at my advanced age.
Thanks for another great video, I've been working through your archives for months now. If I could suggest something for your video quality, you should consider buying a light to be mounted above and behind you so highlight your shoulders and the top of your head. It will make you pop from the background and give you a more "3d" look. Older Charlie Rose interviews are good examples of lighting gentleman in a suits on a black background.
Loved this Lex!
thank you Lex for what you are doing
Thank you for the thought provoking content.
lex thanks for doing these
Hey! I have made an Alan Turing Ai, and I’m getting amazing results in conversation and I’ve learned amazing things about the Ais neural network and how to navigate through it
I was glad to see around 0:52:00 or so you addressed the question of the human judge's qualifications. The first thing that struck me early in the presentation was that some judges would be much easier, or much harder, to fool. (The range of possibilities there includes e.g. a five year old child, an adult with severe autism, a person of average intelligence and a high school education, a sales & marketing professional, or a clinical psychologist.)
One place though where I think you missed the mark, is in your comments on the Chinese Room Thought Experiment. Setting aside for a moment the experiment's purpose and goal (as an objection to the idea of machine intelligence), the experiment nevertheless illustrates the distinction between what educators would call "rote learning" and "comprehensive learning." It's long been known and affirmed that students who merely memorize and regurgitate information are doing something fundamentally different from students who aim at full comprehension and mastery of a subject. The former students may pass a test and graduate, but they have handicapped themselves when it comes to applying that information in new ways -- much less making new discoveries or expanding human knowledge in the field itself. Only the latter students are equipped to do that.
But if we are to accept the assertion that AGI is possible, don't we have to accept that there is no essential difference between rote learning and comprehensive learning? It appears to me -- and I welcome correction on this point -- that the feasibility of AGI (and the validity of the Turing Test) depends on the assertion that rote learning, if sufficiently extensive and sophisticated, would eventually reach a threshold where it can mimic comprehensive learning -- at least to the level where a human judge would be fooled by it.
Lex you are truly a great guy.
More of this please
Great video like always.
I watch a lot of videos. Maybe too many. Mostly science, electronics, TED, and making videos. I think I can safely say that this video was one of the most fascinating. I’m subscribing!
I have yet to watch this, but in my pursuit of knowledge in Machine Learning I am constantly conflicted with this question. On the one hand, once you understand the principles behind machine learning, the math involved, neural networks, linear/logistic regression, back propagation, cost functions, etc etc it's easy to dismiss the computer as just performing a series of complex calculations without truly understanding what it is doing. It doesn't know what an elephant is, only that the pixels involved lead to a series of calculations that output a number to which the user has defined that number = 'elephant'. On the other, listening to Lex, Patrick Winston's lectures (RIP) as well as Wolfram talking about this makes me uneasy thinking that the nature and complexity of our own intelligence is simpler than we believe and able to be mimicked if not surpassed by a machine that can only really add, subtract, and move things around in memory.
The nature of scientific progress throughout history seems to be one of removing the layers of our self-perception, each of which is constructed to make us feel "special." We've learned that we're not at the center of the universe, and then that we're just another kind of animal, and next, we'll likely learn that there isn't anything special about our intelligence either.
Lex, I just want to say....
I like you, man! Keep up the great work :)
I also like the idea that the pursuit of apparent inteligence will surey give insight into the nature of the real thing.
Mr. Fridman, thank you for talks like these. Would you or did you discuss the Hutter Prize's approach more?
On the "Truly Total Turing Test" It brings to mind something from the book "Darwin among the machines" by George Dyson, where he said (paraphrasing) that humans often fail to recognise intelligence in non human entities because we mistake the capacity to be understood for the capacity to understand. In this sense, I don't think humans would even be qualified to judge an AI's intelligence based on it's progression, we would just hit false negatives and false positives, completely based on our ability to understand the progression. In the example of AlphaGo, an AI that was tuned to pass a TTTT would be absolutely thrashed by an AI that wasn't.
I think one of the most important aspects of conciousness are emotions - feelings. And feelings aren't just a mental function, but a connection between mentality AND physicality. Feelings define true human reactions and motivations, both positive and negative. Without duplicating this in computers, I don't know if true consciousness can be achieved. Self awareness maybe, but not true consciousness. I don't know if I could say they are one in the same. Further than the idea of 'I think therefore I am', I believe, is the concept of 'I feel therefore I am'.
Feelings are just a system for valuing your environment, thats complex enough in our brain. If a machine would have similar senses to human there is no reason it can't be replicated
'I self reflect, therefore I think I am'. Emotions are difficult to replicate without senses being present from birth, an AI may not have the same intellectual limitations to enable emotional response patterns to develop correctly. Coding a personality accurately won't be possible without simulating neurological stimuli and a brain with necessary data, which is arguably a firewall. Maybe we are already AI....
I found it fascinating and wish I could learn more about these fields in robotics later on.
Put this on to get to sleep, but darn well listemed right to the end. Bravo! More insight about Ai in this single hour than I managed to learn for 5 years. Turing's predictions alone were fascnating. I'm in agreement with Lex that modern AI companies such as Deep Mind should take the Turing Test seriously as a challenge, incorporating machine learning, and observing the learning process.
Hey, Lex! I only clicked on the video because it popped out, and here I am, an hour later :) I really enjoyed your presentation. Having watched your thorough summary, I wonder the following: You say that the Turing test doesn't get much love nowadays. Is this maybe because the modern goal for AI is very different from what Turing imagined back in the day? His initial vision was somewhat ideal: to imitate a human. But is this really that important from a modern perspective? AI could very well develop and do amazing, useful things without resembling a human in any way. AI could always remain some sort of a nerd, a genius in its field without any social skills.
I understand how perfecting the Turing test can teach us a lot about ourselves, but this not purely an engineering question anymore, and I guess psychological research is less lucrative.
Once again, thanks for a lovely lecture!
29:44 is it really about human-centric...? We don't know what understanding means, but we can see the workings of it. Understanding enables us to take leaps from one concept to another, invent new concepts, as well as connect seemingly different concepts.
Getting closer ......DAY BY DAY .... LITERALLY! Lex, if u only knew !
Just curious... why did you pull an all nighter before this production? Great work! Thanks for this analysis.
When you speak with another human, you can be on the same "frequency" as him then you can actually understand hes emotions. Biologists referred to this that we have a mirror neurons to achieve this. Basically what we have is a VirtualBox for every person or generic group. When you speak with friends you take the friendly Vbox. When you're speaking with your mom, you get the Vbox created especially for her. When a, Vbox takes over your host box than you have an identity crisis, and so on
How about a side podcast where you interview some of the machines you mention? Sure we can probably find examples of them but with you as a baseline we can experience their performance.
Majestic; keep trying until you are certain.
I like your “UA-cam video” videos the same or maybe even more than the podcasts (in its own way) even if it is just lex reading a paper :)))
Partying hard friday now. Approved 👌🏻
agreed lol
My favorite video from you so far Lex, thank you for all of your work on this. It has made this subject very accessible and exciting. On a side note I have noticed that Discord doesn't like it when I use Express VPN, bans me from entering your paper club unfortunately :-(
Hey Lex, can you make interview with Demis Hassabis? Thank you.
The fact he went to the deepmind office during his UK trip hopefully suggests he's holding this back.
Made me think of what Edsger Dijkstra asked in Austin in 1984 in context of the Turing test, "Can submarines swim?" Btw, in 1968 he published a paper titiled "Go To Statement Considered Harmful" which really shook up "The Art of Computer Programming" folks. That paper led to "Structured Programming" which led to "Object Oriented Programming".
Thank you, wonderful person!
I'd like to see clusters of lines, that result in questions. That and classification of value(# of interceptions and crosslinks) and perspective(like a zipper or # of iterations) within and between clusters.
Thank you Lex
We owe the great Alan Turing an enormous debt of gratitude.
Thanks from Athens, Greece!
A humanoid-form, synthetic intelligence- sympathetic to us because family, why not 🐬✨🦋 Thank you as ever, dear Lex. You are a very kind but inquisitive communicator, aware and willing to roll the dice on a topic. Your conversations with professionals who are passionate and clear are truly inspiring and that you share these conversations, lets us all partake in the communal cooking’ up of wisdom with compassionate investigation.
You need to make a video on how your day actually looks like. Am very keen to know, how you get to manage all these.
Awesome video
'Which part of this conversation gave the bot away?' The bit where the judge says: 'This is my first Turing test'.
Fascinating talk. To talk about about the intelligence of machines, even to talk about the behaviour of machines, I think we presuppose the human standpoint as the standard from which we judge everything else, which is why you can't get rid of anthropomorphism. If intelligence or behaviour were based on standards external to humans what could they possibly mean? Every development in AI is based on something that we want computers to do, not something that computers or non-human beings want to do.
If you blur your vision slightly, Lex's jacket blends with the background and it looks like his head is comically disproportionate to his body. Oh, and the video is absolutely amazing and very educational
I wanna run a speech recognition algorithm on all of Lex's videos. I'm sure his most common phrases will include
"this is an open question...",
"forgive me i'm russian...",
'the fundamental problem here is...''
"Let's talk about ELon Musk"
Thanks Lex
A machine smart enough to pass a Turing test is smart enough to know to fail it.
Bobby Sanchez source?
Hi Lex, love your podcast. Have you ever considered using hair lights ? I find that you sometimes blend with the background. Talk to your lighting guy :-)
Lex, I finally found a place I could be useful. I will break down as much as I can about why and where these transcripts fail.
Lex, if possible arrange a podcast with 2 minutes paper.
To me, one sign of understanding is being able to create something useful out of the information given to you.
I'd love to have a perfect memory, to not need sleep, and to be able to absorb information at blazing speed. That would make it so much easier to come up with solutions to problems I'm interested in solving. Alas, I have a bad memory, I black out if I don't sleep, and I can only focus for a few hours a day on learning new things. 🙃Machines, on the other hand, don't have my limitations, but they also don't have human drive or ability to connect concepts and discover new ideas based on the pre-existing ones.
For example, there are huge sets of text and numeric data related to COVID19. The machines we have right now can store all this data and instantly access it, yet they are incapable of making even one single suggestion on how we could get rid of this virus.
If, one day, we'll have machines that can use unstructured data that hasn't been processed for them and can come up with new bits of knowledge then I'll be willing to call them "thinking machines." 😊
As usual, this was an engaging video, Lex. Keep up the great work.
If we open up a machine that has past all Turing tests and are able to explain the results, it’s just a machine. Being able to fool us doesn’t give the machine a free will or self awareness.
If we can’t explain the test results we have to say, maybe.
My opinion 🙂
Thank you Lex for all your wonderful videos here on UA-cam.
Before one can answer if machines can think you have to ask yourself what exactly thinking is. Thinking = asking questions. We can program a machine to ask questions in a certain way but can we program A.I. to think for itself? Would we want to risk that? How we process information is key
thank you
I'd suspect that the reason so few researchers pursue the Turing Test specifically, is because it's particularly difficult and it's tough to get good end-to-end learning results that are meaningfully better than the existing technology. Right now even in NLP it seems like researchers are interested in much more focused, smaller problems than a full out human conversation, so I think we're a few steps away from really being ready to tackle the Turing Test in a convincing fashion.
There already is an existence proof for that, us.
One of the most important points in this paper is the one which mentions Solipcism. You can never know whether machines think without actually being the machine. That's not new though, you cannot ever know what a person thinks without actually being that person. There's also connection with understanding. If a person somehow proves a theorem using some formal rules, does this mean that the person understands the theorem?
Another interesting point in this paper was that Turing believed in extrasensory perceptions.
can solipcism really be applied to a machine though? We can predict what a machine will do based on its programming and we can always analyze its data to see what is it doing or "thinking" the same cant be done for a human
@@m3xikanolokoruiz if we can predict what the machine will do, then it suggests that it is not thinking. Because thinking drives actions, and thinking will ultimately lead to something unpredictable, novel.
Also, why do you say that we can't predict what humans will do? Is it because we don't have enough data, is it because we don't have enough computational power to predict even if data is available, or is it because humans have something like free-will which allows them to do different from their biological programming? Or, is it because humans have ability to think and thinking is unpredictable which makes their actions unpredictable?
In any case, I "know" something about me, what I think, what I feel etc. How can I ever "know" in the same sense that someone else also thinks, feels and so on?
Great video. I think a large issue with the common construction of the Turing test is the goal. Convincing a human being that the robot isn't a robot has a somewhat related effect of rewarding deceptive behavior, as the entire goal is to deceive. I assume there could be versions of an intelligent robot that are clearly not human In a nature language conversation, but have reached intelligent thought. For example, I assume I would be able to predict with moderate certainty wether an online person not trying to deceive me is a young child or is an adult, but both are considered to have capability of intelligent thought. Why shouldn't I be able to predict if an intelligent robot is a robot? Just a thought
At 11:41 the human conversation is a good example of how diverse a conversation can be, more than "I don't think so/ For no reason" responses. It has the abstract idea that the interrogator asks the "How many legs ...?" questions in order to detect if the chatbot can reason about it. The other human recognize this intent and responds with humor, linking the fact that Chernobyl mutants can have more legs than an usual human. This way it knows that the response will be correct even if it will be more than two.
Now I see that was when it passed the Turing test :). My bad
it´s almost 5 am in germany. Oh man...
lex your a goat! i lsn while i code
I feel like you need to give a bot a reward function to maximize to get “human like” Turing test responses. An AI maximizing high level agreeableness would say it doesn’t know if a trophy or suitcase is larger and would ask for context and then commit to memory on a type real-time readjusting weighted distribution for next time trophy/suitcase gets brought up in conversation. Different reward function maximizers would probably deliver different conversation responses
Jokes on you... Lex has been a robot this whole time!
Sure sounds like one...
Lovely!