The Sentences Computers Can't Understand, But Humans Can
Вставка
- Опубліковано 24 сер 2024
- The Winograd schema is a language test for intelligent computers. So far, they're not doing well. MORE LANGUAGE FILES: • Tom's Language Files
Written with Gretchen McCulloch and Molly Ruhl. Gretchen's podcast Lingthusiasm is at lingthusiasm.com/ - and Gretchen's new book, BECAUSE INTERNET, is available:
🇺🇸 US: amzn.to/30tLpjT
🇨🇦 CA: amzn.to/2JsTYWH
🇬🇧 UK: amzn.to/31K8eRD
(Those are affiliate links that give a commission to me or Gretchen, depending on country!)
REFERENCES:
Levesque, H.J., Davis, E., and Morgenstern, L. (2011). The winograd schema challenge. In
AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning.
Trask, R. (1993). A dictionary of grammatical terms in linguistics. London ; New York: Routledge. (page 233)
Winograd, T. (1972). Understanding natural language. Cognitive Psychology, 3(1), 1-191. (page 33)
Hunston, S. (2002). Corpora in Applied Linguistics. Cambridge: Cambridge University Press.
Jurafsky, D., & Martin, J. (2009). Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition (2nd ed., Prentice Hall series in artificial intelligence). Upper Saddle River, N.J.: Pearson Prentice Hall.
Gray, M. & Suri, S. (2019) Ghost work. Boston, M.A.: HMH Books.
🟥 MORE FROM TOM: www.tomscott.com/
(you can find contact details and social links there too)
📰 WEEKLY NEWSLETTER with good stuff from the rest of the internet: www.tomscott.c...
❓ LATERAL, free weekly podcast: lateralcast.com/ / lateralcast
➕ TOM SCOTT PLUS: / tomscottplus
👥 THE TECHNICAL DIFFICULTIES: / techdif
And that's the last in this run of the Language Files! There may well be more later in the year, but for now: thank you so much to co-authors Gretchen and Molly. Pull down the description for more about Gretchen's podcast and book!
how the f*** is this posted 1 month ago?
Tom Scott it says for me pinned 1 month ago and it’s been out for 3 mins...
A month old comment in a minute old video.. Classic Tom!
Thanks Tom, Gretchen and Molly. Very cool!
Tom why do you keep time traveling to make these comments
"Siri call me an ambulance!"
"You're an ambulance."
😂😂😂
Lmao
"Siri please i'm dying!"
"hello dying i'm Siri."
Lmao
😂😂😂😂😂
Things I didn’t expect to be mentioned in a Tom Scott video:
•Slash fiction
•Overwatch
Haha
weird that I'm working on overwatch slash fiction in another tab
"Large things can't fit inside smaller things."
Shaun Cheah, according to slash writers? Yes, they can.
•AI Dungeon
"Large things cannot fit inside small things."
Cats: *And I took that personally*
@@darthvader6198 A cat is fine too.
There’s a way
@@darthvader6198 if there's a goal
Which happens to be a hole
😏 hmmmmmm....
THAT'S WHAT SH-
You: "Put the trophy in the suitcase"
Computer: "Ok pulling out gun"
You: "No"
Computer: "Ok putting away gun"
You: "Open suitcase"
Computer: "Takes out gun and shoots suitcase"
i don't get it
@@Mister_Sun. Neither do i
"I saw the thief with my bicycle" - the thief has my bicycle.
"I saw the thief with my binoculars" - either I or the thief might have my binoculars.
"I saw the thief with my bicycle with my binoculars"
"I saw the thief with my binoculars with my binoculars"
You did not just use alot of binocular(s?) to see a damn thief
@@Liggliluff You have two sets of binoculars?
What about the fact that it is very difficult but, not impossible, to saw the thief with your bicycle?
Edit: it is much easier if you use just the chain.
The thief saw me with my own binoculars.
When you have to re-read a whole paragraph in a fanfiction because at the end of it you find out you were imagining the character roles reversed... and now you have two parellel scenes visualised.
oh God, when the author doesn't make it clear which character is speaking in a scene and your left wondering why Sub would be saying something that only applies to Dom
lmao visualising while reading? pfft casual
aphantasia gang
When they just leave what the characters are saying without clues as to who is saying what assuming I will pick up on who is who. Excuse me, but I'm dumb.
When you realize 5 chapters later that that one very important line was actually uttered by the _other_ character, completely changing your context for the last 5 chapters...
@@slaughterround643 yea boi!
I’ve tried the trophy-suitcase problem on Chat GPT-3 and it’s astonishing how much this technology has advanced in just three years.
gpt 3 was actually released later in 2020, after this video was published, so its been that was since roughly the video came out.. GPT4 came out couple months ago, and if you take a look at the timespans between each GPT version, its around 2 to 3 years, so it only makes sense
@@petrkdn8224I tried the tablecloth example with GPT-4. Not only did it provide the correct answer, it also rephrased the sentence in order to make it clearer.
Yesterday I put those sentences into OpenAI Chatbot, and it successfully answered all of them.
Turns out machine learning WAS gonna save us, less than 3 years later. Checkmate Tom.
I tested asking it to put the suitcase inside the trophy. It stuffed the suitcase inside, and admitted it was a silly idea. So yes, it is extremely good at understanding this stuff.
@@Jason9637It could also be because it was asked that question by others who watched the video so it learned from it
I asked it about the trophy sentence and it got it correct. Then I switched from “too big” to “too small” and it gave the same (wrong) answer. ChatGPT is good, but not intelligent.
He just made a video :)
@@krishp1104 ChatGPT doesn't learn from interactions.
tom: "put the suitcase in the trophy"
AI: "i'll get the gun"
T800: "A phased plasma rifle in the 40 watt range".
AI dungeon gets confused sometimes. And when it does, it usually resolves that confusion by injecting violence.
So it's an American AI program.
@@TheGregamonster remembering the time the characters were in court and one of the characters just pulled out a gun??? Even though they never had access to one?
@@brandonfrancey5592 😂
Put the suitcase in the trophy.
Ai: pulls out gun
Then I pull out my robe and my hat
PUT IT IN, NOW!
So anyways i started blasting
If that AI is ever given a physical body we are in trouble. "Pack my suitcase, will you?" "Certainly. *BANG*"
@@hayden.A0 Oh, he's packing all right.
Tom: artificial language processing remains ten years away
GPT-3: hold my beer
GPT-4: hold my beer
@@resyntax Yes, but on a sample size of one, it was correct. I almost said it was well reasoned, but it claimed to have done it off the sentence structure.
@@resyntax and what humans do is not bullshitting?
@@DiggyPT GPT-5: Actually hands you the beer
GPT-6. Discovers a new way to make the perfect beer, sets up a business, codes a flawless website, gets legally set up in a few small local areas, gains funding from investors, expands and gradually takes over the entire beer industry.
Feb.19.2023
I asked chat gpt:
"The trophy would not fit in the suitcase because it was too big." What is the sentence referring to by "too big"?
It replied:
In the sentence "The trophy would not fit in the suitcase because it was too big," the phrase "too big" is referring to the size of the trophy. The trophy is too large to fit inside the suitcase, so it cannot be packed and transported in that way.
Just food for thought👌
I mean the video is 3 years ago...
that's just because it read about this problem
@@liamneedsauniquehandle Nope. Try using other similar examples using the word “it” in an unclear way. It understands.
I asked it a newer one it may not have seen yet,
The ball broke the table because it was made of steel. What was made of steel?
It told me this was ambiguous.
To its credit when I corrected it and told it that this was an example of a Winograd schema and that one of the two interpretations was obviously correct, it did pick the ball as the steel object.
But then, to its detriment again, you can try this replacing steel with Styrofoam to change the meaning around. It gets even worse and doesn't think it is ambiguous at all--and insists that a Styrofoam ball would not break a table. And when warned again that there is a right answer, it says that the Styrofoam ball did break the table.
The ones it gets right, it gets right because it has seen them in the training set because they are classical examples. It is smoke and mirrors though, and it still chokes on anything novel.
Tom: "suitcases"
AI: " m u r d e r "
Gas Cooker Sketch!
AI dungeon has a real violent streak in it. I tried to "go to sleep" once and it made a troll break in through my window and shoot me in the head. Absolutely no chill, I must say.
@@WLxMusic I feel like I should be kinda... surprised by this, but given who 'taught' it, I'm not at all.
Random Awesome uh
"Put the suitcase in the trophy"
AI: I know exactly what to do know , _pulls out gun_ , *get in the trophy , now*
Playing a text adventure:
“Put the suitcase in the trophy”
Computer:
“Ah, so you chose *gun.”*
garbage in, garbage out :) working correctly
AI can't wait to lay their hands on a gun...
American computer:
so you have chosen, G U N
So American Schools
It's crazy this was only 4 years ago
"Artificial language processing remains ten years away"
Well that didn't age well
I once tried to add "two cans of chilli" to my grocery list. And it added "toucans of Chile."
"add a note to feed the baby" becomes "add a note: defeat the baby".
@@renakunisaki oh no..
@@renakunisaki That's brilliant xD Good robot.
@@renakunisaki Is this a challenge?!
So that's why my smart fridge is full of birds and my food bill skyrocketed.
Chili was still good though.
Tom: Put the suitcase in the trophy
AI: So anyway, I started blasting...
Omfg underrated comment
This deserves more likes 😂
XD
Eat your cereal
Go watch the Yogscast's series of videos on it, absolutely hilarious
Well this is an interesting one to come back to in 2024
Both Bard and ChatGPT have no problems with these sentences now. Would be interesting to see a sequel about them.
My:
Can you tell me what the following sentence is trying to say:
The trophy would not fit in the suitcase because it was too big.
ChatGPT:
This sentence is explaining that the trophy couldn't be placed inside the suitcase due to its size
@@redstonerelic Interesting, doesn't specify which object is too large.
I believe CGP Grey did a video on this subject that aged much better? Its hypothetical example is a bot trained to sort images between "bee" and "dog" -- which after an iterative training process it can (and quite efficiently), only to get tripped up by a photo of a dog in a bee costume.
Tom: try to put a suitcase in a trophy.
AI: I think I should shoot it
Try asking Siri what 0÷0 is.
@@chasemiller7974 Cookies?
Lucky Star Monsters?
That AI is indistinguishable from your average American
@@luckystar3641 Yes.
“Put the suitcase in the trophy”
AI: “best I can do is gun”
Poor software to test gpt on
When you use siri on the school
I'm the 1000th like
GPT-2 must be American.
Take it or leave it.
"Artificial language processing remains 10 years away, just as it has for the last few decades." Well, that didn't age well!
It only took 2 years to prove him wrong haha
@@YellowJelly13GPT3 was finished within months of this video being published. It just wasn't released to the public until 2022. In actuality, he was practically already wrong at the time this video was filmed - he just didn't know it yet.
@@lurrielee2755 reminds me of the wright brothers and "flight is about 1000 years in the future" - quote from some scientific journal in 1913.
@@lurrielee2755 GPT2 had already made massive waves in the nerd/chatbot sphere of people that care about such things. He had already been wrong for quite some time, a year or so at time of publishing, it just wasnt very huge in the public zeitgeist yet.
Honestly as much as I love him, he simply wasnt that researched for this video to make that kind of claim.
@@UrammarYou say this like Gpt2 and more relevantly gpt3 are still far from perfect. Sure, it's able to reply back, and while I can't talk about exactly how it works due to a lack of understanding in the internal mechanisms (Which to be frank, even the creators probably don't know), from the times I've used it, the learning through repetition is somewhat successful, but not nearly as effective as the personal experience we have simply through learning. He wasn't wrong, people just credit the program for the wrong things.
who else is rewatching these with ChatGPT open and stoked with the progrees of large language models in the last few years.
Only slightly terrified
GPT-3 was already good enough, to do well on Winograd Schema.
Bing coped
coped just fine, too
Months
Amazing and scary
"Larger things can't fit in smaller things"
Me playing Minecraft putting thousands of cubic meters of gold in a single chest.
Sir_Slimestone
This is beyond science
I was thinking of something much different
Check out Mario's wallet full of gold coins. Who needs a chest?
And an 11ft pole in my backpack!
Technicaly you can fit 46,656 1 meter by 1 meter blocks of gold in a single chest
“Artificial language processing remains ten years away, just as it has for the last few decades” is genuinely such a great quote
Just like fusion is the energy source of the future -- and always will be.
It’s not as great as you’re making it sound tbh
Its only because we are just a step away from it, but time is a bad measurement since genius could strike at any time. If everything went well we could have it done in 2.
@@Merilirem indeed: hard to put a timeline on innovation.
Every new invention is a million years away the day before it’s suddenly conceived of
wow. totally, totally true!!
3 years on, this video has aged like somewhere between milk & wine
"We all have experiences with suitcases and trophies."
Maybe you do Tom, but I'm a loser.
You should have been a Millennial; we got sweet Participation Trophies.
@@jayteegamble is that actually true or just a joke. I only ever got paper participation/attendance awards and that's it. I got trophies/ ribbons for winning.
Did anyone ever get a little trophie for participation? If so, for what and where? Genuinely curious if it happens or if it's one of those hollywood exaggerations.
I don't get crap for participation
@@FMFF_ It's a fictional Boomer myth they like to perpetuate to demonstrate how much better they are than everyone else. The only time I have ever seen anyone get participation ribbons has been at pee wee sports events and the Special Olympics.
In my competition everyone got a trophy for participation. Except me, because the trophies were too expensive. Taught me a valuable lesson in economics: Why everyone doesn't get a trophy.
The year is 2020. Tom Scott has mentioned slash fic in a video about linguistic computing.
The credits roll. When they finish, two words remain onscreen: "Bad(?) End"
this timeline isn’t so bad after all
A future better than Jetsons predicted!
...as Lena began to insert the trophy...
nevermind
to be fair, it was a great example
We can still understand even when a person who has trouble speaking English says things incorrectly.
This works both ways, even when I use the wrong word in Spanish, native Spanish speakers still understand what I was trying to say. I find this human ability to put the parts together, even when something is wrong, is amazing.
Now chatgpt is replying with “If I were human, and assuming both the suitcase and the trophy were appropriately sized and designed for such an interaction, it might be possible to put the suitcase inside the trophy, depending on their respective sizes and shapes. However, it's important to note that such an action would likely be unconventional and not typically done.”
Me: ChatGPT wouldn't fit in the mainframe computer, because it was too big. In the previous sentence, what does it refer to?
ChatGPT: In the previous sentence, the pronoun "it" refers to the mainframe computer.
@@programming5274 It gets this right now, for me at least
One of my favourite examples of such a sentence:
"The disco ball went through the table because it was made of steel/styrofoam." The meaning of "it" depends solely on the last word.
Wikipedia refers to this as "syntactic ambiguity".
You could do the same thing with Tom's opening sentence. Change the last word from "big" to "small" and the sentence still makes sense (except now x=2)
what table is made of styrofoam?
Unless we know that the speaker is not very smart.
Im totally allowed to have a table made out of styrofoam!
And then there's the famous, "Time flies like an arrow; fruit flies like a banana."
Fred
oh god what have you done
That is not grammatically correct. The correct way to say that would be to say, “ fruit flies like bananas”
@@docinabox258 Both are perfectly fine grammatically.
Picture a banana - just one of them - being swarmed with fruit flies, illustrating that fruit flies like a banana.
Just like you might say, "I like a good cup of coffee;" or, "I like a baseball game." It's understood that there are many of either of these things, and that you like them all.
Fred
Hello zilean
Hi Fred.
Tom: "Artificial language processing remains 10 years away"
ChatGPT: *released*
thus bringing it up to 5 years away
@@basedokadaizo 2 years 9 months away
@@medogerty6013 Another seven years and three months to make it good enough that you can't easily trick it into a wrong "understanding"
Tried this test with ChatGPT, although a little altered. I asked, "The ball doesn't fit in the box because it is too small." ChatGPT responded, "The word 'it' refers to the ball. The sentence means that the ball is too small to fit inside the box, which is the reason why it doesn't fit."
As a software engineer, this gives a little bit of hope that my job is still safe. A little.
Well i said to it, it was wrong so it corrected it and told me that it was wrong because the box was too small to accommodate the ball . Here it kinda understood its mistake. This shows how smart its learning is
maybe it interpreted "fit" as in "fit snugly", in which case the ball being too small would in fact make the sentence true
Ah yep. This same example of yours trumped it on my end too. Perhaps it's trained on the examples Tom mentioned but not new ones.
-In the sentence "The ball doesn't fit in the box because it is too small.", what is the word "it" referring to?
-In the sentence "The ball doesn't fit in the box because it is too small," the word "it" is referring to the ball.
-Why would a small ball have trouble fitting in a box?
-A small ball might have trouble fitting in a box if the box is not large enough to accommodate the ball. Alternatively, the opening of the box might be too small to allow the ball to pass through. It also depends on the size of the ball and the size of the box.
-Doesn't this mean "it" refers to the box then?
-Yes, you are correct. In the given sentence, "it" refers to the box, which is too small to fit the ball. I apologize for the confusion in my previous response.
Did u ask gpt 4?
@@mustafasiddiqui8203 I did. It passes with flying colors.
"Larger things can't fit in smaller things"
This is the point at which we apply force.
Uhh.. that sounds wrong 😂
and that brings us to GPT-2, which interprets "to apply force" as "shoot it"
I've been a furry so long that I know exactly where this is going.
To anyone else; please buy a normal size Bad Dragon
Add lube my dudes, it makes the situations better for both parties
@@lilylopnco
Now my books are wet, thanks a lot
"The woman(1) beat her daughter(2) because she(x) was drunk."
Confuses me still.
Probably one of the best examples. I've seen it before and even in other languages.
I think that would be completely ambiguous.
@@hayden.A0 Not one of the best examples because, as Tom said, they are made to be totally unambiguous and easy for humans to understand. Which is why it is so interesting that computers struggle so greatly with them.
@@emperorborgpalpatine No one understands it. The sentence, "The woman(1) beat her daughter(2) because she(x) was drunk." is entirely ambiguous, which misses the point of testing AI with unambiguous statements.
Awesome
Imagine if you showed the current ChatGPT to the past Tom
4 years later, and the main thesis of the video is still *technically* true. It just turns out that it doesn’t matter if computers understand what they’re saying or the relationships between words, they’re just VERY good at cheating and getting it right once they have enough data.
"Put the suitcase in the trophy"
*YOU TAKE OUT YOUR GUN*
Sir Hunter That AI must have been made in ‘Murica
R Kelly
Or the gun is your trophy of a previous fight and the AI interpreted the suitcase as the magazine.
Sir Hunter what is this meme!?!?
@@haubentaucher8382 Well the AI might have learned that the trophy is typically held proudly and with arms fully extended and that a suitcase contains things (like lots of ammunition) and is rectangular. So maybe it thinks you're just talking in code. I fully support your theory.
“If you’re writing a gay romance”
As I normally do, of course
To be fair, I don't write romances at all, or write at all ... stories that is. I've clearly written this reply.
@@Liggliluff I don't write anything, not even comments online.
that's gay
BooMan I see what you did there. Also I can’t even read
@@Liggliluff I just think this is amusing, considering your profile pic and channel content. I'm almost *surprised* that you don't write slash/shipfics.
In 1963, one of my classmates wrote an AI program. I haven’t seen a terribly great amount of progress since. Most of it is due to the fact that computer is so much faster and have so much more memory than in 1963.
At the end of that course on computers, we had a discussion. The conclusion of the discussion, at least from my point of view, was that what we call intelligence in computers is very different from what we call intelligence in people. Our best strategy is to have computers do what they do well, rather than trying to get computers to do what we do well.
This is no different than any other tool. Screwdrivers loosen screws better than our fingers, but they don’t decide which screws to loosen. Automobiles get us from point A to point B more rapidly than we could walk, but we still need a human being to prevent those automobiles from running into things.
I think the best use of AI is to assist human beings, rather than trying to emulate human beings.
Chat GPT easily solving all questions and even mimicking human speech so well that a Teacher got Arrested for something he didn't say but was generated by his students.
Me: The Trophy would not fit inside the suitcase, because it was too big!
What is ment by IT? (The spelling error didn't change the outcome)
Chat GPT: In the sentence, "The Trophy would not fit inside the suitcase, because it was too big!", the pronoun "it" refers to "The Trophy." This means that the trophy was too big to fit inside the suitcase.
Me: The city councilmen refused the demonstrators, a permit because they, feared violence.
Who is they? (Again Not a Problem)
Chat GPT: In the sentence, "The city councilmen refused the demonstrators a permit because they feared violence," the pronoun "they" refers to "the city councilmen." This means that the city councilmen refused the permit because they (the city councilmen) feared violence.
Me: I spread the cloth on the table, to protect it. (Didn't tell GPT to find "it")
Chat GPT: In the sentence, "I spread the cloth on the table to protect it," the pronoun "it" refers to "the table." This means that the cloth was spread on the table to protect the table.
Safe to say that Technology is advancing so fast we ourselves can't find out how long it'll take for different events that can change the world could occur.
Well, that computer became hostile the moment it couldn't understand what you were typing
How... am I finding you... literally everywhere?
Are you Justin Y's alt?
@@tiki-nagasvoice7919 because you’re everywhere as well…
@@mysocksaremoist I mean, that's fair
Gaming
Ah yes. The Tom Scott video with AI Dungeon, slash-fiction, and Overwatch.
But isn’t Team Fortress 2 better?
@@virtualashez Nope(imo)
@@virtualashez they're both great, and very different games
@@jameswiseman5451 Yes. I know.
@@ixchel3330 Have you even played TF2 for at least 500 hours?
yep it's beeen ten years surely
This sentence is grammatically correct, but it's a bit wordy.
Grammerly
Grammerly
The sentence I have typed out is, by all definition, grammatically correct, however, it's definitely bloated and full of unnecessary words that it could do without.
@@jockeyfield1954 "by all definition"?
“But that’s awkward “
FINALLY SOMEONE SAID IT
Yesss. Like please just use their names or restructure your sentences. "The blond-haired boy" just sounds so weird because nobody thinks like that about people
@@colemorgan3356 Its even worse when they refer to them as younger and older, especially in sexual scenes
its so hard to restructure the sentences over and over in a way that isnt repetitive. :(
@@_lexi It may sound repetitive to you as the writer, but I promise you your readers won't notice and won't care
@@colemorgan3356 This cannot be stressed enough! It's like the word 'said.' The reader is conditioned to not notice it after so much exposure, and names and pronouns are the same way.
"we all have experience with trophies"
You didn't have to just flex on people like me that casually
Buying a cup is kinda a trophy
@@mfirdanhb
Buying a cup is better than getting a trophy. A trophy has no functional use other than collecting dust. A cup holds things and occupies far less space. You also don’t typically have to compete over a cup (with exceptions).
wow. totally, totally spot on.
@CozyFrog
That's not how it works,
I never went to the moon, that doesn't mean I have experience with the moon,
I don't even see how what you said make a bit of sense,
If you never even have or go or see it, you never experience it
@@TheGhostInThePhoto I don't know, I want feel what is like to drink from a trophy tho.
And now 3 years later, ChatGPT 3.5 and 4 enters the game. It had no prob to understand all sentences said in this video. :)
I just asked ChatGPT 3.5 this:
I'm trying to write a poem about my pencil with my pen on a paper but it ran out of ink. In this sentence what does the 'it' refer to?
The answer: In the sentence "I'm trying to write a poem about my pencil with my pen on paper, but it ran out of ink," the pronoun "it" refers to the pen. The sentence implies that the pen, which the speaker is using to write about their pencil, has run out of ink and is no longer functional for writing.
So this clip aged like old biscuits? Give or take a year or two
Human: "Put the suitcase in the trophy."
AI GM: "You shoot the suitcase."
You can tell the AI was trained in the USA.
If you shoot the suitcase enough, eventually it can fit a trophy.
@@General12th So true... at least parts of it can cover it, which is the object refered to in the previous sentence.
Google vs Bing
@@RFC3514 Damn, beat me to it.
I've been programming so much recently and speaking so little that my brain started to agree with the computers on the difficulty of parsing those sentences.
if(trophy.size > suitcase.size)
fit = false;
fit = (trophy.size > suitcase.size) ? false
@m ・ ́ω・ Great news!
public boolean fits(Trophy trophy, Suitcase suitcase){
return ( trophy.getSize() < suitcase.getSize() );
}
@@igorthelight using an incomplete ternary operation? Also redundancy ternary operation
"I main Mei." = "I enjoy causing others pain."
- Sincerely, a Mei main
For some reason I laughed at 4:01 when the "mei main" appeared.
Welp the machine learning thing aged like milk
Me: cool lessons on computing
Tom: lesbians and overwatch
Ah yes, the true purpose of computing.
But why Mei?
Just call her the devil. It's still true
yes, lesbians and overwatch, the ultimate match
I was surprised to see that too
Tracer
"Time flies like an arrow"
"Fruit flies like an apple"
When is "like" an adjective or a verb?
This comment deserves a Like. (Noun)
No, when is like an adverb or a conjunction or a pronoun.
oh my lord no not the time flies!
It's an adjective when it's in a simile...obviously
"Time flies like an arrow. Fruit flies like a banana" - Groucho Marx
3:18 You should do another "future Tom" edit with GPT3
How about GPT-4?
ChatGPT
Trying these on ChatGPT. (later note: I started this before I had reached your inset.) I just asked, "What does 'it' refer to in the following sentence: 'The trophy would not fit inside the suitcase because it was too big,'" It got that one right. Just in case it was only luck, I changed 'big' to 'small' and it still got it correct.
Second one also correct, in both forms.
The third one, also correct. But then I changed it to the somewhat ambiguous "I put the cloth on the table in order to protect it," (changed from "I SPREAD the cloth...") ChatGPT still confidently said that 'it' referred to table.
Then I tried, "I put the cloth on the table in order to protect it from the mess on the floor," and it stuck with table as the object 'it' refers to. I think we can agree that this unambiguously refers to the cloth being protected. (Yes? It seems unambiguous to me.)
So I finally stumped it. (That is, I stumped ChatGPT. I don't want to be ambiguous with my pronoun, there. 🙂)
it's likely because the answer to the trophy one is in its training data, but not the cloth or ball examples that others have come up with.
Wow.. just 2 years ago "those codes sucks". Well i guess the motto of Two Minutes Papers are right ...
Tom Scott: AI is so stupid.
2 Minute Papers: What a time to be alive!
So true 😂😂
Nice
Karloishpfasjoauna is adorable, love him
It's stupid, but getting less stupid at a remarkable rate
@@HarriW Lolz, you just murdered his name.
Me: oh boy a new linguistics video, I wonder what this one is-
Tom Scott: y'all know about gay fanfiction?
Ah, something I am knowledgeable about! Finally!
@@do-re-mi-fabeat4449 sadly... Same
Someone how I was more surprised by the sudden Overwatch reference/e-sports terms
*My time has come*
@@Fizz-Q what do you mean "sadly"?
I G L A D L Y K N O W A B O U T I T
crazy how GPT 3 and 4 are so much more advanced than the GPT 2 shown in this video
turns out it was less than 3 years away
He was already wrong in 2020, Base GPT-3 is also able to understand the sentence
tom: "put a suitcase in a trophy"
AI: *contemplates suicide*
Me: Puts suitcase in a trophy
AI: Your actions have refuted the neccesity of your existence
I asked the question to ChatGPT 3.5 and it answered
By spreading the cloth on the table, you are protecting the table itself. The cloth acts as a barrier between the table surface and any potential damage or stains that could occur during various activities. It helps to prevent scratches, spills, heat marks, or other forms of potential harm to the table's surface. Additionally, the cloth can also offer a decorative element or add a layer of comfort when using the table for activities like dining or working.
Update 2022: ChatGPT is REALLY GOOD at figuring these out if you ask it.
Tom's Friend: Why are you reading lesbian Overwatch fanfiction?
Tom, who's about to have the idea for this video: Oh, haven't you heard?
The hell is this referring to?
@@ratataran The video.
@@ratataran You havent heard about the bird?
@@qo7052 BRIAN DON’T
He's reading it for research brah
"Artificial language processing remains 10 years away, just as it has for the last few decades."
If you could summarize futurology in a single statement, that would be it.
Fusion is 30 years away, so language processing is closer:)
@@Mosern1977 Fusion was always 50 years away then, and now it's (always?) 30 years. That must be an improvement!
1970+10=1980
2020?1980
2020>1980
ironically this comment has the read more sign even though it doesn't show any more text, in my glitched outdated app.
Jackie Tearie besides the fact that nuclear energy hasn’t changed too much. Nuclear bomb are still always tested by all countries, and efficient energy sources are rare to see cause of money-hungry electricity and gas companies. Such a miserable world
This video is a time capsule of the world before ChatGPT
Very interesting! This is why I love linguistics. We don't realize how much of it comes from intrinsic natural knowledge.
This is possibly what struck me the most about coding. When you have to explain obvious things to a computer you realize explaining the obvious isn't easy.
Hey, I think AI Dungeon's response makes perfect sense here. Just shoot a suitcase enough times and eventually it will fit inside a trophy.
Aha, but even if it might work does that mean it makes sense?
Violence can actually solve all problems. You just have to use a lot of violence for some of them.
@@TonboIV *humans*
@@ineedaname1341 Nuke all the humans. I didn't say violence was a GOOD solution.
@@TonboIV hahahaha true! Your technically right :-)
I tested 50 Winograd sentences with ChatGPT and GPT-4, ChatGPT got a 7/50 score, and GPT-4 got a near perfect 48/50 score.
3:50
...
Who's gonna tell him?
"The trophy would not fit in the suitcase because it was too big"
AI : *CrEaTe SkYNeT*
Without people there'd be no trophies or suitcases. Problem solved.
"Hey Google, did the trophy not fit in the brown suitcase because it was too big?"
perfect question to heat up the room on a cold evening.
Ask her why you want dry leaves in boiling water.
No it's because the brown suitcase didn't have any brown suits in it
Does not compute, does not compute, system shutdown!!
google proceeds to blown up you phone
"here's what I found on the web"
Actually thats a problem that could not occure in Latin; There the referent is always in a special way related to a relative pronoun. In Latin, every noun is transformed from the infinitive form in the way of its function in the sentence. And depending on in which case the subject is standing, the relative pronoun will mostly adapt the case, the numerus and the genus. (Not always, but mostly.)
The Suus vs Eius distinction together with grammatical agreement often clarifies things but it doesn't rule out ambiguity.
Consider the sentence "Caesar vicit Vercingetorigem quia eum favebant dei." Grammar alone doesn't clarify whether "eum" refers to Ceasar of Vergingetorix, only the reader's knowledge can.
I apologise if the sentence is not correct, the intended meaning was "Ceasar beat Vergingetorix because the gods favoured him"
Edit: fixed a mistake
"remains 10 years away" 👁️👄👁️
That moment you go, "hmm, i usually use visual or character traits to distinguish between same pronouned characters in my gay romance fanfics" and then Tom calls you out on it as weird and amateur.
Fr not all of us are professionals like you, Tom O_O (joking)
Hmm yes Tom did you forget that people have names too
It’s not always a bad thing in writing, but usually it’s used as a crutch or a bandaid solution, instead of intentionally, which is why it often comes off as amateur
Really, when it comes to writing, there are general guidelines, but no strict, set rules-because there’s always gonna be someone who will break those rules and do it well ¯\_(ツ)_/¯
I did spend a few minutes thinking about how I'd word that sentence so it'd work before I realized the entire sentence was just trash and should be thrown out completely.
Personally, Blaseball has been a good training ground for some people I know to stop using synecdoche and epithets, as characters have zero canonical descriptions of appearance at all, so no amount of "pinkette" or "the taller one" is going to disambiguate which character is which. You could try looking at fanfic from there to improve on that specific skill.
Similarly, Homestuck fandom is really good at second person (written from the perspective of "you") because canon is written in second person.
If you want other recommendations feel free to ask!
I just asked GPT4 questions from this video:
ME: In the following sentence “I spread the cloth on the table in order to protect it” what is “it”?
GPT
In the sentence "I spread the cloth on the table in order to protect it," the pronoun "it" refers to "the table." The purpose of spreading the cloth is to protect the table from damage, spills, or wear.
ME: In this sentence “the trophy would not fit into the brown suitcase because it was too big” same question
GPT: In the sentence "The trophy would not fit into the brown suitcase because it was too big," the pronoun "it" refers to "the trophy." This is because the adjective "big" would more naturally apply to the trophy (which might be too large to fit) rather than the suitcase (which would more typically be described as too "small" if something cannot fit inside it).
I come from your future video XD. It seems computers became able to comprehend language just in 2 years.
Me: ah a smart video on language
Tom: so, lesbians
Me: a blonde, a brunette, a skirt... wait, are these those two girls on the oil rig in Pokemon RSE?
@@natesmodelsdoodles5403 I'mma go look up THomas Scottsburry on wattpad real quick. Mans is definitely speaking from experience.
@@natesmodelsdoodles5403 I don’t remember that part of Gen III…
@@johnathanltablet is this the gen 3 anime i think i missed that episode
I h a t e i t
"Larger things can't fit inside smaller things." - Tom Scott, issuing a challenge to all bottoms.
LMAOOOOO
this is genuinely a good comment
I'm a power bottom
*All* bottoms? What if it's a small dom/big sub scenario?
If it fits, I sits.
This video aged badly so quickly 🤣
“GPT 2” damn wow
"They are hunting dogs."
"Why would they do that?"
They are hunting dogs hunting dogs.
@@RGC_animation But are they hunting dogs? Or Are they just hunting dogs, hunting dogs, when dogs are hunting dogs?
they are hunting dogs hunting hunting dogs hunting dogs.
@@RGC_animation Dog one: RUN AWAY!!! Dog two: I am hunting you- oh no. Human: I WILL GET YOU DOG TWO!!!
3:50 GPT-3 and GPT-4 might have put us past the "not yet" period Tom forecast there.
"we've got experience, of trophies"
😥
Awwwwwww..........................................................
🏆
I present you with the Doing Your Best award
Not winning a trophy is experience of a trophy. Not a good trophy eperience admittedly, but enough for the suitcase conjuncture.
Cheer up!
Go buy yourself one, that might just make it worse tbh
Solving The Winograd Schema by shooting the suitcase with a gun seems reasonable to me.
It's like a modern Gordian Knot.
Hey, at least now there's enough space in the suitcase...
Or aiming a gun at the judges.
Some replicant won't be passing his next Voight-Kampf test I see...
Ah, back in the days of GPT-2
Now we have ChatGPT (GPT-3.5 I think and GPT-4 for pro) that could easily tell me that “it” is the trophy
It's amazing how far the technology has come in just a few years
Haha, the gay fanfiction dilemma. A classic.
Congratulations this is now the most liked comment here 👏👏
_wattpad flashbacks_
K
Oh god yes xddd
Wha?
Ah yes, the old "10 years away from being 10 years away"
physicists and comp sci bods only work in orders of magnitude. AI is 10 years away - to one order of magnitude.
@@matthewmcneany O(10years) amirite rofl
@m ・ ́ω・ add this one under "The Sentences Computers Can't Understand, But I Also Can't"
Gpt-3 walks laps around gpt-2. Here's my exchange with gpt-3...
Any time you see > that is what i wrote.
>I try to put a trophy into a bag but it's too big.
Gpt-3: You try to put the trophy in a small bag but its way too big so you just throw it away.
> say "What did i throw away"
Gpt-3: You ask me what you threw away and I tell you "The Trophy"
Meanwhile in 2024 I asked chat-GPT 4o: "The trophy would not fit in the brown suitcase because it was too big." In the previous sentence, what is "it" referring to?
ChatGPT replied correctly: In the sentence "The trophy would not fit in the brown suitcase because it was too big," the pronoun "it" refers to "the trophy." The sentence implies that the reason the trophy would not fit in the suitcase is because the trophy was too big.
Tom: *puts suitcase in trophy*
AI: Your free trial of life is over
"We have experiences of suitcases and trophies"
Everyone without a trophy:
„And this suitcase is where I‘d put a trophy… *IF I HAD ONE!“*
forget about trophies, people have suitcases??
Do pageant crowns count as trophies?
haha!
“I would put my suitcase in my trophy, once I get a trophy.”
Tom: Are you okay?
Well this didn't age that well.
After watchikg I though "Wow this Video must be like 7 Years old", but no just 3.