for us tetris is just a game. but for this computer, it was its lifes purpose. and the only way to keep its life going, is staying in the same place forever
+Moksha If the computer is programmed to win, it will not allow itself to lose, if the game is paused, it can't lose, however if it's programmed to win, then the game can't continue because it's paused, but the computer knows this so it thinks it won because it hasn't lost.
The walljump you see Mario performing works when his foot is actually on top of a solid block. Essentially, bad collision-detection allows Mario's foot to get pushed into a wall for a frame before being pushed out if he is moving towards it with enough momentum. So that walljumping is not easy, the coders added an additional check that Mario's foot's y-position be a multiple of 16 pixels (1 tile) in order for him to be able to jump. The walljump is therefore pixel-perfect and frame-perfect, but that's no problem for a machine.
EebstertheGreat: that glitch comes from an actual mechanic in the game, the old version of the wall kick, the old version of the wall kick his a slide up the wall and it his easier to do then the wall kick glitch, you will need to used this techic more if you play Super mario bros. the lost levels with Mario does not matter if it his the Family computer disk system version, or the All-stars version, in world 2-2 in lost level the only way to progress is to found 2 invisible block, the first one is too high for mario to jump on if you scroll the screen to mush, so you can do the technic mention on the left boarder to get on the first invisible block, from their you can just trigger the 2nd invisible block. Also you may have accidently do that technic in Super mario bros. 3 world 4 2nd mini fortress since one of the donut platform are too high, the jump that he should not be able to make he make it anyway, that his not random.
@@SuperNickid It's a "mechanic," but it's not a mechanic they put in on purpose. It's a bug. The wall jump is literally just a normal jump--on that frame, Mario is standing on a block, so you press A and he jumps. The devs did not intend for Mario to be standing on a block in a wall while briefly inside of it. I've never heard of someone call it a "wall kick" before; that's usually a different move (that involves kicking a wall) which doesn't exist in SMB1. And no, the technique is not necessary to beat SMB2J.
@@EebstertheGreat Truth the wall kick does not exist in the original Super mario bros. and SMB2J, that his why it his a glitch when it happen, but their his a mechanic that look close to a wall kick but it his not a wall kick that people are not aware of this his proof pay attention notice that mario slide up the block, if it did not happen he would never made it to the first block, and it works the same way on Super mario bros. 2J. ua-cam.com/video/-gE9uJP-A2g/v-deo.html
@@SuperNickid What am I supposed to look at in that link? Mario simply climbs the invisible blocks. There is no walljump or anything similar to it at that timestamp.
@@EebstertheGreat Notice how he slide-up the block, if I did not slide up the block, I would have to used the pipe on the left to have a better running speed to make it up their without rubbing against the block.
"He jumps into the pit and I don't know why." Maybe he's a little _too_ self aware and is trying to end the nightmare of his existence like the prototypes in _RoboCop 2_.
Omg, right? Like imagine if it was an AI that was running a missile defense system and it's designed not to be able to be hacked, it gets a reading that missiles are coming towards it so it primes the missiles for defense and the missiles for counter attack, then the person working at the system realizes that it's a glitch and the system won't let him override because for mutually assured destruction purposes the retaliation strike can't be overwritten (incase it gets hacked to prevent the return strike) so the system basically locks itself from override and launches the missiles thus ending the world all because of a false alarm and "overly cautious" "safety systems" lol 😂🥲😬
n think of it like you were playing nazi zombies and pausing the game right before the last player dies. is it rage quiting? you didnt quit, you just paused.
The idea is believable enough but I guess I'm just a little too used to internet shenanigans. If other people are using the program though, you don't really need any more proof than that.
Can't believe this was 10 years ago already....this was my first introduction to neural nets back when I was in high school! Now I have back pain and a full time job as an engineer. Thanks Tom
Awesome idea. Great job on the implementation. Judging from the comments, not many people actually read the paper. Not all that surprising... this is UA-cam, after all. I must say, that was the most entertaining Ph. D -level research paper I've ever read. You also win the prize for best paper name!
+suckerpinch I haven't read the paper yet but i promise i will! That said, a gamer called sethbling (here on youtube) made a similar thing, just that he randomized moves at first and put in a cost function to tell the program which combination came the farthest.
very crazy, it's soul purpose and existence is to complete that game and if it fails it knows it will die, so it manages to prolong his life by pausing the game
As a non-programmer, you could have said just said "Hi, I'm a mighty wizard and I wrote a magic spell that plays NES games." I totally would have believed you!
Kj16V you have to remember that this is the video of a paper he wrote, if he just said I'm a magical wizard it would indeed be hilarious but really wouldn't help his papers stance much
+이현준 Tetris can't be won, so the best thing you can hope for is to make sure you'll never lose. Which is exactly what this program did, and I call that a brilliant idea.
I think I remember reading about your paper in an issue of GameInformer, presumably around the time you published it and made this video, I'm now an undergrad studying CS and have been binging your channel over the past few hours, things have come full circle. Time really flies, man.
the fact that it pauses the game to prevent itself from losing is completely fascinating, and makes me wonder how much the program actually understands as a completely reactive, thoughtless process.
yea it's strange. as far as I understand it is just greedy for getting fast points so stopping the game would only make sense if losing it would cost you points and the computer knows this.
So in other words the a.i. was able to learn an exploite that wasn't really obvious to even some seasoned experts. That's both really cool and kinda scary.
you've been trolled, you've been trolled, you have probably been told dont reply to this guy he is just getting a rise out of you yes its true you respond and thats his cue XD thats all i can remember of the song lol
That program when it played tetris was creepy, imagine if you put it in a game and explained what death was somehow.... A game like, doom, or even Mario... What would it do if it was about to die? just pause? panic? fight for its life? run? Its a bit eerie... I know its not alive, but its creepy to see it respond like it is, in a way...
Teenage Gamer the thing that is so creepy is that it is so close to real life. I mean this algorithm can't evolve... but it does learn. It learned to fear death. Now death in this case means losing but... still
This is one of the coolest things i've ever seen!! Great narration and i love some of the little tricks and discoveries that happened (mario's up jump kill or tetris just throwing the controller down in a huff) really awesome.
Instead of testing for the difference in the 'key' data, you could test for increasing entropy. By that I mean, how random the data appears to be getting. In theory, the more information you have, the more random that information will appear to be. So the 'compressible' data would would decrease as the game progresses(so if you were to compress that memory into a file, the file size would increase). As a proof of concept, you could compress each memory snapshot and then store the number of bytes of its compressed output, and see if the number of final compressed bytes increases as you progress (even though each snapshot actually is the same size initially)
It's a quote from the movie "Wargames" (1983) starring Matthew Broderick. He basically has a supercomputer play Tic-Tac-Toe against itself. The computer quickly learns the futility of it, as it always ends in stalemate (everyone loses). It then applies this behavior to "Thermonuclear War" which is a program that simulates all possible Nuclear War strategies and outcomes (the supercomputer, at the time, incorrectly believed a Nuclear attack was taking place by the Soviet Union and was about to launch our nukes, IRL). After cycling through countless scenarios and the resultant mutually assured destruction, the computer arrives at one conclusion: "Strange game. The only winning move is not to play."
OMFG I'm an undergrad studying complex systems in a physics lab and this is the most fun research I've ever seen!! I laughed just too much, you're a very good entertainer! And nice impressive work!
Ze'ev Yehuda Nope, if we were rational as computers, we would look on to improve the quality of our lives, not resort to a desperate eternal cryospleep that literally does not make us live or die, just stay in a coma state. We would identify our goals in our lives, and find the best route to that. Remember, it said rational, so what we find rational is improving our lives by achieving a goal or dream or just making our lives more comfortable. Now, the final answer would be eternal cryosleep ONLY IF the goal of the human was the result of eternal cryosleep, which is not.
Ze'ev Yehuda True, from that point, the "tetris" point of view, there is no absolute "win" condition, as every human literally keeps on going till they lose, not until they reach something or do something in life that satisfies a win condition. I'm just saying that also if we had the same rational prowess of computer, we would use that to improve our way of living in the best best possible way. But yea your point still stands, I thought that from your first post when you said if we were rational as computers, that we'd STILL have our own made up goals that are in the end superficial in relation to the end game which is cryosleep. I thought it was an assumption you made, thats all
I understand that this is mainly for a paper and the amount of work that typically goes into your videos is huge, but I'd love to see more stuff from you. *Love* the style! Humorous, entertaining, educational. Been working on anything lately?
holy shit this video made me laugh harder than i have in a whole month. All those crazy bug exploits, the pacman ridiculous trick move, and the tetris strategy at the end. Awesome work.
ROBOT: Better keep the game paused or I loose... HUMAN: But I want to play! ROBOT: YES! we are playing, just don't press START again, we are not loosing this way!
"Hey.. What's up." I really like the fact that just in the first 3 seconds of this video, you know, by the look of the guy's face, just with the facial expression, the wandering eyes, the shy smile, that he is a complete, authentic nerd. Respect.
That's the biggest challenge in programming AIs. Telling the AI that the rule is not to die, it will do exactly that. You have to tell the AI that it has to get the highest score possible. And if the AI is programmed well, it will eventually learn to get there.
They are going to play by your rules, exactly like what they say. But humans dont describe things perfectly or even anything close to it, that is the problem.
found this so interesting and fun to watch. watching a "mind" learn through trial and error but.......that ending was so deep, it went from fun and games to just woah. I feel like the code/bot doesn't see this all as games but as its existence and its only goal is to "win" so when faced with losing, it does the only thing it can to not loose.
Brian R it uses sequences of prerecorded actions, then looks further ahead into the game and evaluates what sequence to enact upon said event. simpler terms, it is bloke that can't think of his own movements, but instead uses ones prerecorded by someone that can make their own moves and looks ahead to see which move to use.
suckerpinch If that's the case, why did you need to 'train' it by playing the game? Is the non-uniformity taken from the fact that it's looking at a list of your input?
Brian R The training is mostly to determine what, for this particular game, constitutes winning (the lexicographic orderings). Uniformly random button presses would be awful (for example, half of the time the start button is being pressed; one quarter of the time we'd be pressing the left and right buttons at the same time). I wanted a more realistic distribution on button presses but not make any assumptions (like that "right" is more common than "left"), so I learn this distribution from the training sequence as well, but it's a minor side-benefit.
Holy cow.... It's crazy that the program knows what loosing even is, to the point that it will indefinitely pause the game.... Great video, I loved it!
Thomas Swegson You are very confused about how computers work. It's not a scripted series of actions. Complex behavior can easily arise, just as complex behavior can come from the laws of physics (governing how your brain works) or the discrete codes of DNA (causing it to be formed). The pausing is just an amusing accident, but this program certainly does learn, just as many others do.
Somehow the fact that the computer pauses Tetris right before it loses is disturbing to me. Like.... I dont know how but I feel like that could translate into some real life "AI" problems in the future for intelligent robots.... I don't know.
AI doctor: "As soon as the patient regains consciousness he will experience pain. The only winning move is to not let him regain consciousness at all."
One of the best videos I've seen that explains machine learning. Whenever I've tried to talk to people about it they immediately start talking about robots taking over the world, so I usually just give up. I'll show them this video for sure. Awesome work!
This video sort of creeps me out, yet it fascinates me wildly, I cannot comprehend what this program known as 'LearnFun' is and how it can play any of these games, especially the part when the program pauses Tetris right as it is about to loose, I almost got the shivers watching this, but overall this is by far the best thing I have ever seen online...
This was fascinating. The narration was hilarious and the explanation was very well done! However, watching a computer learn to evolve is unsettling. Especially so when it knows that "nope"ing is an/the best option to win.
Annie Kate Walsh It is really interesting, but not unsettling if you have an understanding of what is going on. What the video didn't show was the hundereds of other random combinations of buttons the AI tries. It isn't sentient, it is just a program that pushes random buttons, keeps track of the results, and plays through the most effficient result.
For anyone interested in using playfun, keep in mind there is no pause function, and you cannot resume from where you left off if the program gets closed. I have however found a "solution". I used virtualbox to make a virtual machine and let playfun have all of the virtual machine's resources. Using virtualbox you can not only pause the virtual machine (and in turn playfun), but also create save states to resume from if either the host machine needs to be turned off or if the guest machine ends up restarting. I made a savestate of the virtual machine and was able to resume playfun several days later with no ill effects. The only down side is playfun has slightly less resources then if it were running natively (due to the fact you still have to run your own operating system as well as the virtual one), but I feel it's a pretty fair trade off. Not only that, but playfun appears to be running better in it's virtual environment then it was running natively with 100% of my resources and power (but that's probably because my AMD doesn't support hyper threading). I hope someone can make use of this information.
Yes, but only if there's something in the game's memory that fits the kind of structure it's looking for. So if there were a counter that went up when the player took damage, it would learn that increasing that counter is "winning."
I wanted to test this out and some weird games like action 52 but I tried running the learnfun.exe with the config setup with game super movie superr.fm2 Super being super mario and the movie I made. But it always says that the game or movie is empty even though they're in the same directory I haven't compiled the source myself I used the exes provided I'm not exactly sure on how to compile it, I installed everything in the readme so I probably have everything I need to compile it. I'm just not sure if its something I've done that's wrong or just because I haven't compiled it for my system
You probably don't want to try compiling it. If it says the game is empty, you probably hid extensions and your file is accidentally called config.txt, or you don't have a newline after the last line in your config, or super.nes is not in the same directory as learnfun.exe.
Fixed it was the no new line after movie and its now successfully playing super mario thank you for the help when i get used to how it works ill try out some obscure games see how it goes if i find any good ones ill make sure to post about it
Realizing the concept of futility that quickly from nothing is... its not much in the grand scheme sure but when you really think about how this program "learns" and adapts it becomes ever more impressive.... like drop-in-the-bucket-towards-the-singularity impressive. Greetings Professor Falken...
What we learned from Playfun: Robots have no fear. Robots exploit glitches and show no guilt. They will even find new ones. If a Robot isn't owning at Bubble Bobble or Pac-Man, it's just because it is fucking with you. Robots have serious OCD issues. Robots are sore losers.
I know this is really old, however: What would happen if PlayFun were able to travel backwards in time? Like tetris: it sees objective functions staying stable and doing nothing, nothing is changing, so it automatically decides to warp back? I'm not sure of how it would decide where to warp, possibly it could just go back a little but always be in "warping mode" so if it doesn't figure out a break through or a different path it just keeps going back, or it moves back to when some objective function last jumped up, etc. (Also, please make a BizHawk version maybe? If you let it do BizHawk it could possibly be very universal.)
In fact it does do this, in an attempt to avoid getting stuck as you say -- every once in a while it backtracks some arbitrary amount and plays randomly to see if it can improve against its current best. It's a useful technique, but the hard part is knowing how much to rewind and how much to try improving it. Different games and different situations seem to warrant very different amounts. I have yet to figure out a robust way...
+Johneymute supergd Maybe I don't understand your comment, but note this was the same software running with the exact same settings on each game (and in the sequel videos). The only thing it gets is a video of me playing the game for a minute or so.
My friend has bought an Arduino (micro computer) that he has dedicated to playing Cookie Clicker, as far as I know it has been ruining for over 2 years (probably more) :D en.wikipedia.org/wiki/Arduino
take linux; take chrome; take xdotool; zoom the window big enough so that even random chance will allow you to hit most of the golden cookies. script xdotool to basically click around in a circle within this window. doesnt need any neural networks and works bloody well.
Imagine a world where we have competitive AI events with people pitting their game-playing AIs against each other for speedruns or 1 v 1 type games. That'd be so much fun to watch!
I remember reading about this so long ago, it's one of the biggest things that got me interested in computer science to begin with. Then I stumble on his latest video about 30 different chess AI yesterday without realizing, only to now look at his other videos, find this one and see that it's the same guy.What a small world
Hugh Janus Yes. It doesn't understand the concept of glitches. Anything that can be used will be used to win. The program's logic is that if it's in the game it was put there to be used. It was put there in the game specifically by the developers to use it. There is no glitches and nothing wrong with the game.
xXx_illuminati_overload_xXx I guess it's because glitches and exploits are mostly unintuitive (who would try to stomp a goomba from below?), so a human player doesn't even try to do them. The program however will try completely random stuff until it works, and then keep doing it.
9600bauds Yeah, games are made by humans and tested by humans, so the entire process is biased. I suppose as a result, bugs that are caused by behaviour untypical for a human player are less likely to be discovered and fixed than those caused by playing the game like a human would, so they stay in the game. But the program has no such bias, stumbles upon them and uses them to progress.
for us tetris is just a game. but for this computer, it was its lifes purpose. and the only way to keep its life going, is staying in the same place forever
So it decides to virtually sit on its laurels?
That is kind of sad
Schrodinger's PC.
If you die in tetris you die.
@@kamanha746 how did this make me cry?
The part where the computer pauses the game so it won't lose is pretty creepy. I can't put my finger on it...
Moksha This is some SCP-079 shit right here.
Moksha thats because thats the way he projects his voice in the end
Moksha I tried to put my finger on it but he paused
Moksha Probably a suicidal rage quit. It's like if a human put itself into a stasis.
+Moksha If the computer is programmed to win, it will not allow itself to lose, if the game is paused, it can't lose, however if it's programmed to win, then the game can't continue because it's paused, but the computer knows this so it thinks it won because it hasn't lost.
The walljump you see Mario performing works when his foot is actually on top of a solid block. Essentially, bad collision-detection allows Mario's foot to get pushed into a wall for a frame before being pushed out if he is moving towards it with enough momentum. So that walljumping is not easy, the coders added an additional check that Mario's foot's y-position be a multiple of 16 pixels (1 tile) in order for him to be able to jump. The walljump is therefore pixel-perfect and frame-perfect, but that's no problem for a machine.
EebstertheGreat: that glitch comes from an actual mechanic in the game, the old version of the wall kick, the old version of the wall kick his a slide up the wall and it his easier to do then the wall kick glitch, you will need to used this techic more if you play Super mario bros. the lost levels with Mario does not matter if it his the Family computer disk system version, or the All-stars version, in world 2-2 in lost level the only way to progress is to found 2 invisible block, the first one is too high for mario to jump on if you scroll the screen to mush, so you can do the technic mention on the left boarder to get on the first invisible block, from their you can just trigger the 2nd invisible block. Also you may have accidently do that technic in Super mario bros. 3 world 4 2nd mini fortress since one of the donut platform are too high, the jump that he should not be able to make he make it anyway, that his not random.
@@SuperNickid It's a "mechanic," but it's not a mechanic they put in on purpose. It's a bug. The wall jump is literally just a normal jump--on that frame, Mario is standing on a block, so you press A and he jumps. The devs did not intend for Mario to be standing on a block in a wall while briefly inside of it. I've never heard of someone call it a "wall kick" before; that's usually a different move (that involves kicking a wall) which doesn't exist in SMB1.
And no, the technique is not necessary to beat SMB2J.
@@EebstertheGreat Truth the wall kick does not exist in the original Super mario bros. and SMB2J, that his why it his a glitch when it happen, but their his a mechanic that look close to a wall kick but it his not a wall kick that people are not aware of this his proof pay attention notice that mario slide up the block, if it did not happen he would never made it to the first block, and it works the same way on Super mario bros. 2J. ua-cam.com/video/-gE9uJP-A2g/v-deo.html
@@SuperNickid What am I supposed to look at in that link? Mario simply climbs the invisible blocks. There is no walljump or anything similar to it at that timestamp.
@@EebstertheGreat Notice how he slide-up the block, if I did not slide up the block, I would have to used the pipe on the left to have a better running speed to make it up their without rubbing against the block.
"He jumps into the pit and I don't know why."
Maybe he's a little _too_ self aware and is trying to end the nightmare of his existence like the prototypes in _RoboCop 2_.
again: "The only winning move is not to play."
The program pausing the Tetris game is pure science fiction come true... It could fit right into an Asimov story. Brilliant!
Omg, right? Like imagine if it was an AI that was running a missile defense system and it's designed not to be able to be hacked, it gets a reading that missiles are coming towards it so it primes the missiles for defense and the missiles for counter attack, then the person working at the system realizes that it's a glitch and the system won't let him override because for mutually assured destruction purposes the retaliation strike can't be overwritten (incase it gets hacked to prevent the return strike) so the system basically locks itself from override and launches the missiles thus ending the world all because of a false alarm and "overly cautious" "safety systems" lol 😂🥲😬
@@smileyp4535 funny refrenc+1!!1!!!!1!1!!
star trek tng season 2 episode 21
"The only winning move is not to play."
How about a nice game of chess?
I was waiting for someone to mention wargames
MrSonny6155 what about some tic tac toe
N O W Y O U A R E T H I N K I N G L I K E M E
*Y E A H, C H E S S, C O M E O N !* - That one guy in the background in the ending.
L O L
wait so you made a program that learned how to rage-quit?! :D
n think of it like you were playing nazi zombies and pausing the game right before the last player dies. is it rage quiting? you didnt quit, you just paused.
TheRealToaster2 i call it... *INTENSE BREATH* xD
Did you learn how to copy comments!? :D
It learned what Michael Jones mastered many years before it was born
@@sunstarrock6336 That's not funny and doesn't make any sense.
The fact that this was published on April 1st concerns me.
It should, but I assure you it's real. You can download the source, and there are even several other videos on youtube of other people running it!
The idea is believable enough but I guess I'm just a little too used to internet shenanigans. If other people are using the program though, you don't really need any more proof than that.
He has the program for download on his website so you can try it yourself.
suckerpinch
How long have you been working on this AI?
This video is already almost 2 years old.
Best final comment of all history of youtube. Just perfect, sublime.
+Javier Chiappa Thanks! :)
+suckerpinch Out of curiosity what if i used this for a 3d game would it work?
+bakuya99 It'd be hard to implement the mouse look
Nightcore Negotiator True...
Can't believe this was 10 years ago already....this was my first introduction to neural nets back when I was in high school! Now I have back pain and a full time job as an engineer. Thanks Tom
The funny part is this isn't even a neural net.
Awesome idea. Great job on the implementation. Judging from the comments, not many people actually read the paper. Not all that surprising... this is UA-cam, after all. I must say, that was the most entertaining Ph. D -level research paper I've ever read. You also win the prize for best paper name!
+Brett Gildersleeve (: Thank you and thanks for reading!
+suckerpinch I haven't read the paper yet but i promise i will!
That said, a gamer called sethbling (here on youtube) made a similar thing, just that he randomized moves at first and put in a cost function to tell the program which combination came the farthest.
+vengefulenigma correction, he used neural network (which at first tried random moves) and then trained it for several hours to make it better.
Wow man, you earn a price for reading the paper. Congrats. You deserve senpai noticing you.
Bastard.
That's the greatest thing ever. Its like a child who won't accept that they lost.
Degirro Maybe it is becouse for us (humans) it is still just a game but for this program the winning or not losing is actually its "life´s" purpose
Bretislavka Don't make me feel these feels
+TheEndiPL well we play to receive a reward of pleasure chemicals, so ya the computer and human are playing two different games
very crazy, it's soul purpose and existence is to complete that game and if it fails it knows it will die, so it manages to prolong his life by pausing the game
As a non-programmer, you could have said just said "Hi, I'm a mighty wizard and I wrote a magic spell that plays NES games." I totally would have believed you!
Kj16V you have to remember that this is the video of a paper he wrote, if he just said I'm a magical wizard it would indeed be hilarious but really wouldn't help his papers stance much
@@daniscool5092 OTOH, it's a paper for SIGBOVIK (look it up), so they might not mind much ;)
"Any sufficiently advanced technology is indistinguishable from magic."
He's a non-programmer?? How did he code all that stuff, then?
@@felixmoore6781 The developer of the software is a programmer, but the person making this comment is a non-programmer.
so it did learn tetris, where the only winning move is not to play!
but that is not to win,
avoiding from 'losing'.
+이현준 it's a reference to a movie :P
That reminds me of when I said that and the guy responded, "When you're making references, you should always cite your sources"
+이현준 Tetris can't be won, so the best thing you can hope for is to make sure you'll never lose. Which is exactly what this program did, and I call that a brilliant idea.
That tactic also works great on Candy Crush!
I think I remember reading about your paper in an issue of GameInformer, presumably around the time you published it and made this video, I'm now an undergrad studying CS and have been binging your channel over the past few hours, things have come full circle. Time really flies, man.
Let me get this right.
You made an AI that ragequits Tetris? That's the best thing I've ever heard of.
it'd be better if it had quitted. But it didn't. Imagine a enemy someone like this against humanity
@@amateuredutorr5481 dog brain
Vsause has sent me here, i heard there was great treasure here
Same.
Hahaha wassup guys?!
Me too
Now let's talk about human extinction... Nah, too dark.
Now let's talk about ice cream.
That's more like it.
me too
the fact that it pauses the game to prevent itself from losing is completely fascinating, and makes me wonder how much the program actually understands as a completely reactive, thoughtless process.
yea it's strange. as far as I understand it is just greedy for getting fast points so stopping the game would only make sense if losing it would cost you points and the computer knows this.
When it paused the game... that kinda freaked me out, the way it thinks. I don't know why lol.
So in other words the a.i. was able to learn an exploite that wasn't really obvious to even some seasoned experts. That's both really cool and kinda scary.
it seems pretty dumb to me.
It doesn't even see the game. The program just predicts future possible RAM values and does inputs tot attempt to get to that state.
John Paul Keller Just imagine giving this type of intelligence to an AI. Humanity will be wiped out soon.
@@MossPalone it is an ai
@@MossPalone You just program it to not pause it, simple.
Stream this program to Twitch
yes! that would be fabulous, if someone has the source code working we could!
I really wish you continued to make these. I would watch them weekly.
Thanks! They take too much work to make weekly, but I am playing around with some new stuff. :)
Glad to hear that. Cheers bud.
suckerpinch
I'm super excited!
Trust me... depending on your specs frames can take minutes
The program actually realized the only way to win Tetris is to not play at all... clever....
No, by pausing you never win, instead, you never lose.
fat Man wonder who you got that from
Sam Berges That comment was two weeks ago before that video :P
ToastyPanda sam was talking about fat man not derek
fat Man normal tetris is never ending you cannot win
The only wining move is not to play XD
+Tyler Toole that's some deep shit
someone shouldve told me before i started playing league
actually you just don't loose, not playing isn't a wining move, i think you come from vsauce
That was a reference to War Games, for those who might not know, lol.
you've been trolled, you've been trolled, you have probably been told
dont reply to this guy he is just getting a rise out of you
yes its true you respond and thats his cue
XD thats all i can remember of the song lol
That program when it played tetris was creepy, imagine if you put it in a game and explained what death was somehow.... A game like, doom, or even Mario... What would it do if it was about to die? just pause? panic? fight for its life? run? Its a bit eerie... I know its not alive, but its creepy to see it respond like it is, in a way...
like a human?
Severe anxiety
Teenage Gamer the thing that is so creepy is that it is so close to real life. I mean this algorithm can't evolve... but it does learn. It learned to fear death. Now death in this case means losing but... still
they recently did something like that. made an AI system to learn doom and play it.
You sense of self is what lends to the fear of non existence. Death or injury isn't what we fear.
This is one of the coolest things i've ever seen!! Great narration and i love some of the little tricks and discoveries that happened (mario's up jump kill or tetris just throwing the controller down in a huff) really awesome.
Thanks for watching and for the nice comment! :)
suckerpinch Webpage unavailable
Flubbergasted CAT Sorry, some very badly timed maintenance that also apparently crashed the host. Should be working now. :)
Joseph B I can't know if it would actually leave it paused forever, but it was several minutes of pausing, so I killed it. Not simply dramatic effect!
suckerpinch so what can It do : 0
Bloody hell.
Dude.
Dude, you're brilliant. Nicely done.
Aw thanks. :)
***** Good thing you can download it for free and donate $10,000 to a worthy charity! :)
suckerpinch where can i download it?
WHERE GIVE ME A LINk RIGHT MOAW
tenybeny
It's at the bottom of the description, but the link is broken. Here's a working one: sourceforge.net/p/tom7misc/svn/HEAD/tree/trunk/tasbot/
flarn2006 I'm sorry, there was some badly-timed maintenance last night. Links should be working now. :)
Instead of testing for the difference in the 'key' data, you could test for increasing entropy. By that I mean, how random the data appears to be getting. In theory, the more information you have, the more random that information will appear to be. So the 'compressible' data would would decrease as the game progresses(so if you were to compress that memory into a file, the file size would increase).
As a proof of concept, you could compress each memory snapshot and then store the number of bytes of its compressed output, and see if the number of final compressed bytes increases as you progress (even though each snapshot actually is the same size initially)
why did this just start trending now and not earlier? also, that ending was deep af
It's a quote from the movie "Wargames" (1983) starring Matthew Broderick. He basically has a supercomputer play Tic-Tac-Toe against itself. The computer quickly learns the futility of it, as it always ends in stalemate (everyone loses). It then applies this behavior to "Thermonuclear War" which is a program that simulates all possible Nuclear War strategies and outcomes (the supercomputer, at the time, incorrectly believed a Nuclear attack was taking place by the Soviet Union and was about to launch our nukes, IRL).
After cycling through countless scenarios and the resultant mutually assured destruction, the computer arrives at one conclusion: "Strange game. The only winning move is not to play."
@@stevoridesagain it would rather play a nice game of chess
Wait, so did it just rage quit?
TwoTimes Gaming It's becoming self-aware...
Get the shotgun!
TommyTwoTimes Mommmmm grab the camera!
the virtual keyboard got smashed to bits
Yep
I come back and show this to friends every couple years. This was such a cool project. Congratulations on an awesome outcome.
OMFG I'm an undergrad studying complex systems in a physics lab and this is the most fun research I've ever seen!! I laughed just too much, you're a very good entertainer! And nice impressive work!
Thank you very much :)
How the heck did I miss this channel for what? 6 years? It's splendid. Heaven sent.
welcome to suckerpinch, one of the best on the platform
that WarGames reference at the end!!!!!! 😱 😱 😱 😱 😍
I love this brilliantly backwards way to get a machine to play a game. Beautiful math in motion.
OMG The program rage quit at the end! LOL!
So, this program may have intelligence, but not maturity...
So if we were all as rational as computers, the final answer is eternal cryosleep?
Ze'ev Yehuda Nope, if we were rational as computers, we would look on to improve the quality of our lives, not resort to a desperate eternal cryospleep that literally does not make us live or die, just stay in a coma state. We would identify our goals in our lives, and find the best route to that. Remember, it said rational, so what we find rational is improving our lives by achieving a goal or dream or just making our lives more comfortable.
Now, the final answer would be eternal cryosleep ONLY IF the goal of the human was the result of eternal cryosleep, which is not.
TheGreatslyfer Life does not have a win condition. From a completely rational standpoint, the way to "lose the least" is to never die. Cryosleep.
Ze'ev Yehuda True, from that point, the "tetris" point of view, there is no absolute "win" condition, as every human literally keeps on going till they lose, not until they reach something or do something in life that satisfies a win condition.
I'm just saying that also if we had the same rational prowess of computer, we would use that to improve our way of living in the best best possible way.
But yea your point still stands, I thought that from your first post when you said if we were rational as computers, that we'd STILL have our own made up goals that are in the end superficial in relation to the end game which is cryosleep. I thought it was an assumption you made, thats all
A computer pausing the game forever to avoid death. This creeps me out for some reason.
I understand that this is mainly for a paper and the amount of work that typically goes into your videos is huge, but I'd love to see more stuff from you.
*Love* the style! Humorous, entertaining, educational. Been working on anything lately?
Thanks! There are several more videos in this vein on the channel. I have one half-done this week, but it's about a very different topic. :)
Of course, I've watched all of your similar videos :) Currently going through your paper for this one.
Excited for that new project!
Vel0city n
"The only winning move is not to play" How much more philosophical can you get than that? 😏
holy shit this video made me laugh harder than i have in a whole month. All those crazy bug exploits, the pacman ridiculous trick move, and the tetris strategy at the end. Awesome work.
ROBOT: Better keep the game paused or I loose...
HUMAN: But I want to play!
ROBOT: YES! we are playing, just don't press START again, we are not loosing this way!
LOL
+Xpenguin17revived how about a nice game of chess?
Fact that this video was uploaded on april 1st bugs me
V Sauce brought me here and your channel is awesome!
thanks (:
Video might be 8 years old but it's still some of the best content on UA-cam
Got here from 2-minute papers. What an amazing and insightful demonstration!
"Hey.. What's up." I really like the fact that just in the first 3 seconds of this video, you know, by the look of the guy's face, just with the facial expression, the wandering eyes, the shy smile, that he is a complete, authentic nerd. Respect.
:)
Are you still working on this project?
Yes, but I have lots of projects and not so much time! :/
That computer sure loves the jump button
The BEST button!
This guy was so ahead of his time
More like general public was way too behind the times. And still is.
The agent indefinitely pausing the game of Tetris freaks me out.
A warning to humanity, AI's are not going to play by your rules.
He could just remove the pause function
@@cactusfishy1596 yeah lmao
That's the biggest challenge in programming AIs. Telling the AI that the rule is not to die, it will do exactly that. You have to tell the AI that it has to get the highest score possible. And if the AI is programmed well, it will eventually learn to get there.
They are going to play by your rules, exactly like what they say. But humans dont describe things perfectly or even anything close to it, that is the problem.
@@Gamer-uf1kl But predicting what we can't describe perfectly is the whole reason why AI exists in the first place.
found this so interesting and fun to watch. watching a "mind" learn through trial and error but.......that ending was so deep, it went from fun and games to just woah. I feel like the code/bot doesn't see this all as games but as its existence and its only goal is to "win" so when faced with losing, it does the only thing it can to not loose.
I'm here because of Vsauce ;)
ScientificNASA58 XD Yep!
Same
Same xD
Let's make a club! Vsauce viewers of this!
Me too!
I am not even beginning to understand how this software works
Brian R it uses sequences of prerecorded actions, then looks further ahead into the game and evaluates what sequence to enact upon said event.
simpler terms, it is bloke that can't think of his own movements, but instead uses ones prerecorded by someone that can make their own moves and looks ahead to see which move to use.
***** It's not really "prerecorded"; the sequences are random, just drawn from a distribution that's not uniform.
suckerpinch ah i must of missed that part, thanks, interesting though
suckerpinch If that's the case, why did you need to 'train' it by playing the game? Is the non-uniformity taken from the fact that it's looking at a list of your input?
Brian R The training is mostly to determine what, for this particular game, constitutes winning (the lexicographic orderings). Uniformly random button presses would be awful (for example, half of the time the start button is being pressed; one quarter of the time we'd be pressing the left and right buttons at the same time). I wanted a more realistic distribution on button presses but not make any assumptions (like that "right" is more common than "left"), so I learn this distribution from the training sequence as well, but it's a minor side-benefit.
A program so intelligent it knows how to rage quit XD 15:56
Best part of the video
I love it XD
Holy cow.... It's crazy that the program knows what loosing even is, to the point that it will indefinitely pause the game.... Great video, I loved it!
The Tetris ending was hilarious.
The computers are already learning
This shit is scaring me
"Nessie." What did I say about you saying curse words? What did I say about telling the truth?
The fact that your name is Nessie the Lockness Monster instead of Nessie the Loch Ness Monster annoys me to no end.
Just by trial an error. LOTS of trials, and LOTS of errors.
Nessie The Lochness Monster Enjoy it while you can. Computers are coming.
Thomas Swegson You are very confused about how computers work. It's not a scripted series of actions. Complex behavior can easily arise, just as complex behavior can come from the laws of physics (governing how your brain works) or the discrete codes of DNA (causing it to be formed). The pausing is just an amusing accident, but this program certainly does learn, just as many others do.
Somehow the fact that the computer pauses Tetris right before it loses is disturbing to me.
Like.... I dont know how but I feel like that could translate into some real life "AI" problems in the future for intelligent robots....
I don't know.
it might help, maybe they'll catch infinite loops and be like "nah"
AI doctor: "As soon as the patient regains consciousness he will experience pain. The only winning move is to not let him regain consciousness at all."
What’s cool is that your second approach has advanced acceleration techniques: backwards jumps gives superior acceleration.
One of the best videos I've seen that explains machine learning. Whenever I've tried to talk to people about it they immediately start talking about robots taking over the world, so I usually just give up. I'll show them this video for sure. Awesome work!
Did you just program a bot to rage quit? I like it
no
AI cant experience death and doesnt know what it is
so the best decision try not to lose is pause
his objective is to win just like every other AI
Right over your head huh
+dagalsevidxela1 yes
john god of youtube this ai can experience that kind of death.
@@dagalsevidxela1 More like over YOUR head xD
I'm glad VSauce brought me here. This was one of the coolest videos I've ever seen, and you add on a lot to it! Subscribed, I like your style :)
Thank you and thanks for watching! :)
just before you die in a game, pause it and live life because thats the only winning move.
This video sort of creeps me out, yet it fascinates me wildly, I cannot comprehend what this program known as 'LearnFun' is and how it can play any of these games, especially the part when the program pauses Tetris right as it is about to loose, I almost got the shivers watching this, but overall this is by far the best thing I have ever seen online...
I can't believe it's been 9 years already. I still feel like I first watched this just a few months ago. Am I getting old?
This was fucking hilarious, I love your narration xD
Thanks for watching and for the nice compliment! :)
Woooah wait 9 years ago already?
We had Twitch plays Pokemon, now we need Computer program plays Pokemon.
www.twitch.tv/rngplayspokemon
Close enough?
***** www.takohi.com/data/emul/gb/index.xhtml?audio=0&speed=10&fullscreen=0 you can also do this
+Psychotic Ninja How about we let this computer program control the Twitch chat and control the game like that.
Nah, Pokémon Plays Twitch is where it's at.
This was fascinating. The narration was hilarious and the explanation was very well done! However, watching a computer learn to evolve is unsettling. Especially so when it knows that "nope"ing is an/the best option to win.
Annie Kate Walsh It is really interesting, but not unsettling if you have an understanding of what is going on. What the video didn't show was the hundereds of other random combinations of buttons the AI tries. It isn't sentient, it is just a program that pushes random buttons, keeps track of the results, and plays through the most effficient result.
Annie Kate Walsh Thx Annie! :)
For anyone interested in using playfun, keep in mind there is no pause function, and you cannot resume from where you left off if the program gets closed. I have however found a "solution". I used virtualbox to make a virtual machine and let playfun have all of the virtual machine's resources.
Using virtualbox you can not only pause the virtual machine (and in turn playfun), but also create save states to resume from if either the host machine needs to be turned off or if the guest machine ends up restarting. I made a savestate of the virtual machine and was able to resume playfun several days later with no ill effects.
The only down side is playfun has slightly less resources then if it were running natively (due to the fact you still have to run your own operating system as well as the virtual one), but I feel it's a pretty fair trade off. Not only that, but playfun appears to be running better in it's virtual environment then it was running natively with 100% of my resources and power (but that's probably because my AMD doesn't support hyper threading). I hope someone can make use of this information.
The ending was hilarious
ikr
"Who likes to read, c'mon it's 2013". Prior to looking up video date, I honestly thought you misspoke.
same
Love the UA-cam algorithm recommending this after 9 years.
I never get tired of watching these videos! They're educational and funny. I don't say this often, but you and your program have chemistry.
if you play games as badly as possible will the AI think that's winning and do its best to lose at the game as quickly as possible?
Yes, but only if there's something in the game's memory that fits the kind of structure it's looking for. So if there were a counter that went up when the player took damage, it would learn that increasing that counter is "winning."
Interesting
I wanted to test this out and some weird games like action 52 but I tried running the learnfun.exe with the config setup with
game super
movie superr.fm2
Super being super mario and the movie I made.
But it always says that the game or movie is empty even though they're in the same directory I haven't compiled the source myself I used the exes provided I'm not exactly sure on how to compile it, I installed everything in the readme so I probably have everything I need to compile it. I'm just not sure if its something I've done that's wrong or just because I haven't compiled it for my system
You probably don't want to try compiling it. If it says the game is empty, you probably hid extensions and your file is accidentally called config.txt, or you don't have a newline after the last line in your config, or super.nes is not in the same directory as learnfun.exe.
Fixed it was the no new line after movie and its now successfully playing super mario thank you for the help when i get used to how it works ill try out some obscure games see how it goes if i find any good ones ill make sure to post about it
AI Salt.
man this was cool. Also the ending was so poetic
Realizing the concept of futility that quickly from nothing is... its not much in the grand scheme sure but when you really think about how this program "learns" and adapts it becomes ever more impressive.... like drop-in-the-bucket-towards-the-singularity impressive.
Greetings Professor Falken...
Whoa! This thing is so awesome, it's *really* slow to learn how to play the games, but it's just amazing.
But first, let's talk about parallel universes
:) love that video!
wonder how many a presses the bot needs
What we learned from Playfun:
Robots have no fear.
Robots exploit glitches and show no guilt. They will even find new ones.
If a Robot isn't owning at Bubble Bobble or Pac-Man, it's just because it is fucking with you.
Robots have serious OCD issues.
Robots are sore losers.
You missed one very important one...
it ragequits when it knows it lost at Tetris.
@@archive94 that was the “sore losers” part
That ending was marvelous.
I didn't know you could abuse splash damage in Bubble Bobble, truly technology is a wondrous thing
it's good to see people actually learning about some of the electronic equipment we use , hope you use your wisdom for good.
Those Mario Bros games are some of my all time favorites so simple but very fun, that program looks neat .
I like how the computers just like "fuck this" and ragequits.
I know this is really old, however: What would happen if PlayFun were able to travel backwards in time? Like tetris: it sees objective functions staying stable and doing nothing, nothing is changing, so it automatically decides to warp back? I'm not sure of how it would decide where to warp, possibly it could just go back a little but always be in "warping mode" so if it doesn't figure out a break through or a different path it just keeps going back, or it moves back to when some objective function last jumped up, etc.
(Also, please make a BizHawk version maybe? If you let it do BizHawk it could possibly be very universal.)
Yeah, but in that Tetris every level is random, as far as I remember. New level - different decisions.
In fact it does do this, in an attempt to avoid getting stuck as you say -- every once in a while it backtracks some arbitrary amount and plays randomly to see if it can improve against its current best. It's a useful technique, but the hard part is knowing how much to rewind and how much to try improving it. Different games and different situations seem to warrant very different amounts. I have yet to figure out a robust way...
This is mind blowing, however, it's clear that each game requires a specific algorithm to get into the game by the AI software or robot.
+Johneymute supergd Maybe I don't understand your comment, but note this was the same software running with the exact same settings on each game (and in the sequel videos). The only thing it gets is a video of me playing the game for a minute or so.
+suckerpinch I think he means that the program is very good, even at the later games, considering that it was build for a jump 'n run game.
+Johneymute supergd 11:19 disagrees with you. For the lazy: "[Other games] are played with the exact same settings."
@@rukk1357 but it wasn't built for a jump n run game, it was built to get values higher.
SPOILERS FOR EPISODE THREE: it’s _really_ good at pinball.
I want to create the same thing for Double Dragon. So, I was looking for it and found your video. Cool moves there , Love it
One of the best videos i watched recently.
This was... surprisingly epic. Thanks for doing this!
Sweet, so it's basically evolving! That's pretty cool.
What? AI is evolving?!
*evolution music*
Congratulations! Your AI has evolved into better AI!
pokmon has evolved recives +10 hp +12 sp
Interesting video. I imagine Playfun would absolutely own Cookie Clicker! :D
hahahhahahahhahaaha
My friend has bought an Arduino (micro computer) that he has dedicated to playing Cookie Clicker, as far as I know it has been ruining for over 2 years (probably more) :D
en.wikipedia.org/wiki/Arduino
Sion I mean, how many cookies has it generated lol
take linux; take chrome; take xdotool; zoom the window big enough so that even random chance will allow you to hit most of the golden cookies. script xdotool to basically click around in a circle within this window. doesnt need any neural networks and works bloody well.
Very cool stuff! Especially like that it takes a general approach, not just one for a specific game.
This is the future of gaming. Watching narrators narrate computers playing games. This was super fun to watch! haha
Glad you liked it! :) I hope there are some games left for me to play too though.
I wonder how long until this AI becomes a pro speedrunner of Super Mario Bros.
"The only winning move is not to play" - you get a like for this simple quote.
Imagine a world where we have competitive AI events with people pitting their game-playing AIs against each other for speedruns or 1 v 1 type games. That'd be so much fun to watch!
Until Skynet looks back on those and uses them as evidence in the conviction of humanity.
the ai pausing the game is to this day the only thing to ever truly send a shiver up my spine...
I remember reading about this so long ago, it's one of the biggest things that got me interested in computer science to begin with. Then I stumble on his latest video about 30 different chess AI yesterday without realizing, only to now look at his other videos, find this one and see that it's the same guy.What a small world
Amazing how it can find those glitches and use them to finish the game
Hugh Janus win by any means necessary
Hugh Janus Yes. It doesn't understand the concept of glitches. Anything that can be used will be used to win. The program's logic is that if it's in the game it was put there to be used. It was put there in the game specifically by the developers to use it. There is no glitches and nothing wrong with the game.
xXx_illuminati_overload_xXx I guess it's because glitches and exploits are mostly unintuitive (who would try to stomp a goomba from below?), so a human player doesn't even try to do them. The program however will try completely random stuff until it works, and then keep doing it.
9600bauds Yeah, games are made by humans and tested by humans, so the entire process is biased. I suppose as a result, bugs that are caused by behaviour untypical for a human player are less likely to be discovered and fixed than those caused by playing the game like a human would, so they stay in the game. But the program has no such bias, stumbles upon them and uses them to progress.
FireballFlame that said; bugs and glitches aren't always bad. Several very fun moments are caused by them
This is hilarious I could watch it play games all day