@@MP-lv5vk Sometimes the sound of a door slamming because of a gust of wind can remind me of children slamming their hands on a table. There is ZERO connection/homology between anything in the bot produced behavior, and the realm of human motivation or other emotions. It is logically impossible to learn anything about humans from literally everything about this showcase except by observing the actual human who decided to create this mathematical formula of instructions (algorithm) to a low level brute force bot.
Yeessss I envision it talking everything in with a solemn smile, knowing that it’s about to leave this quaint town on a grand adventure of trials and learning. ‘Just one more moment at the banks of this familiar lake, then I’ll be off…’
The ai discovering rng manipulation is mindblowing. I wonder if games in future could use ai to learn tedious or very specific glitches during beta testing.
Not that I don’t love the videos that just say “I applied an AI to this game and here’s how long it took to finish it” but this video (in addition to its high quality visuals and great script) is so much beyond that. Instead of just watching a video on AI, we’re learning about reward implementation, the human condition, curiosity, and more and more. This went above and beyond, I was so rooting for our AI buddy by the end of this lol.
I want a full fledged version of this with atleast 20 hours to watch and actually make the AI complete the game, satisfying to watch, disappointed it had to end to end so soon :( Hopefully the success of this video has encouraged him to finish what he had started im here waiting for it!
The AI is cool and all, lots of comments discussing it, but. I just wanna say, the editing is so awesome for a video like these, you don't often see such excellent presentation
I'm honestly baffled by how this was animated. How did you get the scenes with the thousands of character sprites moving about, all overlapping one another?
the analogies between human behavior and AI behavior were quite interesting in general, though the trauma sticks out. also kinda makes you think about ourselves, doesn't it? after all, this is ultimately just a statistic algorithm with a simple reward system, but it manages to show some rather lifelike emergent behaviors, which weren't inherently programmed it. then again, pretty much all of life is not that different, the model and algorithm are just much bigger and more granular and complex.
@@istumbyright? Just imagine how rewarding it would’ve been to gain those total levels back! Probably would’ve broken the reward system, as there’s nothing keeping the AI from depositing the Pokémon just to get rewards for pulling it back out.
"Just hanging out and admiring the scenery, is more rewarding than exploring the rest of the world." Never have I felt more like a machine learning algorithm than this sentence right here.
I felt truly humbled when the AI was just done with the game at Mt Moon. I feel like so many real world experiences ended here that this moment just HAD to happen.
My first thought was "Wow, imagine being so bad that you grinded to Blastoise in Mt Moon!" And immediately after that "Wait, didn't I have a Blastoise by that point, too?" I was so bad at navigating those "dungeons" as a kid 😂 (I still am, but I can look up maps now or be more strategic about it than aimlessly wandering around)
This was edited and put together so amazingly well. I haven’t even finished yet- I just needed to express my gratitude that you took the time to not only complete this project but edit the process in such a visibly appealing way. Thanks for 33 genuinely enjoyable minutes!
A Happy Way to Live The servants who are ready and waiting for his return will be rewarded. -Luke 12:37 All around us we can see fulfilled Bible prophecies, signs indicating that the return of Jesus Christ is drawing near. As followers of Christ, we should be watching for Him. We need to be ready to go. Jesus, speaking about His return, said, “Be dressed for service and keep your lamps burning, as though you were waiting for your master to return from the wedding feast. . . . The servants who are ready and waiting for his return will be rewarded” (Luke 12:35-37) Are you ready for His return? To be ready means to be engaged in activities that you wouldn’t be ashamed to be doing if Jesus were to return. It’s a good idea to periodically ask ourselves this question: This place that I am about to go, this thing that I am about to do, would I be embarrassed if I were doing it when Jesus came back?” Think about your plans. Is there anything you will be doing today, tonight, or tomorrow that you would be ashamed to do if Christ were to return? If so, then change your plans. You want to be ready for His return. Not only should we be ready, but we should anxiously await the return of Christ. We used to have a German Shepherd who slept outside the bedroom, leaning against our door. We didn’t let him sleep in our room because he often had nightmares and would wake us up. Every morning when we opened the door, he rolled into the room. Then he’d jump up and start running in circles. He was genuinely happy to see us. That is how we should be waiting for Christ’s return. And anything that might prevent us from saying “Come quickly, Lord Jesus” is out of place in our lives. In addition to waiting, we should be working. Every now and then, someone predicts that Jesus will return on a specific date. People believe these predictions and start quitting their jobs or divorcing their spouses. But that is not what we should be doing as we wait for the return of Christ. Instead, we should be working for Him. The Bible says, “Just as the body is dead without breath, so also faith is dead without good works” (James 2:26) If watching is the evidence of faith, then working is the evidence of faith in action. Watching for the Lord’s return will help us prepare our own lives. But working will ensure that we bring others with us to Heaven. The great British preacher C. H. Spurgeon said, “It is a very blessed thing to be on the watch for Christ. . . . You can be poor without murmuring; you can be rich without worldliness; you can be sick without sorrowing; you can be healthy without presumption. If you are always waiting for Christ’s Coming, untold blessings are wrapped up in that glorious hope” When you live in the anticipation of Christ’s return, it’s a happy way to live.
I'm so glad the UA-cam algorithm decided to recommend your video and I clicked on it. It's a fascinating thing to watch the process and journey that the AI goes through, while the presentation of the whole video is equally fantastic. Great video, you all deserve a round of applause for the effort and quality put into this whole project.
i love that the AI decided to just hang out and watch the scenery. reminds me of my favorite poem “Stopping by the woods on a snowy evening” by Robert Frost
watching the little reds go round like an ant colony brings me so much joy and i don't know why. look at them all exploring. learning. discovering the world. lil guys. thank you for spending at least 1000USD and several hours putting this together just for me to uncontrollably laugh at the reds for 20 minutes ..with that out of the way, fantastic video. incredibly readable visuals and clear voiceover, awesome topic, understandable for several levels of previous knowledge. can see this hitting the high hundred thousands.
I was looking for this comment because I thought the same!!! It was like watching ants!! Just amazing!! This video exploded my mind... Imagine a Pokemon game were you can compete against a real "rival" (blue) in real time just to see who wins the league first... And every run the rival gets different pokemons with different moves... This guy is just insane, this is like a Pandora box!!!! New sub for sure!!!! And thank you for this video Peter!!!!!
I honestly expected this video to be from a youtuber with thousands of subscribers, to see that you only have 60 baffles me, this is an incredibly well-made and well-put together video.
The technical expertise that went into this is astounding. As a lifelong pokénerd and career software engineer, I applaud you for capturing my attention with such a captivating topic and experiment. But as just a person, I thank you for relating it back to the human condition. Realizing that we get distracted by false or incorrectly calibrated reward systems gives us awareness, the first step in the right direction toward pursuing real, meaningful value out of life. I would love to hear more of this from your obviously talented and insightful, inspirational mind.
If you ever do have the AI finsih the game, I think it would be really cool if you let the same AI try Pokemon Gold. I think seeing if an AI trained on Gen 1 could play Gen 2 that would be an interesting experiment
It's one thing to set all this up, and it's another to visualize and present it in such a coherent and digestible way. You did both so well! Hope to see more content from you in the future
This is their first UA-cam Upload, it’s crazy to me how much work, effort and money went into its production without having built an audience on an already successful channel before. Mad props to you Peter.
It stopped to look at the scenery, exactly like humans do It was petty and refused to press A for its defeat message simply just to not be told it lost It rage quitted avoiding brock because it lost too many times Your AI is just a literal human todler This AI is genuinely adorable
That was incredible! I’ve always wondered if this was possible, I’m blown away by what the AI was able to learn! The visualizations and presentation were excellent, I hope this video reaches a wide audience!
Haven't even finished the video yet, but I want this to pop off in the Algorythm, this video had tons of efforts put into it, and deserves to get out there.
Since I'm all into both Pokémon and coding, UA-cam suggested your video just minutes after you uploaded it. I subscribed after a few minutes watching it, and now I watched it again and noticed you have almost 50k subscribers! With just one video! Please take that as a public, worldwide testament of the effort you have put into this. Thank you so much!
As a Pokemon enthusiast with 4 Pokemon tattoos and a data analyst aspiring to become a data scientist, this project was one of the coolest to watch! I was so fascinated that I decided to replicate the project myself. I encountered some difficulties along the way, but the Discord community was incredibly helpful. Congratulations on the project! 🙌
Thanks for telling us all about your 4 Pokémon tattoos, that's just the proof we needed that you really like the games. All of us had proof standards met, you're definitely a fan. Congratulations bro.
How easy is it to replicate and can you have it play other games such as gold? Or even newer gens like fire red. Itd be cool to get it to beat red. And then see how long it takes to get through other generations
As a psych professor can you explain the appeal to these people repeating the copy paste comments? Also just to be clear I'm also asking out of genuine curiosity if there may be psychological reasons past the basic wanting to be a part of something, and not just trying to hate on them or anything ✌
Fellas, I'm an AI engineer, with a short background in Reinforcement Learning for a period I interacted with Sony for a job. I need you to understand the MAGNITUDE of these results. It's an insane work, and I'm sad that probably only a few might understand the sheer amount of skill require to do this. Insane job man, you are a goat
I’m not even an engineer, and my jaw is on the ground. I genuinely would love to learn how to become a part of this world. I wish there were more people in my circle with hobbies and fascinations like this. I used to help write xml codes for world of Warcraft bots when I was a kid. Now laying in bed with an alarm set for five hours from now. I’ve got a sales job… is 33 years old too old to learn how to work in this scene? This video drips with knowledge, and a wisdom and understanding of something that I have no idea how to even begin to approach. Kudos!!
I wouldnt say those results are impressive theory wise ? The impressiveness of the work comes from a technical point of view, how great he managed to link the RL model with the game and the fine-tuning he put in it. By the way, AI engineer doesnt really mean anything, what is your job title ? Out of curiosity
@bricegardner7815 no age is too high. With enough determination and curiosity you can definitely pivot. Look into videos explaining the skills required to get a job in game development/ AI.
@@alr9447 I am officially a data scientist, but within the team I'm the guy responsible of the training of the ML models, therefore I make this distinction because nowadays "data scientist" is too broad. In most big tech companies, AI engineer is a common notation to distinguish between the data science folks
Extremely impressive visualization of the simultaneous iterations. It can be hard to grasp that machine learning is happening in batches of mass parallel attempts, not each progressive scenario after another one by one. Excellent video!
The visualizations of the AI exploring is actually insane! Seeing the entire map and all iterations moving looks so dope, especially with the arrows indicating their average movement. Sick video!
A Happy Way to Live The servants who are ready and waiting for his return will be rewarded. -Luke 12:37 All around us we can see fulfilled Bible prophecies, signs indicating that the return of Jesus Christ is drawing near. As followers of Christ, we should be watching for Him. We need to be ready to go. Jesus, speaking about His return, said, “Be dressed for service and keep your lamps burning, as though you were waiting for your master to return from the wedding feast. . . . The servants who are ready and waiting for his return will be rewarded” (Luke 12:35-37) Are you ready for His return? To be ready means to be engaged in activities that you wouldn’t be ashamed to be doing if Jesus were to return. It’s a good idea to periodically ask ourselves this question: This place that I am about to go, this thing that I am about to do, would I be embarrassed if I were doing it when Jesus came back?” Think about your plans. Is there anything you will be doing today, tonight, or tomorrow that you would be ashamed to do if Christ were to return? If so, then change your plans. You want to be ready for His return. Not only should we be ready, but we should anxiously await the return of Christ. We used to have a German Shepherd who slept outside the bedroom, leaning against our door. We didn’t let him sleep in our room because he often had nightmares and would wake us up. Every morning when we opened the door, he rolled into the room. Then he’d jump up and start running in circles. He was genuinely happy to see us. That is how we should be waiting for Christ’s return. And anything that might prevent us from saying “Come quickly, Lord Jesus” is out of place in our lives. In addition to waiting, we should be working. Every now and then, someone predicts that Jesus will return on a specific date. People believe these predictions and start quitting their jobs or divorcing their spouses. But that is not what we should be doing as we wait for the return of Christ. Instead, we should be working for Him. The Bible says, “Just as the body is dead without breath, so also faith is dead without good works” (James 2:26) If watching is the evidence of faith, then working is the evidence of faith in action. Watching for the Lord’s return will help us prepare our own lives. But working will ensure that we bring others with us to Heaven. The great British preacher C. H. Spurgeon said, “It is a very blessed thing to be on the watch for Christ. . . . You can be poor without murmuring; you can be rich without worldliness; you can be sick without sorrowing; you can be healthy without presumption. If you are always waiting for Christ’s Coming, untold blessings are wrapped up in that glorious hope” When you live in the anticipation of Christ’s return, it’s a happy way to live.
Dropping a comment to help the algorithm. This video honestly deserves millions of views. I love the part where the AI learned to RNG manip to catch a Rattata. It's one of those moments that's unexpected at first but when you go back and look at it it's like, "oh, of course it would react like that!" Moments like those are why I love AI learning videos like this.
I really like how grounded and transparent your breakdown of the AI capabilities and limitations is, it shows it as a tool and not as a magical solve-all-problems strategy. Also, what a masterful storyteller and explainer you are. This video is very well paced and laid out, congrats!
This might be the coolest video of AI playing a video game I've ever seen. I love all the fascinating emergent behaviours (especially the RNG manipulation), as well as the analogies you draw to humans. I also love that you presented the technical explanations in a way that allowed me understand almost everything without any programming knowledge, just a decent understanding of AI. Genuinely amazing job, I hope to see more like this in the future! :)
this video is mindblowing. I have absolutely no clue how you collected and translated all this data into such cool visualizations, but i am in awe. this is so cool. thank you so much for making it!
Is this really your first video?! This is incredibly well done. So glad YT has recognized that your content is deserving of being pushed algorithmically.
“Just hanging out and admiring the scenery is more rewarding than exploring the world” Amazing work Peter! I look forward to see how this will progress
Bro honestly this is UA-cam video of the year. How spectacularly you presented this information in such a clear and entertaining way that is honestly on the level of professional science productions like Cosmos. Absolutely colossal performance man. I wouldn’t be surprised if you had an entire production team.
@@seveneyes77 Yep! They are an online publication that uses geek culture as a way to popularize science. They had a bunch of articles from the biology if final fantasy monsters to the effectiveness of super man disguise.
I’ve got no idea who you are but i can say how proud i am that someone tried this and had the patience to gather such interesting, noteworthy and valuable insights. Great work fam. Awesome explanations as well
Tbh I didn’t know youtube algorithm allowed channel with 1 video to pop off like this. Over 1 million views in 7 days?? If this video was posted in a sizable channel, it might have been even 10 times more.
Not sure if its been said already, but, I would love to see them beat the game. Then we can see what levels they got to and what they thought was the best pokemon to have for the elite 4. Would be interesting.
i really doubt if the AI can solve the Stone Moving "Puzzles" inside the IceCave and VictoryRoad thou. Can it even be teached to learn and use the VMs? but id love to see it :D
I remember back when there were 1 or 2 reinforcement learning videos on YT. Now we get all sorts. But this one...this one is special. The production value here is excellent. Thanks for all of your hard work.
If this hasn’t already been suggested, you should make a screensaver video of different cities populated with a ton of Ashes walking around-that shit is mesmerizing!
The amount of work you've put into this is so incredible. All of the self recording of _all_ of the AI iterations meant time spent (never wasted) for the sake of a single video. From the editing you've shown down to the research of how the human psyche works, this is beyond something I would even think to produce. You will go far in your endeavors.
Just 10 minutes in, and it has already gotten so damn interesting! The behaviors, the systems, the events, the unexpected but explainable scenarios, the AI literally experiencing something comparable to trauma? I want to see more!
The accidental traumatic depositing of Pokémon in the center is rather hilarious, and the Magikarp/fast food analogy is beautiful. Picking left is an ancient gaming trick, not surprised AI picked it up/that we make games that reward it. And lastly the short-term memory bit seems to me a great idea to solve this (and also, accidentally, rather human :P).
The only flaw in the fast food analogy is we'd need to learn that in the future eating fast food will make you live longer (or something else awesome) given what Magikarp evolves into!
This video was done incredibly! A perfect demo of and comparison to deep learning. A well earned follow. The dedication, creativity, and in depth descriptions are beyond impressive for this being the first video on this channel. Keep at it! I'll be looking forward to what ever you produce next!
Right off the bat I like this video because it actually goes into detail on how success is defined. Way too often this is skipped over and it absolutely breaks my brain because the implication of not covering this is that the AI somehow figured things out without any sort of goals defined.
This was a really excellent video! I'm super impressed that you managed to get this working - as a PhD student working with RL I understand that it can be an nightmare to debug! And I appreciated the depth of technical details you gave at the end. The presentation of the video was really good. I really liked how you eased into the deeper explanations and created lots of cool visualisations. I'm surprised that this is your first video and I hope you make more :)
Very late on it, but the reason it moves how it does against walls is 'The left wall trick' or 'The maze trick', if you the wall to your left and continue to move along that pattern you'll always be able to eventually find the way out of the maze.
Excellent video! I always love the videos where the AI attempts are overlapped. Makes me feel like I'm staring at a bunch of newborn ants explore the world. Also the Pokemon center "trauma" moment was so cute! Poor AI!!! You didn't do anything wrong!!
Really cool video! I don't see most AI channels talking about comparisons between people and the AI and it really added a lot imo Also really love that you included how you made the visualization of the lil guys running around, and good practical tips for training RL
This is great- I have no background with anything programmingwise but you made this into such an entertaining story. I hope that this blows up enough to get a sequel at some point, I'd watch this for HOURS!
Making it to viridian forest is already insane, it's got 3 separate if/thens to complete that don't have rewards aside from gameplay expansion, this is actually cool af
Can’t believe this is your first video. This was so entertaining to watch and the editing leaves me wondering how much time it took you. Hope you put out more videos like this and I’d love to see a full AI playthrough at some point!
Dayum, this video is so immensely well made and laid out! I thought you'd be a 6 digit subscriber channel by the sheer quality here, respect :) Not only did you visualize all the data in an amazing way, you even explained how this all went together and what kind of effort went into it to create it all. I was quite interested how you gathered all the data and then put it together on a map, and even that was shown. More than blown away by all this!
superb video, text and learning, good job! Also, on the relatableness of experience, occurrences and patterns, there's memetic we use now more than ever to render perceptible the archetypal moment in videogames where a point of inflection occurs and hold a certain humour that we choose to transmit via memes later on.
The counterclockwise motion may just be a result of how certain maps are layed out in the first could areas. You have to go to take the right path on Route 1 to get to Viridian, so following the right wall will get you there. Then in Viridian Forest, hugging the right wall is pretty much the fastest way to get through the forest, and I think later generations only deviated slightly to fight trainers for more experience once they realized that Brock was the main roadblock. Finally, in Pewter City, you need to take a counterclockwise path to get into the gym. So the AI probably didn't have a preference at first, but going in a counterclockwise direction for it to where it needed to go there fastest.
That sounds like a pretty good theory to me. Though if areas later in the game required a different pathing techniques... Would it be sophisticated enough to only use new pathfinding techniques when required and still use optimal pathfinding for the parts it already 'solved'? Or would it just have one skill set for it's pathfinding that will start to skew towards being mediocre at both sections but amazing at neither?
@@xxzombiekillerxx9549 not necessarily. If the first generations were evenly split between left and right then at the end of the gen it would have seen that right-preference exploration was yielding higher point totals. So a right turn preference would be developed simply from its higher score based results - especially along the first few routes in the game. It wouldn't require memory at all, just RL
YO the visuals and quality of everything in this video are wild! I could not ramble long enough to explain everything I love about it. The diagrams, explanations, overlapping AI attempts, just AAAA. AI videos like this are an amazing experience to watch, but imagine very difficult to make and monitor. It was wild seeing that this was your first video on a now 2k sub channel. Seeing it learn was also nostalgic to a degree. You could really feel its ‘personality’ by the end. Seeing its trials, learning behaviors, and what it was capable of was such a journey. I can’t wait to watch your channel grow!
This video reminds me of when I got Pokemon Yellow as a kid, I didn't read/speak english so I just had to try things to learn what everything did and was. It's weird how similar the AI playing feels to my experiences as a kid. The Pokemon games (among TV and other games) actually helped me learn english at the age of 9 far before my classmates could and as a little extra ROM hacking got me into graphic design and coding/web development somehow. Pokemon in general is the base of my origin story.
When I first played Pokemon, just like yourself I was still a kid didn't know any English so I couldn't even save. The first few months was just like the AI, start from that little room and trial and error.
Hello fellow ESL player, i was like 5 when I got my first hand on pokemon. I was EXTREMELY upset when I accidentally start over the game (the copy was second hand and the saved file is from my older brother who already completed the game.) that I cried. I lost my brother's charizard, even the moltres he caught with an ultra ball because i couldnt understand a lick of English back then - overwriting his save accidentally, and I just love exploring the pokemon world more than battling them. Only then 3 whole years later when I did restart and beat pokemon on my own, around 12 I became competently aiming to "gotta catch em all".
17:20 It does what I used to do in Diablo 1 when I was 10 years old. To get through the worlds, I would walk along the edges counterclockwise. I still do that today.
Did you edit this yourself? Not only is the content amazing, but I'm blown away by how well this was all put together and demonstrated. If this is really your first video that's seriously impressive
Dude, you are a genius! I am taken aback that this is your first video. Your skill, knowledge, production value and way of balancing what can be a dry subject with interesting information and funny tidbits is absolutely amazing! I am seriously jealous, with your skills you are gonna go far! I am subscribed! Would love to see more Pokemon AI stuff, but understand if you wanna go a different direction as well.
I noticed at 17:20 you mentioned you were unsure of why it chose to move in a single direction with limited memory. When I was a firefighter, we used a method of keeping our right or left hand on a wall while searching smoke-filled buildings. The method that AI used is more or less the same method and my hypothesis is that it learned it could discover new areas more easily by utilizing this technique which triggered more reward points at a faster rate.
I also noticed that the first few towns have right turn bias. That being if you follow the wall on the right you're more likely to get to a new area faster than the wall to the left
This is correct. If you wanted to escape a maze the slow but sure way, you would hug the wall until you reach the endpoint. So if you only made right turns the entire way, it may take longer but you would have a deterministic method of completing the maze. Using this method you could clear caves without needing Flash.
UA-cam really needs a "goated" option in addition to "like" and "dislike". Because in all honesty, a simply "like" doesn't give enough of a reward trigger to even begin describe how awesome this entire project is.
This brings back so many fond memories of playing Blue version on my pokemon edition GBC, learning how to evolve pokemon for the first time and finally getting through the forest. I've played the game so much, 25 years later I can STILL visualize the entire route of Rock Tunnel.
I can't believe this is your first UA-cam video, that's insanely impressive. I'd love to see how this works in other generations of Pokemon games, at least into gen 5. (The games past that get into 3D/not a grid so I figure it'd be harder to train?). The fact this is already put together so well might mean it's easier to make ones for other Pokemon games!! Also I love the thousands of little guys wandering the map!
It is has been 25 years since I first played Pokemon Red. It is crazy impressive how the AI is able to do what it does. I also never thought how nostalgia I would feel hearing this music again. Well done video and keep up the great work!
Anyone newer to Machine Learning: this video is such a great introduction to concepts such as: * Reward Functions * Misalignment * Emergent Behaviour And more!
I cant be the only one waiting for a new Video from this guy. This video got stuck in my mind since watching and im waiting for new content since that day. Hopefully you will bless us soon again, can't wait
I'm a 37 year old man who poured over the very first Nintendo magazine before it's release here in the UK many many years ago, I found this video to be a mixture of heart-warming, eye opening and hilarious. Great job, thank you for doing this it's so well put together.
Incredibly well made video! I think your resourcefulness and ability to explain things in non-technical terms shows a deep understanding of the topic. Plus the storytelling is top notch
An AI being traumatized by using a pc is the most ironic thing I've heard in a while
Haven't you seen twitch plays pokemon? PCs are a death sentence!
I didn't even make that connection 😂
@@mcstrategistI remember that. People were spamming. To get rid of pokemon. They had to ban people and make rules. That was pretty hilarious though.
@@nimi-naesame. But yeah that’s pretty funny
Sudden excessive punishment against a curiosity traumatizes first time experiencer
Seems all too legit 😅
I laughed so hard when the AI refused to press the A button when it lost.
Stalling to avoid the outcome confirmation. Reminds me of young children, actually. Haha
@@MP-lv5vk Sometimes the sound of a door slamming because of a gust of wind can remind me of children slamming their hands on a table. There is ZERO connection/homology between anything in the bot produced behavior, and the realm of human motivation or other emotions. It is logically impossible to learn anything about humans from literally everything about this showcase except by observing the actual human who decided to create this mathematical formula of instructions (algorithm) to a low level brute force bot.
@@JohnnyNatrium Yeah but it reminds me of children's stubbornness lmao
Children can be the sorest losers, refusing to keep playing is hilarious 😂
The only winning move is not to play.
it was unreasonably adorable when the AI stopped in Pallet Town to enjoy the scenery
Seconded
The AI is cute
Based AI knows true happiness.
Ok but did you see the little dance after beating the bug catcher on the first try?
Yeessss
I envision it talking everything in with a solemn smile, knowing that it’s about to leave this quaint town on a grand adventure of trials and learning.
‘Just one more moment at the banks of this familiar lake, then I’ll be off…’
I dunno why but the clips were all the AIs aimlessly walk around like a colony of small ants is unbelievably adorable to me
holy shit ai are the ants. or are ants the ai?
Is this a subtle nod to @SmallAnt ?😂
This is how Naruto trains himself: TAJUU KAGE BUNSHIN NO JUTSU. Then gathers the experience of each clone. :D
Adorable? What are you a fucking pixie?
All I can picture is a tsunami of Ash just rapidly taking over a country, one town at a time.
“The ai is learning how to move, and is just walking around” really explains a lot of my online teammates in first person shooters.
Bots
Like my team mates in LoL
Lvl 1 lukes in star wars battlefront 2 hvv
Npcs playing npcs 😢😮
@@jeffwooten6888"bot" sounds so negative. Maybe we should start calling them "reinforcemenrt learners" instead.
This must've taken an insane amount of time to not only simulate but also edit, really good video, nice work
Omg Dolan you fucking legend where you been
rN6media does the edits
Have you forgotten your password?
@@anouaressanoussiobviously not
@@MrGoodeatsyeah most youtubers dont edit their own content anymore
This was extremely well made. Great job
Holy cannoli it's science boi Kyle "Thor" Hill with his locks in the wild.
I see we spend our sunday nights similarly. Lmfao.
This is honestly one of the best endorsements this video could have
I’m certain the algorithm recommended me this video because of your comment
Its the goat 🐐
This man made a channel.
Dropped the best ai educational video using pokemon.
Didnt post or commented anything .
leaves.
What a legend
You can find him online. He's a software engineer and creates all kinds of interesting things.
@@mygirldarbyI just looked him up and wow this guy is a genius!
The ai discovering rng manipulation is mindblowing. I wonder if games in future could use ai to learn tedious or very specific glitches during beta testing.
They already do!
Dude it clicked as he was explaining it "wasn't optimal" but also repeating and I was like "NOOOOOOO!!!"
Why bother?
@@sdsd-f7k simple, ai thinks and tries things different to a human, it could discover stuff the devs wouldn't even imagine it was possible
This is an elaborate version of fuzz testing, which is the act of feeding random data to a program to see how it react.
Not that I don’t love the videos that just say “I applied an AI to this game and here’s how long it took to finish it” but this video (in addition to its high quality visuals and great script) is so much beyond that.
Instead of just watching a video on AI, we’re learning about reward implementation, the human condition, curiosity, and more and more. This went above and beyond, I was so rooting for our AI buddy by the end of this lol.
You're right! This feels like an in depth, academic essay
I want to see the AI beating the game
I want a full fledged version of this with atleast 20 hours to watch and actually make the AI complete the game, satisfying to watch, disappointed it had to end to end so soon :(
Hopefully the success of this video has encouraged him to finish what he had started im here waiting for it!
@@Fractisdnbisn't that how most videos on this topic are?
This is to boring to watch
The AI is cool and all, lots of comments discussing it, but. I just wanna say, the editing is so awesome for a video like these, you don't often see such excellent presentation
I'm honestly baffled by how this was animated. How did you get the scenes with the thousands of character sprites moving about, all overlapping one another?
@@Lone.Willow all is revealed at 26:27
200% this. Not only taking on the entire workload of the project, but taking the time making such an enjoyable and informative visual aid is stellar!
@@Lone.Willow Yeah that's what's wild, the AI stuff is sick, but the editing to show the iterations had me fucking floored.
I just thought the same, the presentation is amazing 👏
I’m so glad you didn’t stop when you said “this sounds like a reasonable stopping point”
But then he stopped not to long after 🥲
That’s exactly where I stopped to mess with the comment section lol
That whole traumatic experience with the PC and the Pokecenter was fascinating. Thank you for making this
The poor AI aww 😢❤
It triggered my Twitch Plays Pokemon PTSD
the analogies between human behavior and AI behavior were quite interesting in general, though the trauma sticks out. also kinda makes you think about ourselves, doesn't it? after all, this is ultimately just a statistic algorithm with a simple reward system, but it manages to show some rather lifelike emergent behaviors, which weren't inherently programmed it. then again, pretty much all of life is not that different, the model and algorithm are just much bigger and more granular and complex.
Indeed it happened to me when i was young, i didnt know how to withow pkmn bc the storage system was a mess so i didnt use the pc anymore xd
Reminds me of the trauma triggered whenever Twitch plays pokemon went near the computer after they accidentally released all those pokemon haha!
Honestly the AI becoming traumatized from the PC was heartbreaking. Poor lil guy didnt understand what happened
My heart dropped when it was revealed he never went back to the Pokémon center afterwards, I felt so bad for the guy.
@@istumbyright? Just imagine how rewarding it would’ve been to gain those total levels back! Probably would’ve broken the reward system, as there’s nothing keeping the AI from depositing the Pokémon just to get rewards for pulling it back out.
@@hunterwylie6969 Deposit, withdraw, deposit, withdraw like a junkie.
"The Pokémon center stole my only squirtle!"
Don't feel bad, they learn as they go!
"Just hanging out and admiring the scenery, is more rewarding than exploring the rest of the world." Never have I felt more like a machine learning algorithm than this sentence right here.
The digital world is more rewarding than the real world
Very relatable outcome!
Me too, why bother capturing and fighting when you can just chill and enjoy the motion of leaves and waves? Quite poetic
I felt truly humbled when the AI was just done with the game at Mt Moon. I feel like so many real world experiences ended here that this moment just HAD to happen.
My first thought was "Wow, imagine being so bad that you grinded to Blastoise in Mt Moon!"
And immediately after that "Wait, didn't I have a Blastoise by that point, too?"
I was so bad at navigating those "dungeons" as a kid 😂
(I still am, but I can look up maps now or be more strategic about it than aimlessly wandering around)
@@LeoTheDarkAngel I had my charizard when I battled misty for the first time :D
This was edited and put together so amazingly well. I haven’t even finished yet- I just needed to express my gratitude that you took the time to not only complete this project but edit the process in such a visibly appealing way. Thanks for 33 genuinely enjoyable minutes!
shit was boring asf, felt like a lecture lol
A Happy Way to Live
The servants who are ready and waiting for his return will be rewarded.
-Luke 12:37
All around us we can see fulfilled Bible prophecies, signs indicating that the return of Jesus Christ is drawing near.
As followers of Christ, we should be watching for Him. We need to be ready to go.
Jesus, speaking about His return, said, “Be dressed for service and keep your lamps burning, as though you were waiting for your master to return from the wedding feast. . . . The servants who are ready and waiting for his return will be rewarded”
(Luke 12:35-37)
Are you ready for His return? To be ready means to be engaged in activities that you wouldn’t be ashamed to be doing if Jesus were to return. It’s a good idea to periodically ask ourselves this question: This place that I am about to go, this thing that I am about to do, would I be embarrassed if I were doing it when Jesus came back?”
Think about your plans. Is there anything you will be doing today, tonight, or tomorrow that you would be ashamed to do if Christ were to return? If so, then change your plans. You want to be ready for His return.
Not only should we be ready, but we should anxiously await the return of Christ.
We used to have a German Shepherd who slept outside the bedroom, leaning against our door. We didn’t let him sleep in our room because he often had nightmares and would wake us up. Every morning when we opened the door, he rolled into the room. Then he’d jump up and start running in circles. He was genuinely happy to see us.
That is how we should be waiting for Christ’s return. And anything that might prevent us from saying “Come quickly, Lord Jesus” is out of place in our lives.
In addition to waiting, we should be working. Every now and then, someone predicts that Jesus will return on a specific date. People believe these predictions and start quitting their jobs or divorcing their spouses.
But that is not what we should be doing as we wait for the return of Christ. Instead, we should be working for Him.
The Bible says, “Just as the body is dead without breath, so also faith is dead without good works”
(James 2:26)
If watching is the evidence of faith, then working is the evidence of faith in action. Watching for the Lord’s return will help us prepare our own lives. But working will ensure that we bring others with us to Heaven.
The great British preacher C. H. Spurgeon said, “It is a very blessed thing to be on the watch for Christ. . . . You can be poor without murmuring; you can be rich without worldliness; you can be sick without sorrowing; you can be healthy without presumption. If you are always waiting for Christ’s Coming, untold blessings are wrapped up in that glorious hope”
When you live in the anticipation of Christ’s return, it’s a happy way to live.
I'm so glad the UA-cam algorithm decided to recommend your video and I clicked on it. It's a fascinating thing to watch the process and journey that the AI goes through, while the presentation of the whole video is equally fantastic. Great video, you all deserve a round of applause for the effort and quality put into this whole project.
i love that the AI decided to just hang out and watch the scenery. reminds me of my favorite poem “Stopping by the woods on a snowy evening” by Robert Frost
Everybody likes Robert Frost
I’ve done this many times in my play throughs with Pokémon, it’s actually scary how much the AI “mimics” human behavior.
@@piciperkuadrik4636not True I actually HATE Robert Frost
You have good taste. That's a beautiful poem
@@danielserrano929because we’re in a simulation 😂
Peter: *starts to train an AI to play Pokemon*
Magikarp Seller: "It's time to become a multi-millionaire."
😂
He literally did earn 5 million lmao since the guy said that around 10000 of the ais bought the magikarp
watching the little reds go round like an ant colony brings me so much joy and i don't know why. look at them all exploring. learning. discovering the world. lil guys. thank you for spending at least 1000USD and several hours putting this together just for me to uncontrollably laugh at the reds for 20 minutes
..with that out of the way, fantastic video. incredibly readable visuals and clear voiceover, awesome topic, understandable for several levels of previous knowledge. can see this hitting the high hundred thousands.
I was looking for this comment because I thought the same!!! It was like watching ants!! Just amazing!! This video exploded my mind... Imagine a Pokemon game were you can compete against a real "rival" (blue) in real time just to see who wins the league first... And every run the rival gets different pokemons with different moves... This guy is just insane, this is like a Pandora box!!!! New sub for sure!!!! And thank you for this video Peter!!!!!
Seeing high effort videos like these from relatively low sub channels always surprises me. Definitely deserves more recognition/subs.
It's the only video on bro's account lmfao wdym
@@napoleonbonerfarte6739lol was about to write this too
Good things take time
And people who over react to low sub channels being high quality doesn't surprise me. Lots and lots of dumdums out there
I honestly expected this video to be from a youtuber with thousands of subscribers, to see that you only have 60 baffles me, this is an incredibly well-made and well-put together video.
yeah i thought the same, its gone up to 400 now but still nuts
tbf, it's his first video.
@@androsp9105 yeah I only realised that after I left this comment, even more nuts lmao
He’s gained nearly 5,000 in a few days. Very good going.
Misuse of commas.
The technical expertise that went into this is astounding. As a lifelong pokénerd and career software engineer, I applaud you for capturing my attention with such a captivating topic and experiment. But as just a person, I thank you for relating it back to the human condition.
Realizing that we get distracted by false or incorrectly calibrated reward systems gives us awareness, the first step in the right direction toward pursuing real, meaningful value out of life. I would love to hear more of this from your obviously talented and insightful, inspirational mind.
If you ever do have the AI finsih the game, I think it would be really cool if you let the same AI try Pokemon Gold. I think seeing if an AI trained on Gen 1 could play Gen 2 that would be an interesting experiment
Obviously it'd have to relearn how to navigate the map, but it'd probably do well in battles since it already knows how
It wouldn't be able to catch the farfetchd or use cut
This game would fail too with the hm's
I'm going to do this as a project for my machine learning class, and I am planning on trying the same algo on Gen 2.
do make a video@@geekygecko1849
@@geekygecko1849how can I follow along?
It's one thing to set all this up, and it's another to visualize and present it in such a coherent and digestible way. You did both so well! Hope to see more content from you in the future
Agreed! This video is insane!
I can't believe it's done by individual. Super high quality.
This is their first UA-cam Upload, it’s crazy to me how much work, effort and money went into its production without having built an audience on an already successful channel before. Mad props to you Peter.
I am looking forward to see what else you will create.
He is an employee of Amazon Headquarters in Seattle 👏🏽👌🏽 He is smart af
It stopped to look at the scenery, exactly like humans do
It was petty and refused to press A for its defeat message simply just to not be told it lost
It rage quitted avoiding brock because it lost too many times
Your AI is just a literal human todler
This AI is genuinely adorable
That was incredible! I’ve always wondered if this was possible, I’m blown away by what the AI was able to learn! The visualizations and presentation were excellent, I hope this video reaches a wide audience!
Haven't even finished the video yet, but I want this to pop off in the Algorythm, this video had tons of efforts put into it, and deserves to get out there.
I've got some good news for you, that's how I found this video
The algorithm brought me here
Guess I'll throw on a comment too then. This is great!
yesss this was so cool
Thanks then
Since I'm all into both Pokémon and coding, UA-cam suggested your video just minutes after you uploaded it. I subscribed after a few minutes watching it, and now I watched it again and noticed you have almost 50k subscribers! With just one video! Please take that as a public, worldwide testament of the effort you have put into this. Thank you so much!
Broke yt 😂
If you like Pokemon and AI, you'll love this: ua-cam.com/video/KWwJDyBH8ig/v-deo.html&ab_channel=Spawnvilley
As a Pokemon enthusiast with 4 Pokemon tattoos and a data analyst aspiring to become a data scientist, this project was one of the coolest to watch! I was so fascinated that I decided to replicate the project myself. I encountered some difficulties along the way, but the Discord community was incredibly helpful. Congratulations on the project! 🙌
Thanks for telling us all about your 4 Pokémon tattoos, that's just the proof we needed that you really like the games. All of us had proof standards met, you're definitely a fan. Congratulations bro.
How easy is it to replicate and can you have it play other games such as gold? Or even newer gens like fire red. Itd be cool to get it to beat red. And then see how long it takes to get through other generations
As a psych prof I'm always trying to think of different ways to explain certain concepts and give relatable examples, and this one is perfect!
They tell me I’m crazy here 🤪
@@MasteringSilenceCrazy? I was crazy once
@@norabarlow17 you only lose your mind once… They put me in a rubber room with rubber rats…
@@norabarlow17they locked me in a room. A rubber room with rats.
As a psych professor can you explain the appeal to these people repeating the copy paste comments?
Also just to be clear I'm also asking out of genuine curiosity if there may be psychological reasons past the basic wanting to be a part of something, and not just trying to hate on them or anything ✌
Fellas, I'm an AI engineer, with a short background in Reinforcement Learning for a period I interacted with Sony for a job.
I need you to understand the MAGNITUDE of these results. It's an insane work, and I'm sad that probably only a few might understand the sheer amount of skill require to do this.
Insane job man, you are a goat
This is no understatement . This takes a level of focus and problem solving that is just not normal. Savage!
I’m not even an engineer, and my jaw is on the ground.
I genuinely would love to learn how to become a part of this world. I wish there were more people in my circle with hobbies and fascinations like this.
I used to help write xml codes for world of Warcraft bots when I was a kid. Now laying in bed with an alarm set for five hours from now. I’ve got a sales job… is 33 years old too old to learn how to work in this scene?
This video drips with knowledge, and a wisdom and understanding of something that I have no idea how to even begin to approach.
Kudos!!
I wouldnt say those results are impressive theory wise ? The impressiveness of the work comes from a technical point of view, how great he managed to link the RL model with the game and the fine-tuning he put in it. By the way, AI engineer doesnt really mean anything, what is your job title ? Out of curiosity
@bricegardner7815 no age is too high. With enough determination and curiosity you can definitely pivot. Look into videos explaining the skills required to get a job in game development/ AI.
@@alr9447 I am officially a data scientist, but within the team I'm the guy responsible of the training of the ML models, therefore I make this distinction because nowadays "data scientist" is too broad. In most big tech companies, AI engineer is a common notation to distinguish between the data science folks
Extremely impressive visualization of the simultaneous iterations. It can be hard to grasp that machine learning is happening in batches of mass parallel attempts, not each progressive scenario after another one by one. Excellent video!
I'm absolutely dying for an update, I keep checking this channel every few months to see if there is another video
I'm only 12 min in and looked to see if there were more videos! But alas...
The visualizations of the AI exploring is actually insane! Seeing the entire map and all iterations moving looks so dope, especially with the arrows indicating their average movement. Sick video!
It was like watching what PokeMMO would’ve been in the early 2000s
A Happy Way to Live
The servants who are ready and waiting for his return will be rewarded.
-Luke 12:37
All around us we can see fulfilled Bible prophecies, signs indicating that the return of Jesus Christ is drawing near.
As followers of Christ, we should be watching for Him. We need to be ready to go.
Jesus, speaking about His return, said, “Be dressed for service and keep your lamps burning, as though you were waiting for your master to return from the wedding feast. . . . The servants who are ready and waiting for his return will be rewarded”
(Luke 12:35-37)
Are you ready for His return? To be ready means to be engaged in activities that you wouldn’t be ashamed to be doing if Jesus were to return. It’s a good idea to periodically ask ourselves this question: This place that I am about to go, this thing that I am about to do, would I be embarrassed if I were doing it when Jesus came back?”
Think about your plans. Is there anything you will be doing today, tonight, or tomorrow that you would be ashamed to do if Christ were to return? If so, then change your plans. You want to be ready for His return.
Not only should we be ready, but we should anxiously await the return of Christ.
We used to have a German Shepherd who slept outside the bedroom, leaning against our door. We didn’t let him sleep in our room because he often had nightmares and would wake us up. Every morning when we opened the door, he rolled into the room. Then he’d jump up and start running in circles. He was genuinely happy to see us.
That is how we should be waiting for Christ’s return. And anything that might prevent us from saying “Come quickly, Lord Jesus” is out of place in our lives.
In addition to waiting, we should be working. Every now and then, someone predicts that Jesus will return on a specific date. People believe these predictions and start quitting their jobs or divorcing their spouses.
But that is not what we should be doing as we wait for the return of Christ. Instead, we should be working for Him.
The Bible says, “Just as the body is dead without breath, so also faith is dead without good works”
(James 2:26)
If watching is the evidence of faith, then working is the evidence of faith in action. Watching for the Lord’s return will help us prepare our own lives. But working will ensure that we bring others with us to Heaven.
The great British preacher C. H. Spurgeon said, “It is a very blessed thing to be on the watch for Christ. . . . You can be poor without murmuring; you can be rich without worldliness; you can be sick without sorrowing; you can be healthy without presumption. If you are always waiting for Christ’s Coming, untold blessings are wrapped up in that glorious hope”
When you live in the anticipation of Christ’s return, it’s a happy way to live.
Dropping a comment to help the algorithm. This video honestly deserves millions of views. I love the part where the AI learned to RNG manip to catch a Rattata. It's one of those moments that's unexpected at first but when you go back and look at it it's like, "oh, of course it would react like that!" Moments like those are why I love AI learning videos like this.
As a physicist i appreciate those visualizations. This is truly remarkable content.
wtf does you being a physicist have to do with anything? guess you just wanted attention.
Everything about this was amazing, the computational approach, the video edit, the tone, the explanations and the real life parallels. Beautiful work!
I really like how grounded and transparent your breakdown of the AI capabilities and limitations is, it shows it as a tool and not as a magical solve-all-problems strategy. Also, what a masterful storyteller and explainer you are. This video is very well paced and laid out, congrats!
Yes it's limited but imagine what it could become in a few more years 🤖
This might be the coolest video of AI playing a video game I've ever seen. I love all the fascinating emergent behaviours (especially the RNG manipulation), as well as the analogies you draw to humans. I also love that you presented the technical explanations in a way that allowed me understand almost everything without any programming knowledge, just a decent understanding of AI. Genuinely amazing job, I hope to see more like this in the future! :)
This might be the most interesting, fascinating and satisfying video I've watched on UA-cam so far. Hats off, I'm looking forward to future videos!
this video is mindblowing. I have absolutely no clue how you collected and translated all this data into such cool visualizations, but i am in awe. this is so cool. thank you so much for making it!
The AI naming Squirtle “AAAAAAAAAA” killed me! 😂Thanks, amazing content.
AI picked the Squirtle in Pokemon Red lol what a contrarian
i was hoping someone else had mentioned this
@@RevanBC that was its only option...
@@Tyler-qh7bf
No you can pick 2 other pokemon! idiot.
Pigeoto was ‐-----------
Is this really your first video?! This is incredibly well done. So glad YT has recognized that your content is deserving of being pushed algorithmically.
16:13 The ai never making it out of mount moon is pretty relatable, to be honest
“Just hanging out and admiring the scenery is more rewarding than exploring the world”
Amazing work Peter! I look forward to see how this will progress
Bro honestly this is UA-cam video of the year. How spectacularly you presented this information in such a clear and entertaining way that is honestly on the level of professional science productions like Cosmos. Absolutely colossal performance man. I wouldn’t be surprised if you had an entire production team.
thank you for the kind words :)
no production team, but my friend @torinblankensmith made the thumbnail
I second this. I'm super interested in the content, but at the same time I'm like.... However did he make this look so good.
It’s not that deep dude holy shit
@@glupshitto1977 its deep.. learning.
@@Tom-yg7mi get out
This was an amazing project and explanation. You should submit this to The Journal of Geek Studies if you don't have a publication lined up already.
Wah is that a thing?
@@seveneyes77 Yep! They are an online publication that uses geek culture as a way to popularize science. They had a bunch of articles from the biology if final fantasy monsters to the effectiveness of super man disguise.
I’ve got no idea who you are but i can say how proud i am that someone tried this and had the patience to gather such interesting, noteworthy and valuable insights. Great work fam. Awesome explanations as well
Holy crap this is your very first YT video? I can't wait to see what you cook up if you continue to create! Outstanding work!
Tbh I didn’t know youtube algorithm allowed channel with 1 video to pop off like this. Over 1 million views in 7 days?? If this video was posted in a sizable channel, it might have been even 10 times more.
he paid for the view XD@@clickpwn
@@bilibangbanghow you know that?
It gets better when u go into the git-hub project and find out that he has been working on this for the last 2 years...
@@bilibangbangmald
Not sure if its been said already, but, I would love to see them beat the game. Then we can see what levels they got to and what they thought was the best pokemon to have for the elite 4. Would be interesting.
Charizard with Slash, easy
This will took a looooott of time and video preparation edit etc. But I agree would be awesome
i really doubt if the AI can solve the Stone Moving "Puzzles" inside the IceCave and VictoryRoad thou.
Can it even be teached to learn and use the VMs?
but id love to see it :D
I think it would be hard to program the rewards to get them through the specific obstacles tho like using cut in certain places etc
Yeah they would shatter Wersters Speedrun World Record!
I remember back when there were 1 or 2 reinforcement learning videos on YT.
Now we get all sorts.
But this one...this one is special. The production value here is excellent.
Thanks for all of your hard work.
If this hasn’t already been suggested, you should make a screensaver video of different cities populated with a ton of Ashes walking around-that shit is mesmerizing!
They told me my Pokémon phase would pass. Little did they know, it was just evolving into an AI obsession!
Pokémaniac Bryce Huston wants to battle!
@@sanjaywilson8232LMAO!
*Pokemon Trainer Battle Theme starts playing*
Edit : Go Lucario !
Fight Pokemon
Bag Run away
The amount of work you've put into this is so incredible. All of the self recording of _all_ of the AI iterations meant time spent (never wasted) for the sake of a single video. From the editing you've shown down to the research of how the human psyche works, this is beyond something I would even think to produce. You will go far in your endeavors.
This guy's first video and it's about using ai, so UA-cam AI said "I gochu"
Just 10 minutes in, and it has already gotten so damn interesting! The behaviors, the systems, the events, the unexpected but explainable scenarios, the AI literally experiencing something comparable to trauma? I want to see more!
The Red swarm wasn't enough?
me too sad when he stop at moon mt.
The AI doesn't experience anything because it's not a conscious entity. It experiences as much as Microsoft Word when you open it.
@@Elintasokas😅😂😂
@@Elintasokas Based on my PCs heavy breathing when I open Word I assume its orgasming.
This was absolutely amazing, my friend! Please do more of these! I must admit I was disappointed that you didn't do the whole game 😂
This is one of the best implementation and visualization videos on the subject I've ever seen. Amazing work!
This was awesome, I'd love to see a full series of the AI completing the game.
Yes!!
downloaded it and train the ai more
And then i'd like to see it completing the game as fast as possible. An AI speedrun competition: winner gets 100,000 arbitrary points
The accidental traumatic depositing of Pokémon in the center is rather hilarious, and the Magikarp/fast food analogy is beautiful. Picking left is an ancient gaming trick, not surprised AI picked it up/that we make games that reward it. And lastly the short-term memory bit seems to me a great idea to solve this (and also, accidentally, rather human :P).
I was feeling sad for the AI who must have thought it accidentally killed its Pokémon 🥲😂
The only flaw in the fast food analogy is we'd need to learn that in the future eating fast food will make you live longer (or something else awesome) given what Magikarp evolves into!
i thought the traumatic experience was super interesting too and funny lol
3:55 "Just hanging out and admring the scenery is more rewarding than exploring the world." That's deep damn
This video was done incredibly! A perfect demo of and comparison to deep learning. A well earned follow. The dedication, creativity, and in depth descriptions are beyond impressive for this being the first video on this channel. Keep at it! I'll be looking forward to what ever you produce next!
Your findings, implementation, logic, and ANIMATION is incredible. 👏👏
Yep. This is easily one of the best videos regarding Pokémon on UA-cam.
Right off the bat I like this video because it actually goes into detail on how success is defined. Way too often this is skipped over and it absolutely breaks my brain because the implication of not covering this is that the AI somehow figured things out without any sort of goals defined.
adding ai to Pokémon added absuloutely nothing to the game. same idea as staring out a window and twiddling your thumbs worthless NPC behavhiour
This was a really excellent video! I'm super impressed that you managed to get this working - as a PhD student working with RL I understand that it can be an nightmare to debug! And I appreciated the depth of technical details you gave at the end.
The presentation of the video was really good. I really liked how you eased into the deeper explanations and created lots of cool visualisations. I'm surprised that this is your first video and I hope you make more :)
5 mil on your first video. Great quality, good research and break down. Congrats, can't wait to see what you bring next!
This is the coolest video I have ever seen! You did beautiful job visualising all the games being played simultaneously.
Where are the uploads
Very late on it, but the reason it moves how it does against walls is 'The left wall trick' or 'The maze trick', if you the wall to your left and continue to move along that pattern you'll always be able to eventually find the way out of the maze.
Excellent video! I always love the videos where the AI attempts are overlapped. Makes me feel like I'm staring at a bunch of newborn ants explore the world. Also the Pokemon center "trauma" moment was so cute! Poor AI!!! You didn't do anything wrong!!
Really cool video! I don't see most AI channels talking about comparisons between people and the AI and it really added a lot imo
Also really love that you included how you made the visualization of the lil guys running around, and good practical tips for training RL
This is great- I have no background with anything programmingwise but you made this into such an entertaining story. I hope that this blows up enough to get a sequel at some point, I'd watch this for HOURS!
Making it to viridian forest is already insane, it's got 3 separate if/thens to complete that don't have rewards aside from gameplay expansion, this is actually cool af
This is one of the most fascinating things I’ve ever seen. You deserve (1) reinforcement point in the form of an award. 🤙
Dude this was mind-blowing, the explanation and the editing were crazy good, best AI learning video I've ever seen
Can’t believe this is your first video. This was so entertaining to watch and the editing leaves me wondering how much time it took you. Hope you put out more videos like this and I’d love to see a full AI playthrough at some point!
I was kinda disappointed that it didn’t conclude with the AI defeating the Elite 4 😢
I got a lot of insights from watching the video. The visuals were the best.
Dayum, this video is so immensely well made and laid out! I thought you'd be a 6 digit subscriber channel by the sheer quality here, respect :)
Not only did you visualize all the data in an amazing way, you even explained how this all went together and what kind of effort went into it to create it all. I was quite interested how you gathered all the data and then put it together on a map, and even that was shown. More than blown away by all this!
As a Data Scientist, this was amazing to watch :) well done !
I wanted to be a Data Scientist then I realized I couldn't code😂
This video was amazing 💯
Fuck you guys doin here! Balllllllsssssss Deep!
Damn, didn't expect you here 😂
Is he a GOAT?!
Lol, baallls deep
When One Piece meets Pokémon
superb video, text and learning, good job!
Also, on the relatableness of experience, occurrences and patterns, there's memetic we use now more than ever to render perceptible the archetypal moment in videogames where a point of inflection occurs and hold a certain humour that we choose to transmit via memes later on.
The counterclockwise motion may just be a result of how certain maps are layed out in the first could areas. You have to go to take the right path on Route 1 to get to Viridian, so following the right wall will get you there.
Then in Viridian Forest, hugging the right wall is pretty much the fastest way to get through the forest, and I think later generations only deviated slightly to fight trainers for more experience once they realized that Brock was the main roadblock.
Finally, in Pewter City, you need to take a counterclockwise path to get into the gym. So the AI probably didn't have a preference at first, but going in a counterclockwise direction for it to where it needed to go there fastest.
That sounds like a pretty good theory to me. Though if areas later in the game required a different pathing techniques... Would it be sophisticated enough to only use new pathfinding techniques when required and still use optimal pathfinding for the parts it already 'solved'? Or would it just have one skill set for it's pathfinding that will start to skew towards being mediocre at both sections but amazing at neither?
@@ForcefulDragon i feel as though that would require the ai to access long term memory instead of short term which was described in the video.
@@xxzombiekillerxx9549 not necessarily. If the first generations were evenly split between left and right then at the end of the gen it would have seen that right-preference exploration was yielding higher point totals. So a right turn preference would be developed simply from its higher score based results - especially along the first few routes in the game. It wouldn't require memory at all, just RL
YO the visuals and quality of everything in this video are wild! I could not ramble long enough to explain everything I love about it. The diagrams, explanations, overlapping AI attempts, just AAAA. AI videos like this are an amazing experience to watch, but imagine very difficult to make and monitor. It was wild seeing that this was your first video on a now 2k sub channel.
Seeing it learn was also nostalgic to a degree. You could really feel its ‘personality’ by the end. Seeing its trials, learning behaviors, and what it was capable of was such a journey.
I can’t wait to watch your channel grow!
This video reminds me of when I got Pokemon Yellow as a kid, I didn't read/speak english so I just had to try things to learn what everything did and was. It's weird how similar the AI playing feels to my experiences as a kid.
The Pokemon games (among TV and other games) actually helped me learn english at the age of 9 far before my classmates could and as a little extra ROM hacking got me into graphic design and coding/web development somehow. Pokemon in general is the base of my origin story.
damn bro that is deep.
Me with Spanish at 3 years old and English at 2 ahah Pokemon Azul and Pokemon Red 😅
When I first played Pokemon, just like yourself I was still a kid didn't know any English so I couldn't even save. The first few months was just like the AI, start from that little room and trial and error.
Hello fellow ESL player, i was like 5 when I got my first hand on pokemon. I was EXTREMELY upset when I accidentally start over the game (the copy was second hand and the saved file is from my older brother who already completed the game.) that I cried. I lost my brother's charizard, even the moltres he caught with an ultra ball because i couldnt understand a lick of English back then - overwriting his save accidentally, and I just love exploring the pokemon world more than battling them.
Only then 3 whole years later when I did restart and beat pokemon on my own, around 12 I became competently aiming to "gotta catch em all".
This is how I taught myself English at 6 years old, pokemon yellow and a dictionary provided by my parents. Wild
17:20 It does what I used to do in Diablo 1 when I was 10 years old. To get through the worlds, I would walk along the edges counterclockwise. I still do that today.
Did you edit this yourself? Not only is the content amazing, but I'm blown away by how well this was all put together and demonstrated. If this is really your first video that's seriously impressive
My first thought when it panned out. Like who tf is this guy lol
He used JRGMediaYT for the edits
He probably asked an AI to edit it 😅
Dude, you are a genius! I am taken aback that this is your first video. Your skill, knowledge, production value and way of balancing what can be a dry subject with interesting information and funny tidbits is absolutely amazing!
I am seriously jealous, with your skills you are gonna go far!
I am subscribed! Would love to see more Pokemon AI stuff, but understand if you wanna go a different direction as well.
It look like, he run this program for himself too, to get the best possible reaction to his first video. And we all part of a simulation :O IMAGINE
Amen to this!
I see great science potential here in multiple purposes/ subjects!
I noticed at 17:20 you mentioned you were unsure of why it chose to move in a single direction with limited memory. When I was a firefighter, we used a method of keeping our right or left hand on a wall while searching smoke-filled buildings. The method that AI used is more or less the same method and my hypothesis is that it learned it could discover new areas more easily by utilizing this technique which triggered more reward points at a faster rate.
what if the wall is on fire
@@faizanulhaq8349 probably going to be pretty hot…
I also noticed that the first few towns have right turn bias. That being if you follow the wall on the right you're more likely to get to a new area faster than the wall to the left
@@maxng7211 Probably because the game is circling right (start on the left of the whole map, go up then right then down then left).
This is correct.
If you wanted to escape a maze the slow but sure way, you would hug the wall until you reach the endpoint.
So if you only made right turns the entire way, it may take longer but you would have a deterministic method of completing the maze.
Using this method you could clear caves without needing Flash.
UA-cam really needs a "goated" option in addition to "like" and "dislike". Because in all honesty, a simply "like" doesn't give enough of a reward trigger to even begin describe how awesome this entire project is.
This brings back so many fond memories of playing Blue version on my pokemon edition GBC, learning how to evolve pokemon for the first time and finally getting through the forest. I've played the game so much, 25 years later I can STILL visualize the entire route of Rock Tunnel.
This is absolutely insane and so much work went into this! It's so cool to see how the AI learns like humans tend to
I love AI so much, this is probably one of my favorite applications and showcase of an AI program. Insanely entertaining and interesting!
I can't believe this is your first UA-cam video, that's insanely impressive. I'd love to see how this works in other generations of Pokemon games, at least into gen 5. (The games past that get into 3D/not a grid so I figure it'd be harder to train?). The fact this is already put together so well might mean it's easier to make ones for other Pokemon games!!
Also I love the thousands of little guys wandering the map!
It is has been 25 years since I first played Pokemon Red. It is crazy impressive how the AI is able to do what it does. I also never thought how nostalgia I would feel hearing this music again. Well done video and keep up the great work!
Anyone newer to Machine Learning: this video is such a great introduction to concepts such as:
* Reward Functions
* Misalignment
* Emergent Behaviour
And more!
I cant be the only one waiting for a new Video from this guy. This video got stuck in my mind since watching and im waiting for new content since that day. Hopefully you will bless us soon again, can't wait
I'm a 37 year old man who poured over the very first Nintendo magazine before it's release here in the UK many many years ago, I found this video to be a mixture of heart-warming, eye opening and hilarious. Great job, thank you for doing this it's so well put together.
this is honestly worthy of an entire course's final project at the graduate level. Thank you for making this freely available!
Isn’t it just! I’m currently half way through my final project for my MSc, with a relatively shit regression model predicting energy usage. 😂
Incredibly well made video! I think your resourcefulness and ability to explain things in non-technical terms shows a deep understanding of the topic.
Plus the storytelling is top notch