It doesn't know what to do so it acts chaotically. Humans do the same. We just also get distressed because our brains recognise the problem. We don't act chaotically because we're distressed, it's just that acting chaotically and getting distressed often have a common cause - we don't know what to do.
I never thought I would hear the sentence "There are, in fact, 54 other dinguses training in the exact same level at the exact same time" that caught me off guard
I was coming to the comments to ask exactly this, @Dingus_labs can you please show us Dingus going through all the levels with his new found knowledge?
@@mondongoloco7902 Oh it's a little fella that appeared on an image I AI generated. Full image: i.pximg.net/img-original/img/2023/02/12/21/34/03/105319111_p10.png
It's incredible how fast this ppo setup learns. I've never seen anything like it. Reinforcement usually takes much more trial and error and training time.
Yeah 55, he did still learn super fast, I didn't have too many inputs or actions for him to perform which does help with learning but it was still far ahead of what I expected
when dingus finished all the levels i think it would have been interesting to see you put him back at level 1 and see how all his training thus far would have affected him from the start.
There's definitely a risk of overfitting, where the AI has become to specialised for a specific task and struggles to solve more generic problems. Particularly for the last level where he trained on it for a long time and learned to stay very close to the centre.
Maybe try something where instead of just static stages you set parameters and generate a seed kind of like a rogue like? That way it could potentially force the AI to try and use general information rather than fitting to a stage.@@Dingus_Labs
It would be interesting to see a course where it starts with a path forward, but that path is not possible. It is required for you to backtrack behind the start to progress
@Dingus_Labs I could imagine it working if it knows things like going from point A to point B (aka start to finish). Basically have it have a target location to go to (like the top of the hill here) and once it does that, the map changes its "finish" location. I'll use emoticons here as a diagram. 🟨 map ⏹️ end location 💗 Dingus 🟢 ball 🟦 obstacle 🟥 off map 🟪 reward location 1️⃣2️⃣3️⃣4️⃣ segmented finishes (Basically larger rewards or finish locations that change places once reached) Part 1 🟥⏹️⏹️⏹️⏹️⏹️🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 2 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟥🟥🟥🟥🟨🟪🟨🟥🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 1 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨1️⃣🟨🟦🟨4️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨3️⃣🟨🟥🟥 🟥🟥🟨2️⃣🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 2 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨1️⃣🟨🟦🟨3️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨2️⃣🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 3 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨1️⃣🟨🟦🟨2️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥 Part 3 - round 4 🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥 🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟦🟨2️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟪🟨🟨🟨1️⃣🟨🟥🟥 🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥 🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥 🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
I think you should've added a level where the boulders rolling in the opposite direction are randomized, cuz it looks like it just memorized the pattern in which they're rolling down instead of associating the boulders with moving out of their way
the problem with that is that dingus has no vision. He's like a little blind man with a ball for a walking stick. if he loses it, he has no idea where he is
@ it literally says in the first sentence of the video that it is deep reinforcement, which means he does have vision and that vision has nothing to do with the ball
With these AI trainers that have levels, I really would enjoy if at the end of the entire training sequence, we could get a nice all--previous-levels-in-one level that Dingus could do. It would be so satisfying to see all the training pay off on what we've already seen to be really hard at first become easy at the end
having the ability to actually read its environment seems to have drastically increased the speed of the learning process. it at the very least seems a substantial improvement over 'blind' learning
@@speenta4879 its when the ai has access to only the 'success/fail' information of its task. without some means of 'seeing' its environment the only thing the ai can do is spam random inputs until it gets a reward for accomplishing one of the preasigned tasks. when you give the ai the ability to determine what objects exist around it, it can actually use that information to make decisions. when you give a blind ai a new level its basically impossible for it to beat it first try. but if the ai can 'see' and is very well trained it can do a level its never seen before and first try it with relative ease
@@speenta4879 "most" non-blind ai is so unusual that im actually surprised when i hear about one that does. theres a decent handful that get some very basic stats feed into it, but one that can actually see stuff around it even somewhat like a player playing a video game. then again i dont keep up with learning ai very much anymore... so maybe thats changed
I now want to see a video of Dingus dealing with openings where Dingus can walk through but the ball cannot, and things like marble rails where Dingus can't walk on but the ball can, so basically, Dingus needs to learn to let go of the ball. That would be VERY interesting to see and another thing I want to see is alternate paths. Have a LOT of paths and see which one Dingus prioritizes? The easiest? The fastest?
wow it's kind of a crime you don't have more subs!!! i'll have to keep an eye out for more dingus content, watching this video and cheering on a cute little ai pushing boulders delighted me very much
He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
What algorithm did you use? Also, are you able to open source the boulder pushing environment? I'm researching hiearchical reinforcement learning and planning and your boulder environment could be a pretty useful benchmark.
I use ppo, If you're keen to learn deep reinforcement learning hands on, I highly recommend mlagents toolkit for unity, it comes with tons of examples, already set up example environments and proper readmes with set up steps. Awesome for learning!
@@wyndhamwynne2480 I'm apprehensive to share any repos because I do make use of some commercially purchased textures and models, and they restrict redistributing, which means I'd have to find a different way to store those files and share, and people using the repo would still need to replace those missing assets with something else, I don't yet have a simple and quick solution to that issue.
@@Dingus_Labs that's a shame but understandable. Personally, I'm not interested in any models or textures, or even trying to reproduce locally, I'd only interested in reading through the code and learning more about deep reinforcement learning and specifically ppo. It's mainly the architecture design (width & breadth) for particular applications and the cost/loss function and back propagation to weights & bias adjustment that is unclear to me. Any references in these areas to good learning material would be greatly appreciated!
It would be cool if u made this into a game where people train their own ai on their own maps and then u release a map and see who trained their ai to beat the map the quickest or last the longest in a tounament. I think it would be pretty fun. Great vid btw.
He never stops trying (until the specified Max steps have been hit on training the model). By visualisation do you mean the rays coming out of dingus early in the vid?
I have a fun idea you could make a live stream of dingus pushing up a boulder a long track while chat messages can cause boulders to roll down and mess with him
I've thought about some live stream stuff! I'm not in a position to do it right now but I'm hoping to have the opportunities sometime in the coming months.
Would have liked to see him go back to the other levels after beating the last one to prove he actually learned all the skills instead of overfitting to each level. Am also curious to see the source code for the network
this video has been popping up in my recommended for 3 days and I've avoided it, I didn't want to watch a video of a cube pushing balls, finally I gave in and ended up watching almost 15 minutes of a cube pushing a ball, damn you Dingus
You could probably get him to not go down the slide easily if you really wanted by adding a minor negative reward function to Dingus's height, but honestly I thought it was cute how he went down the slide too and makes a lot of sense considering the potential of the boulder sliding off the side.
Does learning later courses negatively affect earlier ones? e.g. if you redo everything with the latest model, will he smash it first time on each course, or still mess up and have to learn a bit more?
@Dingus_Labs: He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
One thing i think would have been cool was a course where he has to go down hill beckwards then back up. Wonder he that would work with the reward systems?
Im fairly confident that dingus just brute forced most of the levels instead of actually learning because (read more) A: he was repeating a never changing level B: movements were mostly VERY dsimaler to the last generation C: When you double checked his boulder dodging, it completly failed D: It was predodging obstacles before even spawning
yeah, you're probably right but I did notice it did some real learning as well. at 5:36, he stops at the end of the platform, which is something I believe he learned. alternatively, it could just be he picked the best one from every parallel instance and this was just the one that happened to stop.
Movements being similar to last generation isn't too unexpected, changing hyperparameters to encourage more experimentation would have resulted in more variance per generation, I was fortunate that he managed to get through everything with my initial settings. The predodging obstacles is very interesting for sure. And yeah, having static levels does invite brute forcing, I did make Dingus spawn in a random rotation each respawn to add some small amount of variance per run but it's not enough to counter it.
Not only did you make a great video, not only did you create an intelligent ai, but you even came up with an original game idea! This actually looks fun, in a getting up or jump king kind of way!
I'm not convinced the AI isn't just memorizing inputs to beat the levels. It seems like it remembers how to move the ball forwards, left, and right, but aside from that, it doesn't look like it's using it's senses to actually determine anything. It looks like it's just memorizing inputs.
Dingiphus enjoys pushing his boulder, it generates dopamine (or the virtual equivalent) in his brain. Therefore, one sees him happy upon watching him complete these challenges and doesn't have to imagine it.
great video! it would be good for the boulder to spawn at a somewhat random x-position every time so the AI doesnt learn to rely on it spawning in the center.
Something I always would want to see in this kind of video is some kind of abstraction of the inputs for the AI so we get some insight into the context it considers. Like "is it aware of the slopes, if so, is it an abstracted into levels of slopyness or an realistic value" or "is it aware of its level of control over the stone" And not only out of sheer curiosity, but also since people often want to compare AI learning with human learning, and in that regard the question wether it is "very sloped" or "sloped by 60°" seems like an important difference.
“Ah…a visitor…indeed. The kingdom of heaven has long since forgotten my name…and I am eager to make them remember. So come forth child of man… and…die” - Sisyphus prime 20XX
One must imagine Dingus happy
Legends say he was happily pushing boulders even before he was put into this strange simulated environment.
One must imagine dingus upside-down, its hilarious
@@Dingus_LabsHmmm...
I knew this comment will be here as i saw the title of the video.
Dang it. He beat me to it
Give Dingus a top hat that progressively gets taller every video.
YES
but wait nono one that gets taller everytime he gets a reward
@@paper177 then one no longer needs to imagine Dingus happy, for he is actually happy
free hat
@@paper177is that dimmagingus, owner of the dimmsdale dimmadingus?
it acts so convincingly distraught whenever it loses the ball
He lost his ball. Why do you think he’s not distraught. I would freak out if I lost my ball too.
He so sad 😢
boulder*
It doesn't know what to do so it acts chaotically.
Humans do the same. We just also get distressed because our brains recognise the problem.
We don't act chaotically because we're distressed, it's just that acting chaotically and getting distressed often have a common cause - we don't know what to do.
@@alansmithee419 so you're saying that dingus is the next stage in human evolution?
I never thought I would hear the sentence "There are, in fact, 54 other dinguses training in the exact same level at the exact same time" that caught me off guard
Dinguses, or Dingi? (Din-guy?)
Please have seggs with me no brainer gamer
@@rennegaddefoxxedinguses is an actual word so let's go with that until we get comfirmation
@@frowner_and_co cactus/cacti ... octopus/octopi ... Dingus/dingi. /s /jk
your right! the plural of dingus would be dingi I didn't think about that...@@rennegaddefoxxe
I always enjoy these machine learning videos but at the end I always wish they'd run the trained AI through all the levels/courses again.
Definitely. Want to see the difference between the new and final ai
Wtf is ur pfp
yeah, or maybe a final level with all the elements from previous levels
I was coming to the comments to ask exactly this, @Dingus_labs can you please show us Dingus going through all the levels with his new found knowledge?
@@mondongoloco7902 Oh it's a little fella that appeared on an image I AI generated.
Full image: i.pximg.net/img-original/img/2023/02/12/21/34/03/105319111_p10.png
It's incredible how fast this ppo setup learns. I've never seen anything like it. Reinforcement usually takes much more trial and error and training time.
What inputs did you use?
I have a feeling that there's a couple hundred instances running in parallel.
55, he says so
Yeah 55, he did still learn super fast, I didn't have too many inputs or actions for him to perform which does help with learning but it was still far ahead of what I expected
@@Dingus_Labsit seems like dingus doesn’t struggle with overfitting, thats something ive never seen on any other ai vid. This is super cool dude
when dingus finished all the levels i think it would have been interesting to see you put him back at level 1 and see how all his training thus far would have affected him from the start.
One must imagine Dingus happy. Or speedrunner
Or combine all levels into a mountain.
For sure. I believe reinforcement learning is pretty prone to forgetting.
There's definitely a risk of overfitting, where the AI has become to specialised for a specific task and struggles to solve more generic problems. Particularly for the last level where he trained on it for a long time and learned to stay very close to the centre.
Maybe try something where instead of just static stages you set parameters and generate a seed kind of like a rogue like? That way it could potentially force the AI to try and use general information rather than fitting to a stage.@@Dingus_Labs
I can easly imagine AI taking over the world and having this guy to push boulders for the rest of his life.
One must imagine Dingus Labs happy.
It would be interesting to see a course where it starts with a path forward, but that path is not possible. It is required for you to backtrack behind the start to progress
Yeah, depending on how the model has trained and how the reward function is programmed it can really struggle to adapt to situations like that!
So a maze
@Dingus_Labs
I could imagine it working if it knows things like going from point A to point B (aka start to finish). Basically have it have a target location to go to (like the top of the hill here) and once it does that, the map changes its "finish" location.
I'll use emoticons here as a diagram.
🟨 map
⏹️ end location
💗 Dingus
🟢 ball
🟦 obstacle
🟥 off map
🟪 reward location
1️⃣2️⃣3️⃣4️⃣ segmented finishes (Basically larger rewards or finish locations that change places once reached)
Part 1
🟥⏹️⏹️⏹️⏹️⏹️🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟪🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 2
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟥🟥🟥🟥🟨🟪🟨🟥🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 1
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨1️⃣🟨🟦🟨4️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨3️⃣🟨🟥🟥
🟥🟥🟨2️⃣🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 2
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨1️⃣🟨🟦🟨3️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨2️⃣🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 3
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨1️⃣🟨🟦🟨2️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨🟪🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
Part 3 - round 4
🟥🟥🟥🟥🟥⏹️⏹️⏹️⏹️⏹️🟥
🟥🟥🟥🟥🟥🟥🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟦🟨2️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟦🟨🟨🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟪🟨🟨🟨1️⃣🟨🟥🟥
🟥🟥🟨🟨🟨🟨🟨🟨🟨🟥🟥
🟥🟥🟨🟢🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨💗🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟨🟨🟨🟥🟥🟥🟥🟥🟥
🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥
I think I have a bunch of Dingus cubes running my brain
Well, use reinforcement learning on them for the next 3 years.
I think you should've added a level where the boulders rolling in the opposite direction are randomized, cuz it looks like it just memorized the pattern in which they're rolling down instead of associating the boulders with moving out of their way
the problem with that is that dingus has no vision. He's like a little blind man with a ball for a walking stick. if he loses it, he has no idea where he is
@@Slappydafrog_that makes no sense? Thats just not true and has nothing to do with the comment you replied to
I was going to leave this same comment. I would also want to see things like the ball/dingus spawn points on the start platform being randomized
@ it literally says in the first sentence of the video that it is deep reinforcement, which means he does have vision and that vision has nothing to do with the ball
@@Slappydafrog_ 0:06
the robotic urge to push a boulder up a hill for eternity
I dont remember pushing this boulder
In a way, we’re all just like Dingus
this was the most emotional video i've ever watched
watching dingus fall off at the very end made me cry
and when he succeeded i was so proud of him
😢
With these AI trainers that have levels, I really would enjoy if at the end of the entire training sequence, we could get a nice all--previous-levels-in-one level that Dingus could do. It would be so satisfying to see all the training pay off on what we've already seen to be really hard at first become easy at the end
So theoretically, if you let the AI run for long enough and teach it to get rewarded with faster times it would become the perfect speedrunner
In theory. I’m not 100% sure you could teach an AI to use glitches or exploits, nor could it find them on it’s own
@@Delmworks I’ve seen reinforcements algorithms frequently find glitches/ exploits. I’m an ML Engineer.
the fastest sisyphus
having the ability to actually read its environment seems to have drastically increased the speed of the learning process.
it at the very least seems a substantial improvement over 'blind' learning
I don’t know much about this subject what do you mean by “blind learning”?
@@speenta4879
its when the ai has access to only the 'success/fail' information of its task. without some means of 'seeing' its environment the only thing the ai can do is spam random inputs until it gets a reward for accomplishing one of the preasigned tasks.
when you give the ai the ability to determine what objects exist around it, it can actually use that information to make decisions.
when you give a blind ai a new level its basically impossible for it to beat it first try. but if the ai can 'see' and is very well trained it can do a level its never seen before and first try it with relative ease
@@brentramsten249 oh that’s interesting so some ai doesn’t get information on it’s surroundings and only learns based on trial and error?
@@speenta4879
"most" non-blind ai is so unusual that im actually surprised when i hear about one that does.
theres a decent handful that get some very basic stats feed into it, but one that can actually see stuff around it even somewhat like a player playing a video game.
then again i dont keep up with learning ai very much anymore... so maybe thats changed
@@brentramsten249I've seen more "non-blind" AI than "blind" AI. I have no idea what gave you that idea.
6:35 It's literally like Naruto training with shadow clones and then absorbing all their experience 😂
Somewhere there's an operator looking at you on screen and commenting on it as if you were named Dingus and constantly messing up. Awesome video!
I now want to see a video of Dingus dealing with openings where Dingus can walk through but the ball cannot, and things like marble rails where Dingus can't walk on but the ball can, so basically, Dingus needs to learn to let go of the ball. That would be VERY interesting to see and another thing I want to see is alternate paths. Have a LOT of paths and see which one Dingus prioritizes? The easiest? The fastest?
car
wow it's kind of a crime you don't have more subs!!! i'll have to keep an eye out for more dingus content, watching this video and cheering on a cute little ai pushing boulders delighted me very much
Dingus: what is my purpose?
*You push rocks up hills*
Dingus: Oh god
"The Dingus you are watching is not the only Dingus training at this time." THERE ARE MULTIPLE DINGI!
woah, this is some high quality content, this man deserves more subscribers
agreed
People never show the final AI doing all the levels again
He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
@@Dingus_Labs I'd love to see that!
Dingus master of the shadow clone jutsu using 57 clones to speed up is learning pace. One must be proud of this dingus and his vast chakra reserves.
You have shamed, and embarrassed dingus with these unusual trials. He and his brethren will come back in full force to extract vengeance.
What algorithm did you use? Also, are you able to open source the boulder pushing environment? I'm researching hiearchical reinforcement learning and planning and your boulder environment could be a pretty useful benchmark.
I use ppo,
If you're keen to learn deep reinforcement learning hands on, I highly recommend mlagents toolkit for unity, it comes with tons of examples, already set up example environments and proper readmes with set up steps.
Awesome for learning!
He used 54 other dinguses
@@Dingus_Labsdo you have a shared repo for this project, which I could learn more about PPO RL from?
@@wyndhamwynne2480 I'm apprehensive to share any repos because I do make use of some commercially purchased textures and models, and they restrict redistributing, which means I'd have to find a different way to store those files and share, and people using the repo would still need to replace those missing assets with something else, I don't yet have a simple and quick solution to that issue.
@@Dingus_Labs that's a shame but understandable. Personally, I'm not interested in any models or textures, or even trying to reproduce locally, I'd only interested in reading through the code and learning more about deep reinforcement learning and specifically ppo. It's mainly the architecture design (width & breadth) for particular applications and the cost/loss function and back propagation to weights & bias adjustment that is unclear to me. Any references in these areas to good learning material would be greatly appreciated!
It would be cool if u made this into a game where people train their own ai on their own maps and then u release a map and see who trained their ai to beat the map the quickest or last the longest in a tounament.
I think it would be pretty fun.
Great vid btw.
I just watched some of your first videos and this one. Your editing and commentary these days is a vast improvement! Keep up the good work!
Interestingly this felt like a news segment
You dingus! You've automated the indomitable human spirit!
Also, how'd you make this visualization?
He never stops trying (until the specified Max steps have been hit on training the model).
By visualisation do you mean the rays coming out of dingus early in the vid?
I have a fun idea you could make a live stream of dingus pushing up a boulder a long track while chat messages can cause boulders to roll down and mess with him
I've thought about some live stream stuff! I'm not in a position to do it right now but I'm hoping to have the opportunities sometime in the coming months.
bro created an intelligence purely for it to suffer eternal punishment
One must imagine dingus happy, the preprogrammed reward of push a boulder up hill is enough to fill a mans heart
Would have liked to see him go back to the other levels after beating the last one to prove he actually learned all the skills instead of overfitting to each level. Am also curious to see the source code for the network
this is really cool! i'd love to see one of these AIs try going up against harder and harder bullet hells
this video has been popping up in my recommended for 3 days and I've avoided it, I didn't want to watch a video of a cube pushing balls, finally I gave in and ended up watching almost 15 minutes of a cube pushing a ball, damn you Dingus
im so happy this video did so well, i love your content and you need more people watching you!!
one must imagine dingus happy. 😔
Wow this is amazing, more people should see this!
i need one of those youtube boxing tournaments between Dingus and Sisyphus
Adorable character, good editing, good narration. Watch this man blow up.
You could probably get him to not go down the slide easily if you really wanted by adding a minor negative reward function to Dingus's height, but honestly I thought it was cute how he went down the slide too and makes a lot of sense considering the potential of the boulder sliding off the side.
Awesome video I would expect this quality from a much bigger channel! Video blowing up in 3 2 1…
You should have more subscribers, this is amazing! Also I'm surprised that this is the video that got more than 10K views. Good job!
I appreciate there wasn't the same patronizing basic explanation of machine learning at the beginning of every AI video on youtube here, great vid
This could easily be turn into a game
this man is criminally under-rated
Fr
Im glad this video got blessed by the algorithm. Ill be watching all of your videos now! Good stuff ❤
"Think you outsmart the biggest dingus in the room?"
Does learning later courses negatively affect earlier ones? e.g. if you redo everything with the latest model, will he smash it first time on each course, or still mess up and have to learn a bit more?
@Dingus_Labs: He would very likely struggle, the changes to the model after he's finished a level means he'd likely need to relearn for a bit when he goes into a different level
One thing i think would have been cool was a course where he has to go down hill beckwards then back up. Wonder he that would work with the reward systems?
the all mighty algorithm has led me to this video! i subscribed instantly. I hope to see your channel grow more and more!
for 200 subs the production qualitys amazing man
Thank you! I'm trying my best to get better with editing!
Does dingus "see" the boulders coming toward him? Does he "see" the whole map?
Im fairly confident that dingus just brute forced most of the levels instead of actually learning because (read more)
A: he was repeating a never changing level
B: movements were mostly VERY dsimaler to the last generation
C: When you double checked his boulder dodging, it completly failed
D: It was predodging obstacles before even spawning
yeah, you're probably right but I did notice it did some real learning as well. at 5:36, he stops at the end of the platform, which is something I believe he learned. alternatively, it could just be he picked the best one from every parallel instance and this was just the one that happened to stop.
@@maxwellbenedict5167 or luck
Movements being similar to last generation isn't too unexpected, changing hyperparameters to encourage more experimentation would have resulted in more variance per generation, I was fortunate that he managed to get through everything with my initial settings.
The predodging obstacles is very interesting for sure.
And yeah, having static levels does invite brute forcing, I did make Dingus spawn in a random rotation each respawn to add some small amount of variance per run but it's not enough to counter it.
Not only did you make a great video, not only did you create an intelligent ai, but you even came up with an original game idea! This actually looks fun, in a getting up or jump king kind of way!
Getting up? you mean Getting Over It?
@@lance499 oh yeah, you’re right.
Dingus: "what is my purpose?"
DL: "You are sisyphus"
Dingus: "MY GOD."
I'm not convinced the AI isn't just memorizing inputs to beat the levels. It seems like it remembers how to move the ball forwards, left, and right, but aside from that, it doesn't look like it's using it's senses to actually determine anything. It looks like it's just memorizing inputs.
Bro you’re blowing up!! Congrats man :)
I feel sick as all hell right now, but watching this i imagined Dingus happy and that made me happy :)
please do more like this. much better than the previous video formats
Great video! Subscribed.
I just have few suggestions :
1) add a generation counter
2) show how well the final model deals with the first few levels
Yo man, your content is great.
Dingiphus enjoys pushing his boulder, it generates dopamine (or the virtual equivalent) in his brain. Therefore, one sees him happy upon watching him complete these challenges and doesn't have to imagine it.
Dingus looks like it's ready to check my vibe.
Welp, instant sub. Please continue doing videos like this one
This could honestly work as a game
*clicks on the video
*sisyphus meme theme starts playing
love these videos keep making em
WE MAKIN' IT OUT OF TARTARUS WITH THIS ONE 🗣🗣🗣🔥🔥🥶🥶
great video! it would be good for the boulder to spawn at a somewhat random x-position every time so the AI doesnt learn to rely on it spawning in the center.
really high quality videos for such a low sub count. love this and love how this guy experiments with ai
dingus makes me emotional in a way i never thought was possible
Something I always would want to see in this kind of video is some kind of abstraction of the inputs for the AI so we get some insight into the context it considers.
Like "is it aware of the slopes, if so, is it an abstracted into levels of slopyness or an realistic value" or "is it aware of its level of control over the stone"
And not only out of sheer curiosity, but also since people often want to compare AI learning with human learning, and in that regard the question wether it is "very sloped" or "sloped by 60°" seems like an important difference.
even the sisyphus’s job is getting automated by ai
This video has earned my subscription
Give dingus legs so he needs to learn how to walk and jump.
I'd love to see more of your Dingus
Your are disgusting-ly underrated how tf are ya not at 100k?!?! Look ya have a bright future on UA-cam
“ This dingus.. To hold.. Me? “
Congrats, your hard work has paid off and you've been selected by the UA-cam algorithm!
Commenting to make UA-cam recommend me more stuff like this. Good video!
Dingus finna build some pyramids soon 🧐
One must imagine dingus educated in the art of pushing boulders up insane hills.
You got the algorithm jackpot, gained 50 subs in a few minutes
"One must imagine Sisyphus happy. "
I saw your channel name and within the first 10 seconds of this video I subscribed.
Excellent shit mate
now we need to play "me and the birds" by duster in the back
One must imagine dingus happy
Oh god I can already hear the countless people imagining Dingus happy
2:07 Just like me when I played the 8-ball missions in the Splatoon 2 Octo Expansion DLC.
sysiphus was kind of a dingus so this seems appropriate
We need a Dingus Prime.
“Ah…a visitor…indeed. The kingdom of heaven has long since forgotten my name…and I am eager to make them remember. So come forth child of man… and…die” - Sisyphus prime 20XX
one must imagine ai pushing a boulder up a hill
We live
We love
We Dingus
But honestly very much enjoying the videos man!! I am actually very much considering making some fan art of dingus lmao
He's hard to get just right in art.
All those straight lines, and also circles!
This would be one hell of a rage game if you decided to make it one