Not meant to take away from the achievement(its a big big achievement), but something to keep in mind. The dota game wasn't complete dota, for example not every hero was available (15 out of 115 heroes), no summons or illusions, no scans, 5 couriers that are invulnerbale per team instead of 1, no divine rapier, no bottle. While some of these may seem like a insignificant change, it greatly reduces the complexity and strategy potential of the game. Enough that people argue that it's not "real dota" yet. (ps. Dota doesn't stand for defence of the ancients because of legal reasons, and it is not a rts)
also not "some of the best players", if I recall correctly it was just a team of dota analysts. the story might have been completely different if bots played against vp or liquid with their signature heroes, who are currently the 2 strongest teams in the world.
Michael_Keehl They will try sth like that at TI this year i think. Of course the hero pool will still be limited but they will play against pros... And i mean... 99.95th percentile is "some of the best players in the world"
That isn’t “completely” true. The limitations on the game they played tends make ‘death ball’ strategies and relying on reaction speed the optimal approach to winning. It’s already understood that software is much better at reaction speed tasks than humans (even if the AI reaction speed is artificially limited, it could act at the very edge of that speed limit). You can see this particularly in the games that were played by the fact that instant usage of curse attacks by one of the support units was key to winning several of the team fights. The additional game mechanics that were removed from this competition are in the game to minimize the effectiveness of death ball strategies. By limiting the game mechanics that were in play, they have changed the nature of the problem that AI had to solve. While its anticipated that they will be able to add additional complexity to the model and improve its ability to play ‘more’ of the game, it’s also not certain at this point that reinforcement learning will continue to perform well as the type of problems the AI has to solve continue to expand. When the AI has to handle secondary objectives and can’t death ball its way to victory, it might not continue to perform as well. Or it might, that is what makes the research interesting - but it would be hyperbole to at this point to say that AI ‘beat’ humans in DOTA. Its also not completely clear at this point that the reinforcement learning approach has allowed the AI to outperform other methods of automation such as rules based approaches that are typically used in game development (and those other methods have substantially lower resource costs as of now). It was a good result, but AI superiority in DOTA is far from established as it is in chess and go.
I've never had anyone take me in-depth on the actual techniques and thought process and decision algorithms behind OpenAI. Great video man, appreciate it. I had to pause quite a few times and replay bits to fully make sense of them and grasp them. It makes me sad tho they said they're no longer going to dedicate anymore time or release any more DOTA2 AIs.
Your hustle is freaking unreal man. Sweating through your shirt at the end of this video ;). Seriously though, anytime I have a question about ml, ds, or anything ai related, you’ve done a video on it, and your delivery of content has improved soooooo much. Been watching for a long time and just wanted to say I truly appreciate the value you are providing to the world. I’ve heard you say that making this stuff is your life’s purpose and I can see that it’s well in alignment. Cheers
Great explanation! Thank you! Imho it's crazy how many ML-algorithms they manage to 'combine' to make this happen. Looking forward to the day when OpenAI beats the TI-Winners in a show match of 'full' dota
Love your videos mate. From months of following your videos I’ve been able to implement a real world model to optimise underground mining (gyro crusher) equipment using a deep LSTM with PPO. Keep up the awesome informational videos!
Great, informative video. Ummm...Siraj? Is there a CONSTRUCTIVE version of DOTA that Alfred Nobel would approve of that I could train a multi-agent system on? E.g. A Cliodynamics, cooperative, positive sum, non 4X variant of Freeciv and Age of Empires. I'm interested in the application of A.I. to Livingry. I want to create a moderator for Buckminster Fuller's World (Peace) Game. PPO reminds me of Bayesian analysis and comments made by Edwin T. Jaynes regarding certainty factors in Expert systems.
hi siraj, im curious is the input literally like in the video ? it should be using CNN right? if it isnt (which is the input from the game itself like in the video). i considered that as cheating. the AI should be identified itself the object2 and data2 in the game. which is human do btw u can ignore this if im too stupid
Hi Siraj, please consider the data-ink ratio in your videos. The golden summations over the wavy background may have been too much. Looks pretty but really I couldn't get a single thing of what you said while looking at it. I mean, focusing throughly and reading the topic I could... but, to retain the concept, neither the summations nor the decorations helped. Even more, the decorations may have distracted me.
I think you missed out on the most important aspect of OpenAI Five, being that the five bots operate completely independent of each other to achieve a shared goal (only receiving information from the other bots by the state of the environment and their actions). They explained the mechanism behind creating cooperative bots by implementing both individual and team awards and assigning a "team spirit" weight to the bots to balance these two. The actual purpose and real-world application behind this research is not just having usable AIs trained with unsupervised learning, but that they can create cooperative AIs that function together (to some degree) for a greater good.
I was watching that game live and cheering up for the machinei love your channel siraj please make more videos about AI in games (chess, poker, dota, etc)
I don't think it takes years. I've only been studying ai for a few months and I pretty much understand what Siraj is talking about. And I'm by no means very smart. I've just recently started learning LSTM's, so this is relevant. I think if I continue to take online courses and watch guys like Siraj, that I'll pick it up before too long. The only problem is we're at a point where there are a lot of advances in new ai developments, and that is why it's important to have people like Siraj who can keep us from getting lost.
Siraj Raval Not really i only know a little bit about AI atm and dont think you could have done better in 13 minutes! It is very rare to see a well edited video that has actual usefull explanation in it thats not just an analogy.
I literally remember watching your first Python video and you had like 250 subs, but your content was solid, niche, unique, and well put together (obviously still very rough but above avg)--however I did not sub at that time because only 1 video so far really resonated with me, however I've been on UA-cam since 2005 and I can fuckin tell good from nothing special to bad and I've been damn near flawlessly right. Then you continued with steady releases, continued content, and building your community and fast forward to 2018 and you're killing it. Good stuff. I've been thinking of getting into building an AI for the game Super Smash Bros Melee. There's a huge competitive scene around it and I've played it forever, but I just can't let the thought go that it'd be the perfect game. Small ass stage, small number of movements, no variables (on certain maps) but literally the character in front of you. Just a bare screen with 2 characters on it that move around. AND you can emulate it on PC and play with keyboard controls lol. I want to start on it but can't find the final bit of motivation but maybe I will sometime this upcoming month. I read something about a university with a group made something like that and they were fairly successful beating at least ranked players (not top 50 I believe, but still impressive). Before I go, sub'd and liked. Loved this and hope you keep detailed stuff on OpenAI and DOTA, I hope they play a full fledged TI boot camped, ready team at TI. Like a Chinese team against full out OpenAI with all items and characters open etc. Hell even if hero limited should be fine. BTW, I've fully given up hope on the human side unfortunately.....I believe it's going to soon become impossible to beat OpenAI...I mean it's played near a thousand years worth of DOTA.....ffs man. Some of the strategy the AI was doing was fuckin insane....it's like--it made SO MUCH SENSE seeing them do that, but no one actually did. People are fuckin afraid. Why afraid of full 3 lane swap by 6min in the game. Why afraid of giving supports full farm and rotating like they did. Keeping Lich against SF the whole game to deny creeps and exp. Having Axe do deep dives of wave cutting in unique manner, going for stats no regen, stacking 3 at weird spots and grouping at unorthodox times....it all seems weird because we're taught to play and watch DOTA from pros a certain way, but there's like this whole Pandora's Box of untapped and near infinity strategies that humans just haven't made enough time or progress to flesh out and explore all possibilities and routes. And also when they were losing and AI was doing literally every bit of maximize efficiency to deter that by giving up calculated risks and plays to further a prolonged goal in the future in an unknown territory of infinitily changing variables...to win playing live. I also believe the leader of OpenAI is tired of talking about it in terms of DOTA, he's known in his mind he's shit on DOTA and hates it and wants to move on to the next thing--to him I say, OpenAI can't play against every situation yet and I'm sure humans will find flaws to exploit. At the end of the day, AI is no more than biased optimization.
Hi Siraj, awesome explanation on what RL algorithms OpenAI used in training its agents againts DOTA - i.e. PPO. Just to clarify here - so this is different from what DeepMind used in AlphaGo correct - I recall it was RL using MCTS? Can you make a video to explain both in more detail and the differences please? Tq
The AI just won vs some casters, they lost every game on main stage even vs the "worst" pro team that didn't even qualified for the international main event. Don't get me wrong i love see how fast AI learn and how approach the game and i hope they keep developing those bots but we will say they are better than best dota 2 pro players when they really are, for now they are only better than a casual highrank (not toprank/"immortal") player. Nice video!
Took me about two years to figure out that Reinforcement learning was behind all advances in AI like this or intelligent robots like autonomous vehicles and drones. I feel a little down for not having the time I had back then, but with AI, EVERYONE can buy time. Hope so
This is very interesting. One thing I am wondering is whether open AI is receiving better input data then the players. The players are just playing from what they see, is the bot working with information that can be all extracted from the screen, or does it have more knowledge about the game state than the human?
Empiricists Academy no its the same as the bots.. they only see as much as human players see.. thats why the bot still need the uses of some sort of scan ability.. the only advantage that the bots had was quicker response time than human players. bcos human still need to click mouse button & press the keyboard..
Do you know how to solve the warding problem? The bots place wards randomly but how do you tell the bots to place wards that spot people going for a kill? Is it enough if you just tell them placing wards = less kills for the opponent and then let them play 180years of dota?
Oliver Schneider with enough training the bots will learn to place wards in useful places because those games will have a higher chance of winning and useless wards are just a waste of money and will thus have a lower chance of winning
the AI is dumb, places wards under tower/where you already have vision. If the ai spends gold on stat items instead, that will lead them to having a higher winrate than when placing wards. That means the ai wont learn how to ward?
Always hated when youtubers beg for subscriptions before I have even seen the video. How am I supposed to know if the content is any good to deserve a subscribe.
Visualized Math i think ai in fps games will be extremely unbalanced. AI is always automatically better than players at mechanical skill at an almost perfect rate, and in FPS games, mechanical skill is everything when its perfect. A bot would just instantly headshot anyone they see the moment they see a pixel of them
But not a deep rl bot. A deep rl bot would not only *learn* to aim, but to also strategize properly. This would also be much complex to execute, as the outputs of the ai would have to be discrete.
@@Funny9689 It would still be insane good after training. Of course it would be nice to see strategy but they would need to lower accuracy or reaction time. Even in Dota they probably make bot a little slower because it would be too good
50 years from now we might not have generals but an AI controlling tank and troop movement, which enemy targets to prioritize, optimized trajectories for aircraft to follow to bypass air defenses...pretty scary!
dota isn't really markovian. it is partially observable so it's really pomdp. it isn't markovian in a sense that you may observe the game-state but inside the fog your enemies is doing something different. for that reason you need lstm to integrate over some history to do-away with the partially observable aspect of the fog.
Good content, it`s hard to find this balance btw amnt of details and length/accesibility, but there are SO many excessive effects and, sorry, but little bt awkward jokes.. All random font popup effects, particles.. 9:40 - kinda hard to focus on your words when you blend into the background and half of screen is some bright (unrelated?) vid..
Nice Video. Very impressiv what openAI managed to do. But to be fair, at the moment they play only with 5 defined heros. I am exited when they will manage to win a regular 5vs5.
I've got a question, is the feat here that it won or that it at all capable of playing a multiplayer game? Because I recall Heroes 3 had a pretty tough AI, although the game did "cheat" towards the AI side to make it stronger...
I think this video just introduces some basic concepts about policy gradient. It doesn't dive into the five architecture so much. But it is a good video😁
Love your stuff, Siraj, never afraid to turn the learning volume up to "11"... but these animations are "ppt transitions on acid", and nearly giving me a seizure (seriously, at the very least they are terrible for folks w/ even minor adhd... I need to close my eyes to listen & read the slides on "pause".) Equations can be "fun" without the *extreme* s p e c i a l e f f e c t s ...
That's an interesting fact (180 years a day), do you any source for that? About the design of a civilization game, that sounds like a good idea. However, we need to be comprehensive to design such a game so that we understand what each AI actually trying to do and rather than we deciding naval campaigns, I would let AI decide which aspect of the development is more important for the sustainability of the civilization.
yeah, the goal of both game is same "destroying enemy throne". I have never played LOL. but I hear dota2 has better mechanics and requires more time and skill to master.
great work still needs improvement AI just master part of the game ( probably 20%) without beating the best teams out there and playing against random 6k-7k MMR players with some limitations
Dota is a strategy game. League of Legends is more of a MOBA. This misunderstanding is common for those who don't play dota above the 3K bracket (~Archon IV) and just play casually. However, in League, there is very little room for strategy in comparison to Dota.
Dota is a MOBA. Yes, there is a lot of strategy involved but that doesn't change the fact that the genre of the game is MOBA as opposed to RTS. AoE, CnC etc are RTS games.
Some idea of how I can use openai for web scraping to create a dataset of any website. Someone told me that I could use universe but I see that the repository is already archived, and in Openai I only see examples for games.
It is not finished yet! AI still needs some rules to defeat the pros! NO Divine Rapier or Bottle; No summons or illusions; Without using a scan! The couriers are immortal and they are 5 for each bot and you must not use them for scan territories! And also there are limited hero pool! only 10 heros when Dota has over 100 heros! SO I Believe AI is far away from conquering this game!
All of the movement in the frame makes me feel a bit nauseous, especially the part where you have an actual ocean in the background. Still, the content was interesting enough that I couldn't stop watching, only now I feel a bit sick.
"the best Dota players" is abit of a exaggeration, if we compare those players to a GO ranking system that human team would be 2-3dan at most if I was being generous, its like saying a c league soccer team is one of the best teams. Yes this is true if you look at all soccer players, but in the pro scene those c league soccer teams arn't really relavent. In addition there were many restrictions on the game (some would say that wasn't even Dota since there were so many restrictuons.) which made it eaiser for the Ai. Later when the AI played a real team at TI, during the games the pro team exposed alot of the AI's weaknesses and made the bots look kinda stupid, although this is still a mayor step for AI and I am sure next year there will be alot of progress.
Well, you really do not know the DOTA2 game. The Bots only beat some lame commentators. The bots still cannot learn the fundamental core of the game, even though they played against themselves for a gazillion years. The reason they beat the commentators is they have better control over the heroes.This is still a big big achievement, and what the 5 team should do, I think, is to let the bots learn from the replays. Simulate the same environment of a replay and let the bot learn from humans and train based on that.
For those people mocking about playing dota2 with a controller... that seems that haven't read anything about RL. Controller stands for : agent and system for: environment, while reward and state are sensory and feedback signals. This are control theory concepts which not so many people without an engineering background understands.
Not meant to take away from the achievement(its a big big achievement), but something to keep in mind. The dota game wasn't complete dota, for example not every hero was available (15 out of 115 heroes), no summons or illusions, no scans, 5 couriers that are invulnerbale per team instead of 1, no divine rapier, no bottle. While some of these may seem like a insignificant change, it greatly reduces the complexity and strategy potential of the game. Enough that people argue that it's not "real dota" yet.
(ps. Dota doesn't stand for defence of the ancients because of legal reasons, and it is not a rts)
CapelaGames Yeah would have been too much for this year i guess. Next year they will probably have the whole game if they continue to do the project.
also not "some of the best players", if I recall correctly it was just a team of dota analysts. the story might have been completely different if bots played against vp or liquid with their signature heroes, who are currently the 2 strongest teams in the world.
Michael_Keehl They will try sth like that at TI this year i think. Of course the hero pool will still be limited but they will play against pros... And i mean... 99.95th percentile is "some of the best players in the world"
It doesn't matter. Once the AI model works it can be trained for any variation.
That isn’t “completely” true. The limitations on the game they played tends make ‘death ball’ strategies and relying on reaction speed the optimal approach to winning. It’s already understood that software is much better at reaction speed tasks than humans (even if the AI reaction speed is artificially limited, it could act at the very edge of that speed limit). You can see this particularly in the games that were played by the fact that instant usage of curse attacks by one of the support units was key to winning several of the team fights. The additional game mechanics that were removed from this competition are in the game to minimize the effectiveness of death ball strategies. By limiting the game mechanics that were in play, they have changed the nature of the problem that AI had to solve. While its anticipated that they will be able to add additional complexity to the model and improve its ability to play ‘more’ of the game, it’s also not certain at this point that reinforcement learning will continue to perform well as the type of problems the AI has to solve continue to expand. When the AI has to handle secondary objectives and can’t death ball its way to victory, it might not continue to perform as well. Or it might, that is what makes the research interesting - but it would be hyperbole to at this point to say that AI ‘beat’ humans in DOTA. Its also not completely clear at this point that the reinforcement learning approach has allowed the AI to outperform other methods of automation such as rules based approaches that are typically used in game development (and those other methods have substantially lower resource costs as of now). It was a good result, but AI superiority in DOTA is far from established as it is in chess and go.
I've never had anyone take me in-depth on the actual techniques and thought process and decision algorithms behind OpenAI. Great video man, appreciate it. I had to pause quite a few times and replay bits to fully make sense of them and grasp them. It makes me sad tho they said they're no longer going to dedicate anymore time or release any more DOTA2 AIs.
Your hustle is freaking unreal man. Sweating through your shirt at the end of this video ;). Seriously though, anytime I have a question about ml, ds, or anything ai related, you’ve done a video on it, and your delivery of content has improved soooooo much. Been watching for a long time and just wanted to say I truly appreciate the value you are providing to the world. I’ve heard you say that making this stuff is your life’s purpose and I can see that it’s well in alignment. Cheers
I watched the games live. Was pretty great to see!
This is the best speech pace, no too slow, neither too fast, you are getting the hang of it.
Great explanation! Thank you!
Imho it's crazy how many ML-algorithms they manage to 'combine' to make this happen. Looking forward to the day when OpenAI beats the TI-Winners in a show match of 'full' dota
Love your videos mate. From months of following your videos I’ve been able to implement a real world model to optimise underground mining (gyro crusher) equipment using a deep LSTM with PPO. Keep up the awesome informational videos!
Great, informative video.
Ummm...Siraj? Is there a CONSTRUCTIVE version of DOTA that Alfred Nobel would approve of that I could train a multi-agent system on? E.g. A Cliodynamics, cooperative, positive sum, non 4X variant of Freeciv and Age of Empires. I'm interested in the application of A.I. to Livingry. I want to create a moderator for Buckminster Fuller's World (Peace) Game.
PPO reminds me of Bayesian analysis and comments made by Edwin T. Jaynes regarding certainty factors in Expert systems.
God! That was intense.........Thanks buddy.
A great explanation about RL + LSTM + Markow decision process 👌👏💪
you've taken your graphics/diagram game to a whole nother level, man. Props! What software do you use to make your videos?
....... and asmr videos. Pretty wild, weird and unpredictable
Smart and elegant. As always, thank you for explaining it to us in an understandable way. your work is very appreciated. Kudos to you.
hi siraj, im curious is the input literally like in the video ? it should be using CNN right? if it isnt (which is the input from the game itself like in the video). i considered that as cheating. the AI should be identified itself the object2 and data2 in the game. which is human do
btw u can ignore this if im too stupid
Holy shit. Didn't expect you to do a Dota video!
Nice! I have to learn more about ppo!
im going to ignore the fact you said "valve creators of dota" because the rest of the video was great :) keep it up
Hi Siraj, please consider the data-ink ratio in your videos. The golden summations over the wavy background may have been too much. Looks pretty but really I couldn't get a single thing of what you said while looking at it. I mean, focusing throughly and reading the topic I could... but, to retain the concept, neither the summations nor the decorations helped. Even more, the decorations may have distracted me.
Really interesting point. Can you give me an example of some educational math content you enjoy?
damn Siraj you've put some weight, great vid btw
I think you missed out on the most important aspect of OpenAI Five, being that the five bots operate completely independent of each other to achieve a shared goal (only receiving information from the other bots by the state of the environment and their actions). They explained the mechanism behind creating cooperative bots by implementing both individual and team awards and assigning a "team spirit" weight to the bots to balance these two. The actual purpose and real-world application behind this research is not just having usable AIs trained with unsupervised learning, but that they can create cooperative AIs that function together (to some degree) for a greater good.
I was watching that game live and cheering up for the machinei love your channel siraj please make more videos about AI in games (chess, poker, dota, etc)
Even though I've had the idea before, Siraj predicted AI ASMR.
Im not going to pretend that i understand more than half of the video. But what i understand seems interesting
ugh. i'lll get better at explaining this dont u worry. not ur fault its mine
I don't think it takes years. I've only been studying ai for a few months and I pretty much understand what Siraj is talking about. And I'm by no means very smart. I've just recently started learning LSTM's, so this is relevant. I think if I continue to take online courses and watch guys like Siraj, that I'll pick it up before too long. The only problem is we're at a point where there are a lot of advances in new ai developments, and that is why it's important to have people like Siraj who can keep us from getting lost.
Başaran Badur Yeah i agree. I doesnt seem like he explained it poorly.
Siraj Raval Not really i only know a little bit about AI atm and dont think you could have done better in 13 minutes! It is very rare to see a well edited video that has actual usefull explanation in it thats not just an analogy.
I pretended that I understood the other half and it made me feel really smart for a few minutes 😃 you should try it! 😉
I literally remember watching your first Python video and you had like 250 subs, but your content was solid, niche, unique, and well put together (obviously still very rough but above avg)--however I did not sub at that time because only 1 video so far really resonated with me, however I've been on UA-cam since 2005 and I can fuckin tell good from nothing special to bad and I've been damn near flawlessly right. Then you continued with steady releases, continued content, and building your community and fast forward to 2018 and you're killing it. Good stuff. I've been thinking of getting into building an AI for the game Super Smash Bros Melee. There's a huge competitive scene around it and I've played it forever, but I just can't let the thought go that it'd be the perfect game. Small ass stage, small number of movements, no variables (on certain maps) but literally the character in front of you. Just a bare screen with 2 characters on it that move around. AND you can emulate it on PC and play with keyboard controls lol. I want to start on it but can't find the final bit of motivation but maybe I will sometime this upcoming month. I read something about a university with a group made something like that and they were fairly successful beating at least ranked players (not top 50 I believe, but still impressive). Before I go, sub'd and liked. Loved this and hope you keep detailed stuff on OpenAI and DOTA, I hope they play a full fledged TI boot camped, ready team at TI. Like a Chinese team against full out OpenAI with all items and characters open etc. Hell even if hero limited should be fine. BTW, I've fully given up hope on the human side unfortunately.....I believe it's going to soon become impossible to beat OpenAI...I mean it's played near a thousand years worth of DOTA.....ffs man. Some of the strategy the AI was doing was fuckin insane....it's like--it made SO MUCH SENSE seeing them do that, but no one actually did. People are fuckin afraid. Why afraid of full 3 lane swap by 6min in the game. Why afraid of giving supports full farm and rotating like they did. Keeping Lich against SF the whole game to deny creeps and exp. Having Axe do deep dives of wave cutting in unique manner, going for stats no regen, stacking 3 at weird spots and grouping at unorthodox times....it all seems weird because we're taught to play and watch DOTA from pros a certain way, but there's like this whole Pandora's Box of untapped and near infinity strategies that humans just haven't made enough time or progress to flesh out and explore all possibilities and routes. And also when they were losing and AI was doing literally every bit of maximize efficiency to deter that by giving up calculated risks and plays to further a prolonged goal in the future in an unknown territory of infinitily changing variables...to win playing live. I also believe the leader of OpenAI is tired of talking about it in terms of DOTA, he's known in his mind he's shit on DOTA and hates it and wants to move on to the next thing--to him I say, OpenAI can't play against every situation yet and I'm sure humans will find flaws to exploit. At the end of the day, AI is no more than biased optimization.
Hi Siraj, awesome explanation on what RL algorithms OpenAI used in training its agents againts DOTA - i.e. PPO. Just to clarify here - so this is different from what DeepMind used in AlphaGo correct - I recall it was RL using MCTS? Can you make a video to explain both in more detail and the differences please? Tq
correct, yes i have so much reinforcement learning content coming you will be so pleased
Siraj Raval cheers Siraj, really looking forward to it. By the way I am also your school of ai dean 😀
The AI just won vs some casters, they lost every game on main stage even vs the "worst" pro team that didn't even qualified for the international main event. Don't get me wrong i love see how fast AI learn and how approach the game and i hope they keep developing those bots but we will say they are better than best dota 2 pro players when they really are, for now they are only better than a casual highrank (not toprank/"immortal") player.
Nice video!
Thank you for speaking slowly. Though i still lost you toward the end. Nevertheless, it was a great video to watch!
Took me about two years to figure out that Reinforcement learning was behind all advances in AI like this or intelligent robots like autonomous vehicles and drones. I feel a little down for not having the time I had back then, but with AI, EVERYONE can buy time. Hope so
Will it be available for the public in the future?
It would be fun to play these bots offline at home.
Good to se you make another open ai episode , you have made at least 10 times more sbput deepmind
true. just wait for the RL stuff coming
HEY SIRAJ WHAT DO YOU THINK ABOUT JULIA LANG?
I watched game live too, It was amazing.
Great video as always :-)
Can you share the Dean list with details so that we can contact them regarding issues,doubts we come across
Can do you a tutorial on SerpentAI? it allows you to use any game as environments for training reinforcement learning algorithm.
you're enthusiasm for A.I. encourages me like your Mr. Trejo pic. Thank you!
Can you compare openai dota vs alpha star?
This is very interesting. One thing I am wondering is whether open AI is receiving better input data then the players. The players are just playing from what they see, is the bot working with information that can be all extracted from the screen, or does it have more knowledge about the game state than the human?
Empiricists Academy no its the same as the bots.. they only see as much as human players see.. thats why the bot still need the uses of some sort of scan ability.. the only advantage that the bots had was quicker response time than human players. bcos human still need to click mouse button & press the keyboard..
Btw the international tournament is going on right now with over 24m prize pool. You can watch it on twitch
Do you know how to solve the warding problem? The bots place wards randomly but how do you tell the bots to place wards that spot people going for a kill? Is it enough if you just tell them placing wards = less kills for the opponent and then let them play 180years of dota?
Oliver Schneider with enough training the bots will learn to place wards in useful places because those games will have a higher chance of winning and useless wards are just a waste of money and will thus have a lower chance of winning
the AI is dumb, places wards under tower/where you already have vision. If the ai spends gold on stat items instead, that will lead them to having a higher winrate than when placing wards. That means the ai wont learn how to ward?
Always hated when youtubers beg for subscriptions before I have even seen the video. How am I supposed to know if the content is any good to deserve a subscribe.
Siraj is now focused on OpenAI
I want to see what would happen if the openAI agents were in an openAI league where they were randomly matchmade with each other like alphastar
Playing dota with a controller LUL
Casual AI
life is absurd
wow open ai! Dota 2! Hope that I can see FPS AI bots in the future!
Visualized Math i think ai in fps games will be extremely unbalanced. AI is always automatically better than players at mechanical skill at an almost perfect rate, and in FPS games, mechanical skill is everything when its perfect. A bot would just instantly headshot anyone they see the moment they see a pixel of them
it's called aimbots
I plan to develop one specifically for Apex Legends in 1 year so if u still learn ML/DL/AI , we can work together on this project
Playing Dota with a controller? GGWP. No but seriously, nice video Siraj :D
Neural network algorithm needs rewards, human needs rewards too...Matrix
Cool.
Good video, but why did you take sample videos of dota from 5 years ago, it has a much cleaner UI now=)
Ok I understand that DotA isn't the main focus of the video but what's with the 2012 in-game video footage when explaining what the game is.(2:38)
Tbh I’m just waiting for the time they’ll do the same with tf2, those could be some interesting AI battles
Its called aimbot :D It makes no sens in fps games because computer will always be better at aiming
But not a deep rl bot. A deep rl bot would not only *learn* to aim, but to also strategize properly. This would also be much complex to execute, as the outputs of the ai would have to be discrete.
@@Funny9689 It would still be insane good after training. Of course it would be nice to see strategy but they would need to lower accuracy or reaction time. Even in Dota they probably make bot a little slower because it would be too good
There is absolutely no sense to make an AI bot for a shooter game.
Godly perfect accuracy > great strategy
aways
mateus b12 I was focused more on the interesting stuff, such as rocket jumping, sentry buildings and hauling, being a spy overall, etc.
You are incorrect stating that OpenAI is running on Google Cloud. OpenAI runs on Microsoft Cloud (Azure).
As someone who's very deep in Starcraft 2 research, hearing you call DOTA 2 an rts really hurts.
Yay, you posted it
please dont do what you did in 11:30
im talking about the way the text appears, it doesnt look good
lmao it really is bad
50 years from now we might not have generals but an AI controlling tank and troop movement, which enemy targets to prioritize, optimized trajectories for aircraft to follow to bypass air defenses...pretty scary!
actually, a lot of those tasks are performed by computers right now, even space travelling
Siraj, can you please do a review on lobe.ai? It seems to be getting pretty popular...
So, Can i expect dota2 hero builds in future vids??
Feels proud to be a Dota Player
Hello AI it's siraj..
wait, doesn't using an RNN contradict the Markov property?
dota isn't really markovian. it is partially observable so it's really pomdp.
it isn't markovian in a sense that you may observe the game-state but inside the fog your enemies is doing something different.
for that reason you need lstm to integrate over some history to do-away with the partially observable aspect of the fog.
@@dewinmoonl yeah, that makes sense.
OpenAI is really opening AI
Good content, it`s hard to find this balance btw amnt of details and length/accesibility, but there are SO many excessive effects and, sorry, but little bt awkward jokes.. All random font popup effects, particles.. 9:40 - kinda hard to focus on your words when you blend into the background and half of screen is some bright (unrelated?) vid..
Nice Video. Very impressiv what openAI managed to do. But to be fair, at the moment they play only with 5 defined heros. I am exited when they will manage to win a regular 5vs5.
Another resource:
medium.com/@evanthebouncy/understanding-openai-five-16f8d177a957
I've got a question, is the feat here that it won or that it at all capable of playing a multiplayer game? Because I recall Heroes 3 had a pretty tough AI, although the game did "cheat" towards the AI side to make it stronger...
5:53 - hahha according exclusively to those 3 training images AI wouldn't know whether Shibe the Doge is doge or cate
with this pictures it would
2014 international???
"A team of some one of the best player in the world"
Later half of the video "WHAT ARE THOSE?!"
Genius
ASMR VIDEOS PogChamp, i cant anymore XD
Siraj check PECULIUM, the first AI powered savings platform over blockchain
Human players in 2018 OMEGALUL
I think this video just introduces some basic concepts about policy gradient. It doesn't dive into the five architecture so much. But it is a good video😁
Oh yah you are starting that
Love your stuff, Siraj, never afraid to turn the learning volume up to "11"... but these animations are "ppt transitions on acid", and nearly giving me a seizure (seriously, at the very least they are terrible for folks w/ even minor adhd... I need to close my eyes to listen & read the slides on "pause".) Equations can be "fun" without the *extreme* s p e c i a l e f f e c t s ...
Algorithmic is very presidential.
I think it's time that Deepmind's and OpenAi's Al should compete with each other!
Saurabh Farkya Sounds like a fun idea
They're completly different AIs designed for a different purpose
love it
I understand that but my point was to make a common ground for them to compete.
That's an interesting fact (180 years a day), do you any source for that? About the design of a civilization game, that sounds like a good idea. However, we need to be comprehensive to design such a game so that we understand what each AI actually trying to do and rather than we deciding naval campaigns, I would let AI decide which aspect of the development is more important for the sustainability of the civilization.
Is LOL and DOTA2 pretty much the same game? in which ways is it different
yeah, the goal of both game is same "destroying enemy throne". I have never played LOL. but I hear dota2 has better mechanics and requires more time and skill to master.
LOL is like a boring version of Dota
Yes in the same sense that Mortal combat is the same game as Street Fighter. Or Red Alert is the same game as StarCraft.
typo just now, they lost against the first team
bro you are sweating... like crazy...
OpenAI 5 (52gpu's 25.600cpu's
I know some of those words
great work still needs improvement AI just master part of the game ( probably 20%) without beating the best teams out there and playing against random 6k-7k MMR players with some limitations
Dota is not an RTS (realtime strategy) it's a MOBA (massive, online battle arena), just putting that out there =]
Dota is a strategy game. League of Legends is more of a MOBA. This misunderstanding is common for those who don't play dota above the 3K bracket (~Archon IV) and just play casually. However, in League, there is very little room for strategy in comparison to Dota.
Dota is a MOBA. Yes, there is a lot of strategy involved but that doesn't change the fact that the genre of the game is MOBA as opposed to RTS. AoE, CnC etc are RTS games.
This algoritem has a bit of a problem with long term strategy
Some idea of how I can use openai for web scraping to create a dataset of any website. Someone told me that I could use universe but I see that the repository is already archived, and in Openai I only see examples for games.
Sup with his shirt? 🤨
10:10 Sounds a lot like me.. I'm fucked.
oh my god, what the heck! Why are the clips you're using so outdated? >.> That gui is so Dota 2 Beta lmfao
That was probably Purges guide to dota 2 which was very old. He still showed the Open AI match with the latest UI.
Capitalist cant really be considered as "one of the best player of the world".
Likes to dislikes ratio is infinity.
very nice, but dota is been out for years and you got footage from pre-beta? lel
It is not finished yet! AI still needs some rules to defeat the pros! NO Divine Rapier or Bottle;
No summons or illusions;
Without using a scan!
The couriers are immortal and they are 5 for each bot and you must not use them for scan territories!
And also there are limited hero pool! only 10 heros when Dota has over 100 heros! SO I Believe AI is far away from conquering this game!
LOL they are writing these AI scripts in Python :D
So u play dota?
All of the movement in the frame makes me feel a bit nauseous, especially the part where you have an actual ocean in the background. Still, the content was interesting enough that I couldn't stop watching, only now I feel a bit sick.
"the best Dota players" is abit of a exaggeration, if we compare those players to a GO ranking system that human team would be 2-3dan at most if I was being generous, its like saying a c league soccer team is one of the best teams. Yes this is true if you look at all soccer players, but in the pro scene those c league soccer teams arn't really relavent. In addition there were many restrictions on the game (some would say that wasn't even Dota since there were so many restrictuons.) which made it eaiser for the Ai. Later when the AI played a real team at TI, during the games the pro team exposed alot of the AI's weaknesses and made the bots look kinda stupid, although this is still a mayor step for AI and I am sure next year there will be alot of progress.
Well, you really do not know the DOTA2 game. The Bots only beat some lame commentators. The bots still cannot learn the fundamental core of the game, even though they played against themselves for a gazillion years. The reason they beat the commentators is they have better control over the heroes.This is still a big big achievement, and what the 5 team should do, I think, is to let the bots learn from the replays. Simulate the same environment of a replay and let the bot learn from humans and train based on that.
For those people mocking about playing dota2 with a controller... that seems that haven't read anything about RL. Controller stands for : agent and system for: environment, while reward and state are sensory and feedback signals. This are control theory concepts which not so many people without an engineering background understands.