Joseph Suarez Thesis Defense - Neural MMO
Вставка
- Опубліковано 13 тра 2024
- The defense of my PhD thesis on April 11, 2024: a summary of all of my work on Neural MMO from 0.x to 2.0. Advised by Phillip Isola, additional committee Eugene Vinitsky & Pulkit Agrawal. Dedicated to my parents Jose and Patricia Suarez.
Oh cool - didn't expect anyone to see this here! There's a Neural MMO discord. You can chat with me here: discord.com/invite/BkMmFUC
For my new project on bringing these advancements to all of RL, you can start github.com/pufferai/pufferlib or come chat at x.com/jsuarez5341 and discord.gg/puffer. Lots of announcements soon!
UA-cam algorithm is finding an hidden gem for me today
Yum Yum Yum num num num THx
Same
@@upsidedownChad same
100%
its only days ago...
Ari K if you see this you need to learn to mute.
FACTS. HOLY
Me and my homies hate Ari K
I can not imagine sitting ina meeting for that long and not double checking if I was on mute or not at least 5 times. Just sitting their not caring at all is the level of not giving any fucks that I want to get to. Still rude tho
he really ruined the video
well, I also blame whoever recorded this for recording it in a way that allowed Ari K. to take over the recording screen if he wasn't muted.
Bro said screw it imma do my phd on runescape
Idea for this legitimately came from runescape
@@neuralmmo I knew it! Congrats, man!
I was looking at the items to see if you'd copied Runescape assets. Some looked similar but I think you've avoided any law suits for now xD
@@eltyo340 There were some in the early days, but just as an homage, like using runes instead of bullet points
@@neuralmmo Are you sure it didn't come from the Zulrah farming bots?
Whattt I went to high school with Joseph, wild to just get this in UA-cam recommended 😂 huge congrats!!!
Oh hey, Alex? DM jsuarez5341 on discord/x!
this is so cute ❤
Good for us who are recommended this video.
Thank you all! 100k is absolutely nuts on a thesis defense. Can't wait to share what's next. Not the end of NMMO!
The intersection of neural networks and MMOs have always been something I wanted to pursue but there wasn't anything quite like what I had in mind. The closest one to this was OpenAIs work in Dota 2 which after seeing it, made me pursue a specialization in artificial intelligence during my undergrad.
After seeing this (by some stroke of fate from the youtube algorithm), I feel like this is the direction I want to go for my graduate studies. Thanks for pushing forward innovations in this space. Congratulations on your defense as well.
I was deving nmmo 1.0 like 20 feet away from the dota2 team at openai. Most cracked team I've seen, was awesome
There’s an Ari K in every zoom meeting
My faith is renewed in the YT algorithm
God damnit Ari K.
For real, incredibly rude and distracting to attend an hour presentation unmuted.
They will forever be tarnished as an asshole, as this guy gets super popular lol
There was a lot of stuff to handle with a/v... room wasn't quite set up right. I didn't want to hard mute everyone since one of my committee was on Zoom, and I forgot that Zoom switches focus like that
Thats what im saying. shut uuppp
@@neuralmmo if you could make the video just freeze for the frames when ari if interfering with the feed, it would make a much better viewing experiance.
Here before this dudes starts his own OpenAI,
Same
heheh I'd love to start a 10 person engineering heavy industry lab on core simulation and RL tech
@@neuralmmo I'm working on my physics PhD and started self-learning AI out of passion. In particular, I'm really into AI for video games. I've been wanting to do a very similar thing with my career. Maybe we'll cross paths in the future :)
@@coach_tae_ Do you like Skyrim with AI NPC's? What are your thoughts with how far that could go?
@@nilsolsen8727 I'm guessing you mean story games guided by LLMs? If so, I know a group working on a similar project now and that others are integrating this idea. I think it's the future of story games for sure. Not my area of interest though. I tend to like AI focused on optimizing gameplay. Working on a Pokemon Blue AI right now, for example. Just to see how it learns and tries to optimize the game.
This is the best thing UA-cam has recommended recently
To have this continuous research objective from such an early stage is insane, very inspired - awesome work
lifegoals.
I'm only 10 minutes in but this guy seems very knowledgeable and confident on the subject and topic is fascinating.
Congratulations Joseph ! It's a great achievement !
Kudos my dude, looks like you'll be going a long way!
Been tracking Neural MMO for some time now. Congratulations!
I’m also a Joseph, nice to see another be recommended by the YT algorithm. I also wrote my paper in the realm of reinforcement learning.
This was an interesting watch :)
Congratulations Dr. Suarez!
Glad to be recommended this video!
Follow + star my latest project on making RL work easier: github.com/pufferai/pufferlib
Or come chat: discord.gg/puffer
This is exactly what i want in my feed.
This is really incredible progress in five years!
when passion meets profession! always a pleasure to see
You rock dude, stay awesome and humble
Great work! Congrats, really inspiring
Beautiful PT
This looks very promising, definitely earned a sub!
The use of ECS style patterns baked into the NN architecture is really interesting.
Seems really obvious in retrospect, but nobody was doing it
I've thought about ECS or EC style approaches but couldn't think of anything like you did! Not as obvious as you might think, requires good interdisciplinary knowledge and creativity. I think you're going to have an incredible career. Congrats on the PhD.
@@alexthemle Thanks! One of the things that helped me realize this was actually using Unity's ECS to write an ultra high-perf renderer for NMMO early in development
legendary youtube pull, incredible work Joseph.
Thanks!
This is so interesting, what a cool project!
this was a great presentation!
As a Uni student, seeing this is inspiring and motivating, great stuff and great presentation.
Bless the youtube brain
Congrats Joseph, appreciated your contributions to the RL Discord server as well :)
This is insanely good!, I hope I can make a good thesis such as this one, Congratulations from Argentina!!🎉
This was great, thank you.
This is fascinating. Very interesting presentation
This is amazing! Thank you for sharing your work w the world. As a relatively new software engineer (2 years) and an avid gamer, this is so inspiring and makes so much sense. My mind was going wild thinking of the future implications of this and how this can be used. I know it's a long shot, but if you ever need any help, please let me know. Im so impressed.
It's all open-source run through Discord! We have people who are now core who just showed up and started helping
Don't know why the UA-cam algorithm recommends this video. Anyway, congrats, bro
Great stuff!
Its nice to see someone did this. as a undergrad student who is really eager in the topic, i would love to have some motivation to pursue a life aroud these topic. But it is hard to do something when u dont have anyone to talk to or lead to in the area. this video gave me some motivation. thank you Joseph.
You don't need people to tell you how to do things. You can just do them. Early years of NMMO were me building stuff in very stupid ways as an undergrad because I didn't know any better. But I don't think any mentors could have made me learn any quicker than just working hard and failing often
@@neuralmmo Actually you are right. i will do my best. just sometimes i feel lost, that's it. Thank you for sharing your experience. :)
@@HesitantOne You never stop feeling lost. You just get to the point where you know more about a very specific topic than anyone else, so at the least, you're the best qualified person to possibly do the work
@@neuralmmo at the least :) thanks a lot Joseph.
you are awesome and i hope you know that
Congrats man
I'm playing runescape while watching this on the side. I love seeing this game crop up randomly
Very cool! I think there's a lot of potential in abstract/high-level multi-agent systems like this.
Damn man 1 hour defense presentation is nuts
awesome work!
Powering up Pokemon soon!
Congrats!
Oh gosh this is gold
this is awesome!
So this is how the simulation begins
Well done!
Wow, amazing!
Thank you-you are now important.
You did excellent work with the AI advancements and the defense. I will be looking into how I may be able to contribute to this project. Well done!
Really well done 👍
bro looks great in the suit dawg. Excited for this 1
Hadn't worn that tux since high school!
Hey, a fellow Suarez here! really cool stuff!
A really well done presentation!
I've been exploring a RL application for complex multi agent environments, but have ran into many similar type hurdles as you outline here (with far fewer clever solutions). Specifically, i see a use in industrial/operations applications which maybe more traditionally fall under the operations research umbrella. Your presentation reignites that fire and gives me additional resources to explore!
You'll love pufferlib then. It's a lot of my insights on how to make stuff work in NMMO bundled into a general use library for whatever env you want
Damn! This is absolutely awesome! I've been writing accelerated agent sims since I was 14 with the intention to create something like this one day. This video is one for the history books, great defense!
Thank you!
@@neuralmmo I briefly looked at your code and knowing scientists and post grads I'm stunned. The years have paid off and your earnest passion shows. Looking forward to testing it out more in depth, incredible work again, I hope your flame burns evermore brightly.
@@GRAYgauss The code base has gotten very bloated tbh. Wait until you see some of my new stuff
impressive talk
This guy sounds so confident. Jesus.. lemme buy this product. I'm in. Good job Dude your future is bright
If only it were a product! Working on some new stuff now with PufferLib (pufferai.github.io / discord.gg/puffer)
As an MMO enjoyer I approve of this! 😎
The state tensor map thingy is very cool!
It's one of those things that is obvious in retrospect but definitely wasn't at the time
Love this. I just missed the huge AI boom when I went to uni. We did things such as image recognition with CNNs, some audio recognition with RNNs and some SNN for classifying directional movement etc.
Would love to make a similar project. Maybe I will come back to academia to get a PHD but unfortunately I doubt it.
Thanks algo. No idea what’s going on but I enjoy this head doing stuff.
Let's go Joseph!
50k views and counting on a thesis defense is wild. Congrats man!
I honestly did not expect anyone to see it here... I posted it on Twitter first and then did this just for archival!
I'm not in academia, but is this guy above average in terms of speakers? I get this is his defense, but I'm just really impressed.
Well, I guess at this point in his career he really would already be an expert, he just looks really young so it's kind of throwing me for a loop.
I'm studying ML as a hobby, never looked into RL at all. The first few minutes of this talk alone captivated me and made me decide im going to look into RL.
Thanks! Am 27, have been working on AI since ~2013 and RL since ~2017
Oh, and RL is going to get much, much easier soon!
Have to say, this dude speaks so eloquently, possibly the best speaker i have listened to so far!
Definitely an above average speaker
What an incredible topic, work, and defense. You did a wonderful job in a field I love watching evolve. I hope you are not done yet! Subscribing to follow your future work.
Thank you! Lots more coming soon
Legendary algorithm pull
Multi agent learning that results in the formation of hierarchies is what teaches artificial entities how to use force for correction via stochastic control… hasta la vista, baby!
I have made DSS to help me to choose which units most important to upgrade based on some criteria but my lecturer/teacher didn't like it. Watch your great job make me wonder to continue my works or not.
Lack of game knowledge is a hard communication barrier with most of academia
I understand nothing but this is amazing.
hehe, I pity those who followed after him
This is so damn cool
Hey ! I just discovered your channel and in fact, your entire project and I'm really happy to see someone actually doing what has been proding around in my mind for a long, long time (except I had neither the means nor the knowledge or really the motivation to achieve what you did, what you've accomplished here is highly impressive).
I'm a professional game developer, currently working off of Unreal Engine. Seeing this, I get very inspired about everything of course but I'm thinking on whether creating a more, shall we say, "proper" representation layer for an ulterior version would have any value. It would also be an opportunity to bring the MMO's game mechanics into the hands of Human players (you might think this is too simple and primitive to get engagement from human players but just the ability to play against advanced AI agents would be quite something already I think), which might also have some value for training AI, perhaps by attempting to "snapshot" human actions into policies and using that as some sort of training dataset / start point ?
Anyway, I'm barely even a layman in all this so I might be saying complete nonsense. But I figured I'd ask, I really want to toy around with machine learning aswell one day, just haven't reached this stage in my life / education yet :)
Full scale NMMO proper would be cool. MMOs are just so hard to develop... I didn't go this route because it would be 95% game dev and 5% research
@@neuralmmo Oh yes it would have been a bottomless hole in your time budget. The concept has to be proven first :)
In the context of current world affairs, this presentation is actually very chilling.
These agents are pretty chill. They run around and play games
i am currently doing my masterthesis in architecture on the topic of the interaction between digital emergent enviroments and the interactions of avatars, i stumpled on your video through coincidence but its amazing what kind of work you did and what is currently being researched on in the direction of digital enviroments.
Thanks! Much more coming this summer
this is good stuff
I think theres something inside us all that wants to do something like this. It's so cool. Were you inspired by carykh's youtube videos from 2017 that play around with this? They look pretty similar to your original setup. I tried to reimplement his stuff on my own since i was a freshman in college and it was the first nn evolutionary program id seen, but i got a bit out of my depth and gave up lol
no hadn't seen it actually. First got the vague idea for something like this in 2014 or 15... tried again in 2015, and then again in 2017. By then, I knew enough to get it to stick
Had this same concept cooking in my head since ChatGPT day 1, but I’m broke and my MacBook died while I was in a Warsong Gulch!
What a marvel to see it outside of my fantasies. Great work!
NMMO is older than ChatGPT :)
Great find from the algo! As a a uni student I definitely need to check out your papers now. I wonder how this would work if you add a communication system to the agents by hooking up llms as a brain (for communication and task assignment), like in the ChatEval paper.
My work intentionally does not consider LLMs in the main loop like that. RL models have millions to tens of millions of parameters. LLMs are 10s of billions. You lose all the advantage of having infinite data by not being able to consume it. We do use LLMs in niche areas, like for the task encoder, where we can precompute once
subbed!
Seeing this in the future coupled with Agent base attributes (like intelligence, strength, dexterity etc.) And then see the interactions between agents really excites me!
For example, we have a blacksmith character and a Librarian. Maybe the blacksmith is trying to teach the librarian how to forge a weapon, but the librarians Strength and Dexterity attributes are too low or dont meet the requirement so they fail to learn it in an ample time, match that with a mood or frustration of some sort and they might give up learning it all together.
Im excited to see how this blossoms!!
Emergent specialization. Definitely a main area of interest for me
@neuralmmo I'm happy to hear we think similarly! I'm very excited to see how this project blossoms in the future :)
When the UA-cam algorithm actually works:
He looks very confident and the world seems going to change...
At the least, RL is going to in the next couple of months
oh, no offence, this is really a good job and presentation actually. It makes sense to me someday Open World Games would have more vivid NPCs (game agents) as proposed, we will see and anticipate, gamers' blessing
@@tkingless That's definitely going to happen and is one of the more fun applications. I have high hopes that RL is going to play a much larger role in AI long term
I would like to use Neural MMO to explore my research on epistemic agents
big ups
i could use sponsorblock to cut the flashing screens up but it would mess with the audio.
Man this is exciting, jealous I'm not in school
And I'm happy to be out of it!
fascinatinggggggggggggggggggggggggggg
Id love to see an update of human vs model improvements, where humans can take an active role in the simulation. I have a feeling that your agents would see an exponential increase in productivity and faster optimal behavior onset. perhaps decode to a graphical representation on a small scale and then squash the humans behavior patterns into weights and use them as training agents against the more randomized approaches. it seems like powerful kindling. either way, great oration and a very cool topic! hope to see you more in the algo
I do wish I had made it human playable, but the amount of human data you would get is like nothing compared to how much they train in sim
congratulations joseph! I'm an undergrad and share a similar passion for games, RL and Neural Networks much like yourself, any chance you'd willing to mentor me or even drop some advice for someone just beginning to navigate this field, your research is truly inspiring!
Drop by the Neural MMO discord! would be happy to chat
Nice
I am Recommended this, Am I worthy enough 😂
Hey quick question. How did you do the Figures and Diagrams at 18:10 ? Congratualtions btw!
Google slides with a very picky advisor!
would the new Bend language have any application here? the parallel sorting itself may be worth a look with so much going on in the simulation. applicability here is beyond me though
Not unless people start implementing full games on GPU. I do have some cool stuff coming for performance though
Really interesting how he didnt follow the conference approach, he genuinely just looked at his own interest. I think Unity3D and U real can create realistic versions of these games, and also having humans and agents could be interesting. If humans interface with agents there might be interesting things to learn .
The 1.3-1.6 client was in Unity3d. Some of the demos in the video show it. But you don't use it for the game state, just the rendering side