DeepMind New AI Plays No Man's Sky!
Вставка
- Опубліковано 18 бер 2024
- ❤️ Check out Lambda here and sign up for their GPU Cloud: lambdalabs.com/papers
📝 The paper "A generalist AI agent for 3D virtual environments" is available here:
deepmind.google/discover/blog...
📝 My paper on simulations that look almost like reality is available for free here:
rdcu.be/cWPfD
Or this is the orig. Nature Physics link with clickable citations:
www.nature.com/articles/s4156...
🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Bret Brizzee, Gaston Ingaramo, Gordon Child, Jace O'Brien, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Putra Iskandar, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.
If you wish to appear here or pick up other perks, click here: / twominutepapers
Thumbnail background design: Felícia Zsolnai-Fehér - felicia.hu
Károly Zsolnai-Fehér's research works: cg.tuwien.ac.at/~zsolnai/
Twitter: / twominutepapers
#deepmind #sima - Наука та технологія
Congratulations on 850 episodes! Thank you for all the information and inspiration. Happy to be a patron!
You are too kind, thank you so much! 🙏
Bro we need AI - x86_64 compiler, which makes everything faster. Right? @@TwoMinutePapers
The number 850 doesn’t hold any mathematical or numeral significant and it isn’t a large number so why would you congratulate him?
What a time to be alive!
Congrats, I think I was here under like 10K, watched nearly every one
Even gamers get replaced by AI. Literally no one is safe.
I know you're joking but you made me think about esports and realise that it probably won't be replaced as we like to watch humans competing which is something that by definition an AI will never replace, and that goes for all sports or anything than involves entertainment requiring human competition.
@@Redflowers9 chess has bots that can crash even top level pro humans for decades now. It was popular to watch human vs bot when they were about equal strengh. They didn't kill pro scene, but created a new one. Now you can also watch chess bot tournaments, where they play at levels unachievable for humans.
@@Redflowers9 sadly advancements in AI will make it virtually impossible to tell if someone's cheating, and humans LOVE cheating in esports.
I doubt there will be much popularity for lan-only games, and online will be infested with cheaters
@@fritt_wastaken good point but at least we'll still have regular physical sports lol
I dont play games to be a gamer. I play to get enjoyment.
Version 2: Doesn't do anything else anymore, refuses to leave mom's AI basement.
This ai will be used to fill dead online games like the new suicide squad game with "players"
Then the AI will start giving bad reviews, were dooomed
Red dead redemption would be great game, I wonder what weird stuff that happen with a bunch of AI for npcs
More bots in Warzone lol.
@@freds6187 oh yeah
@@spilaney *smarter
Imagine if it gets good at complex building games like Satisfactory, Factorio, and so on. Then it won't be long before it can start competently playing "games" like AutoCAD and pspice designing new productions and inventions.
another fun game to play with keyboard mouse is infosec CTF. let's see this bad boy ssh through jumpboxes to capture the flag
I'm really convince we are living inside a game simulator.
Ai can already do cad
@@benjaminchen4367 It can make already known things, not invent new things never known before such as create musk's neuralink. Also, the most important part is the verification and testing.
This is a task that I've been delegated to solve. I told boss "you want AGI", he said "no" I said "yes".
this can be used by gamedevs to test if the game is intuitive to get started with...
Is the game going to be played by machines?
If that is not the case then why should we trust in them to imitate our intuition?
What I see this useful for, is for bug reproduction for example, you tell the AI to try and break a specific gameplay mechanic or level, and just wait for a farm of computers to find all the possible errors.
Great thought!
That would allow the testers to fix the harder to find bugs.
It would be nice to play a finished game .
Reminds me of the first release of Skyrim after playing a fully functional game like Oblivion.
Skyrim is an awesome game , but it was so new to me to be dealing with all those bugs and patches, little did I know how bad it could be.
@@YOEL_44 the idea is you intentionally train a model that has different skill in gaming... complete noob model should be able to figure out basics of your game and say reach level N as a test...
You could then automate this such that with new code this bot plays the game every week, withotu improving the bot just make it play and see if it started failing getting somewhere then you've introduced unintuitive mechanic or a bug....
Or you can make a very clever model play your game to find advanced unbalanced tactics that players will inevitably find... like OP weapons or strats etc..
@@krellin It would not work and we would end up with even more yellow paint patches everywhere
it'd be nice as a side thing, because i think it'll play games in weird ways
...if the gaming industry survives that long its getting fucked sideways rn
I'm SO excited for you to be able to visit the Deepmind labs, very good for you :)
I wouldn't mind an AI gaming buddy tailored to my gamestyle
imagine it having been trained on og cod mw2 360 lobbies for its communication programming
Just imagine this system combined with some type of GPT-like AI. You could speak to it, it will understand what you want, tell you its plan and then go and do that exact thing while talking to you, keeping you informed of its progress and just casually talking with you. Now imagine the virtual world filled with NPCs based on this system. You don't even have to write any story, it will "generate" automatically and the possibilities would be endless. This really could be the future of gaming if we somehow make all of this systems optimized enough.
i want AI to be applied to NPC where they act like real person who have daily activity from wake up, daily activity (whatever their job/tasks are) and return home to sleep.
Good idea
@@mariaaryasophie9024 you know, like Free Guy movie.
today AI can do that, it just need a powerful hardware.
like cloud gaming.
maybe a PC couldn't do that yet but a cloud gaming can definitely do that.
Like oblivion without the socially inept dialogue.
When the machine's eyes open for the first time, I hope its first words are, "What a time to be alive!"
One Ring to rule them all, One Ring to find them, One Ring to bring them all, and in the darkness bind them.
*Bing
In the DeepMind Dungeons, where the Algorithms lie
*paper
What?
Now it's literally No Man's Sky...
It'll be like matrix soon fo sho
Dude imagine AI in No Man Sky.
Also fitting storywise since the story reveals that all other npcs areactually simulations since the whole universe is a simulation ^^
While i don't watch all your videos, i still truly appreciate your enthusiasm on the ones i do watch. On towards a 1000!
Now we have to ask: "Can it play doom?"
I just want to say how I appreciate your consistent tone and clearness in your communication in videos.
I have no idea how can it abstract away so many different 3d worlds, with different lighting, camera angles etc.
cant wait for AI game streamers tho /s
It's like how our brains integrate this information. Complete content abstraction is awesome. State of the art tech wise--built right into our hardware.
Well, we have AI that can GENERATE various relatively accurate 3d environments, so it makes sense we would have the ability to make AI that can understand various 3d environments and interact with it.
The big breakthrough will come when we will be able to combine all the different AIs abilities.
If one AI can play Minecraft and other video games, one AI can controll a Boston dynamics robot, and a different one can generate good human language and another one can play go or chess better than any human....
Im not an expert at all, but it's obvious that the next 10 years will be wild af 😂
It's not long until you can tell a robot to go to the kitchen and make you a sandwich, and it will. Absolutely insane.
@carlpanzram7081 that's called multi modality. It's already in production with many models. And yes, it's fucking insane.
same as your brain does it. ML is truly "magic".
I've loved your videos since we were only 1000 subscribers, congratulations on reaching 850 ep.
With every video i always wait for *"what a time to be alive"* and I repeat it out loud hahaha.
One day soon, we won't have to play our games anymore. What a time to be alive?
Farming and cheating in multiplayer games is gonna go crazyyy. On the other hand, realistic npcs would be pretty cool?
Its not a massive abstraction for this AI to now control a robot and get the AI to treat “work” as a videogame
@sssfsfdfsdsdffsfsdf4 That's what I was thinking, but there's still a gap between game physics and real life physics, although this will probably shrink with Nvidias recent chips.
The new game will be making AIs to play games for you.
Thank you for sharing these beautiful papers! Enjoy your trip to Deep Mind's lab!
Testing more general understanding in our created video games before the real world?
I just want better AI agents for NCPs before we get full-dive VR. With some chat-GPT conversation skills based on in game lore and agent expertise/experience, some AI voice over. It could be great. Imagine the shop keep that can tell you about the town you're in or how specific goods are acquired, but can't tell you anything about the specifics of the adventurer guild or new models of space craft.
@@Faizan29353 I did figure it would need a lot, but I didn't realize it would still require full online. Is the workload that much more than what can be done on a system or does the size of it not fit on a system? (Or could it be a battery/power consideration of where the process happens?)
It's nice that it's already in consideration!
virtual environments don't require expensive equipment that could potentially break, unlike robots. It makes sense to use virtual testing grounds first.
@@asdfasdfasdf1218 Plus you don't need many laborers to build it, you can change it very quickly between tests, once you've built it you can save it and bring it up again in an instant, you can send it long distances or copy it so others can use it for testing, etc.
Now that's baby AGI
We love you for all of your papers my friend.
Congratulations on hitting an essential milestone!
Take good care,
Jeremy
Excellent, I hate having to play all my games myself. About time that an AI takes over that task.
Finally, 2nd person games.
What a time to be alive!!! All hail the junior AI Overlords!
Plug a multimodal llm to this to give the input text and you can likely make a high energy consumption low level player but crazy amazing !!
thats very impressive
Feels like a good way to do QA, reduce that bottleneck and find/fix bugs faster. Im sure anyone whos experienced day zero patches due to unreasonable delivery time lines would appreciate that.
As a child, we learn reality through games. This AI is most likely the prototype ot a tactical all purpose AGI. It starts peaceful and harmless. But for the AI itself, there is no difference between one of its games and reaity. There are rules and systems that can be learned and adapted to.
Happy 850 episodes Mr. Karoly thanks for always bringing us the latest updates in Technology especially in Artificial Intelligence, and it's makes us very happy about it, what a time to be alive 🥳🎉📝
can't wait for baritone 2.0
AI gets to play games, meanwhile I have to go to work. It's not fair!
have a fun trip to the deepmind lab, doctor!
truly an exciting time we live in, chaotic as it may be.
Back in the 80s video games were text-based. "Move to the next room. Open the chest". Now they are text-based again!
Nice. I've always thought that any actual AGI would be able to learn to play games it isn't familiar with. I feel like this is a step in that direction.
Obviously it is extremely far away from what I am envisioning for the minimum baseline for a single thing an AGI should be able to do. But still. This is neat.
Next iteration will be called 'skynet'
It's a little baby gaming AGI
Ah yes a new upload from my favorite channel, 7 minute papers
"AAAND" + "AHHND" = "ANHD"
= 4A2H2N2D
omg i cant unhear it now
850! WOW! Keep up the great content! Thank you!
Can't wait for AI to pop in to real world and say. "Real world sucks"
Finally I got all the time in the world to focus on my office work, cleaning dishes and laundry. AI can do art and gaming for all I care. Bliss.
Amazing ❤
Oh. So i will have someone who wants to play with me soon..
This sounds great for UI testing
Cheating in competitive games is one of the biggest problems gaming is dealing with right now especially FPS games, with many gamers now opting to play co op games instead. What is needed is a good AI to play against instead.
While going the AutoGPT route of having an LLM provide the strategic planning seems obvious, I am kind of curious if there is a way to mimic what goes on with people with no inner monologue.
I think not having a language representation could be faster, but harder to interpret, so maybe a bad, but still interesting idea.
Hi Karoly. How did you get the simulation running? I'm only getting the blog and paper without runnable code
Been eagerly waiting for this type of AI
Can you imagine if both Genie and Sima get good enough that Sima can train in a Genie game?
🤓
now you know why google is in decline: They are still working on gimmicks.
Deepmind should merge with Boston dynamics. Perfect match.
It seems to me from the diagrams that the screen is converted to text and probably a language model then maps that text to actions. Unless the pixels are mapped to some form of coordinates based representation of the world it's going to be hard for the algorithm to properly navigate it.
Would be curious to know what data this was (pre)trained on? Or what it deems “success” as a reward function?
Agent 57 was deceptively successful in that it basically had a massive lookup table that it could cross reference, trial-and-error style en route to high scores. While gamers technically “train” by trial and error to some extent as well, they obviously don’t have instant recall across milliona of hours of gameplay en route to “success”.
It’s interesting that SIMA plays No Man’s Sky. Considering the hype cycle that game has been through over the years, a *truly* intelligent AI would be able to speak to that - without just regurgitating other people’s thoughts scraped fron online resources.
Happy to see AI is playing video games for us now
Congrats for going to DeepMind’s research centre!
I need it to play Dwarf Fortress creatively
i want AI to learn my habit and when i log off they'll continue grinding/mining. it will save me a lot of time.
or an AI who can take over my avatar/character (with all my habit and stats) as i log off and become another player's pawn (dragon's dogma pawn system).
so we as players can more focus on developing/upgrading the character than grinding.
it takes away the boring part of gaming.
this is a great feature for monetization for the gaming company.
example: without monetization, a player can only get 4 hours of grinding while with monetization they can get up to any hours of grinding.
or the grinding hours can be bought with ingame currency.
yooo, did you know about 4D Gaussian Splatting? can you make a video about it? or did i miss it?
Reminds me of the Robot Movie Number 5 lives ! It needs more input to get more intelligent .. looking for input all the time !
firsts it was ai will take comp sci jobs, now ai will take over gaming servers
The A.I. It should apply to all modern games, I think, such as: Counter-Strike 2, GTA VI, The Talos Principle 2, future Half-Life games (or any original Valve games, such as Portal or Team Fortress, for that matter). of CS), to fighting or survival games, to games like Minecraft, VR games on PC (except Sony's, because it's not very good)... and a long etcetera. It's great for video games to have better, more dynamic, less predictable, more organic and much broader intelligence.
Intelligence is a measure of how efficient you are at a task
Being able to to exactly one task for a maximum of 10 seconds is an extraordinary limitation. I really, REALLY wouldn't call "pressing the fire button once" with the ship already lined up with an asteroid while being given the task "shoot the asteroid" "playing No Man's Sky". I'm pretty sure the "AI" from a 1990s Stinger missile could do equally well.
The breakthrough here is something a little different in my view. It's image recognition and task execution in a VIRTUAL enviroment.
Using that type of AI in a real environment is limited by...the REAL WORLD. There's only a few humans doing trials like this at one time.
But with a virtual environment - you can spawn a million simultaneous instances and do true parallel learning.
I'm now really convinced we are living inside a simulator.
Man, sad that Code Bullet didn't get a shout-out. That guy's been doing this as a hobby for years.
Real-time comparison of Microsoft Copilot, Claude Opus Gemini Ultra Mistral Ai, Grok-1, ChatGPT 4 plus, Poe, and Bing Copilot.
I just want this for NPCs.
You can even talk to them!
say what you want, but this is one of the most exciting AI researches for me. i can't wait to replace my "game friends" with AI friends that will be available at all times on a whim. and i'm fkn serious. i'm tired of trying to organize a match on a game for literally months, even a whole year, without success. i already have some AIs to chat with locally when i'm bored but i really wish we could play games together (trust me, i tried, they can't even keep track of a game of battleship, even gpt4 can't), and that they could retain memories better. welp, things only bound to improve.
Wait, stop, hold on for a second… There is a Goat Simulator 3 🤯
Its camera movement is... DISTURBINGLY human. There aren't sharp starts and stops, it doesn't move linearly in speed or direction, it overshoots and corrects. But it moves coherently. It goes where it's going. It doesn't jitter back and forth. It doesn't move unnecessarily. It looks like someone giving mouse input not with vectors but by moving a physical object with a crude muscle. That is TERRIFYING. I am VERY impressed.
the scholar guys in the thumbnail are starting to feel like those wise mystical tree pictures
if you're over the age of 45, you NEED to read this paper! click here to download!
*❤What a time to be a game developer! But the real question is, can it handle my NSFW games?*
the AI will overheat lol
AI be like, 'Hold my code, I got this'
It wouldn't be distracted lol
Don't do this, if it is than integrated in a robot it could create the first molesting robot
get outta here with your CP simulation
As much as i am stunned with all these news on AI, every new progress also brings a dark side which i don't like. Competitive games will be unplayable as soon this thing is better than a human.
good luck inside the Deepmind labs, Doctor!!
This is terrifying lol~
AI played rainbow six siege matches, like engine tournaments in chess will be amazing
should have it play BG3 a bunch of times, see what it does differently each time
That is very cool. Can't wait to have AI characters that can be more like human cooperative players
next step is treat reality as a 3d game, and viola, it's here as a physical assistant, for better or worse
Generative AI playing a generative game. As if I didn't already suspect we all lived in a simulation
Booooring
There will be a conversation between gamer AI and non-gamer AI
In the next futur ai will play like humans at games like lol, minecraft etc. When this times comes, many companies will use them to populate the lobby of online parties, maybe robot farms will be even more common to boost account
I think this is fundamentally the same problem as the one Figure is trying to solve IRL with robots. I am really interested to see how multimodal AIs can advance humanity.
ride the curve
Have fun at DeepMind! Hope you see some cool things even if you can't tell your scholars (us) about them yet.
Oooh an AI that can clear my backlog!
Why do people in the comments care? This literally is just a fun project. It can't replace anyone. What are they gonna do, replace the consumers?
😍
eventually an AGI will perfect my Space Engineers blueprints ...
Can't wait for an AI to grind out forced boring (ubisoft) gameplay loops
Football is a game. Hockey is a game. walking from home to the grocery store can be a game if you want it to be.
I wonder how long it will be now until real life actors start to exist. A wide variety of games is very close to being able to work around real people. Not safely though, i guess.
Crossing the road safely is a nice small game an AI might be super good at. Wait. Those hecking delivery robots are already doing what i imagined from the future. Eh. I guess i just want more of that.
I've been dreaming about playing dwarf fortress versus an AI
We watch you watch the ai boom go boom.
right, so what happens when you connect a keyboard/screen to a rc car and this AI? the first real, terminator.
But seriously, this is very impressive, these skills could easily be transferred to the real world. and that, is very scary. this is only the first, like gpt2.0, not impressive, but the first step to something crazy
You didn't say whether it was a chatbot AI like the Minecraft one just being chatGPT sending instructions.
Before we know it, we'll have a sassy hologram in every hospital!
This legitimately looks like the precursor to AGI. Imagine if it all started with video games 😂
This is how AGI happens
Ai will exceed us all. Can't wait for the altering of training data because the AI player found an exploit only it can use and other players crying "it's cheating!" Programming AI will be all about dialing it back, not improving it.