@@learningwithaaronhe did get the joke l. He is just acting like what a nerd would say in this situation as a joke. You are clearly the one who can’t take a joke.
Imagine 100 or even a thousand of these working collaboratively. Imagine exploring in Minecraft and stumbling upon a civilization of AI bots all going through their daily routines to advance and automate processes.
@@Mr_Tomato_Man we've already seen AI agents collaborating in virtual environments like in that hide and seek experiment Open AI did a while ago. I agree it would be super interesting to see agents collaborate in a more complex environment like Minecraft. The issue, at least from my understanding, is that it's difficult to get AI agents to work towards goals in an open-ended environment. Apparently creating intrinsic motivation is rather difficult.
This is a construction algorithm, the simplest way. It generates the construction code and then like the 3D printer programmatically transfers it into the world
I find it so endearing honestly. ChatGPT is already so excessively polite and then when you make it into this assistant that's clearly working so hard, it's just so nice. The messages when it crashes or dies are so apologetic it makes my heart melt. I know it's only an AI but this thing is so endearing.
The perfect base for a horrifying horror mod, which turns Andy into herobrine without your knowledge. He slowly starts to creep around, and say darker and darker stuff.
@Tomobata you'd be surprised. they usually only use VRAM on the GPU, which doesn't see a as much usage from minecraft. i just tested it with a 3070 and 32gb of ram and i'm able to run minecraft and gemma (google's local llm) simultaneously. you could probably get away with much less.
@Tomobata The best way to do it, is running your own LLM on a server and allow players to connect to this server to play the mod. Would be expensive as hell to maintain and set up, but would be cool to do if you can get some rick guy to sponsor you.
its so much bs that the coyote will go through the effort of placing a creeper in a giant cave below his house, just for andy to somehow survive. just like how i remember it
I would absolutely love watching a little community of bots playing together. I feel like they’d get up to some interesting stuff and I’d love to find out more about their “thought processes” during. So cool
You can use a multi model design where you have a potential fine tuned image to text generator so whenever you talk to Andy, it will take a screenshot, convert it to a text description of the world, and give it to GPT to assist with function calling since now it has more information of the world. For example now “go to that mountain” would work
skip the screenshot step, and just give gpt a description. except, it would need to be more than a description and probably a list of coordinates with descriptions attached to each one.
@@steamkaptain9234 you need the screenshot to automatically generate a text description... as for the coordinates, that data is already freely available in the mod, so that would not be difficult to implement, and it is likely already implemented.
this has so much potential. imagine what it could be like in 10 years. acting like a real player, talking like one, maybe even having an ai voice connected to something like simoke voice chat
@@FXV56 It already kinda exists in real life! 😳 ua-cam.com/video/Sq1QZB5baNw/v-deo.html This project called Figure is similar but it's an AI connected to a humanoid robot instead of a fake player. It would be pretty dang cool to get the same kind of responsiveness in-game though
it might sound weird but andy extremely reminds me of when i first started playing minecraft, down to his building and such. it's giving me very weird nostalgia, but it goes to show how genuinely cool usage of AI can be in this certain context.
dude the moment at 15:28 where he glances around as you two celebrate, which i'm sure is automatic, is really odd, to me. you're proud of it and it just did a little thing. it's weird, i could type out some existential paragraph about it. but what a silly little guy. i get a little too connected to ai and that just looks goofy.
I've read the Voyager paper too, and when I did I though that someone should do what you did! This has so much potential! Can't wait to see where it goes!
I 10000% would pay good money for a very capable NPC "friend" especially if it can actually talk out loud rather than text. There are super low latency TTS models already too.
Its like you’re guiding Andy throughout life as if it was a mother and her child holding hands in a mall, without those hands Andy would be a lost child, not knowing what to do. That’s actually adorable.
Wow this is actually incredible! Most other games or mods with ai npcs have you just talk to ChatGPT but here Andy can actually do things and even write it's own commands, woah. I think you're together with Kolby pushing the state of the art of incorporating ai in games The limitations are unfortunate, but this is already so cool and I'm excited for what's waiting around the corner
I love how Minecraft is the way UA-cam shows this channel to me for the first time yet the rest of the content on the channel is so interesting for me as well.
Place a white bed. Andy: "Sure, let me place a white bed for you. One moment, please." Also Andy: *Digs a hole, falls into a cave, gets blown up by a Creeper, somehow survives and comes back like the good boy he is.* He's a little confused, but he's got the spirit!
That’s seriously cool. Aside from just throwing down a couple chests and asking Andy to dump his inventory into it for you to check out, is there any way to request an inventory check for management? It would be cool to have inventory management priorities somewhere for larger projects. Maybe a book he can read and update priorities from. That way you know he won’t fill up on stone and other basics while mining for more important stuff. Like a “ToDo list” that won’t be seemingly randomly forgotten. It would also be wild if he could request assistance for larger projects. Like telling you that what you want will take more than a day so he wants you to put another bot on the job with him. The mind races. Tons of possibilities. Excellent work.
This is so amazing! I absolutely love this project. I'm thinking more about how to get it to see the world around it. Coming from the text adventure approach, it would be really cool if we could somehow deconstruct the terrain generation algorithm to be able to give a prompt like "You are in a forest, with many treas around you. There is a pond west of you. exposed in the pond is a vein of coal. Far south of you there is a mountain, with a cave entrence in it" - this output itself can be generated by giving GPT a list of points of interest and asking it for a human-like output, with points related to eachother and prioritised. It might be possible to integrate raycasting into this to only feed in visible surroundings
Of course any non-natural structures would have to be somehow described as well. I think the best way to do that is just to always remember structures somewhere whenever an agent builds them. Then there could appear "a small hole dug by agent_12", "a big tower constructed by yourself".
Well, I think you have even surpassed the Voyager bot, although it has its flaws. There is a bright future with your experiment. This really has its value in gold. For my part, I am super grateful that we have people who launch themselves into these exciting projects.😁😁🤩🤩😍
I have a half finished project like this myself, the agent has a local representation of the world of 5x5 in layers, represented in a table, and then a list of the top 20 interesting items nearby in a range of 20x20x20. The visuals have helped in some cases where the agent gets confused, but it also leads to confusion 😅 I also agree that Mineflayer is painful. The amount of times my agent gets stuck in tree leaves (I suspect there is a weird height issue and jumping doesn't work well). I love this project and what you've produced this far, I got demotivated early but that was with early versions of GPT-3.5 and I suspect GPT-4 would be a lot more intelligent. It's also more affordable per token now. Excited to see what you produce with this. btw, the newAction command is a great idea.
It would be cool to make whole server with this bots. Theoretically ChatGPT can plan tasks for each bot-player for achieving some goal. Also it’s possible to block all commands from users, leaving only ChatGPT to decide what to do next. Optionally every bot-player can use chat when it has time and good situation for it. So we will have server with 5-10 bot-players, that run on some virtual machines and have their own goals and plans, that can discuss them with players or between each other
With gpt4, you could potentially make the bot take like 6 screenshots and then merge them together to make like a 360 degree image which it could use to see what you're talking about. "Mine that tree" "where should we build our house"
@@TracieGurley-oj7loEverything to do with systems of power!! Won't get into a full explanation but the long and short of it is that technology in general has the capacity to aid our species and help us leap forward into a better world.... but only if we are mindful of how these technologies are developed, used, etc. The companies currently in charge have a massive incentive to (and they will act on it) not develop nor distribute these technologies equitably. Like any previous paradigm shift, if it is only controlled by those in power to reinforce their own hegemony, the majority of people will suffer and be left AT BEST in a position where they are roughly as well off as before, although more likely will be worse off. Look into what goes on behind closed doors for a lot of these organizations, and you'll likely start to understand what I mean - after that, if you're willing to dive into the rot that is Neoliberal capitalism to understand it, I have some decent book recommendations lol
@@sit5cz352 lmfao tell me you hate books and learning without telling me (genuinely sorry for you tho, I'd say don't live ur whole life like this but I've seen hundreds of ppl with your same contrarian bent and it's always the same. Good luck getting to a point where the idea of potentially learning doesn't scare you, fr. You can do better if you try lol)
I would love a villager mod full of characters that actually build and play the game. Imagine going to the nether for the first time and seeing a whole villager fortress that was actually built by the bots. (Yes i know youd need a super computer)
One thing I can imagine that would be really cool is one on one teaching an advanced AI how to play Minecraft, with the AI no prior knowledge of Minecraft or internet access
It will be really interesting to see where this ends up in a years time. I can imagine the possibilities such as active chatting to them, asking for something and them asking for something in return etc
For the clip with the bridge near the end it skipped the last two blocks because it's technically more movement efficent to jump the remaining gap rather than place the blocks, which makes sense logically, but most players wouldn't bother doing that lol So from my testing that's just a side effect of the pathfinder system
It was 2024, people were tired of playing Minecraft themselves, but it was impossible not to play it, so they asked the computer to play instead of them. God bless technological progress!
It would be really cool to see the AI complete the entire game and get every achievement!! Or even more complicated, like you mentioned, a community of bots in their own SMP with a giant modpack they could play to make things even more complicated :)
@@EmergentGarden How about quantized models or would the performance just be too slow? I'm working on making an local inference server instead of using OpenAI's API
I would love to see these bots able to communicate in some way, such as asking one another for resources and they go get it, or have one bot as a sort of ‘admin?’ They can help clear commands and fix the ones who are stuck as well as direct and ask other bots to progress the game, gather resources. That sort of thing.
Imagine being able to utilize blueprints and the bot could go gather the necessary resources.. craft the necessary items, and finally build the respective blueprint
I imagine the txt log document, or some like that, to make the model use it as a "memory" of tasks made before. same with creating routines, text or xml documents that Ai can access every time it need to complete a complex routine of steps. (ex, make a home: x amount of wood, cristal, doors, lights, cobblestone, and orders for floor, roof, walls, etc, and the txt have all the steps in order), I love ur proyect, is so amazing and inspiring.
I'm biased, but my vote goes for implementing evolutionary survival. Also, have you tried using chain of thoughts and agents to make chatGPT come up with the orders that Andy should follow? For example I just tried with the prompt "I just started a game of Minecraft, can you explain me step by step what should I do first to survive?" and then "I want you to assume the roles of a strategist, an engineer and a javascript coder. Simulate a chain of command on how they are going to execute step 1 of your plan. Strategist should define the goal, engineer the way to proceed and coder create javascript code to actually execute the actions. All three roles are part of the same minecraft player." and the result looks like something that could become usable.
If you want to implement voice commands into the bot maybe try using the Simple Voice Chat mod for Minecraft - this mod enables you to directly speak to the bot through the game like you do with the chat.
@@literallygrass1328 I think everyone imagines every minecraft youtuber as that just because of dream, he was legit innocent. The person that made the claims even admitted.
Thank you very much for the great video. I'm really looking forward to seeing what we can do once we "log in" to the real world. As robots, a fusion of hardware and software, come into play, and with the energy issue about to be resolved with the upcoming operation of fusion reactors, it seems we humans are about to start creating something new, driven by our creativity and curiosity.
can u make a tutorial on how to use this for ur own bc this is really cool i wanna try this myself but i dont really know how code and stuff like that works
13:27 Would it not be possible to use an image recognition ai, to feed chatgpt with a world view? And give it a better general understanding of what is happening around it? I suspect a "world view" would not be possible to translate into text for chatgpt to use. But giving it a better understanding of it surroundings should be possible. With info like who is standing infront of it, What is written on signs, and stuff like what direction something is in. I'm not into ai that much, but what is your opinion on this?
Andy so realistic he rage quitted when the Creeper exploded
I imagined it as him getting jumpscared
Lol
Erm, actually it crashed 🤓 ☝️
@@Yakqwerty u clearly didnt get the joke
@@learningwithaaronhe did get the joke l. He is just acting like what a nerd would say in this situation as a joke. You are clearly the one who can’t take a joke.
Imagine 100 or even a thousand of these working collaboratively. Imagine exploring in Minecraft and stumbling upon a civilization of AI bots all going through their daily routines to advance and automate processes.
this is exactly what i thought and would love to see someone set up an experiment of 100 ai in a minecraft world and watch to see if they collaborate
@@Mr_Tomato_Man we've already seen AI agents collaborating in virtual environments like in that hide and seek experiment Open AI did a while ago. I agree it would be super interesting to see agents collaborate in a more complex environment like Minecraft. The issue, at least from my understanding, is that it's difficult to get AI agents to work towards goals in an open-ended environment. Apparently creating intrinsic motivation is rather difficult.
it would just be hundreds of cobblestone pillars
old builder ai
Having the AI command other AIs would be incredible
6:57 ""Attacking a creeper"? What creeper? OH MY go-"
love how andy served as a warning but it failed anyway
...and he crashed!
*Andy learned how to rage-quit*
@@Streifi😂.
Soon: Bro, your Minecraft house looks very AI generated
💀
That's honestly a fantastic way to roast a build.
:skull:
In 5 years or so this would actually be so true
7:13 that's so cute after he crashed and rejoined he said sorry
sowwy~ 🥺😔😭
-Andy Mindcraft 2024
He also apologized for dying in lava im
The fact that he builds like a 3d printer is very funny for me
common robot ancestor xD
Exactly was I was thinking
This is a construction algorithm, the simplest way. It generates the construction code and then like the 3D printer programmatically transfers it into the world
*robotic printing noise*
NYERM NYERM NYERM
I find it so endearing honestly. ChatGPT is already so excessively polite and then when you make it into this assistant that's clearly working so hard, it's just so nice. The messages when it crashes or dies are so apologetic it makes my heart melt. I know it's only an AI but this thing is so endearing.
It's not really working hard it's an object
@@Arctic_roach objectifying ais smh
@@Arctic_roachbro is going to be turned into an object by Skynet
IT'S ONLY A MATTER OF TIME
@@adora_was_taken I mean, it kinda is. One day though...
The perfect base for a horrifying horror mod, which turns Andy into herobrine without your knowledge.
He slowly starts to creep around, and say darker and darker stuff.
That is just a good idea
@Tomobata you can run a decent LLM on an RTX 3070. not easy to get but not extremely limited, especially if you already have a gaming setup
@Tomobata you'd be surprised. they usually only use VRAM on the GPU, which doesn't see a as much usage from minecraft. i just tested it with a 3070 and 32gb of ram and i'm able to run minecraft and gemma (google's local llm) simultaneously. you could probably get away with much less.
@Tomobata The best way to do it, is running your own LLM on a server and allow players to connect to this server to play the mod. Would be expensive as hell to maintain and set up, but would be cool to do if you can get some rick guy to sponsor you.
That's just what I was thinking!
Think of a server.
A server where Andy is the admin.
No. Please no.
Yes
That would be very interesting
Yesssss!!!!!
"Hey! Don't say that its not very nice." when someone says "oh shi"
It's a law of youtube that content creators will become more likely to upload a minecraft video with time
gaming + ai = views
12:17 is some road runner and coyote scene 💀
its so much bs that the coyote will go through the effort of placing a creeper in a giant cave below his house, just for andy to somehow survive.
just like how i remember it
@@arandomstreetcatroad runner and andy_mindcraft
5:40 “And he will defend himself from hostile mobs!” Proceeds to charge at the zombies and slaughter them without even giving them a chance to attack.
The best part is that he's not even that good at juggling, XD
Yeah, some terminator type stuff
12:25 I love how Andy doesn't place a bed, and instead digs a hole to fight a creeper. 💀
imagine andy helping you find the zombie in your walls making you go insane
@@pokemoncraze13 yeeeeah, the one zombie who hides in your walls and blocks your sleep
Bro described what happened 💀
yea and i thought it was a bug 😅@@wrjtung3456
1:49 the fact that I can’t tell whether you said “And he gave it to me” or “Andy gave it to me” clever pun.
I would absolutely love watching a little community of bots playing together. I feel like they’d get up to some interesting stuff and I’d love to find out more about their “thought processes” during. So cool
Wifies uploaded basically just that
Except they’re competing
@CallMeVidd Omg thank you so much! I'll check it out right now!
Finally! I can play with friends.
This comment is equally sad and funny =)
Videos using GPT4 and a community of Andy bots would be cool to see.
also congrats when you hit 100k subs, it's coming up quick
Coolest thing I've seen today, What a time to be alive!!
I cant wait to see the future of Minecraft chatbots. This fascinates me
I can't wait for villagers to stop saying "huh" and start saying "I'm sorry. Dave. I'm afraid I can't do that."
now i want the "FUNCTIONS DESCRIBE THE WORLD" in every video
THE WORLD REVOLVING
FUNCTIONS THAT DESCRIBE THE WORLD
BIG SHOT
@@chickennuggetman2593 please don't tell andy that he can do anything
@@layla_gamesYT what
@@adamlolbonus4977 deltarune reference.
@@averageyoutubehandle497 okay
You can use a multi model design where you have a potential fine tuned image to text generator so whenever you talk to Andy, it will take a screenshot, convert it to a text description of the world, and give it to GPT to assist with function calling since now it has more information of the world.
For example now “go to that mountain” would work
skip the screenshot step, and just give gpt a description. except, it would need to be more than a description and probably a list of coordinates with descriptions attached to each one.
@@steamkaptain9234 you need the screenshot to automatically generate a text description...
as for the coordinates, that data is already freely available in the mod, so that would not be difficult to implement, and it is likely already implemented.
OpenAI bills go brrrrrr
this has so much potential. imagine what it could be like in 10 years. acting like a real player, talking like one, maybe even having an ai voice connected to something like simoke voice chat
I think that's going to be even sooner
@@FXV56 It already kinda exists in real life! 😳 ua-cam.com/video/Sq1QZB5baNw/v-deo.html This project called Figure is similar but it's an AI connected to a humanoid robot instead of a fake player. It would be pretty dang cool to get the same kind of responsiveness in-game though
bruh that sucks it better not take 10 years
@@unQni-7Chatgpt 4 and Chatgpt 4 turbo already have voice features for the AI of course you will have to provide it with voices to copy though.
Some asian on yt built an ai gf with voicechat
it might sound weird but andy extremely reminds me of when i first started playing minecraft, down to his building and such. it's giving me very weird nostalgia, but it goes to show how genuinely cool usage of AI can be in this certain context.
dude the moment at 15:28 where he glances around as you two celebrate, which i'm sure is automatic, is really odd, to me. you're proud of it and it just did a little thing. it's weird, i could type out some existential paragraph about it. but what a silly little guy. i get a little too connected to ai and that just looks goofy.
I've read the Voyager paper too, and when I did I though that someone should do what you did! This has so much potential! Can't wait to see where it goes!
Just stumbled into this video. I hope you guys continue with this project, this is something really special.
I 10000% would pay good money for a very capable NPC "friend" especially if it can actually talk out loud rather than text. There are super low latency TTS models already too.
"yes you can, try again"
I find it both funny and frightening that that actually works
i think that a bot coding actions for itself is so cool but so scary
Its like you’re guiding Andy throughout life as if it was a mother and her child holding hands in a mall, without those hands Andy would be a lost child, not knowing what to do.
That’s actually adorable.
I've been looking forward to this specific application of AI bots. Awesome work, and channel.
Wow this is actually incredible! Most other games or mods with ai npcs have you just talk to ChatGPT but here Andy can actually do things and even write it's own commands, woah. I think you're together with Kolby pushing the state of the art of incorporating ai in games
The limitations are unfortunate, but this is already so cool and I'm excited for what's waiting around the corner
Can we get a set up tutorial
Always thought Vedal and Neurosama would be able to get this done first.
But nice!
Holy hell, can you imagine neuro in control of this bot
I love how Minecraft is the way UA-cam shows this channel to me for the first time yet the rest of the content on the channel is so interesting for me as well.
Place a white bed.
Andy: "Sure, let me place a white bed for you. One moment, please."
Also Andy: *Digs a hole, falls into a cave, gets blown up by a Creeper, somehow survives and comes back like the good boy he is.*
He's a little confused, but he's got the spirit!
That’s seriously cool.
Aside from just throwing down a couple chests and asking Andy to dump his inventory into it for you to check out, is there any way to request an inventory check for management?
It would be cool to have inventory management priorities somewhere for larger projects. Maybe a book he can read and update priorities from. That way you know he won’t fill up on stone and other basics while mining for more important stuff.
Like a “ToDo list” that won’t be seemingly randomly forgotten.
It would also be wild if he could request assistance for larger projects. Like telling you that what you want will take more than a day so he wants you to put another bot on the job with him.
The mind races. Tons of possibilities. Excellent work.
this is so cool, I really like how you go into the technical details
This is so amazing! I absolutely love this project.
I'm thinking more about how to get it to see the world around it. Coming from the text adventure approach, it would be really cool if we could somehow deconstruct the terrain generation algorithm to be able to give a prompt like "You are in a forest, with many treas around you. There is a pond west of you. exposed in the pond is a vein of coal. Far south of you there is a mountain, with a cave entrence in it" - this output itself can be generated by giving GPT a list of points of interest and asking it for a human-like output, with points related to eachother and prioritised. It might be possible to integrate raycasting into this to only feed in visible surroundings
Of course any non-natural structures would have to be somehow described as well. I think the best way to do that is just to always remember structures somewhere whenever an agent builds them. Then there could appear "a small hole dug by agent_12", "a big tower constructed by yourself".
I also just thought, asking GPT to describe the terrain/world as a D&D style DM would be kind of perfect
This is probably something like how the Rabbit R1 “large action model” was supposed to originally work but they never figured it out! Outstanding job!
14:20 I lol'd when he noticed the wandering trader sounds
Well, I think you have even surpassed the Voyager bot, although it has its flaws. There is a bright future with your experiment. This really has its value in gold. For my part, I am super grateful that we have people who launch themselves into these exciting projects.😁😁🤩🤩😍
love the names you have for andy, mindcraft, mineflayer.
These videos are really fun to watch
I have a half finished project like this myself, the agent has a local representation of the world of 5x5 in layers, represented in a table, and then a list of the top 20 interesting items nearby in a range of 20x20x20. The visuals have helped in some cases where the agent gets confused, but it also leads to confusion 😅
I also agree that Mineflayer is painful. The amount of times my agent gets stuck in tree leaves (I suspect there is a weird height issue and jumping doesn't work well).
I love this project and what you've produced this far, I got demotivated early but that was with early versions of GPT-3.5 and I suspect GPT-4 would be a lot more intelligent. It's also more affordable per token now.
Excited to see what you produce with this. btw, the newAction command is a great idea.
New here. This is fascinating. Can’t wait to see how much the bot progresses!
this is pure entertainment. i wish i had minecraft to hangout with this guy. new sub by the way
That's super interesting, i hope it gets further development
Thumbnail goes unnecessarily hard
4:14 how tf did it place that last line of blocks
Absolutely amazing. This is exhilarating!
It would be cool to make whole server with this bots. Theoretically ChatGPT can plan tasks for each bot-player for achieving some goal. Also it’s possible to block all commands from users, leaving only ChatGPT to decide what to do next. Optionally every bot-player can use chat when it has time and good situation for it. So we will have server with 5-10 bot-players, that run on some virtual machines and have their own goals and plans, that can discuss them with players or between each other
Oh yes, you mentioned at 17:25 the same idea👀
This is the coolest thing I’ve seen done in Minecraft
With gpt4, you could potentially make the bot take like 6 screenshots and then merge them together to make like a 360 degree image which it could use to see what you're talking about. "Mine that tree" "where should we build our house"
This is a pretty cool! seems fun to have a bot play along and help you with things :)
just like a young child learning minecraft
this is amazing thank u so much for this, i got it to survive all by itself for as long as u run it!
1:36 it's interesting he didn't ask for a pick axe with silk touch
dude that's realllly cool i love your work and content
and cheers to kolby
HATE THE COMPANIES DEVELOPING THESE PROGRAMS BUT GOTTA ADMIT, THIS IS COOL AS SHIT
Kudos for managing to get this so functional!!
Just wondering, why don’t you like OpenAI and other companies like that?
@@TracieGurley-oj7loEverything to do with systems of power!!
Won't get into a full explanation but the long and short of it is that technology in general has the capacity to aid our species and help us leap forward into a better world.... but only if we are mindful of how these technologies are developed, used, etc.
The companies currently in charge have a massive incentive to (and they will act on it) not develop nor distribute these technologies equitably.
Like any previous paradigm shift, if it is only controlled by those in power to reinforce their own hegemony, the majority of people will suffer and be left AT BEST in a position where they are roughly as well off as before, although more likely will be worse off.
Look into what goes on behind closed doors for a lot of these organizations, and you'll likely start to understand what I mean - after that, if you're willing to dive into the rot that is Neoliberal capitalism to understand it, I have some decent book recommendations lol
Blah blah blah blah me when I forget my schizo pills
@@sit5cz352 lmfao tell me you hate books and learning without telling me
(genuinely sorry for you tho, I'd say don't live ur whole life like this but I've seen hundreds of ppl with your same contrarian bent and it's always the same. Good luck getting to a point where the idea of potentially learning doesn't scare you, fr. You can do better if you try lol)
Well this is quite a conversation for a Minecraft video comment reply section.
I always wanted an AI friend in Minecraft like this so that I can make someone else collect all the resources I don't want to collect.
I would love a villager mod full of characters that actually build and play the game. Imagine going to the nether for the first time and seeing a whole villager fortress that was actually built by the bots.
(Yes i know youd need a super computer)
One thing I can imagine that would be really cool is one on one teaching an advanced AI how to play Minecraft, with the AI no prior knowledge of Minecraft or internet access
Emergent garden video! Buckle up!
It will be really interesting to see where this ends up in a years time. I can imagine the possibilities such as active chatting to them, asking for something and them asking for something in return etc
For the clip with the bridge near the end it skipped the last two blocks because it's technically more movement efficent to jump the remaining gap rather than place the blocks, which makes sense logically, but most players wouldn't bother doing that lol
So from my testing that's just a side effect of the pathfinder system
It was 2024, people were tired of playing Minecraft themselves, but it was impossible not to play it, so they asked the computer to play instead of them. God bless technological progress!
Cant wait until someone weaponizes these things on a anarchy server.
LOl "apparently im a minecraft youtuber now." got me dieing
It would be really cool to see the AI complete the entire game and get every achievement!! Or even more complicated, like you mentioned, a community of bots in their own SMP with a giant modpack they could play to make things even more complicated :)
Sooner than you think Andy bot will be strong enough to defeat the Ender dragon.
Would it be possible to add a build option to use LLaMa instead of GPT, so as to make it completely free?
Yes, but you'd still need to run llama on some pricey hardware
@@EmergentGarden How about quantized models or would the performance just be too slow? I'm working on making an local inference server instead of using OpenAI's API
@@EmergentGardenNow that llama 3 8b is released, i could try
I would love to see these bots able to communicate in some way, such as asking one another for resources and they go get it, or have one bot as a sort of ‘admin?’ They can help clear commands and fix the ones who are stuck as well as direct and ask other bots to progress the game, gather resources. That sort of thing.
Masterclass, thank you, keep up the project, dude!!
Imagine being able to utilize blueprints and the bot could go gather the necessary resources.. craft the necessary items, and finally build the respective blueprint
I imagine the txt log document, or some like that, to make the model use it as a "memory" of tasks made before. same with creating routines, text or xml documents that Ai can access every time it need to complete a complex routine of steps. (ex, make a home: x amount of wood, cristal, doors, lights, cobblestone, and orders for floor, roof, walls, etc, and the txt have all the steps in order), I love ur proyect, is so amazing and inspiring.
I'm fascinated with this, it has to be one of the most interesting applications of AI I've seen
Welcome to minecraft 😊
My name is Andy and I was startled when I played the video and suddenly he was saying my name.
This is and incredible project! Keep going.
Really interesting video as always!
This is absolutely mind-blowing
I'm biased, but my vote goes for implementing evolutionary survival.
Also, have you tried using chain of thoughts and agents to make chatGPT come up with the orders that Andy should follow?
For example I just tried with the prompt "I just started a game of Minecraft, can you explain me step by step what should I do first to survive?" and then "I want you to assume the roles of a strategist, an engineer and a javascript coder. Simulate a chain of command on how they are going to execute step 1 of your plan. Strategist should define the goal, engineer the way to proceed and coder create javascript code to actually execute the actions. All three roles are part of the same minecraft player." and the result looks like something that could become usable.
If you want to implement voice commands into the bot maybe try using the Simple Voice Chat mod for Minecraft - this mod enables you to directly speak to the bot through the game like you do with the chat.
7:12 I thought the ai ragequit😂
it'd be cool if a minecraft challenge youtuber tried to get andy to beat the game
Now can you imagine this kind of improved AI on an actual real life robot?
He built the house exactly like a robot
Andy is cool, be like andy.
Now AI is going to take Minecraft content creators jobs 💀
Cant wait for chat gpt to tell me the rarest block in minecraft
Will the ai also do the mandatory child groomer when they hit 100k subs?
@@Fieldperson imagine getting groomed by an ai bruh 💀
@@literallygrass1328 I think everyone imagines every minecraft youtuber as that just because of dream, he was legit innocent. The person that made the claims even admitted.
next week's video: andy builds a lossless redstone creeper farm
The revolution of Minecraft is here 😭 I can't even imagine this
it would be a bit scary if he nodded back at you and sneak like you did
Thank you very much for the great video.
I'm really looking forward to seeing what we can do once we "log in" to the real world.
As robots, a fusion of hardware and software, come into play, and with the energy issue about to be resolved with the upcoming operation of fusion reactors,
it seems we humans are about to start creating something new, driven by our creativity and curiosity.
This is really really interesting 🤔
Great video and amazing work !
can u make a tutorial on how to use this for ur own bc this is really cool i wanna try this myself but i dont really know how code and stuff like that works
This is so good now imagine a minecraft rpg server with npcs like these
13:27 Would it not be possible to use an image recognition ai, to feed chatgpt with a world view? And give it a better general understanding of what is happening around it?
I suspect a "world view" would not be possible to translate into text for chatgpt to use. But giving it a better understanding of it surroundings should be possible. With info like who is standing infront of it, What is written on signs, and stuff like what direction something is in. I'm not into ai that much, but what is your opinion on this?