Friend, thank you very much for the video! I really want to start learning Unity and repeat what you did in this video! But unfortunately I have absolutely no experience in this and I live in Russia, which causes some difficulties when working with the ChatGPT API, but I will try to figure it out. Thank you very much again for the video! I really hope that you will make another video again so that I can understand everything in more detail. Have a nice day to you!
@@sgt3v Thanks for the answer my friend, I will try later when I meet Unity to watch your videos and understand what you did to connect ChatGPT to the Unity project. Do you have any step by step instructions on how to do it or is the video enough to do it?
Thank you~Sorry.But I want to ask a question for avatar moving.After I created a character and adjusted the camera target, I wanted to move the character using the keyboard to move left and right(Only jumping can be used at 21:10), but I still couldn't do it. What's wrong is this?
Error Message: You exceeded your current quota, please check your plan and billing details. even for a new acount, I haven't used it , In very first use how can it say that quota exhausted ?
Thanks for the video. Simple "if this then that" on events. How would you take this one step further to instructing the NPC? For instance "Pickup that gun" or "Let's sit here and talk about it." Obvious way would be to map animations names to a database and search the database to find a similar animation and trigger it. But old school compared to what ai offers. Would be awesome to not even trigger animations but have the character animate via AI based on instructions. Combination of AI4Animation and ChatGPT perhapse?
for some reason, my IDE (VScode) doesn't automatically add "using" statements at the top of the file. At 47:25, you get a suggestion "PlayerInput (in UnityEngine.InputSystem)" and it automatically adds "using UnityEngine.InputSystem" to the top of the file. mine doesn't show that PlayerInput is an option. any suggestions?
got it to work, you need to enable the setting in C# extensions. search "Intellisense for available 'using/import's in C# with Visual Studio Code" and the Stackoverflow post gives the step by step.
51:30 The character's position is not moving because the "Character Controller" is controlling the position. To control the position, the "Character Controller" component must be changed after enable = false. Example code: if (playerTransform.TryGetComponent(out CharacterController characterController)) { characterController.enabled = false; playerTransform.SetPositionAndRotation(standingPoint.position, standingPoint.rotation); characterController.enabled = true; } else { playerTransform.SetPositionAndRotation(standingPoint.position, standingPoint.rotation); }
ur problem is solved now I'll introducing AI CHAT DEEPAI the AI CHAT DEEPAI is so powerful and much better than OPENAI since start 2015-2018 and u can show ur picture and video and anything u want
Could you make a video about bureaucracies, like contracting a plan, how much it cost, the estimated ideal usage for each plan? I don't know if I'm asking for something off scope, but I think it would be nice to know these things as well.
I would love you to use Salsa 2 to power the NPC you are talking to. That would take care of animating the NPC and feeding it the returned text would drive the speech.
wow this is great!! I've tried this since last week and have some problems..can you help me out? 1. time 1:07:51 : when I set the fuction of "close" button from NpcDialog , it doesn't appear 'Recover' window in the Event trigger.
Hi, as written in the description different videos are covered in different branches and this videos content targets video-1 branch on github. Changes are shown in the next video.
This is the most basic example. But man the possibility. One of my favorite games of all time is morrowind. Because of the freedom and how you could ask everyone about almost everything you have ever been told. Ofc they all have limited pre made responses so talking to the npcs is not interesting by modern standards. But using OpenAI a system like that would suddenly come alive and every single npc could have a life like conversation with you that remembers things in context. Add AI driven text to speech and the future is a wild place for gaming. Only question now seems to be if this can scale. Also is it limited to running on a server or can you run it on your desktop one day. Really cool things ahead.
I want to show some more examples of this with some so-called prompt engineering tricks and show how NPCs can be made more context dependent and sophisticated. I think it will have its first adoption in game studios as a assistant to writers. Then we will see smaller models pretrained for the context of that game.
Thank you for your video. I have a problem. In my case, unity tab 'ready player me' - I can see only 'check for updates' option. I can't see avatar loader, settings option. why?
You can restart the Unity Editor see if the issue is still there. Or check the package manager if RPM Avatar Loader is installed, if not please re install the Core package again while there are no errors in console.
@@sgt3v Thank you for your quick reply even though you must be busy. The same symptoms appear when you restart unity. We've also started several new projects and installed RPM, and the same problem occurs. The difference from your video is that if you mouse over the "Ready Player Me" tab at the top of the unity screen, there are no "Avatar Loader" and "Settings" items, only the "Check for Update" items, and secondly, there is no "Packages-Andras Attendant" (glast) point in "package manager" after RPM installation. Only three subtopics of Packages-Ready Player Me are installed,
@@Robloxzz This means that Avatar Loader package was not installed, can you see it in the Package Manager? Other wise you can manually install it via git URL available in our GitHub
Hi Ievgen, I plan to use NVIDIA text-to-anim to do exactly that, it does not have Unity integration so that part is also onto me to figure. Might take a while.
It really helps😆I have already subscribed. That's brilliant!!!!!BTW, how to solve the problem of failing to teleport the avatar to the standing pointing?
@@sgt3v are you thinking about a video in which instead of using the chat window of openai, you use a TTS and a mouth lipsync to show the character speaking? I mean it would be nice to have a NPC that can act and speak as a specific character (a pharaoh or a medieval knight ecc....), generate a text from open AI, convert it with TTS and lipsync animation on his face.
@@sceltadelcamper I am working on Nvidia ACE to bring that support to RPM SDK, that would use ARKit blendshapes and have better results. But alternatives are of course possible.
have also made a c# api a while ago, have basicly same structure what allowed me fast to start with yours because that basicly merged automatic with my existing haha nice nice
I tried doing a webgl build.The game runs perfectly in the editor but for some reason in the build once we reach the boundary near the avatar it doesn't switch to the second camera thus nothing happens. Please can anyone help me fix this??
Hey Sarge, i am Data science and Ai student, i have to make an NPC within 19 days from now as a decertation project. i have never used unity, i am getting error while adding open ai unity package. i am really afraid. can you please help me out. otherwise i will fail.
Thank you for providing this great resource. I updated the Player Ready Me character on their website and it didn't load the latest version within Unity. Must the avatar be reimported entirely in Unity? By the way, if you adjust the "Character Arm-Space" animation in Mixamo, the avatar arms will not intersect the body. I found a value of around 70 works well. Of course, this will depend on the actual animation.
My friend please tell me where I can find the setting font color and background text when the chat window opens with the NPC? Right now when communicating with NPCs, the background of the user's text is highlighted blue, but I don't know where to change that.
@@sgt3v Thanks for the tip I found this setting, but it appears for some reason only in project launch mode and when entering text, but in the editor mode it is not. Anyway, I'll try to figure it out and find it, thanks a lot for your awesome work! Very much looking forward to more videos on this topic :)
if u want create ur own games and apps I'll introducing AI CHAT DEEPAI the AI CHAT DEEPAI is so powerful and much better than OPENAI since start 2015-2018 and u can show ur picture and video and anything u want
There's a pause in the animation when sending the message and again after receiving the reply. Is there any way to eliminate the pause in the animation? Basically, can an idle animation run unaffected by the background operations?
@@sgt3v Thank you for your insights. The problem actually occurs while the animation is looping. When the button is pressed to send the question to Chat GBT the looping animation freezes for a few seconds. Then the animation freezes again when the reply is return. It starts again when the reply is spoken.
Hello my friend I'm still doing the project you showed in this video and ran into a problem I can't solve. Maybe you know the solution ? The game character moves in one place (walks in place) when starting the scene until the run key (Shift) is pressed or until the camera angle is changed with the mouse, after that the game character continues to move, but if you stop it and resume walking it will also walk in place until the keys are pressed. How can this be solved please help?
Hi Max, it might be that you missed one of the steps while setting the Character up. Its kind of tricky to get that right, you should set the camera follow after setting the character.
Great stuff keep it up! just one thing about the video, your recording bitrate is very low, for 1080p the image is still pixelated so I encourage you to record with higher bitrate (BandiCam is mad, it utilise the power of the GPU) ^^
Thank you for the feedback Spartinus, I make the videos quickly single take with a quick edit. Not prepared at all, sorry about that. I use OBS and record pretty much with the default settings, I will take a look at the bitrate settings. 🙏
@@sgt3v bro one more feedback, I know how great this is but as I can see it's not mainstreamable, for example I can't use it for mobile other platform like VR , and even Desktop build is annoying because that other person have to has the directory in his local disk, so im wondering what are the use cases of such project other than testing it urself(as of the dev)
hey bro, nice video. i tried to do the same but it always showing auth.json not found in the unity3d console. i alredy put the file in the user section, but its not detecting. is it because my unity3d editor in diffrent drive?
I just started getting this error on a chat response: JsonSerializationException: Could not find member 'warning' on object of type 'CreateChatCompletionResponse'. Path 'warning', line 2, position 12. Has anything since last week changed with OpenAIApi.cs. This line seems to be an issue: data = JsonConvert.DeserializeObject(request.downloadHandler.text, jsonSerializerSettings);
@@Ji-sz5sk Thank you! I'll give this a try. Adding "public string warning { get; set; }" within Packages>OpenAI Unity>Runtime>DataTypes under "public struct CreateChatCompletionResponse : IResponse" also fixed the issue.
My friend hello. Please advise, for a few months everything was working fine I logged in to my project sometimes to check that it was working. But today when I tried to use dialog with NPC an error appeared: "Error Message: You exceeded your current quota, please check your plan and billing details. Error Type: insufficient_quota". I understand that the free token limit is used up, but what should I do in this case ? I need this project to demonstrate my thesis project and it makes no sense to switch to a paid subscription period. If I won't use it for some time, will it be possible to use it again ? Or do I have to do it all over again and a new account ?
@@sgt3v The thing is that I do not have the ability to pay, I am from Russia and our bank cards do not allow you to pay on a third-party service. Is there any way I can temporarily extend the use for free? Perhaps if I wait some of the tickets will be restored or it doesn't work that way?
This has been a wonderful set of tutorials so far and I'm so close to getting everything to work. However I seem to run into problems right after using the editor to apply the settings to my character model, after fixing the cameras and loading into the scene the camera will follow my character but the character struggles to move initially with WASD. They walk in place for a while, holding the walk buttons will eventually get them to move. Any ideas what's causing this?
Hi, This is a problem mostly originating from Unity TP Controller. Please make sure you manually set the camera follow object, which I mention here 20:08
@@sgt3v Hi, thank you for the response, sorry for the late reply, it was night for me. I had went to double check and both my Follow and Look At are properly set to my character, the camera works fine as it follows and rotates around the character however the character still walks in place.
@@Welavix Make sure the "Min Move Distance" value within the Character controller is set to 0. Initially, mine was set to 0.001 and the character wouldn't move.
My friend I solved this problem so that I selected the main player and in the field of methods of inputting controls instead of the value Any I specified the value of the keyboard and mouse and the problem went away. What I still can not cope with is that on the stage in the editor all the objects are well visible only at close range, but if you fly away when the scene is not running, they seem to be in a fog.
incredible. Is it possible to give those NPCs specific knowledge base? I want to create a trainer and I need them to be able to converse but also stick to the factual information without making things up. How can I achieve that?
@@sgt3v thanks for your response! I am also wondering if it would be possible to assign a bank of information to objects in the scene and have an NPC talk about it when moving from one object to another. Do you think it's possible?
Where would we want to keep the openai api key if we want to build to iOS? Would love to just add the api in a input field after building to phone. Haven’t figured out how to do it with the sdk yet . Any help would be amazing ! Thank you for bridging the ai gap to us creators!!!
Hi, thanks for this video :) I was trying to build it for android. I made the following changes to the configuration script. It works in Unity editor but when built to the device it does not. Do you know why that's happening? public Configuration() { var myData = new { api_key = "added_my_key here" }; string jsonData = JsonConvert.SerializeObject(myData); Auth = JsonConvert.DeserializeObject(jsonData, jsonSerializerSettings); }
Probably best would be to remove the exceptions in config file that it looks for the auth.json, that might be causing the issue. Are there any error messages?
@@sgt3v for this particular method there were no errors messages. I looked through the other scripts, I was not able to locate where the Auth file was actually being used. Also, I tried placed the auth file in StreamingAssets and provided that path, but that did not work. Got the error as “no key was found” or something like that.
Hey, I've been trying to look over your code but I'm just learning c# and sometimes it's not clear to me why some things are done in a particular way. Could you either make a video listing the resources you used to learn c# and unity or a reply with some good textbooks or lectures. Thanks for the content. I don't just want to copy your code but I want to generate it from scratch.
to understand ur problem code I'll introducing AI CHAT DEEPAI the AI CHAT DEEPAI is so powerful and much better than OPENAI since start 2015-2018 and u can show ur picture and video and anything u want
So from what I can see, you need to pay for the use of GPT in Unity, in the form of tokens which is kinda makes this non viable for any large scale projects since you would need to get players on a subscription base to pay for tokens to then have NPC's speak to you.
Hey, Sarge! Thanks for your good video on Unity attach with OpenAI, it's a good tutoral for me to learn it! By the way, I want to make a game that the player can order the NPC AI to walk, jump, dance, and the npc and also do these motions, do you think it is possible to make it?
Hi Joseph, thank you. Glad to hear it was useful. I made a followup video to this one about prompt engineering you can check it out, it is kind of about a similar topic.
@@sgt3v thanks! I have seen it, so basically if I want to catch the walk order, I need to use the same steps like the End_Conversation part in the next video right?(but I think it need to do lots of same steps HAHA
@@josephhan3954 Yes exactly, you can have a chain of commands in the prompt, since it can analyze the sentiment and produce result with that it is better than just parsing user speech and deciding. For example what I said might be: "never jump." or "don't you think of not jumping now." if I just check that string has "jump" and/or "don't" to decide the jump command, its gonna fail in both cases. But If I tell ChatGPT to end the response with JUMP_COMMAND if the user response indicates that the player character should jump, then it will understand it regardless word plays.
Unfortunately won't work online until we have a very compact and local ai model that can be shipped with unity. GPT3 itself is a very very large ai model.
@@sgt3v Yeah it would be too good to be true. But now that chat gpt gets is so mainstream and everybody is using it will it affect the time that the npc takes to answer? Or when their website is down due to the extreme traffic, will this still work? Thanks for the answer btw
What I predict is there will be platforms, even native support in game engines to train very compact and small models that is focused on the games content.
@@sgt3vI added an text to speech module, with a female voice. If I relay with every query "you are a women", it creates a strange bias where for example if I ask for a joke he will only make woman based jokes. What is the correct way to ask chat gpt to respond in a certain way (like he is a woman or like he is a 1950 bank teller)?
Hi @lefteron6804 I am currently working on a service product to make this much easier and possible, if you are working in a game or tech company and need help with this, you can reach out to me via email and I can show you what it can do. You can check it our here neocortex.link
@@sgt3v It's actually for my theses in university. I've made a detective game and i would like to make a mode where you can organicly ask questions to the npcs.
Checkout my new project Neocortex for simplest way to integrate Smart NPCs in your games
neocortex.link
I've been dreaming of that type of AI NPC in a RPG for years ! Now we're here ! Incredible !
Assuming you’re into text adventures and haven’t heard of AI Dungeon, that’s a great place to play a text based rpg with ai
Thank you, Sercan. This was extremely useful on many levels. I look forward to future videos!
Really happy to hear this!
A video I didn't know I needed to see. Thank you, subbed!
This is one of the coolest ideas i've ever seen
This is cool Secran. Thank you for sharing to us.
Glad you found it useful!
Again awesome video, I am following your all videos. Every video is amazing, thank you!
Friend, thank you very much for the video! I really want to start learning Unity and repeat what you did in this video! But unfortunately I have absolutely no experience in this and I live in Russia, which causes some difficulties when working with the ChatGPT API, but I will try to figure it out. Thank you very much again for the video! I really hope that you will make another video again so that I can understand everything in more detail. Have a nice day to you!
Thanks Max. You might wanna try using VPN if the OpenAI services are blocked.
@@sgt3v Thanks for the answer my friend, I will try later when I meet Unity to watch your videos and understand what you did to connect ChatGPT to the Unity project. Do you have any step by step instructions on how to do it or is the video enough to do it?
I had another video before this one, that should help: ua-cam.com/video/MQfVCY9qgEU/v-deo.html
Thank you~Sorry.But I want to ask a question for avatar moving.After I created a character and adjusted the camera target, I wanted to move the character using the keyboard to move left and right(Only jumping can be used at 21:10), but I still couldn't do it. What's wrong is this?
Great video .
This was really helpful
Glad it was helpful!
Error Message: You exceeded your current quota, please check your plan and billing details.
even for a new acount, I haven't used it , In very first use how can it say that quota exhausted ?
Thanks for the video. Simple "if this then that" on events. How would you take this one step further to instructing the NPC? For instance "Pickup that gun" or "Let's sit here and talk about it." Obvious way would be to map animations names to a database and search the database to find a similar animation and trigger it. But old school compared to what ai offers. Would be awesome to not even trigger animations but have the character animate via AI based on instructions. Combination of AI4Animation and ChatGPT perhapse?
These are also possible, I am working on a complete course explaining and teaching these.
Great Tutorial Man, Thank you!
FOOTBALL MANAGER PC GAME AND GTA NEEDS THIS.
Thanks for sharing
I lol'd at the end when we see the female NPC talking animation for the first time, she looked like she was annoyed and rude to him hahaha
9:03 I signed up for this site, but the Unity SDK doesn't show up in the left menu. What should I do?
Thank you so much! This was a great tutorial.
for some reason, my IDE (VScode) doesn't automatically add "using" statements at the top of the file. At 47:25, you get a suggestion "PlayerInput (in UnityEngine.InputSystem)" and it automatically adds "using UnityEngine.InputSystem" to the top of the file. mine doesn't show that PlayerInput is an option. any suggestions?
got it to work, you need to enable the setting in C# extensions. search "Intellisense for available 'using/import's in C# with Visual Studio Code" and the Stackoverflow post gives the step by step.
@@stevegreen5638 Yes, VS Code, VS Studio and JetBrains have unity packages which help with this.
Sercan hocam gerçekten teşekkürler detaylı bir video hazırlamışsınız ellerinize sağlık
51:30
The character's position is not moving because the "Character Controller" is controlling the position. To control the position, the "Character Controller" component must be changed after enable = false.
Example code:
if (playerTransform.TryGetComponent(out CharacterController characterController))
{
characterController.enabled = false;
playerTransform.SetPositionAndRotation(standingPoint.position, standingPoint.rotation);
characterController.enabled = true;
}
else
{
playerTransform.SetPositionAndRotation(standingPoint.position, standingPoint.rotation);
}
ur problem is solved now I'll introducing AI CHAT DEEPAI the AI CHAT DEEPAI is so powerful and much better than OPENAI since start 2015-2018 and u can show ur picture and video and anything u want
Could you make a video about bureaucracies, like contracting a plan, how much it cost, the estimated ideal usage for each plan? I don't know if I'm asking for something off scope, but I think it would be nice to know these things as well.
I would love you to use Salsa 2 to power the NPC you are talking to. That would take care of animating the NPC and feeding it the returned text would drive the speech.
Link for salsa 2 ?
Hi Bob, I do not know what Salsa 2 is and google search resulted only in dance videos. :D
@@sgt3v Unity Asset Store from Crazy Minnow software.
@@arpu4967 Unity Asset Store from Crazy Minnow software
No need. Ready player me's built in voice to lipsych component on their runtime avatars is good enough. Uma is great but is not as universal as RPM.
.
just subscribed .. thanks ,this awesome buddy
I'm sorry to ask you so often.
Unlike your character can moving in 20:43, my character is not moving in the direction key.Why is that?
Hi! I have the same problem.Did you solve it?
wow this is great!!
I've tried this since last week and have some problems..can you help me out?
1. time 1:07:51
: when I set the fuction of "close" button from NpcDialog , it doesn't appear 'Recover' window in the Event trigger.
Hi, as written in the description different videos are covered in different branches and this videos content targets video-1 branch on github. Changes are shown in the next video.
@@sgt3v ok! thnx much!! this is an awesome project!!
This is the most basic example. But man the possibility. One of my favorite games of all time is morrowind. Because of the freedom and how you could ask everyone about almost everything you have ever been told. Ofc they all have limited pre made responses so talking to the npcs is not interesting by modern standards. But using OpenAI a system like that would suddenly come alive and every single npc could have a life like conversation with you that remembers things in context. Add AI driven text to speech and the future is a wild place for gaming. Only question now seems to be if this can scale. Also is it limited to running on a server or can you run it on your desktop one day. Really cool things ahead.
I want to show some more examples of this with some so-called prompt engineering tricks and show how NPCs can be made more context dependent and sophisticated. I think it will have its first adoption in game studios as a assistant to writers. Then we will see smaller models pretrained for the context of that game.
Thank you for your video.
I have a problem. In my case, unity tab 'ready player me' - I can see only 'check for updates' option. I can't see avatar loader, settings option. why?
You can restart the Unity Editor see if the issue is still there. Or check the package manager if RPM Avatar Loader is installed, if not please re install the Core package again while there are no errors in console.
@@sgt3v Thank you for your quick reply even though you must be busy. The same symptoms appear when you restart unity. We've also started several new projects and installed RPM, and the same problem occurs.
The difference from your video is that if you mouse over the "Ready Player Me" tab at the top of the unity screen, there are no "Avatar Loader" and "Settings" items, only the "Check for Update" items, and secondly, there is no "Packages-Andras Attendant" (glast) point in "package manager" after RPM installation. Only three subtopics of Packages-Ready Player Me are installed,
@@Robloxzz This means that Avatar Loader package was not installed, can you see it in the Package Manager? Other wise you can manually install it via git URL available in our GitHub
@@sgt3v Thank you very much. The solution is installing "glTFast". Thank you!!
@@Robloxzz glTFast comes with RPM SDK.
so if i use it for personal use its completely free also is it unrestricted like allowing nsfw conversations.
I subscribed. It remains to teach the avatar to recognize the voice, make a project for VR and it would be incredible!
Hi Ievgen, I plan to use NVIDIA text-to-anim to do exactly that, it does not have Unity integration so that part is also onto me to figure. Might take a while.
It really helps😆I have already subscribed. That's brilliant!!!!!BTW, how to solve the problem of failing to teleport the avatar to the standing pointing?
Hi, I am follower # 1000.... Nice video
Thank you, really appreciate it.
@@sgt3v are you thinking about a video in which instead of using the chat window of openai, you use a TTS and a mouth lipsync to show the character speaking? I mean it would be nice to have a NPC that can act and speak as a specific character (a pharaoh or a medieval knight ecc....), generate a text from open AI, convert it with TTS and lipsync animation on his face.
@@sceltadelcamper I am working on Nvidia ACE to bring that support to RPM SDK, that would use ARKit blendshapes and have better results. But alternatives are of course possible.
Just released a tutorial on text-to-speech! ua-cam.com/video/rdHqRRzltTo/v-deo.html
have also made a c# api a while ago, have basicly same structure what allowed me fast to start with yours because that basicly merged automatic with my existing haha nice nice
ReadyPlayerMe has the best avatars that exist!!! 👏
this is great!
I tried doing a webgl build.The game runs perfectly in the editor but for some reason in the build once we reach the boundary near the avatar it doesn't switch to the second camera thus nothing happens. Please can anyone help me fix this??
How can you put chatgpt into an avatar? Where it talks to you in real time?
Is there a way to train the ai to answer question beased on a script
Hey Sarge, i am Data science and Ai student, i have to make an NPC within 19 days from now as a decertation project. i have never used unity, i am getting error while adding open ai unity package.
i am really afraid. can you please help me out. otherwise i will fail.
i cant rotate the standing point i dont know why
Thank you for providing this great resource. I updated the Player Ready Me character on their website and it didn't load the latest version within Unity. Must the avatar be reimported entirely in Unity? By the way, if you adjust the "Character Arm-Space" animation in Mixamo, the avatar arms will not intersect the body. I found a value of around 70 works well. Of course, this will depend on the actual animation.
There has not been much change in how avatars are imported, so it can stay the same.
I got rotation to update, but never position...
Do you think is it possible to give the chatgpt npc a network behaviour interacting with network objects?
Amazing video, thank you
Theoretically, this should be possible with the correct code implements. Someone who understands programming well may be able to do it in a weekend.
My friend please tell me where I can find the setting font color and background text when the chat window opens with the NPC? Right now when communicating with NPCs, the background of the user's text is highlighted blue, but I don't know where to change that.
You can edit the message prefabs for that.
@@sgt3v Thanks for the tip I found this setting, but it appears for some reason only in project launch mode and when entering text, but in the editor mode it is not. Anyway, I'll try to figure it out and find it, thanks a lot for your awesome work! Very much looking forward to more videos on this topic :)
Hello, Did you train ChatGPT by certain document or prompt to be customized for NPC?
if u want create ur own games and apps I'll introducing AI CHAT DEEPAI the AI CHAT DEEPAI is so powerful and much better than OPENAI since start 2015-2018 and u can show ur picture and video and anything u want
awesome! Is there a demonstration using speak to text softwares or AI to talk chatgpt within game?
I am planning to show how to use Amazon Polly with this.
Just released a tutorial on text-to-speech! ua-cam.com/video/rdHqRRzltTo/v-deo.html
Hello, I'm trying to follow this tutorial but for some reason the ready player me package is never installed succesfully
I just solved, just installed the rpm package from disk instead of git
There's a pause in the animation when sending the message and again after receiving the reply. Is there any way to eliminate the pause in the animation? Basically, can an idle animation run unaffected by the background operations?
You can cut and blend the animations in Unity, I tried to keep it as simple as possible in the tutorial.
@@sgt3v Thank you for your insights. The problem actually occurs while the animation is looping. When the button is pressed to send the question to Chat GBT the looping animation freezes for a few seconds. Then the animation freezes again when the reply is return. It starts again when the reply is spoken.
Thank you Sercan.
You're welcome Elio, hope it was useful.
Hello my friend I'm still doing the project you showed in this video and ran into a problem I can't solve. Maybe you know the solution ? The game character moves in one place (walks in place) when starting the scene until the run key (Shift) is pressed or until the camera angle is changed with the mouse, after that the game character continues to move, but if you stop it and resume walking it will also walk in place until the keys are pressed. How can this be solved please help?
Hi Max, it might be that you missed one of the steps while setting the Character up. Its kind of tricky to get that right, you should set the camera follow after setting the character.
thank you !! managed to create a nice dragon character who can chat nicely) Thank you so much
You're welcome!
@@sgt3v and may be you can make a small tutorial how to set up my key so that the app works on android? is this possible?
@@mmilamix Will do!
U R the GOD. Thanks bro :)
Great stuff keep it up! just one thing about the video, your recording bitrate is very low, for 1080p the image is still pixelated so I encourage you to record with higher bitrate (BandiCam is mad, it utilise the power of the GPU) ^^
Thank you for the feedback Spartinus, I make the videos quickly single take with a quick edit. Not prepared at all, sorry about that. I use OBS and record pretty much with the default settings, I will take a look at the bitrate settings. 🙏
@@sgt3v cheers and good luck on your dev journey
@@sgt3v bro one more feedback, I know how great this is but as I can see it's not mainstreamable, for example I can't use it for mobile other platform like VR , and even Desktop build is annoying because that other person have to has the directory in his local disk, so im wondering what are the use cases of such project other than testing it urself(as of the dev)
@@Spartinus a web service would solve the problem in that case. When I have the time I will also show how that part could be moved out of Unity.
@@sgt3v Great! i'll be waiting for that patiently.
hey bro, nice video. i tried to do the same but it always showing auth.json not found in the unity3d console. i alredy put the file in the user section, but its not detecting. is it because my unity3d editor in diffrent drive?
you can use the api key directly in OPenAIPI ctor as a paramater.
Hi,how can i solve this issue：Cannot implicitly convert type 'UnityEngine.Quaternion' to 'UnityEngine.Vector3'，thanks！
XYZ representation of a Quaternion is Euler. You can use Euler type to access axis of a rotation.
I just started getting this error on a chat response: JsonSerializationException: Could not find member 'warning' on object of type 'CreateChatCompletionResponse'. Path 'warning', line 2, position 12. Has anything since last week changed with OpenAIApi.cs. This line seems to be an issue: data = JsonConvert.DeserializeObject(request.downloadHandler.text, jsonSerializerSettings);
I had the same problem. Use another model. For example, gpt-3.5-turbo or gpt-3.5-turbo-0613
@@Ji-sz5sk Thank you! I'll give this a try. Adding "public string warning { get; set; }" within Packages>OpenAI Unity>Runtime>DataTypes under "public struct CreateChatCompletionResponse : IResponse" also fixed the issue.
The issue is fixed in the latest version 0.1.15
My friend hello. Please advise, for a few months everything was working fine I logged in to my project sometimes to check that it was working. But today when I tried to use dialog with NPC an error appeared:
"Error Message: You exceeded your current quota, please check your plan and billing details.
Error Type: insufficient_quota".
I understand that the free token limit is used up, but what should I do in this case ? I need this project to demonstrate my thesis project and it makes no sense to switch to a paid subscription period. If I won't use it for some time, will it be possible to use it again ? Or do I have to do it all over again and a new account ?
You will need to pay for API after some point.
@@sgt3v The thing is that I do not have the ability to pay, I am from Russia and our bank cards do not allow you to pay on a third-party service. Is there any way I can temporarily extend the use for free? Perhaps if I wait some of the tickets will be restored or it doesn't work that way?
@@MariMaxVR Maybe it might be bought via Azure, which provides OpenAI access.
@@sgt3v Thanks, I'll try to find out and pay.
This has been a wonderful set of tutorials so far and I'm so close to getting everything to work.
However I seem to run into problems right after using the editor to apply the settings to my character model, after fixing the cameras and loading into the scene the camera will follow my character but the character struggles to move initially with WASD.
They walk in place for a while, holding the walk buttons will eventually get them to move. Any ideas what's causing this?
Hi, This is a problem mostly originating from Unity TP Controller. Please make sure you manually set the camera follow object, which I mention here 20:08
@@sgt3v Hi, thank you for the response, sorry for the late reply, it was night for me.
I had went to double check and both my Follow and Look At are properly set to my character, the camera works fine as it follows and rotates around the character however the character still walks in place.
@@Welavix Make sure the "Min Move Distance" value within the Character controller is set to 0. Initially, mine was set to 0.001 and the character wouldn't move.
@@williamowens6049 Man you have no idea how much this helped me
Important question! If i wanted to add other NPC's that use the same system but have different prompts, how would i go about that?
In this video I go into similar subject: ua-cam.com/video/usrxIUGK9Gc/v-deo.html
You can have different sources of info to feed into the system.
@@sgt3v wow thanks!
My friend I solved this problem so that I selected the main player and in the field of methods of inputting controls instead of the value Any I specified the value of the keyboard and mouse and the problem went away. What I still can not cope with is that on the stage in the editor all the objects are well visible only at close range, but if you fly away when the scene is not running, they seem to be in a fog.
There should be a game object called global Volume in the scene you can disable it.
incredible. Is it possible to give those NPCs specific knowledge base? I want to create a trainer and I need them to be able to converse but also stick to the factual information without making things up. How can I achieve that?
You can provide that factual info in the initial message, then it should mainly use it.
@@sgt3v thanks for your response!
I am also wondering if it would be possible to assign a bank of information to objects in the scene and have an NPC talk about it when moving from one object to another. Do you think it's possible?
Where would we want to keep the openai api key if we want to build to iOS? Would love to just add the api in a input field after building to phone. Haven’t figured out how to do it with the sdk yet . Any help would be amazing ! Thank you for bridging the ai gap to us creators!!!
The best is to keep it server-side. You might also prevent your endpoint from getting flooded with spam requests there.
@@sgt3v Thank you! Working on it this weekend.
Is there a similar video to replicate this project on Unreal Engine 5?
Not that I know of but probably someone can port it there.
Hi, thanks for this video :)
I was trying to build it for android. I made the following changes to the configuration script. It works in Unity editor but when built to the device it does not. Do you know why that's happening?
public Configuration()
{
var myData = new
{
api_key = "added_my_key here"
};
string jsonData = JsonConvert.SerializeObject(myData);
Auth = JsonConvert.DeserializeObject(jsonData, jsonSerializerSettings);
}
Probably best would be to remove the exceptions in config file that it looks for the auth.json, that might be causing the issue. Are there any error messages?
@@sgt3v for this particular method there were no errors messages.
I looked through the other scripts, I was not able to locate where the Auth file was actually being used.
Also,
I tried placed the auth file in StreamingAssets and provided that path, but that did not work. Got the error as “no key was found” or something like that.
Hey, I've been trying to look over your code but I'm just learning c# and sometimes it's not clear to me why some things are done in a particular way. Could you either make a video listing the resources you used to learn c# and unity or a reply with some good textbooks or lectures. Thanks for the content. I don't just want to copy your code but I want to generate it from scratch.
to understand ur problem code I'll introducing AI CHAT DEEPAI the AI CHAT DEEPAI is so powerful and much better than OPENAI since start 2015-2018 and u can show ur picture and video and anything u want
So from what I can see, you need to pay for the use of GPT in Unity, in the form of tokens which is kinda makes this non viable for any large scale projects since you would need to get players on a subscription base to pay for tokens to then have NPC's speak to you.
OpenAI API is a paid service, being a paid service does not make it non-viable. What makes most sense is delegating the cost to your users.
Hey, Sarge! Thanks for your good video on Unity attach with OpenAI, it's a good tutoral for me to learn it! By the way, I want to make a game that the player can order the NPC AI to walk, jump, dance, and the npc and also do these motions, do you think it is possible to make it?
Hi Joseph, thank you. Glad to hear it was useful. I made a followup video to this one about prompt engineering you can check it out, it is kind of about a similar topic.
@@sgt3v thanks! I have seen it, so basically if I want to catch the walk order, I need to use the same steps like the End_Conversation part in the next video right?(but I think it need to do lots of same steps HAHA
@@josephhan3954 Yes exactly, you can have a chain of commands in the prompt, since it can analyze the sentiment and produce result with that it is better than just parsing user speech and deciding.
For example what I said might be: "never jump." or "don't you think of not jumping now." if I just check that string has "jump" and/or "don't" to decide the jump command, its gonna fail in both cases.
But If I tell ChatGPT to end the response with JUMP_COMMAND if the user response indicates that the player character should jump, then it will understand it regardless word plays.
@@sgt3v I will keep this in mind, thx a lot!
hi Sarge,
does this work for webgl as well?
Just uploaded a video about mobile and WebGL support.
ua-cam.com/video/DjX8mjBCMcg/v-deo.html
eline sağlık çok iyi olmuş
This is amazing thank you. Does it also work offline? And if not is there a way?
Unfortunately won't work online until we have a very compact and local ai model that can be shipped with unity. GPT3 itself is a very very large ai model.
@@sgt3v Yeah it would be too good to be true. But now that chat gpt gets is so mainstream and everybody is using it will it affect the time that the npc takes to answer? Or when their website is down due to the extreme traffic, will this still work?
Thanks for the answer btw
What I predict is there will be platforms, even native support in game engines to train very compact and small models that is focused on the games content.
@@gianniskaragiannis4300 Yes seems like API got slower since they introduced paid pro version. It will still work but not as fast as before.
@@sgt3v Yes that seams to be the way. I think Ai Dungeon is a very good version of this. Do you believe there is any way to import it into Unity?
How to instruct chat got to act in character?
Hi Dragos, can you elaborate what exactly you mean?
@@sgt3vI added an text to speech module, with a female voice. If I relay with every query "you are a women", it creates a strange bias where for example if I ask for a joke he will only make woman based jokes. What is the correct way to ask chat gpt to respond in a certain way (like he is a woman or like he is a 1950 bank teller)?
Do you have to have a paid plan for this?
Hi @lefteron6804 I am currently working on a service product to make this much easier and possible, if you are working in a game or tech company and need help with this, you can reach out to me via email and I can show you what it can do. You can check it our here neocortex.link
@@sgt3v It's actually for my theses in university. I've made a detective game and i would like to make a mode where you can organicly ask questions to the npcs.
@@lefteron6804 I see, I will also add a free tier for such use cases for education and hobby projects but it's not there yet.
.git is not added to the package manager
To use Unity Package Manager you will need git installed:
git-scm.com
I have ERROR
so very so interisting tuttorials
so help me this
is this free?
Ready Player Me and the plugin are, OpenAI api is for for 3 months with a quota.