Creating Talking AI Characters in VR and Android | Convai Unreal Engine Tutorial
Вставка
- Опубліковано 5 сер 2023
- Learn how to add AI NPCs in Unreal Engine for Android and VR applications using the Convai plugin. This tutorial walks you through the installation of the plugin, creating interactive characters and integrating them, adding actions and scene perception, and finally creating the application for Android VR.
GDrive link for the ConvaiReadyPlayerMe and ConvaiOVRLipSync plugins:
drive.google.com/drive/u/2/fo...
Subscribe for upcoming demos and tutorials, and join our Discord channel. / discord
Sign up at convai.com to try it out.
Write to us at support@convai.com for any queries, bug reports, collab, or business-related engagement.
Follow us on your favorite platforms to stay in touch:
Discord: / discord
LinkedIn: / convaitech
Reddit: / convai
Twitter: / convaitech
Instagram: / conv.ai
Facebook: / convaiofficial
TikTok: / conv.ai
ArtStation: www.artstation.com/convai - Ігри
I'll second the request for a Unity version for Quest 2. These recently posted videos are very much appreciated! ❤🎉
I'll third the request for a Unity version! Most of the VR developers are using Unity not Unreal Engine. Would be super helpful!
@@malorum Hello, great to hear about the requests, I would like to understand more what would be good for all of you. Please set up some time in my calendar to discuss: calendly.com/nyla-convai or email us at support@convai.com
everyone has been pretty clear, we just want to see these same videos but using Unity@@convai
Thanks! Looking forward to see whats next!
Thank you! We'll make sure to make a lot more exciting content!
well this looks freakin dope, really awesome plugin!
bro the tutorial is good, i am new in the comunity but is too easy understand❤
Would you please do a tutorial of the same topic for Unity
Wow!!! Great video
Thank you!
Do you have something like this but for Oculus in Unity? These videos are great! thank you.
Hello, for the time being we do not, but the base plugin is compatible with Oculus and you only need to change the character controller to make it work
Whenever I try to record my audio using Unity and the Oculus, the whole headset freezes and 95% of the time it doesn't record anything@@convai a verified project would be an excellent, simple thing to add to make everyone's life easier and keep people using this plugin. as it stands right now, I have emailed support ( 3 days ago with this issue and no one has gotten back to me. I am now considering other options)
Amazing! Does tutorial relate to an Oculus Quest HMD? How would this technology package onto such a headset and still work remotely? Or do you need to have the Oculus Link attached?
Absolutely, there's no need for an Oculus Link. You can simply package the project and install it directly onto your Oculus Quest HMD with no further steps required!
Tried to package the project to exe but it did NOT launch
I got this massage: Plugin 'ConvAI' failed to load because module 'Convai' could not be found. Please ensure the plugin is properly installed, otherwise consider disabling the plugin for this project.
ANY FIX
Hello, sorry to hear about your issue. Please email us at support@convai.com for help
hi, I'm trying to do the same thing on unity, but it doesn't seem to work, how can I connect oculus quest 2 to convai on unity?
Could this be used with a meta xr mixed reality template?
HI, can you do it for AR also please?
Can you please have AR NPC in unreal ?
Pleaseee a tutorial for Unity with the quest 2
Hello, we would love to hear more about your use case. Mind setting some time to chat on my calendar: calendly.com/nyla-convai
my character is not moving tried everything that you said in the video
Mostly helpful! Only one problem im getting. My AI wont turn towards me. So if I say to follow me, he does, but when i run behind him, he runs backwards. seems his rotation is locked but nothing in constraints is checked
Hello, this sounds like the animations may be backwards or something is off with the physics.
Hi, we have made a very recent update to the Convai ReadyPlayerMe plugin which fixes this issue, you can find the updated plugin in the following GDrive link: drive.google.com/drive/u/2/folders/11jAzxkrjUj7FBMeM8w3ENii6KKH-gB5K
Hi, thanks for this video. Should we follow the same instructions with plugin v3.0.1? In my case (UE 5.2.1 on Windows 11), the chat widget doesn´t show in the viewport preview and I cannot trigger voice recording with the T key or text with Enter, pressing F10 also doesn´t work. Tried with controller input action wearing a Focuse 3, but also didn´t work. The character updates fine with the one from playground.
Hi @llupiy, yes the same instructions work for all plugin versions and UE versions, what you're explaining indicates that the player blueprint is not inheriting from the ConvaiBasePlayer, please double check that you followed the steps correctly. Additionally, we always encourage developers to understand the plugin on a lower level, so I would recommend going through this guide here which explains how to set the components without having to change the parent class:
docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides-v2-under-development/getting-started/simple-talking-cube
please tell me how to add the chatbox in the VR mode, I want the user can view the text.
Hello, please email us at support@convai.com
How can one implement this for Meta Humans instead of ReadyPlayerMe Avatars? Will a new tutorial come out?
We have this metahuman tutorial: ua-cam.com/video/HHJvY9dmwwg/v-deo.html
And Unity?
Hi, thank you for this tutorial! But there is a problem, once I change the Parent Class to Convai Base Player, the Snap Turn function will be disabled, then I can't turn my view with my joystick in VR. Could you please check this issue? Thank you very much.
Oh! I found a solution. Just need to go to VRpawn details and change Use Controller Rotation Yaw to Use Controller Rotation Pitch.
After making the suggested adjustments to the VRPawn blueprint, the VR navigation is still broken. The left stick used to turn does not respond as it should.
Try to undo the changes one by one until they start working again, and then you will be able to debug which step caused the issue
@@convai Thanks for the response. The issue starts right after changing the VRPawn's Parent Class. None of the changes after change or fix not being able to turn with the left stick. I'm using Unreal 5.2 and a Oculus Quest Pro
If you close rotation yaw in vrpawn blueprint class default, it is work@@trolatrolatrola
@@trolatrolatrola hi there, I am having the same issue it stop the turn control, i have try in two different projects one is a default template and still happening, not a clue of what make it, did you realize what was happend? thanks in advanced
answering to my self, in case someone need it. the problem is that when we change to convai Base player, our "use controller rotation yaw" become true and it need to be False, just in the blueprint character click in vrpawn and in the Details section look for "use controller rotation yaw" and make it false
My UE 5.2 dosent start conversation with AI character after speaking t. Even after typing in textbox and hitting enter it doesn't reply (4:48)
Could you please check the output logs for any errors? It might be something straightforward like a missing API key or character ID.
If you were not able to tell what the error is from the logs, then please share with us the output logs after interacting with the character over our support email support@convai.com.
Anyway of doing an open mic?
Hello, do you mean having it so that it can always pick up a voice? Not having to activate it with the keyboard?
@@convai exactly, I was looking around but no avail.
@@EarthlyF0rg0ten There is a way, but it is still in beta with not much guarantee that it will work well. At 2:22, instead of changing the player class to ConvaiBasePlayer change it to ConvaiPlayerWithVoiceActivation.
@convai thank you, I see it's kinda buggy if I found out anything I'll report back.
This didn{t work for me sadly as soon as I installed the plugins the memory no longer register my voice properly and they just repeat themself endlessly any tips or help with this ?
It seems as a microphone issue, you can test your microphone by pressing F10 while in play mode. Let us know how it goes!
@@convai I mean it was working fine with the mic, installed the pluggins and now I cant not even see what I say in the register of the memory which is a bit confusing. is just a blank space of dialog I said and then her answer, bit weird gonna try in a clean project.
@@convai did what you said but is not working idk what happened tbh, it was working and now all it says is "how can I help you" doesn{t matter what I say
Basically, when I Speak/Talk it doesnt transcribe into text as it should she just blindly answer the same over and over.
@@convai
@@MrrPancake To troubleshoot your audio recording issues, first press F10 to access the settings menu, then click on "Test Record" and speak into the microphone. Once you've finished speaking, press "Stop" and listen to the playback to see if your audio has been successfully recorded. If you can't hear your audio, go back to the F10 settings menu and select a different microphone before repeating the test. Should the issue persist, it may indicate a problem with your microphone hardware; double-check to ensure it is functioning correctly.
For further assistance, you can send your output logs to our support team at support@convai.com.
How much does it cost
Hello, feel free to check the pricing page from here: convai.com/pricing
Do you have a limit of 100 questions per day on your website? Do I also have the same restrictions in my game?@@convai
It works but my metahuman keeps looking at the floor, occasionally she looks up
Promo`SM
UE5 in visual studio 2022 when i build my game, it's showing
Severity Code Description Project File Line Suppression State Details
Error C1083 Cannot open include file: 'ConvaiDefinitions.generated.h': No such file or directory Ai_NPC D:\UE5Main\UnrealEngine-5.1_Source\Engine\Plugins\Marketplace\Convai\Source\Convai\Public\ConvaiDefinitions.h 7
Hello, please follow our guide here over github for building the plugin from source
github.com/Conv-AI/Convai-UnrealEngine-SDK?tab=readme-ov-file#installation
@@convai Thank you
You're welcome! @@hasstv9393