AI NPCs with Reallusion Character Creator | Convai Unreal Engine Tutorial
Вставка
- Опубліковано 17 лип 2023
- In this tutorial, we show how to integrate Reallusion Character Creator with Convai in Unreal Engine. Using Convai, we can make the characters not only talk to us but also perceive the environment and carry out actions.
We start by creating a Reallusion character and setting up an Unreal Engine project, and then configure it to work with Reallusion characters. We dive deeper into the tutorial by adding lip sync and animations to bring our characters to life. We demonstrate how to download and install the necessary Convai files, modify blueprints, and seamlessly synchronize character dialogues with realistic mouth movements.
But the experience doesn't stop there. We guide you through enabling dynamic movement and navigation for your characters, allowing them to follow you or interact with objects in the scene. From conversations to physical actions, your characters become virtual companions that respond intelligently to your commands.
Subscribe for upcoming demos and tutorials, and don't hesitate to join our Discord channel
Sign up at convai.com to try it out.
Write to us at support@convai.com for any queries, bug reports, collab, or business-related engagement.
GDrive link for the Realusion Animation Blueprint and Extra Convai plugins:
drive.google.com/drive/u/2/fo...
Follow us on your favorite platforms to stay in touch:
Discord: / discord
LinkedIn: / convaitech
Reddit: / convai
Twitter: / convaitech
Instagram: / conv.ai
Facebook: / convaiofficial
TikTok: / conv.ai
ArtStation: www.artstation.com/convai - Ігри
love that you guys support Reallusion characters, this is perfect for my workflow. Thank you!
Just what we were looking for! Excellent, thanks.
Great to hear! Would love to see what you do!!
Awesome workflow 🎉
Hi, all seems to work. Thank you for a very useful tutorial. the only part that was tricky was that the position of the object (e.g. box) to move to seems to work some times but not always. Perhaps this is to do with Nav Bound volume position of the object but I am not entirely sure. However I am happy that most things work smoothly.
Hey, glad to hear the tutorial helped! For the issue you mentioned, ensure you have the latest Convai plugin (2.6.1). It might resolve the inconsistency.
Hi @@convai , does the ability to move to a reference object only possible for CC4 convai avatars or can Metahuman convai avatars as well?
Hi, Great development. I have been using it for 4 month and it seems super cool but limited due to only two animations possible in the blue print. is it possible to import more than 2 animation types from CC4 ( currently idle and walking)? if possible could you help make a tutorial to enable the NPC to have animation to sit, lie down, run, jump etc. it would really help the functionality and scenarios that the NPC can be used.
Hello! I tried your tutorial with metahumans but it seems that ThirdPersonController loses all references when changing inherited class ConaiBasePlayer. This plugin is suppose to work only in first person?
Can you please specify a bit what you mean when you say changing inherited class ConvaiBasePlayer and for the second part the plugin works for both first person and third person.
Yes, at 8:05 when the class is changed, I loose all my movement in the character along with character itself@@mrigankadhar8204
Hello, I'm sorry to bother you. When using the latest UE5.3, all the plugins used are 5.3. Due to the difference between the blueprint foundation and the ones in the video, I don't understand why every time a character speaks, it becomes an A-pose
Hey iam very thankfull for the videos, i lerned so much about blueprints with these videos. Let me ask for one especific video? I want so much see the plugin convai with the plugin ALS, I tried one tutorial of you guys, but the tutorial is with one pawn player, and i cant find these pawn player in the ALS sample.
Thank you, looking into this!
@@convai iam very grateful 🙂
Hi. Can I use my RVC voice model in this pipeline? so that my character can speak with special voice
Hello, good day. I have a problem. I'm using Unreal 5.2, and I started with a tutorial from 4 months ago in which you use the MetalHuman characters. I'm also using the Covai character found in the Unreal Marketplace. So far, everything is going well; the character speaks and can have a conversation. However, my issue arises when I reach this specific video and follow the steps to make my character perform actions. Initially, I can make it follow me, but when I ask it to move towards an object in the scene, it says it will do it but remains in place and doesn't move.
The differences I can notice are that you are using Unreal 5.0 and a different type of character, requiring additional steps to make it move compared to the previous tutorial. Could it be because of these reasons that my character doesn't perform the "move to" action? (I have added the objects to the list within the AI with their names).
Thank for yout help.
Hello,
Good day to you too! It sounds like you've done some impressive work on your project so far.
The issue you're experiencing could potentially be due to the navigation mesh not being properly generated around the object. When this happens, your character might try to move, but fail to find a valid path to the object.
A simple way to check this is by pressing 'P' on your keyboard. This should display the navigation mesh, and if everything is working properly, the entire area should appear in green.
I hope this helps. Let me know how it goes, and don't hesitate to reach out to our Discord or support email for any questions.
Email: support@convai.com
Discord: discord.gg/VUdhn7qb
@@convai Hey, unfortunately, I'm facing the same issue....this is very weird as I made sure that the navigation mesh is properly generated. Any idea what else could be causing the problem? I actually followed this exact tutorial to the T on an empty project as it didn't work in my existing project as well. :(
@@amanywahba4049 Hello, I have the exact same problem! I have tried lots of ways, but could not make it happen...
Hello everyone, the latest plugin release has resolved the issue, and actions are now performing with much greater accuracy. Thank you for your patience!
LipSync is not working properly, what could be the cause?
When I press the play button, sometimes I get a message like "LipSync is not working, please refer to youtube".
I set it up the same way as in the tutorial.
Sometimes it would still give this message even if it was installed, will be fixed in the next release. However, does the lipsync work on the characters? Is there any other error messages?
@@convai On the screen you will see this statement in yellow text." No LipSync compornent detected Please watch the following tutorial on adding LipSync:ua-cam.com/video/n5zjnN1W41M/v-deo.html"
This is followed by "Device Set Successfully" in green text.
The following are the cautionary and error messages that can be seen in the output log
The following are the caution and error statements that can be seen in the output log.
・LogPackageName: Warning: DoesPackageExist called on PackageName that will always return false. Reason:Input"" was empty
・Error: Ensure condition failed: CurrentNum == InitialNum [File:D:\RocketSync\5.2.0-25360045+++UE5+Release-5.2\Working\Engine\Source\Runtime\Core\Public\Containers\Array.h] [Line: 232]
・Error: Array has changed during ranged-for iteration!
・Error: [Callstack] 0x00007ffe5ad4ca84 UnrealEditor-Convai.dll!UConvaiEnvironment::RemoveCharacter()
・Error: [Callstack] 0x00007ffe5ad4ebfd UnrealEditor-Convai.dll!UConvaiEnvironment::execAddCharacter()
Would you please do a detail video with Unity with OVR plugin?
We'll be working on this soon!
@@convai Thanks!
How can i this on macbook pro m2?
Can triger events ? Let say NPC ai ask about password and if password correct open door or trigger event …
Is that possible ?
Yes definitely, we are planning on a tutorial on how to do that and more soon!
@@convai cant wait !
Bro how can I make him talk using Oculus quest 2 like is there any option we can change the T button to any button of Oculus?
I wonder if it’d be possible to combine Convai with something like ElevenLabs AI.
We support ElevenLabs' voices!
Thanks for the tutorial...is would be great to see a packaging tutorial ...i have tried and i keep getting convai plugs error while trying to run my package on windows
Hello! Could you tell us what issues you're facing with the packaging process? You can also send an email to support@convai.com with your packaging logs attached, so we can assist you better.
Edit: For quicker help, join our Discord community. You'll find tips and get faster responses from all the excellent developers there.
Discord Link: discord.gg/ZAY83Hs7
@@convai Thanks for your quick response .....Plugin 'ConvAI' failed to load because module 'Convai' could not be found. Please ensure the plugin is properly installed, otherwise consider disabling the plugin for this project....i have installed the plugin manually(am using unreal 5.2) and now and now am getting a differenterror message ........UATHelper: Cooking (Windows): ERROR: System.ArgumentException: An item with the same key has already been added. Key: ConvAI
PackagingResults: Error: System.ArgumentException: An item with the same key has already been added. Key: ConvAI
@@ExtraNodes , thank you for elaborating on the issues.
For the second error, it appears that the Convai plugin may have been installed twice. Please uninstall one to resolve this.
Regarding the first error, we need more context to understand it. Did this issue occur with the marketplace plugin? For more in-depth assistance, please email us at support@convai.com. It will allow us to address the issue more efficiently.
@@convai thanks...i will do an email right away.
@@ExtraNodes I had the same problems with the windows builds, after several trials I solved them by deleting the ConvaiReallusionContent folder , the initial version worked but had the problem with the avatar rotation, the latest update creates problems for me with the convai plugin and the build doesn't work. It is fine for me right now because I am using the avatar without making it move.
Hey Thanks,
Can you please shared the google drive link because I cant find in the description?
Hello, the drive link is now added to the description of the video.
Thanks for the update😊
Hello please tell how to pickup that cube pickup action.. I'm working on this i need help..
Hey, check the demo map inside the plugin, it contains a use case on how to pick up items
The OVR Lip Sync Plugin was working and then it stopped working and I can't find it again. Do you have any idea how I could find or activate it again?
yeah same with me- it does not seem to work at all- I tried re doing in version 5 and version 5.1 noluck lots of blendshape errors using the Kevin character
When I hold T, my UE5.2 dosent record anything...even if I type anything, the convai based character doesn't respond
Hi, does any errors appear in the output logs? could you share the whole output logs with us via email support@convai.com so we can further investigate? Thank you!
The Following member variables in blueprint /AnimBlueprint/Game/ConvaiRealusion/Convai_Realusion_AnimBP.Convai_Realusion_AnimBP"have invalid type. Would you like to remove them? Blendshape Map Ref This is the error I get will not compile the Convai_Realusion_BP
have you found any solution for this?
the character is following facing in one direction, not turning to the direction it goes, I dont know what i did wrong
Hello, thank you very much for reporting the issue, we have made a recent update to the animation blueprint which fixes it, you can find the updated animation blueprint from the same GDrive link: drive.google.com/drive/u/2/folders/11jAzxkrjUj7FBMeM8w3ENii6KKH-gB5K
Is the tutorial also available for Unity?
We will be releasing the Unity equivalent tutorial in a few weeks. :)
hello, please tell me how to make the widget not always displayed, and when I approach the NPC, and when I leave it disappears, how to do this? Help
Hello! We're currently in the process of creating several guides, some of which are aimed at addressing UI customization. We plan to share these over the course of the next one to two weeks. Stay tuned!
@@convai understood, thank you
@@convai ?
Errors in The BluePrint lipsync and movement will not workat all-
Hello, did you utilize the lipsync plugin linked in the description? If so, please contact support@convai.com or chat with us on discord: discord.gg/VBHFd7sFb8
Me gustaría que no me apareciera ese hod gigante de chat y que solo al acercarse puedas hablar con el bot, alguien sabe como?
Hay formas de modificar esto, pero mientras que no sea enterprise support, o haya contrato, en nuestros terms of service no dejamos que se remueva. Si quieres saber mas, mensajeanos a support@convai.com
hello, please tell me how to make the widget not always displayed, and when I approach the NPC, and when I leave it disappears, how to do this? Help
We will be adding this to documentation!
@@convai where?, do not understand