Conversational AI in Unreal Engine Quick Setup Guide | MetaHumans, ReadyPlayerMe, and more | Convai
Вставка
- Опубліковано 26 бер 2023
- How to set up Convai's Unreal Engine plugin and use the demo project.
Full tutorial playlist: • Convai Unreal Engine I...
Sign up at convai.com to create your own character.
Find the UE plugin here - www.unrealengine.com/marketpl...
Write to us at support@convai.com for any queries, bug reporting, collab, or business-related engagement.
Discord: / discord
LinkedIn: / convaitech - Ігри
📌HEADS UP!: We've removed basic lip-sync from recent plugin versions. Learn how to add lip-sync to your MetaHuman or Reallusion characters by following the steps in this guide: docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides/creating-metahuman-characters/adding-lipsync-to-metahuman-from-plugin-version-3.0.0-or-later. Don't miss out!
In the creation of brueplints I do not have a convai character I need help and that I followed the step by step someone else happens I need help please
Hi@@nacho6247, if you do not see the Convai Character blueprint, then ensure that the plugin is correctly installed and enabled in the project.
@@convai thanks 👁️👄👁️
How to make follow me on unity
is possible to use sequencer to have this kinda conversation and render out by movie queue?
Always waiting for new features and updates! Keep going convai you guys and your developments are great!
Do you have an orgy fest there or something?
Thank you!
Works perfectly in 5.2 as well! I made a project in 5.1 and opened a copy of that project in 5.2.
I hope you will have more views, likes, and attention in the future. You are such a revolution maker 👍👍
Awesome! I can't wait to try this out.
Helpfull tutorial, thanks a lot! Very cool! Though I would recommend people to back-up their project up front since putting back the class settings to character is pretty difficult. In overall very happy with the result! Awesome feature.
This is the future can’t wait for the voice to action tutorial grabbing weapons and cooperate in a fps situation
This is awesome, would be great to see these NPCs trained by developers for more unique and robust interactions. I know there's definitely competition in the NPC-AI space with InWorld. It's great to see all the technology coming out, hope to the audio quality improves to be more natural (like Spotify's DJ AI).
Just noticed they acquired Sonantic, would be great if they licensed that tech out.
@@Native_Creation We are working on improving the voices while maintaining the lowest latency for the smoothest experience possible. :)
@@convai great to hear!
this is AMAZING ! i like this plugin and I want to learn as soon as possible!
Thank you!
you are smart and done amazing work!
Hi there. This is epic!
Great tutorial and so easy to follow. I'm needing a few talking AI characters for a game I'm working on.
So, all I really need are some modular characters from the Marketplace and this free Plugin? You provide the animations etc?
i would love to see convai and elevenlabs do a colab
Incredible !
how can i remove the chatbox, when the player is not nerby?
Just tried it Awesome work. For Me it seems I have to put a 200 to 300 ms delay in the voice 0utput to get in sync. How would i implement?
maybe a new video?
Hi, is it possible to also type in case player does not use a mic? Also, is the update for 5.2 coming? Keep up the great work! :)
Yes, it is possible to use text input as an alternative to a microphone. As for the UE 5.2 update, it is indeed on our roadmap.
THis is so cool! Can be used in a different language or it can only be used in English?
Fascinating. Is there a possibility to change their behavior on the fly depending on certain values? For exemple have a good standing system affected by a player's actions that would impact how obedient or polite the AI is.
Yes, you can update a character's backstory on the run.
Honestly just waiting for an ElevenLabs API plugin for our own voices and everything will be complete. Also Android for the Convai Plugin lol
bro you know!! Im just hoping they listen to this request. I've messaged both elevenlabs and convai. I swear they would change the game if they did a collab.
how do i interact with convai metahuman without changing the player's current parent class in class settings? Ive looked thru the ConvaiBasePlayer Blueprint as previously suggested but im not finding a solution. The Ai model is in game and doing the animations as the tutorial shows. My problem is im currently using horror engine on unreal engine 5.2, changing the parent class breaks everything already interactable. The T key is already mapped for another item. Ive searched google for your discord channel as well but only finding broken links.
Hi! I see you're having an issue integrating your custom player blueprint. No worries, we've got you covered! If the comments in the ConvaiBasePlayer blueprint doesn't clear things up, our tutorial series dives deeper into the plugin blueprints: ua-cam.com/video/iGn_WMjfU-A/v-deo.html. This should give you a headstart, using the ConvaiBasePlayer as a reference.
Need more help? Join our friendly Discord community: discord.gg/tCwBz8qv7.
Stay tuned! We're writing a guide on easy integration with various player blueprints. It'll be up on our docs soon.
it only works with first person template?
Is there any tutorial to TPC?
Hey guys. I had success when creating a new project, but am having issues whed adding a metahuman into my existing project. Character looks good, but I don’t see the Talk Window that usually pops up. I've added the two lines of text in the .ini file also. Don't I need to add other files to tell my project what to do when I hit the T button to talk? I followed your tutorials, but these are only for new projects aren't they? The convai plugin is good to go also. I have only pasted the Content folder from the Convai folder into my project. Don't we need the other folders also?
It didn't work for me either at first. In the video we are told to add the two lines in the beggining. But in the convai website, we are told to add it in the end. I did it and it solved the problem.
After getting this all sorted, would it be much more work to replace the mesh with my own, it has no mouth component, I'm fine with only the audio and the animations
how can i interact with the npc's without changing my player settings? changing my player blue print class breaks a lot of the interactions i already have set up, like grabbing a flashlight or working a door... is there a workaround?
No worries, you don't need to change your player blueprint! Just check out the ConvaiBasePlayer blueprint. We've added comments and hints to make it easy for everyone to understand and copy the needed logic into their own blueprint, the same goes for the AI character blueprint. Feel free to ask us or the community via our Discord channel or support email if you need any help. Happy Development! 🎮
Discord: discord.gg/tCwBz8qv7
Support Email: support@convai.com
@@convai thank you very much for the fast response. I'll be looking forward to getting this right tomorrow!!
what about giving him a quest? and have player talk to him with text like preset responses , like an rpg game, how do i tell the AI what role he will handle in the game and what he will respond to the player ?
ANy chance for a redo of this one for 5.3 , including maybe a bit more detail on the Metahuman import aspect ? love sent.
Well done team Convai! Love your work. I tested it out today and although not perfect, it was quite clever. Maybe I need to do some tweaking on my end and provide more information for the characters. They didn't always follow specific commands. Still good fun. Keep up the excellent work and look forward to what you bring to the table next. Exciting times!
ua-cam.com/video/1OgBy_mBVfs/v-deo.html
We are improving features daily! Great work!
@convai
Thank you. No worries. Is there any way I'm able to use convai to control an animal using voice commands? It's for a UE5 game I'm working on. I have the dog and mesh etc.
Any idea how to change the microphone source in Unreal Engine? My voice doesn't get picked up, I had the same problem in Firefox, but I selected a different source and it worked. Also, I'm using Unreal Engine 5.2 so maybe that is the problem? Everything else works great. Edit: Found it, press F10. It works now :)
Glad you found it! Excited to see what you make!
In the creation of brueplints I do not have a convai character I need help and that I followed the step by step someone else happens I need help please
I nees help it says something went wrong and check the output logs
Fantastic, thank you. I cant seem to get the mouth to move on my Metahuman. Tried a custom character and a default and neither seem to work
This happens on the last version, you have to add a Convai Face Sync component in the metahuman blueprint
Is the AI limited to the characters backstory or can it find information it needs and fill in any gaps? for example if you set it as a historical character, could it scrape the internet to find information it needs?
You can add context to its knowledge using the knowledge bank. There you can upload anything that would be relevant to your character's conversation
hi, Is it possible to make package project for windows? it gives me "failed" and does not specify the error :/ thanks
Sorry to hear you're having issues with packaging for Windows. Could you email the output logs to support@convai.com so we can help you out? Thanks!
sent! thanks! @@convai
I don't understand I follow the protocol to the letter it worked before but since the unreal 5.1.1 update it no longer works
How do i get them to follow me. This doesn't seem to work as explained above anymore. I see there is a reference action, but I am not sure how I would get him to follow a spawned character
Can we integrate our custom STT and TTS api
Can this be used with your own characters as I have a vampire girl character I purchased off epic market place
Awesome easy to follow tutorial. Although mine fails and gives off a error. lol
For the metahuman don't go to class settings and change it to base character just add the chatbot from components. I had the same issue.
Hey I am facing the same issue. Can you please explain how can I make the chatbot work with the metahuman @@Clydiie
@smrititrivedi2494 I have a tutorial on my channel that explains everything its called quick and easy way to make ai npcs in ue5 please watch that
Is there anyway to use the NPC in a 3dps game as an enemy?
@6:34 _"If you have your own player class, you can inherit the components from this class into your own blueprint, we cover this more in our tutorials that take a deep dive into this functionality."_
Have you solved it by any chance?
Hi there ! Nice, how can we use that with a non metahuman skeleton ? thanks !
Hello! We support Reallusion and RPM as well, but if you have something very custom you might need to create your own animation blueprints, take a look here for a very simple implementation: docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides-v2-under-development/getting-started/simple-talking-cube
when i change the parent class on third person character i lost all inputs. The character and camera dont move. UE5.1
To use this ai... Need to change character to convai base player... But if i want to use character how so?
Well, this is frustrating, never fails I follow step by step instructions, and then I come to a part where I don't have an option or setting shown in the tutorial, either I missed something that was glossed over or something's changed in the process and I don't have the updated process. I get to the point where you are creating the blueprint, I noticed that when you say "inherit" you're blueprint becomes a pawn, but when I click on the same option, mines becomes a face, and instead of a box I see a cylinder. When i go over to the character to look for the char ID I don't see it at all. Was there a specific way I'm supposed to select the convai base player, because whenever I click on it, it stays as the face, so I don't know if that's what the issue is. But I'm on 5.01 (my 5.1 isn't working which is another issue altogether) but I should in theory be able to do this in 5.0 correct? I'm not seeing the Char ID field anywhere though....
Nevermind, folks if you're having the same situation or issue that I was having, just know to start off it's easier to go to the first person template, that's why I couldn't see the char ID, for those more well versed in UE correct me if I'm wrong, but there is a way to add it to third person. It was easier to just go to fpv first as I was then able to enter the char ID, I was having a dumb moment hah
what? bro, i am having the same problem you were, but i can't understand your solution at all xD what do you mean, "go to the first person template"? where? idk what that is or why it wouldn't be referenced in the video.. @@legacylee He just created a Template in the content browser, that's it. I am in a third person map. update: ok! i figured it out xD. Accidentally skipped the step where you have to restart unreal engine after editing the INI file.
@@xiaobai7060 damn I've been working on so much stuff I forgot about this we're going to be creating a UA-cam stream where we're going to drop stuff on all this pretty much anything Ai and CG related I'll be looking to have guests on it and what not will take a look at a lot of these things in depth and break it down
I'm already using parent from another plugin for PlayerCharacter, how to i add canvai features to a player without setting a parent?
Hello, you can read below guide that explains from scratch step by step how to add conversation capabilities without ever changing the parent class.
docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides-v2-under-development/getting-started/simple-talking-cube
Teach me please.
Metahumans I created
There is no default for details, and Convai's CharID cannot be entered.
What should I do?
Don't change the class settings for metahuman just add the chatbot from component. I have a tutorial if you are interested.
Hie how can i hide the talk menu when im not with a pnj
in blueprint creation i dont have a convai character to choose did i do something wrong?
Me sucede lo mismo
How can i hide the talk box
I couldn't login to the Quixel Bridge on UE5. Seems like many people are having the same issues☹
Hi, there might be an issue with the Epic account, you can try to contact Epic or just a wild guess you can try a different account.
For me the lip sync is working but not the gestures ( body movements )
How to set"hould T to talk"
Azure TTS voices??
hi,when i create the blueprint class i only get convai environment and convai actioncontext
Hi, the latest plugin update is still taking place as of this time. Give it a day or two and it shall appear on the marketplace.
@@convai same problem here, we have to wait for a new update on the marketplace ?
We are still looking into it with Epic, until it is fixed, you can do a manual engine install as explained in the following docs:
docs.convai.com/api-docs-restructure/plugins-and-integrations/unreal-engine/download-and-setup#engine-install
But make sure to uninstall the version from the marketplace to avoid conflicts.
The update is live now and should appear on the Epic games launcher.
Wow
i cant find the character id on unreal i have the id but i cant find where to put it??
hi same here
Hi@@user-pu4ns3iy6n, you can check the following guide (step 4) on where to find the input field for the character ID
docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/guides/creating-metahuman-characters-1/adding-lipsync-to-metahuman
It works but for some reason my metahuman keeps staring at the floor when I come near
Hello guys! When i try to open the newest convai metahuman demo project (where i think is the version can use your newest updated features) but when i try to open it i got an error about the convai OVRlipsync that is not supported by UE versions newer than 5.0 and im unable to get the 5.0 i just get the 5.0.3 and i cant open it because is made for a newer version but when i try in this or in the 5.1.1 i got the errors mentioned.
UnrealEditor.exe - Entry Point Not Found.
I came here trying to learn how to upload my own voices to the character. This is the closest title I've found to what I'm looking for, but it still doesn't answer my question.
Please email support@convai.com to get access to the closed beta to clone your own voice. Thanks.
it is not working wit 5.2.1
Sorry for the inconvenience. Could you share the logs with us via Discord or support email support@convai.com?
the character does not follow me how to fix it?
Hello, make sure you have the navigation bounds set correctly, you can also try out our Actions sample project: drive.google.com/drive/u/1/folders/1UWgQMHTTSq4O45PtKCyr8sjZbbkq-2_o
command follow me not works in metahuman character
Hi @dimaerox5133, there was an update on the backed regarding actions, the Follows action will start working again in a short while. For now you can modify the action name from Follows to Follows this temporary fix will get it working for you.
Now works thanks ConvAI for great product!
can we just typing word in chat dialogue, not a voice
Yes, that is indeed possible
"Hi, is it possible to package a project in Unreal Engine? And how, please? Thank you very much!"
Hello! Of course it is possible, are you facing any issues packaging? we have a troubleshoot guide here: docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/troubleshoot-guide/missing-unreal-engine-tool-set-in-microsoft-visual-studio-toolchain
Also make sure to follow our installation guide and install visual studio and its components as specified here: docs.convai.com/api-docs/plugins-and-integrations/unreal-engine/installation
Let us know if you face any issues :)
tried for hours follwed every word, nothing.
Hi @Slenderirl, We're sorry to hear you're encountering issues after following the tutorial. It's important to us that you get the help you need. Could you provide some more details about the problem you're facing?
Also, our Discord community is a great resource for real-time support. It's a dynamic group of over 1000 developers, all well-versed in Convai, and they're usually quick to offer assistance and advice. You can join us here: discord.gg/76ptuBcRGC.
We're here to help, so please let us know more about your issue, and we'll do our best to resolve it promptly.
Until languages other than English are not supported - this will not be used on the market, since games are published with at least 3 languages localizations, and 8 on average.
nice. but u bond ur game to service of other company... and that never end well.
so complicated also you do everthing so fast that i cant follow i dont know how to use UE but i wanted to try setting up my caracter but you were going so fast and skiping a few basic steps that i didnt know about it. I wish there was a file that did all of this for me like plug and play just needed to insert the API keys.
Hello, we appreciate your feedback and understand that finding the right balance between catering to beginners and experienced developers can be challenging. We strive to make our tutorials as user-friendly as possible. In response to your request for a plug-and-play project, we have one available here:
drive.google.com/drive/u/4/folders/1HNcghI9SG1NpCUaJWRX9Yh28HUF00-U0
Simply download the project, and you'll only need to insert your API keys to get started.
@@convai thank you for all your support i hhope that you guys go far! ❤
@@LogitechXibanga Thank you for your kind words! If you ever encounter any issues or need assistance, feel free to reach out to us at support@convai.com.