Unity Tutorial: Voice Interaction for Android and iOS
Вставка
- Опубліковано 12 вер 2024
- Today Matthew Hallberg shows us speech to text and text to speech in Unity for Android and iOS so you can add voice interaction to your apps or games.
PingAK9's plugin: github.com/Pin...
Stay in Touch with Third Aurora
--------------------------------------------------------
Website: thirdaurora.com/
Facebook: / third-aurora
Instagram: / thirdaurora
Thank You Man ! in 2023 I was searching on Unity Store and Google for a plugin for the Android project. Working 100%. Thanks again.
The permission window for microphone is not popping in the app so the speech to text is also not working any solutions?
@@nickwinbritto7674hello have you fix the problem? I can't see it too
Great Tutorial, I love the explanation !
I'm here because Matthew stopped uploading tutorials on his channel😥
Since a couple of months ago this plugin causes a crash on recognized speech when the pop-up is not enabled. Does not seem like a fix is on the way.
Hello, I have a problem when I run the sample application on a samsung s22+, when I press, it starts to recognize the text perfectly, but, when I release, the application crashes. Any idea what can happen? thanks and great job!
Same issue, let me know if you figure it out
a temporary workaround is to re enable the "is show popup android" bool on the speech to text script, then just use the popup modal. This resolves the error. Not perfect though, still want to figure out what's causing it. I noticed in the code that when the bool is false, it calls a java class from com.starseed.speechtotext.Bridge, so maybe it's a java issue
@@generichuman_ Hello!! I had the same issue and thanks to your comment, it worked. Have you found out how to make it work without enabling the popup? Thanks
how to make it work without enabling the popup?
Hello. This tutorial helped me a lot. but STT isn't running. I think StartRecording works, but the application is forcibly closed for some reason when the StopRecording. I want to analyze this, but i can't see the plugin. Do you have any solution? My phone OS is Android 11.
@A Editor Any solution?
@A Editor Same issue i am facing
Same issue i am facing
Any solution😢😢😢
Great tutorial. Very helpful. Is there a way to start listening to speech without pressing a button?
That’s what I want to know, I switched from Windows to iOS and everything’s wonky.
did you get this figured out? I've been trying to but I'm pretty new to coding in c#.
@@SilverEchoDreamer ya i did the same it works
Hi @@vedantkadam2147, Could you share how did you do it please ?
Hello I couldn't do without STT android popup. Popup does not work when false. Please help me
Great tutorial but when i come to run STT the app crashes. I am developing on Unity 2019 and running on a Pixel 6A. I have triple checked the code and it is the same as yours... so any idea how i debug? Nothing appears in console before it crashes... TTS works great
thanks for any help you can offer
Have you been able to find a solution? I am coming across the same error
@masterChange8155
Have you been able to find a solution? I am coming across the same error
@@hassanbennajma5155 no - i ended up buying a solution as wasn't getting anywhere on this - one small issue flummoxed the whole thing
'PBXProject.GetUnityTargetName()' is obsolete: ''This function is deprecated'. There are two targets now, call GetUnityMainTargetGuid() for app.
When i change the line of code to string targetName = PBXProject.GetUnityMainTargetGuid();
it says: error CS0120: an object reference is required for the non-static field, method or property 'PBXProject.GetUnityMainTargetGuid()'
Trying to get it to work on Android + Vuforia
The permission window for microphone is not popping in the app so the speech to text is also not working any solutions?
When used for Oculus Quest 2, Text to Speech functionality line of code --- TextToSpeech.instance.StartSpeak(message);
On executing getting error as -
"W/TextToSpeech: speak failed: not bound to TTS engine"
Any idea how to resolve this ??
Not working at all at the moment. Assume the repo needs to be updated which will require a Java rewrite and recompile.
I really thank you for the tutorial!
It is GREAT!
hello someone can help me? in my 2020 unity version, permission microphone not appear, can somebody help me fix this? thank you
Wait...... Android and IOS has STT and TTS for free built-in?!!!!
Thanks for the tutorial!
Hi everyone! First of all, thanks for the tutorial! TTS is working fine, but STT isn´t. What could it be? My OS is Android 11 and the LANG_CODE i´m using is 'es-ES', i don´t know if that affects
same
for me, it opens up google's tts and then shuts it
same
Same : (
same 🥲
Hello friend, I hope you're very well.
Excellent your video.
Is possible use ML-agents in this project?
If you answer is "yes", Could give me some resource, video... As help, PLEASE..
I need information for i begin
hi Matthew, is this possible to speech in other language?
That's amazing bro! Thank you very much ^^
thanks for the tutorial sir. is there any way to save the audio after converting the speech to text? thanks a lot
Is it possible to make it work in the Unity Editor?
Hello good afternoon, I have the following problem: when the start listening is executed in the mobile build, it crashes. I'm not sure why this could be happening. Any ideas?
thanks man for the explanation! really good!
Encountered an unexpected glitch..ish?
When the volume is up the recognition takes almost ten seconds to recognize, without volume it's instantly, I'm trying to check why is that happening
Plugin not work anymore - github project just make crash app.
Great video! Is there a better way to handle STT and TTS on iOS with Unity in 2023?
Got this working now, its great :)
I would like to know how to mute the beep notification sound (uses ringer volume on Android) when it starts & stops recording. Thanks
i want to know the same...Did you find a solution for this?
@@MagicalSAMElectronics no unfortunately, I suspect it's something that would need to be done within the plug-in
@@ashlington79cool...thanks
I have the same doubt... Can anyone help, please?
How can I compare the spoken words, e.g. like using Windows.Speech; with: "if (args.text == m_Keywords [0])", where what is said is compared with the entered sentence from the array "[0]" and the if query is then executed if it matches?
I get this error saying "probably the android SDK is read only", which doesn't seem to be the case. Something is wrong with the plugins or assets. There're no compilation errors. It's just build errors. Anybody know to fix them?
Hi, thanks for the tutorial. But it doesn't seem to be worlong for me. The app builds and the google microphone popup works, but it does not show the recorded audio on the text component attached. This leads me to believe that it's not working properly
Hi, I did the exact thing as in the video but my android app is not asking for microphone permissions. Can anyone help?
Same issue ?? Did it fix
Great VIdeo! Really appreciated!
How much extra size does this give your app?
This work fine but i need audioclip so that i can perform lipsync. Thanks
Hi Matthew, nice tutorial on that native textspeech plugin. Pls help, trying to find how to make the tts voice be feed to SALSA lipsync model, as the voice is playing through the native plugin. By the way, I have tried it with EASY AR SDK and text to speech working cool. Just making it work with SALSA lip sync, can you share some guidance on it.
Just saw this recently, have you solved this problem yet? grateful!
how to fix the crash when it detects what you said? please help me coz this for my capstone project :(
HELP PLEASEEEEEEEEEE................................
How can I compare the spoken words, e.g. like using Windows.Speech; with: "if (args.text == m_Keywords [0])", where what is said is compared with the entered sentence from the array "[0]" and the if query is then executed if it matches?
THANKS IN ADVANCE
HI bro.Did you solved problem?Can you give advice
Amazing tutorial. The TTS voice on my Android device sounds really weird, nothing like yours. Is there any other language code I can use besides "en-US"?
I found the reason. It's the TTS voice lib on my phone.
Hello admin, now I start learning develops android apps, I done every steps that you show on this video, everything look fine until I clicked record voice on my phone, the program has crashed, it seems there is only voice recording that leading to this. After it showed what I speak then it crashed I don’t know how to fix this 😢😢😢
9:40 Thank you for the tutorial!
Why do I click two out, one down and one up, and then when importing to the phone, press the blue square and it will flash back
Awesome, am I able to add voice command to a button, example: start game, then the game starts?
First of all big thanks for the tuto. I have a question. For the TTS French language is available?
The permission window for microphone is not popping in the app so the speech to text is also not working any solutions?
Hi , thanks for this tutorials.
I got a XCode build error :
Undefined symbols for architecture arm64:
"_OBJC_CLASS_$_SFSpeechAudioBufferRecognitionRequest", referenced from:
objc-class-ref in SpeechRecorderViewController.o
"_OBJC_CLASS_$_SFSpeechRecognizer", referenced from:
objc-class-ref in SpeechRecorderViewController.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
any idea ?
Thanks
Mat.
same
i got the exact same error, did u handle this error ?
please tell me how
Add speach recognition platform in your xcode your problem will be solve
Trying to include this in an existing project but can't get it to work. Got the Plugins (deleted androidmanifest) and SpeechAndText folder as mentioned but when I test it on my app it won't play anything. Just in case, I also build an apk from the UnityProject included and that works.
thank you!. Does it use Google speech services under the hood? and is it free for thousand of users? and can it be offline?
thank you for tutorial, how to build with high API(ex:24), I build with arfoundation, but your plugin have API low
Amazing man. How to reduce the speaking speed in start speaking cube?
Hi. This works on all android devices, but ios 14 also comes with a black screen and the application does not open.
There is no problem with ios 13.
Great tutorial. Any resources or leads to use the generated audio from text to speech for lip syncing?
Microsoft Azure can provide text to speech voice generation along with visemes data for lipsyncing
Was just wondering if it works on pc as well? Thinking of making a robot baddy that can dynamically say your name.
Hey, thanks. When I copy the plugin and SpeechAndText folders to the asset folder i get this errors: 1- 1 - The type or namespace name 'iOS' does not exist in the namespace 'UnityEditor'
2 - The type or namespace name 'PBXProject' could not be found (are you missing a using directive or an assembly reference?
make sure your version has ios support available
Greate tutorial. but not working on recent android version device with STT. I Couldn't find the reason :(
Is it possible to get this to work for a pc application I only need the text to speech function but I need it to read the text of a specific part and then speak it while it updates. Any help on this would be greatly appreciated.
Hi thanks for your tutorial. May I know is there any way to use a UI Button or Image as button instead of 3D cube? Thanks in advance!
hello, r u know how to make it now? im stuck in this problem, can u help me how to use button or image instead 3d cube? thans
Little tip, the thing you are trying to do has nothing to do with speech to text, it's plain vanilla Unity functionality, so instead of asking the creator of a very specific Unity video how to do something basic and hope that he has time to get back to you, just google or UA-cam "How to create a UI button in Unity". You'll get a million results and you won't have to wait. Learning how to find the information you need quickly is your biggest asset as a developer. 99% of my job as a developer is googling shit.
Hey, man. Do you think it's possible to achieve continuous speech recognition with this plugin?
At the moment I coded my application to call StartRecording() as soon as it presents an error callback. But this is not very ideal.
Thanks for the tutorial!
(keep in mind, not tried this on mobile, so milage may vary)
You could probably run it through the microphone command parsing loop you are using, have a command_level variable that can switch between command_level=0 (for detecting listen command), command_level=1 (for basic commands) and what not. have the "Listening" method run on a loop from the start, but have it start with command_level=0. Use whatever string query system you are making use of within a switch(command_level) block. It should not be too difficult to port that over to mobile (again, not tried mobile voice commands yet myself).
Hey man, thanks for the video. I can't seem to find where the sample is that includes the text to speech and speech to text files. Could you help me find those files?
how i can remove the sound of start listening is started and end sound? please
Is there anyways to have this working for phrases? I can get the audio to be picked up but not sure how to have phrases impact gameplay
I'm working on something similar, you could have it simply look for certain keywords in the string that stores the voice input, then fire a corresponding callback based on the word/words detected. It won't be very modular, or very extensible right off the bat, but it's a starting point
@@amuzak9063 Did you just use an if statement to have the code look directly for that string? Honestly, I'm not super strong in coding but I did manage to get it to work pretty seamlessly using Unity's Native Voice recognition stuff and on PC but still struggling to get it to work on mobile.
how to add/change to local language? and how to check available language? thanks
It doesn't run with vuforia platform .
Interesting that you commented this cause I just had the same issue today! Will update you with a fix as soon as I figure it out. Are you getting Vuforia won’t init error?
Yeah that was a main bug couldn't understand why it didn't work so did couple research like changed the License of Vuforia 2-3 times and then installed some XR plugin and rotation to potrait mode from player settings and also added a toggle button at end so while it detects the ground it doesn't create a problem on touch for the cube . It worked for me now and now I think I'm ready to complete my project thanks for the TTS and STT plugin it works perfectly now...
Can you help with combining image tracking of Vuforia with AR foundation I think there are limitations using both together ..
@@MatthewHallberg I am having the same issue. Been trying to combine Vuforia image tracking and this TTS plugin.
@Hardik Shah having the same issue, any luck?
Hello Matthew Hallberg! Sorry for my English, I live in Brazil and I'm starting in Unity and I want to develop for IOS, I have an Ipad Pro 2020 and I know that I need a MacOS to use XCode after Unity, my doubt is: I can do the entire development process of do I play on my Windows PC and later on MacOS to use XCode or do I need to do the entire process on MacOS (install ynity on Mac too)? Another question is: I have a very low budget to buy a Mac now, if only to run XCode what is the minimum or cheapest I need to have it? (Macbook Pro, Air, Macmini)? I don't even know which old model runs Xcode. Thank you!
Maybe you can try downloading MacOS to a Virtual Machine! And even rent a Virtual Machine (For example, Paperspace, that is a high quality perfomance virtual machine and you can pay for use hour). And also, sorry for my English, I'm a native spanish speaker haha
Can someone help me?
How do I display a message (eg, "I didn't hear what you said") when the user doesn't say anything and so the system doesn't hear a word?
Hi admin, how to do if I want to change scene using voice command?
Use "using UnityEngine.SceneManagement;"
then on voice command do
"SceneManagement.LoadScene(*SceneName*);*
That should work
@@themeatynoob3886 thankz admin 😊👍🏻
Got an error SpeechToText does not contain a defintion for instance. Pls help
make it Instance, not instance with the letter case sensitive
This works perfectly on iOS when launched using Xcode debugger. But will show a blank screen and crash after a few seconds if run directly on the device. Tried Unity 2019 and 2020 and two iOS devices. Any ideas?
On android its crash too. I think its make crash on many systems.
My Android too, any idea how to fix
I my phone it not ask for microphone permission
I think this plugin is really buggy. Anyone has an update to make it work? I would appreciate it
It ONLY workeble for android and no WINDOWS support ?
how can i pass speech context phrases for better transcription for speech to text api from C# script ?
Is it still valid in 2023 ?
For Android, when I click the microphone button in the Scene I get ERROR: Insufficient Permissions. Then when I let go, I get ERROR: Client Side Error. I have already allowed the Microphone Permissions. Can anyone help? SOLVED: I had to re-enable Voice in my Google App permissions
does any one has the problem with textmeshpro not updating?
Something must have changed. Nothing works. The APP comes up. Then nothing works.
i try install speech.apk from plugin but SpeechToText doesn't work
Bro how did you add ui text to the voice controller.
I subscribed a method to the OnDoneCallback but it's never called, does anyone know how to solve this?
is it possible to use this code in PC? (MacOs specifically)
thanks
Thank you very much!!!
can this be used in vr too?
Hi, how record voice in android smartphone ( unity3d )
Can I run it on Nreal Glasses Computing Unit as .apk file?
Text to speech is working on my android, but when i press the button for speech to text the application crashes..any fix for that?
Hey which phone are you using please respond? For me it doesn't work at all It shows the pop up for the voice mic but does not update the text box
@@Need2pee Google pixel 3.
Hi, is it possible to implement the speech for android in MAXST AR?
Perfect practice
" 'TextToSpeech' does not contain a definition for 'onDoneCallBack' and no accessible extension method 'onDoneCallBack' accepting a first argument of type 'TextToSpeech' could be found" HELP PLS!!!
Hey its not 'onDoneCallBack' with "B" to Back but back so its should be 'onDoneCallback'
Great tutorial! Thanks! Anyone know how to get rid of the beep notifications sounds?
Hi, did you ever work this out? its very annoying :D
For now, the only way I know is to set my phone on silent.
Likely not possible. It's a security/privacy feature of the phone (same as the shutter snapping sound whenever you take a picture with your phone)
will this work with oculus?
How in the world do you get this to work for iOS??????
Thank you so much
Hey this is a freat tutorial, but when I tried to do speech to text, it starts pop up a google interface, and not changing my text :(
nvm, he does explained it, I wasnt really listening XD
Can we use in oculus quest 2?
You are my HERO
Hey I did more or less everything like you but when I cloud build on ios I keep getting this :
41: ▸ ❌; Undefined symbols for architecture arm64
42: ▸ ❌; ld: symbol(s) not found for architecture arm64
43: ▸ ❌; clang: error: linker command failed with exit code 1 (use -v to see invocation)
Am I the only one ?
it's worked fine under IOS 15...
Not working in IOS 15
Anyone solved this..?
what about webgl
Can you show how to do this with JUST voice commands like the user says “jump” and the player jumps in game, like a voice button.
Fantastic tutorial.. Thanks.. I am also struggling with the same issue (voice commands) .. !!
if you still need an answer what I did is add this to my OnFinalSpeechResult method
if (result.Contains("command"))
{
//do something cool
}
@@klortak did you ever run into this error? "Undefined Symbol: _OBJC_CLASS_$_SFSpeechRecognizer" on build.
Don't forget to add the Speech frame work to your unity framework in xcode fellas(for ios)
Thank you so much,@@klortak!
Can you help me once more?
When the user says, for example, "left", one thing happens, when he says "right", something else happens, and when he says any word other than "left" or "right", a third thing happens.
But I want a fourth thing to happen when the user doesn't say anything. For example, the message "I didn't hear what you said".
Do you know how I can do this?
loved it
how to make using mp3 file?
Apparently nobody is having any problem which makes more nervous, why whenever i access the scene, the app instantly crashes ? :( i have no clue where i went wrong
You are on iOS right? The app runs fine on iOS when started with xcode but as soon as I run the app by tapping the icon on the device the screen goes dark and about 10 seconds later it just crashes. The Unity logo doesn't even appear. Weird. Let me know if you figure it out.
@@THEJABTHEJAB that's weird Xcode doesn't even load it for me it tells meSFSpeech Recognizer Error, iOS is such a pain, I wish Steve Jobs was still alive so he could see how they ruined his company.