How To Implement Real-Time Voice Translation In Unity With HoloLens 2

In today's mixed reality video I show you a real-time voice translation prototype I created for the HoloLens 2, this prototype uses Microsoft Cognitive Services Speech Translation to get the voice from the built-in microphone, send it to azure, and display the translated results in a MRTK window.
You can download the SpeechSDK for Unity here:
github.com/Azu...
Also the example shown in this video is available in Patreon today with "Early Access To Source Code" tier / dilmerv
Device sponsored by Enklu www.enklu.com
Recommended Unity Asset For Today is "VR Interaction Framework"
assetstore.uni...
** Support me by Subscribing to the channel as it will help me in bringing a lot more content **
bit.ly/2NmnFMu
Get realtime indie development tips from me in Twitter
/ dilmerv
Subscribe to my indie development videos & game releases newsletter
www.dilmergame...
Blog about Indie development and startup
www.dilmergame...

[@dilmerv]

In the next video I plan to add more options and update the menu to use a hand tracking menu for quick language selection, also if you want to see more prototypes with mixed reality let me know in the comments below !
Thanks everyone

[@runningyang3051]

Great！！ I always think that voice-based interaction is the best interaction

[@dilmerv]

Thanks and I agree ! voice makes it so easy to work with :)

[@pushthebutton4602]

Great, now I only need a Hololens! :)

[@dilmerv]

Cool and this works in the Unity editor with the simulator as well ! thanks for your comment :)

[@jiayichen6490]

where did you find the GUI from the MRTK demos? Somehow I can't find it. Thanks

[@overthinker1935]

Hi Dilmer! I got a project that involves detecting objects with Hololens2 and the user needs to put the object in a simple box and the program detects that the object is in the box. Do you have any ideas how can i detect a real object using HoloLens2?

[@sandipjoshi8121]

Hi, I want to apply this technology to make an utility that can help the people to enjoy movies created with other language voice can be listened in their own language. can we work together for this venture?

[@dilmerv]

Thank you for your kind comment and hopefully you found someone to help you out !

[@waihanlim6699]

This feature must be required in anywhere, if not we have a hard time typing in the air.

[@dilmerv]

Absolutely 🔥

[@barreradev]

funciona con la primera versión de hololens?

[@dilmerv]

En realidad no lo he probado con la primera versión, pero podrías intentar ;)

[@nationbot8748]

Nice Video, keep it up!
Can you show us how to implement the other way around - text to speech?
I used a sample, but I only get it to work in Unity, if I build it into the HL2 I'll get a connection failure and no audio.
greetings!

[@dilmerv]

Great idea and thanks for your feedback !

[@BR14Nx]

SpeechSDK link is out of date

[@dilmerv]

Thanks for the info, I just updated it and you can also find it here github.com/Azure-Samples/cognitive-services-speech-sdk/tree/master/samples/csharp/unity/virtual-assistant

[@memelab2007]

Is there a similar speech-to-text solution that would work for Oculus Quest 2?

[@dilmerv]

Yes I did the same for Oculus with MRTK github.com/dilmerv/MRTKOculusQuest2VoiceTranslator

[@memelab2007]

@@dilmerv Cool! Thank you, I will check it out!

[@陳宇靖-w1r]

hello where to find this example code Thank you

[@dilmerv]

The demo is available here github.com/dilmerv/HoloVoiceTranslator but more likely it needs to be updated at this point.