I Was FLOORED. Realtime AI Translation & Voice Cloning!
Вставка
- Опубліковано 9 вер 2024
- Experience the magic of AI with Meta AI's latest innovation in real-time translation and voice cloning! This video dives into seamless, multilingual communication breakthroughs, demonstrating how AI is revolutionizing language barriers. Join us in exploring this transformative technology.
▼ Link(s) From Today’s Video:
✩ Try the realtime Speech Translation Demo Here: seamless.metad...
✩ Download the Realtime Translation Models: github.com/fac...
✩ Meta Blog Post: ai.meta.com/re...
► MattVidPro Discord: / discord
► Follow Me on Twitter: / mattvidpro
-------------------------------------------------
▼ Extra Links of Interest:
✩ AI LINKS MASTER LIST: www.futurepedi...
✩ General AI Playlist: • General MattVidPro AI ...
✩ AI I use to edit videos: www.descript.c...
✩ Instagram: mattvidpro
✩ Tiktok: tiktok.com/@mattvidpro
✩ Second Channel: / @matt_pie
-------------------------------------------------
Thanks for watching Matt Video Productions! I make all sorts of videos here on UA-cam! Technology, Tutorials, and Reviews! Enjoy Your stay here, and subscribe!
All Suggestions, Thoughts And Comments Are Greatly Appreciated… Because I Actually Read Them.
-------------------------------------------------
► Business Contact: MattVidProSecond@gmail.com
PLEASE! Let me know how good the translations are! Hard to believe this is all happening in just 2 seconds of delay!!
Spanish worked very well!
They sound too Mexican, though
The Spanish translation is basically perfect. Tone meaning and word choices couldn't be any better, I don't think.
SO so SO voice french cloning !!!!!!!! You got the BEST A.I. channel !!!! So amazing to feel everuday your so so essential passionate vibes bro !!!
They never show Icelandic in ai stuff I noticed
As a native Spanish speaker I can tell you the translation was accurate even the very fast one.
Great to hear!!
@@MattVidPro lo confirmo. The ice cream sentences in spanish are absolutelly PERFECT.
All the spanish translations are perfect, i cant detect one simple error, son absolutamente increibles.
El unico error que se me ocurre es "voy a llorar por todo el suelo" pero es literal lo que dice
I'm not a native speaker, and didn't even get good spanish grades in school, but I think the translation of the channel name was a bit off. For what i heard, it says "de la ia mattvidpro", which implies that the channel is owned by an ai called mattvidpro, but I could have heard wrong, like I said, I don't speak spanish, but my native language is very close to spanish, I speak brazilian portuguese.
AI is moving so fast that I can't even dream of what we'll have in a year. You showed me so much I wasn't aware of. Honestly the best AI channel imo. :) When you showed me suno I got hooked at using its Custom mode to write my own lyrics (some memes are on my channel). All thanks to your video. :D
Thank you so much! Probs gonna do another suno vid soon! Maybe a live!
@@MattVidPro Yeah, do it! The new custom feature lets you define the music style and lyrics. It's awesome. Don't forget the three rules of Suno:
1. You usually only have time to write 8 lines per generation. That's a 4-line [Verse] and 4-line [Chorus]. If your lyrics are too long, there will not be any voice in the generated audio.
2. To make a longer song, click "Continue from this song" or something in the generation's dropdown. Then you get the custom UI to write more lyrics.
3. To make sure the song ends, add [Song end] at the bottom of your last section.
Have fun! It definitely blows me away what it can do. And I can't even imagine 2.0! :D
We aren't even starting yet.I have a couple open source projects of my own which will be kicking pandoras box very soon
Wait ..wait..Suno has custom mode?
It would be absolutely mindblowing if they could combine this technology with that HeyGen translated lip-sync or something similar, even at the cost of a bit more delay
Absolutely! Forgot to mention tht
If the AI language translation software doesn't delay the entire video feed 2 seconds, wouldn't it end up looking like a poorly-dubbed foreign martial arts film where the actors' lips move long after you've heard the words...making it seem disjointed and unintentionally funny?
Hopefully we can toggle languages for all youtube videos soon. Imagine how many more views people would get. Thanks for putting in the work to demo it for this video, great job Matt.
thats just matter of a year or two. 🔥
This is the future.
I'm looking forward to that.
Real time translation was working for a lot of videos on youtube . But ... then they vanished ??
Maybe it was resource heavy for youtube but it was great !
@@ajitsen6927I can still use it, but it's kinda bad. Maybe it's only available with Premium?
As a Spanish-speaking person, the translation sounds pretty accurate to what you said real fast. This is amazing technology. It means soon everyone will be able to speak with everyone else across the planet, no matter the language, and everyone will understand everyone else.
It's not cool to me. It makes us robots :) We should just learn languages..
The fact that it can translate both languages, but the EXPRESSIONS as well is amazing. I can't WAIT to see what the future holds for this tech. Thanks for sharing.
I am french and I confirm French demo is Insane😍🇫🇷
The first German output was indeed very decent. One can tell it's generated but the pronounciation was accent-free German indeed. The second output was rather subdued. I don't know whats scarier: that the system could replicate your voice to that degree from 9 Seconds of audio input or that it could understand extremely jumbled German input and translate the words you meant to say into English :-)
It kind of messed up the placement of the words in the first German example. But what can you do, weird placement is somewhat a core aspect of our language.
Wow, your German is pretty impressive (I can confirm since I'm a native German speaker)!
I didn't know you can speak German and that you were German the entire time :)
its gonna be so weird when we all have AR glasses with built in headphones that automatically caption and translate everything in real time.
can't wait to use this for anime
@@EonSoundnow that is something I'm willing to see
@@EonSoundwait til this man discovers that subbing and dubbing is a technology that has existed for decades upon decades
@@Lord_Drakostar Except dubs are vastly different and awful.
@@EonSound ??
Dude the Spanish ice cream one was literally verbatim perfect, unreal!!
It's not learning your voice then applying expression, it's cloning the voice you're speaking in. Try changing expressions mid record to see. Still, this is crazy how good it is!
This is one of the best vids Matt has ever produced, it's awe inspiring, practical and you can have hands-on experience (no boring papers). Matt, how is AI at translating English text to other language text - I always wanted to write a book in Spanish or Italian.
Crazy to imagine this tech will be built OFFLINE into our phones or even directly into an earbud at some point in the near future.
The birth of a Universal Translator.
Absolutely!
If actors weren't going to be replaced by AI soon, you'd be a very good actor!
Appreciate that 😂
you in alanta in the state it avaible there lets gooo@@MattVidPro
TVs in the not too distant future may have the ability to dub any movie or TV show into nearly any language. What would be especially great about that is it should be able to capture the emotional performance of the original actor.
Matt, I’ve never heard anyone speak Spanish, or German, quite like you. It’s… amazing. Great vid
The spanish translation is perfect. TIP: When you review translators, you can record the translation and then play it back to translate back to English. It is still funny hearing you try, though. Oh, btw, you did say but instead of dog. XD (Perro = Dog, Pero = But)
It would be interesting to see if you gave it a sentence where dog would make more sense in the context if it corrected it.
Mind-blowing!
imagine this being incorporated into something like earbuds
yes
that would be cool
for sure!!!!
They was on sale years ago
This is insane! Fantastic to see that this is already here I have been dreaming about this exact thing for like 2 years and it's here thanks Meta
the fact it can translate with the emotion is just the cherry on top of the already mindblowing tech!
Wow! That whispered one was indeed soooo much creepier in Spanish! Hilarious!
Thanks Matt for keep us all up to date with all this exciting AI . This looks just amazing not long until we have a universal translator so language will no longer be a blocker .
Love the video!!! I think another good thing to try would be saying something with an intermittent laugh or grunt to see how it handles non verbal sounds we make in communication!!
Finally, the universal translator from Star Trek. Next, the food replicator.
I could really use something like this. My partner is Persian and while she speaks English, much of her family doesn’t and when they get together I have no idea what they are talking about and she will often translate for me but it breaks the flow of conversations at the table. I would LOVE to just be able to understand them and join in the laughter seamlessly. I’m sure that Farsee is not high on their priority list but I can still see it coming soon enough.
So what from I can tell with German and Spanish the translation is pretty good, it sounds a bit like the actors in like an audio play would? I think that's the best I can describe it, a bit overemphasized but very comprehensible.
The french to english was perfect
Edit: when you did english to french, to me it sounded like a mix of your voice and basic translation voice. It's still easy enough to guess that it was translated by ai
Being a Spanish speaker myself, I have to say, the accuracy is spot-on. However, it does come off a tad neutral, almost like a movie translation. Nonetheless, it's truly impressive. I often collaborate with Americans, and integrating this into our video conferences would be a game-changer.
Translation in Spanish at 7:48 was perfect. Prountiation was like at 70% but the traslation was at 100%
Did it not sound more Mexican spanish than Spanish spanish ?
this is probably a really good tool for learning languages as you get back your own way of expressing yourself
Years ago I told everybody AI with live translation using the original voice of the speaker will be huge. Can’t wait till streaming devices build it in or people mod to add it. Watch any show from any country with the intended voice just in your language.
В России уже три года эта функция встроена в браузер)
Смотрю иностранных блогеров и новости на русском языке с гендерным разделением озвучки.
Hi, bro! The Spanish translation was perfectly correct. Thank you for sharing the cutting-edge technology that appears instantly. This is the best AI channel to stay updated all the time.
World peace can finally be achieved with this ai translator lol xD
As a speaker of English French and Spanish I can tell you it’s very good.
I want to have this as an app . Would be mind-blowing.
Imagine translating ASMR 😂
Were on the cusp of star trek type translation. If meta has their way, I bet we will all be wearing tiny devices in our ears that translate for us.
This. It would be really cool (even if it was an App running on the background of your phone in your pocket).. and you could set preferences for which Languages you want it to listen for and auto-translate. It could always be running in the background,. if you run into some public-space situation where you need translation, you'd have 15seconds of cache already ready (kind of like a Car security cam that's always recording just in case you need it). Brings up some interesting privacy concerns, but the benefits far outweigh that. I recently moved to Portland, Oregon and there's a lot more diversity here. I often hear multiple languages on the sidewalks every day as I walk around.
I think before too long you will just be able to tell it the language you want to listen in for any UA-cam video. And it will render it in that language, potentially even translating the lip movements.
It would have to be smart though, to handle a video like this one, and know not to translate everything said into the requested language.
The might be one of the most significant half decade in human history
I hope someday they'll make a mask with this ai model implemented into it, so you can translate your speech in person in real time
French and Spanish speaker here - they’re both AMAZING!
Hilarious and VERY impressive!
Thank you for the kind words!
The German version sounds good, but it's as 'natural' as your feigned 'emotions' - like an actor. The intonation during questions is off. Your German was both entertaining and chaotic, yet it was intriguing to see what the model could decipher. Well done! You've created an exciting video! 👏👏👏👏
The French is still not up there, on some words like TOUS which we pronounce differently depending on context, which it fumbled.
But it's so much better than what I've heard elsewhere.
It's insanely good. I'm just starting to be able to carry on convoys in Spanish without my translator but I'm gonna play with this. Game changer!
as a frenchman, this is really well done.
Have you played around with the Magnific AI upscaler yet? I used it on a few of my older composite images I didn't like, and it suddenly made them look like a real photo. It was insane
German wasn't bad, it had some issues with abbreviations like "AI" to "KI" - but as a way to communicate over language barriers this is a very solid start.
Ai is Ki in German ;) kuenstliche intelligenz
@@pigdogcheap1773 Ja, aber die Abkürzung wurde falsch ausgesprochen.
This is great. This is the hardest I've laughed at your videos. You really had fun and worked this model. Good times ( in every language).
That reverse translation back to english looks fun as
For me as a german, both german versions of your voice sound similar. The model voice's lower frequencies are slightly more dominant.
It has this weird reverb but the translation is on point
absolutely! this is prob the cost of the near realtime latency
I wonder if I can find a good use for this.
Holy translation! That sounded more than good. Very funny video Matt! I laugh a lot with your reading and translations.
The Spanish translation is so top-notch!
I really hope the new version is noticeably improved over the previous versipn. Im trying to translate a light novel series from japanese to english and ive been usinf chatgpt for it thus far. With very advanced prompt engineering you can actually get insanely good translations but its very time and labor intensive. Translated half a book with this method and i dont know 1 word of japanese. Im working on a program to automate a lot of it using a locally hosted sql database and the gpt api. Should be done with it soon. If this new seamlessm4t model is even comparable to prompt engineered chatgpt that will save me a lot of time.
Btw you have mad scientists eyes.Like if you stopped blinking, you could hypnotize fools😂
I've been waiting for this for 22 years.
... I should probably just have studied, though. :D
the ice cream one was amazing,not a single error in translating
but yeah all of them are perfect (spanish speaker)
This was the best video I have seen in recent months. This made my day.
Can’t wait for Eleven Labs’ version of this tech
As a native spanish speaker I think the pronunciation here it's really good but it's messing up on translating correctly a little on some parts, what I mean is that yeah, I can still get what's the meaning of the phrase, but that's just not how a native spanish speaker would say it
For example here: 10:30 it says "te has desconectado", the correct translation would be " Te has desuscrito"
Yeah same for french, pronunciation is good but the translation is a bit basic (it's like they translate word by word)
As a native french speaker, the french sounds much better than the Spanish but doesn't sound natural either
can't wait until we can load voice samples into a guitar pedal style box, then just load our favorite voices to speak into a microphone and have it convert live.
we can try use google translate to voice pronunciation ... so probably we can do some of real test for translation is right or wrong
0:42 The term you were missing here is "Babelfish"
My son is learning Italian. I asked chat gpt to test his skills by asking questions (with Voice). And it was able to ask questions in Italian and understand his answers in Italian
It's indeed impressive how it picks up accents
I speak spanish and the spanish voice sounds like a native spanish girl!! And your spanish translation Matt sounds ok too with a little spanish accent from spain well for the purpose I totally understand the words in spanish, the translation is very good!!
So I guess we don't need to learn other languages now. All of the arts and world language teachers in my school are kind of freaking out.
This is just insane and so cool! wow!
Very impressive! Please, someone give Matt an Oscar for best actor. 😅
Crazy. Thanks for sharing
The french one is really good. Doesnt sound robotic
The Spanish AI translated the English word mouse to house, but the French was perfect. I don't know German.
Hi, well for me all I would want is it spoken back to me in perfect English that I could understand not necessary in my voice or anybody's voice, a robot voice would be fine just as long as it can understand what was sead in the other language perfectly? Great vid mate loved it, take care and have a great day!
Great French Matt!! 🔥🔥🤟🔥🔥
Regarding Spanish, translations were very literal, a human translator wouldn't translate your lines that way. A human translator would adapt them so they sound natural, as if they had been spoken in Spanish originally. Impressive tech. What I'm very curious about is what amount of horsepower on Meta's mainframes is required for these AI translations. Also how much energy is required for this task.
My beloved wife is a professional interpreter. Thank God she will be retiring next year. This is going to destroy her industry.
There were some errors in all the languages. They have to do with the intricacies and differences between the two used languages, e.g. "you" that you used in English was meant as plural ("vous" in French), but was translated into "tu" - singular meaning of "you"
Spanish is very good, yes. Es muy buena la calidad de la traducción.
I can confirm that your German was real German. Awesome! Unglaublich! :)
French guy here : translation is okayish, but there's some sort of a little english accent.
It's still pretty impressive for some use cases.
It’s cool when it works it tends to crash a lot but this is just a demo this is cool.
😮 Amazing
Thanks Matt! 🙏🏼
Matt, Will you be using this for video dubbing with the video dub feature on UA-cam? it'd be interesting to see how well it could work for internationalization...
Universal translators on the way in due time
Wow, useful method and thanks for your sharing. Besides, I know a plug-in called Immersive Translate can also translate video subtitles into multiple languages in real time.
Я вообще в реальном времени смотрю ютуб на 9 языках независимо от наличия субтитров. Достаточно чтобы там был голос или песня.
Translation in French is not yet perfect, but pronunciation is spot on, regardless of the little robotic touch in the information. Very impressive!
You mean spot on if you live in Montreal, right?😅
@@ArnaudMEURET it sounds more like a bilingual English-speaking person to me.
@@AlienetteColdfire Definitely Canadian French to my ear, but the point was mostly that it would never pass as France French to a native speaker. Disclaimer, my experience with Canadian French is mostly from watching hours of Têtes à claques. 😅
Not Canadian nor French enough in my opinion, but regardless of the accent, the pronunciation is passable!
Love it!
lol call center workers will be out of work soon
would be nice for someone to work on a webui or discord bot for this.
9:47 OMG HOW DID THE LIKE THINGY HAVE A RAINBOW OUTLINE
I don't believe in real time translation between languages with different sentence structure, like japanese with the question particle at the end vs others at the start. Whole sentences? Yeah, but not before ending them
If they make it work with there own VoiceBox model it will going to be game changer 😮😮😮
Hi, Matt, good to hear you in French! The voice and the tone were great; yet the translation was a little bit too litteral. Totally understandable, though.