Dub Video or Audio into 29 Languages with ElevenLabs Dubbing Studio
Вставка
- Опубліковано 4 жов 2024
- ElevenLabs has just released the dubbing studio. While ElevenLabs has offered dubbing for the last few months, the Dubbing Studio adds some new features... like the ability to EDIT the dubbed results to get it just the way you want. You can adjust the voice settings, change the timing (to make the new language version fit correctly) and a lot more.
With the Dubbing Studio by ElevenLabs, it has never been easier to take your existing content and duplicate it in any or all of 29 languages to reach a much larger audience. You can upload the audio or video you want to dub - or you can just tell ElevenLabs where it lives on a number of popular platforms - give it the source and target language and the AI does the work.
It separates the background music or noise into one track and each speaker onto their own track! That's awesome because it means you can upload a finished video or audio and don't have to break it down first into separate parts!
Then, for any track or individual clip, you can tweak the transcript, redo the translation, regenerate the audio in the target language, and export.
👉 Get started with ElevenLabs:
excelerator.te...
I am an affiliate which means that I may receive a small commission, at no additional cost to you, if you end up making a purchase after following my link. I sincerely appreciate you using my link!!
👉 Get started with ElevenLabs:
excelerator.tech/getelevenlabs
Great Video Explanation! Looking forward to trying it!
Have fun!
UA-cam will have a dubbing feature built into studio next year, be interesting to see if it will be similar to this. If viewers can select which language they want to listen in then it might not be necessary to set up separate channels for different languages.
I've heard that too and you're right - it would be awesome if all videos were just available in multiple languages on the same channel!! Indeed, some of the AI tools are being built-in to the platforms we already use and will make some of these things obsolete as 3rd party tools.
Now I can dub any cartoon without having to hire a production company or team to translate the cartoon. The problem is that you have credits per month and if you have 2 hours you have to wait a whole month for the credits to renew and then you can continue dubbing.
True. If you're on the creator plan or above, you can buy more credits. Starter and below cannot.
finally, thank you ! you described it 100% better then eleven labs and influencers with 1m+ subscribers
You're too kind!
@@excelerator It's true !
thanks for the great examples, much appreciated
Glad it was useful
yeah hesa good kid
I just watch your video. Just curious if I can dub a whole 1 hour video using the paid plan? Because whenever selecting a video it says 500 mb or 45 minutes
There is a 100mb and 45-minute limit on how long the clip can be that you upload. You might try splitting the clip, start with the first section and then, in the dubbing studio, click the plus button to the right of the timeline to add a clip and try to add the second section. If the limit is truly per clip, this should work. If the limit is actually per project, it won't.
I did see its a very compelling option, I find it expensive per minute. For those who want to use them for a regular basis. They will need to purchase a high monthly plan. Looking forward to it, as AI hardware permits cheaper processing power per audio min. But its uber cool tho.
I agree, the cost could rack up quick for regular usage, especially for longer projects and may or may not produce a sufficient return. Like you said, the cost will hopefully come down as hardware costs decrease.
@excelerator. Hi, I really like your videos. I’ve been learning a lot! Are there any classes out there for those who want to learn how to use text to speech, dubbing, etc?
So happy to hear you're getting something out of my videos!! As for classes, I'm sure they exist but I don't know of any personally (or even specific instructors) so I can't give any recommendations. My sources of knowledge have been UA-cam and trial-and-error (mostly error 😃.) But if I see something that looks good, I will surely pass it along to you.
@@excelerator thank you for replying
Very clear and informative. Thanks. Does it dub in the other direction? From a foreign language into English?
Oh, and what about music? If the voice sings, does the foreign language version sing? On key?
I haven't tried it myself, but it does allow you to select other languages as the source language and then select English as the target language. So that should work.
I can't find anything from ElevenLabs on using dubbing for music. My guess is that it would not work (or not work well), but I could be wrong.
Great video
Glad you enjoyed it
Can i dub from English to English? Maybe changing tonality from one voice to the others? Or English speaking with accent to native English speaking voice.
I just did a little testing with this in response to your question. I was able to dub from english to english. By default, it used 'clip clone' (a voice clone of the speaker in the original video) but I was able to edit the voice settings and change the voice for the new English translation to any voice from my voice lab. But, the lip movements and words weren't perfectly matched in the video/audio output. It wasn't necessarily out of sync, because I could see certain words and syllables hitting at just the right time, but some some facial movements didn't match the new speech well. Hope that helps. This wouldn't be a problem if you weren't trying to ensure lip-sync. But, if you don't lip-sync, using ElevenLabs speech-to-speech tool would probably be a much easier way to achieve a voice change/cleanup. Hope that helps!
what about multiple voices like for say a documentary or a news piece
It can handle that. I included an example from elevenlabs in this video where they had 3 speakers.
It is okay but I found that I had to make many adjustments to arrive at the finessed voice that I wanted. I was disappointed that each adjustment cost me credits, so it is too expensive in this respect. If you were able to fine tune the adjustments that you needed and then only pay for the final generated voice it would be worth subscribing. Unfortunately the current model is prohibitive. There is going to be a lot of competition for this type of app as AI evolves and becomes smarter - prices will have to become equally competitive to attract or retain subscribers.
I do hope the pricing falls as competition increases and resources become more efficient.
The whole point of dubbing is to ensure the lip, syncs with the new language audio.
It somehow makes the audio in the new language, and the original video clip, the same length so you don't end up with audio drift, but it does not lip sync.
@@excelerator lol what's the point then
I guess to make the video understandable to folks who only speak a different language, even though it will be obvious that it was dubbed since the mouth movements won't align. I've seen a lot of movies and videos that do this. But, I agree, lio sync would be ideal.
Does dubbing use minutes? Meaning, if I record a 5-minute clip and then dub it into 5 languages, did I just use 5 minutes or 30 minutes?
Yes, dubbing uses your character balance. You can get a discount if you click the box to let elevenlabs put a "dubbing by elevenlabs" watermark on the video.
Where is result video??
I'm sorry. I don't understand your question. 🤔
I can't find the option " create a dubbing studio" is it in the paid version only ?
Yes, the dubbing studio is only available in paid plans.
Can you translate youtube videos?
Yes. This is probably not the most efficient way to translate a video you want to watch in a different language, but if you want to translate one for publishing, this is the tool.
Hello sir Can i ask
Can it dubbed the Voice of Multiple speaker "Woman and Man" in Movie and Recognize it Without changing their voice?
Yes, it will dub the voices (for all of the speakers) into whatever language you've selected.
@@excelerator Hello sir I just recently tried it however the dubbed voices was only on the top of the original voices I can clearly hear the original English Language isn't it should be remove and overcome by The Filipino Language I chose to Translation it
Or should I use the paid version 11$ to make a clearly dubbed adaptation
That is not the expected behavior. I just tested converting english to Filipino and the output was in Filipino language only with no english (original language) heard. The dubbing studio is available in any paid plan. But, even without the dubbing studio, you can still use dubbing even in the free plan you just wouldn't have the dubbing studio to edit, adjust, and work with the tracks. If you are using dubbing studio (again, any of the paid plans have it) you might want to look at the tracks and see if you can isolate where the original language is and why it's bleeding through the generated dub. I hope that helps. If not, maybe ElevenLabs support can get you fixed up.
Preciso assinar para ter acesso ao Studio?
Yes. Dubbing studio is only available in paid plans.
@@excelerator Qualquer assinatura, eu consigo acesso ao Studio?
@@excelerator Obrigado por responder 😃
Yes, even the cheapest paid plan (starter for $5/mo) includes dubbing studio.
Glad to help!
Does it sync lips?
No, it does not.
@@excelerator do you know any good lip sync tool
I've looked at plenty but I can't say any are good. Even those that use preset characters have room for improvement. And the ones that try to lip sync a person in a user supplied or generated character... You may get the lips to move somewhat realistically, but they either keep the rest of the face still or have some odd movement.
@@excelerator oh okay, but which is at least better than rest?
Runway does a halfway decent job. CapCut just revamped their lip sync offering, but I haven't tried it out yet. They did well with lip sync in their AI characters, so there's hope.