ua-cam.com/video/ca9rcQYTIS0/v-deo.html Trippy Wav2Lip demo of paintings reading Alice in Wonderland - ua-cam.com/video/ca9rcQYTIS0/v-deo.html Become a member ua-cam.com/channels/mGXH-jy0o2CuhqtpxbaQgA.htmljoin 👍 Support on Patreon www.patreon.com/WhatMakeArt Tip Jar to Support the Channel paypal.me/pxqx?country.x=US&locale.x=en_US www.venmo.com/JimmyKuehnle Paypal: @pxqx Venmo: @jimmykuehnle
Wow! Great tutorial! I tried running it and it looks like the Google Colab gets tripped up on dependencies. I am getting a ton of red errors near the end right before the "Let's Try It". Even tried their updated version which wasn't any better.
Oh my goodness. I just watched FIVE more different videos explaining Wav2lip lip-sync that are twice as long as yours and much more confusing. Your video is definitely the best video describing this procedure in a succinct, precise, concise and simple way. I am so thankful that I stumbled on your video first as it is definitely the clearest and simplest explanation that I have seen given. You really should have more subscribers as your explanations (as noted in comments) are the best.
Although this was a very helpful video I have spent over 30 hours trying to get it to work unsuccessfully. Most of my mistakes included things you mentioned such as leaving no spaces in the labels for the audio and video, for example. However there are other problems that I could not easily solve even after watching your video several times in slow motion. These were 1. when I went to paste my google code into the box to authorize me using it, it would not allow me to paste it. When I used the keys control and V it caused an error until I realized I had to press them instantly together for a split second. 2. As I could not get it to work, I downloaded Google Drive thinking that was my problem. I then discovered that Google drive on my Lenovo computer does not make any distinction between small "l" and capital "L" so it indicated they were the same file. I had to uninstall google drive from my computer so that I had the correct files in the cloud. 3. I observed that you moved your files "Kennedy" and "ai" over into the Wave2Lip file but then later I realized you "had to change" the file names as you inputted them as that. I thought this was a later option and realized I needed to change my file names to what they are in the program "input_audio.wav" and "input_vid.mp4" . In spite of all my eventual corrections, I am still not provided a result that I can download even though it appears to have processed it all the way through. So I will keep comparing your video of the computer language you show to figure out where I went wrong. I only mention this stuff in case a complete newbie like me who knows nothing about computer code has similar problems. thanks.
When using control and v to paste. You can always press and hold control as long as you want and then press v to paste. You don't have to do it at the same time The same is true for using control plus c to copy. Yes, case insensitive file systems can make a problem for making sure something works on a case-sensitive system. The file naming could work better. If you go to the GitHub site, there are some other versions of Google Collabs that other researchers have made. They may be a bit more user friendly. I recommend going to the original GitHub site linked in the description and exploring. Appreciate the feedback and hopefully you get a result that works.
Holy cow dude! I think The Mona Lisa took the win here. It seems characters that are farther away from the camera, look a lot more convincing with deepfake technology. Characters closer to the screen, or with larger mouths tend to have more artifacting (if that's what its called?) For example, the presidents chin kept glitching out, and sometimes the lips seemed to fuze together for a frame or two. But overall this is amazing stuff. And the more this is done, I assume the more refined and better the software will get at doing this.
even though u were doing it on mac os and im on windows 10 it was still the exact same and easy to follow unlike some tutorials which are completely different so gg for that :)
Thanks for the feedback, good point, I should probably boot into Windows more often and record tutorials in Windows so people can see the different OS user interfaces
You need to convert a single PNG image to an MP4 and a video editor. You can just stretch out the PNG for as long as you want. There are video converters online as well
@@WhatMakeArt ive been expirmenting with a few different files i have on my computer and ive come across an odd problem. once the process was complete i looked in the results folder and saw tht there wasnt any file there. i looked back over the steps but couldnt find any error codes. do you have any idea what went wrong?
Hey! This used to work but I keep getting this fail TypeError: mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given Any reason known why?
Haven't tried wav2lip in a while, some of the dependencies may have changed since the original Collab notebook was published by the creators of wav2lip
For everyone thats getting the ^C error at the end of the output then here is what I did to fix the issue. I maxed the length to > 30 seconds and dropped the framerate to 30 from 60. My resolution was still 1080p. After that it actually started rendering.
Face not detected? Damnit, I guess the model doesn't work so well for anime characters... Nice video by the way, easy to understand and follow! I didn't run through any problems, so thanks!
Maybe try shortening the clip. Also make sure when you're edited it, your video editor didn't add a single blank frame at the end of your clip. If there's just one frame the entire video that doesn't have a face then it won't work try it with a short version that you know has faces in every single frame.
You just need to download the file on your computer to be able to put it in your Google Drive. Download it how you would save a video from a web page or any file from a web page. It could be under safe page as or just save. You could also try to download via Google Drive.
Make sure you do a test with the sample footage provided in the description. Then make sure that your video has a face in every frame. It should work if you have an MP4 and a wav file
Likely missing a, or a; where you changed the code. Best thing to do is reload the page and then try it with the example audio and video files linked in the description to eliminate variables
I really appreciate your video but was wondering if you could potentially inform me or help me with an issue I'm having. Is it possible to input 1080p footage and have the program work? Everytime I put in 1080 footage, the "Now lets try!" stops really early, giving me no result. It just spits out the usual stuff, and then "^C". I've done a couple of tests, where I put in 720p footage, it works, then I put in the same exact footage, just scaled to 1080p, and it no longer works. If you have found it to work with 1080p footage, please do tell me your secrets friend, should I be exporting the 1080p footage in some sort of way that the program can read it better? For reference I'm cutting and rendering the footage in Adobe Premiere, but I also have Handbrake to do any modifications if you are familar with that program. I'd appreciate just about any help you could have, I've been working on this on and off for about a week now and can't seem to crack it. Thanks.
Try running it through media encoder to export an H.264 You can also try and coding it with FMPEG It may be running out of available RAM at the larger size. Since you have it working at 720p you know there is a face in each frame so that shouldn't be the problem. Try a shorter version of the video to avoid any out of memory problems.
The algorithm was trained on lower res footage so you might only get it to work with low res footage unless you retrain the model with the info on GitHub
The code worked once and it wouldn't work anymore in the "Now lets try!" section it keeps saying "Using cuda for inference.Reading video frames...^C" even tho every frame has a face and the names are correct please help thanks
@@WhatMakeArt thanks for the reply. I think i figured it out. I didn't have enough ram, but the thing is the first time i did it i used a 4 min video and it works
It kind of depends on how much RAM Google lets you use for that session, it can vary depending on how much your account has used the GPUs that month or recently, glad it worked
It seem it doesn't work anymore. I used this for years and I never had any problem. Now it doesn't work. It always says "cannot stat '/content/gdrive/MyDrive/Wav2Lip/wav2lip_gan.pth': No such file or directory" in STEP 3 of "Get the code and models" section. It seems something changed in the source code. It says there's no "wav2lip_gan.pth" in "Wav2Lip" folder but it should be in "Wav2lip" (with lower case L) folder.
Not the author at all, their contact information is on the GitHub page linked in the description. On that GitHub page. There are also some online GUIs that are drag and drop and work fine, I recommend trying those
so does this alter the video or only sync your audio to match lip sync? Im curious if we can simply change the voice on an existing video without altering video, for instance on a cloned audio file. Does the audio file then have to be the exact lenght? how does the machine know where each word goes in sequence with the video? thanks.
Ah, I understand... I don't know of any program that swaps voices If you have a cloned audio file of a voice then just use that and it will update the video to match
great video! Quick question though; I follow all the steps and I'm careful to make sure everything is done properly, but it only seems to generate a result whenever I try the more lo-res version? When I try the first version, I only get the message: "Using cuda for inference. Reading video frames... ^C" and nothing pops up in the folder. The same happens when I try using more padding. But when I try using resize_factor, then it goes through the entire process to generate a result. I've tried this by inputing different pictures (as .mp4 files), and audio, and this has been the case every time. Any idea what that could be?
It might be running out of RAM on the GPU, try the same MP4 file but just a few seconds long, and make sure there is a face in every frame, if it takes too long to process then the Google collab will time out
@@WhatMakeArt I can confirm that a shorter clip did indeed work, and also clips that were already lo-res, so I think you're correct about the RAM on the GPU
Wave2lip works on videos with faces. If you have a motion comic that has a video file of the character then you can animate its face with wave2lip. If you don't have an animated character art face, you could use the first order model to animate it. I'm making a tutorial on how to use the first order model to animate artwork and drawings.
Since there's only a certain amount of free processing time this practice is to break your video up into parts and then link it back together afterwards. You can overlap the cut points so you get rendered video before and after each transition to make it more seamless
hey it's a great video. by the way, 1) my output video has been rotated 180 degrees. could you gimme some clue? 2) and I've tried with self-record with my camera's phone, eventually the script's output told that something like "can't recognize the face". Why is that? thanks in advance.
Stupid question but, I didn't quite understand what I need. I need one picture, and then one mp4 video of someone talking and then sync them? Can the mp4 video be only visual or does it also need an audio of someone speaking? TIA!
You just need an MP4 video of someone's face. There needs to be a face in every frame of the video. The person can be talking or it can be a video of a still image. Then you need a .wav audio file. You upload the video and the audio file and then the video will be lip synced to the audio file.
MAN! BIG THANK YOU!!! THANK YOU SO MUCH! I made some mistakes to begin with but i got it wowrking!!! Thank you thank you thank yoU! I see you responding to everyone's questions. You're a legend!! While I'm here. Any idea on how to execute the tensorflow uninstall [Y enter] command??? I don't have a background in coding. I can't get past that stage in the new updated notebook... I don't know if the notebooks make any difference. i suppose not.
When you get to that point after pressing the play button it will pause, then just press Y on your keyboard and then press the enter or return key Then you have to wait for a bit
Like always awesome video ! Shame tho that the lips keep moving when the speaker doesn't speak when using a video, but do it correctly when using a picture... Hope they can enhance that part.
Yes, it works perfectly with a still mouth such as us still image. One way to make it work well is to find footage of someone where they're not talking much and then it'll make their mouth open when the words happen in the wav file. Another option is to time your speaking to when they were originally talking and then the pauses will line up.
Hi thanks for this. I don't get a code when I link my google account and it doesn't display a URL to follow when I click the play button. It just pops up a window and I select my account. It doesn't seem to be linked afterwards as it can't find my Wav2Lip folders. Any thoughts?
Best thing to do is reset the cash on your browser and make a new collab session. Make sure you're logged into the correct Google account. You can reset the cash by holding shift and then clicking the refresh button on Chrome You can also clear out all cookies and session IDs to make sure you have a fresh start then it should work
ouch too complex , but great idea , i hope someday we get the software in GUI version as drag and drop, i hope you come up with it soon! since then lets watch some podcast on jefferson!
That's frustrating, are you using a tablet or mobile device? Sometimes those have issues If you go to the original researchers' GitHub page they have links to some drag and drop options
This was really helpful. I wonder: 1) can you save the notebook to G-drive and keep using it without running the initial steps, and 2) is there a way to keep the notebook RAM from getting exceeded. Any idea from your experience?
Yes, you can save a copy of the notebooks to your Google Drive and it'll keep most of your settings, you still have to reinitiate a session, but it can save a lot of time I think the only way to have more RAM access is to sign up for a pro Google Collab account
Thanks mate !!! Much love from New Zealand mah man... now I can try some Rick & Morty voice overs well that's the plan half way into your video and its very well explained and you made it easy ( tho there's always one person the easier you make it the harder they find it hahaha )
This is great, it’s on a painting witch is very photo real but still a painting. Do you believe this would work on a puppet or a action figure as long as the face is visible
Do we have to have a spret audio thing. because my audio is in my mp4 file. and my two files are saved as untitled project (1) and the second one is untitled project (2) . so do i put untitled project (1) in the input video
Your MP4 video file can't have sound in it but that sound won't have an effect on the lip syncing. If your sound is in another video you need to export that sound and save it as a wav file
If you use a video editor such as Adobe premiere you can save the file as a wav. You could also extract the audio with FFMPEG and then edit it in audacity. Both of those programs are free anf open source and have instructions online.
Thank you it worked like a charm , still i don't know if the part where you allowed acces to your drive is safe so can you know how we desactivate this authorization please ?
I believe that access is revoked after the session is terminated You could also use a separate Google account that doesn't have your personal information to avoid any security issues
@@WhatMakeArt Thank you for your fast response and also for the quality of your content , thank to you I found a funny and easy way to make speak my 3D models =D
how do you make the text reappear in the cell? I tried to fill in my google drive code but as soon as I ran that cell the text and bar disappeared within the cell
You might need to make a new session in Google Collab. go to the manage sessions in the upper right and terminate all sessions. Close your browser tab. Then open up a new Google code app and then you should be able to edit the text as you need to.
That's just to make sure everything is working. If it works with the example files then something's wrong with your files. If it doesn't work with the example files then there's something wrong with the collab page
What if I want to make another video a different day? The first one I made worked just fine, I saved the Google collab file on my drive. The next day when I tried to make another video, I added the files to the folder nad replaced the names on the Google collab file and it didn't work, now it says: "/bin/bash: line 0: cd: Wav2Lip: No such file or directory" on the second line in "Now let's try"
Remember each time that you start a new session you need to reconnect your Google Drive. That's where you click on the link and copy the code and paste it in. Your drive will only stay connected for a short amount of time so you need to reauthorize it. Easiest way is to restart your browser go to the code lab page and start from the beginning and do all the steps
What Make Art Yes understand that wasn’t my intention, just fun with friends etc, but I was still wondering how to get what I would like the video to say if you get what I mean , for example , Tom Cruise wishing my wife a happy birthday, hope your understanding what I’m trying to say, thank you.
Yes, I understand. You would need to find a video clip of the actor. Sometimes it works best with the actor's mouth not moving sometimes it works best with them talking. You can look at the audio waveform of the clip and time your speaking with when the actor is moving their mouth. You can use audio recorded by you and it will work fine to make the actor's lips move based on the words you say. If you want the audio in the actor's voice you need to have a large sample of their voice and use a general adversarial network to create an audio print but that is beyond the Wav2Lip technique.
i did it again but it's stuck at the third step of 'Get the code and models' and it says _cp: overwrite '/content/Wav2Lip/checkpoints/wav2lip_gan.pth'?_
Hard to say, sometimes it's best just to restart the browser and then really double check that all the files are the right folders, and everything is spelled correctly. Frustrating but it'll eventually work if you follow the instructions exactly
this does not seem to be working anymore. i tried using this google collab to make new content and it will not generate a result anymore. what is going wrong? is this just a me thing? if you look at my past content, i have successfully done this before?
Could be a problem with the new video file, try it with a file that you know worked in the past or with the example file from the description to eliminate all variables
@@WhatMakeArt I just tried it with video files I know have worked in the past, and have used in videos in the past and it sill will not work. It tells me "TypeError: mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given."
That sounds like you have a missing; or, in your argument line for the video files or the parameters added I would start with a completely blank collab page, get the original one from the GitHub page then make sure you have no typos when you add your video files, if you miss a quote or a, or a; then you will have an error Frustrating when something unknown is causing it not to work
when I run the last cell in "get the code and models" section i get this error: "cp: cannot create regular file '/content/Wav2Lip/checkpoints/': No such file or directory" but I have already connected my drive to the colab and already named the folders in the drive "Wav2lip" and "Wav2Lip", any help?
Did you run all previous cells and put the pre weighted model file in the folder in your Google Drive? The is likely some step overlooked or a timed out remote file. It's not much help but I would start a new session with the original collab from GitHub and try again.
You can just click up in the top right where it says manage sessions and terminate all sessions Then close your browser and reopen it, and you can go to the original GitHub repository and open up the collab page again, then everything should be reset and you can start a new session
@@WhatMakeArt I’m trying to sync a song to a video of a guy talking about something unrelated to song, but when there is silence in the song, the guy continues to talk, any tips?
Might be a dumb question, but I'm guessing the .mp4 has to match the .wav length? Or if I put a video shorter than the audio length then the program will expand it?
Don't have to be the same length but you have to have at least enough audio for your video. I haven't done an experiment recently but I think it truncates your video if the audio audio goes silent.
Is there an updated version of how to do this today? Like a lot of other people trying this in the collab notebook throws tons of errors. If someone has an updated video of the gotchas and how to get around these, would be helpful.
Yes, that would be great, I don't have time to make a new version of the original researcher's colab notebook now, but if a community member doesn't do it first, I will give it a go
I don't think there is a parameter. I would use a video editor to crop out the section you don't want. Then after Wav2Lip you can recombine the footage. Here is a face detection cropping tool on GitHub - github.com/1adrianb/face-alignment
This is wonderful, thank you! They seem to have updated their Colab page and the new version does not match your amazing tutorial. After I paste a link to my uploaded audio, and upload a video, it always says "cannot find custom.mp3". I clicked the "show code" button where I can see this file path, but it's way too complicated for me. I have found their old version and I will try that with your tutorial but maybe you could do an updated video for the other version?
Yeah, the new collab version is in some ways easier and in some ways more confusing. I think overall it is easier That's a good idea to make a new video. I'll look into it
@@WhatMakeArt Thanks so much for the reply. I thought I was finally getting somewhere with their old colab, but after processing for 41 minutes and reaching 100%, it reported: FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/wav2lip_gan.pth'. Any ideas?
Double check that you have the proper filename and folder name with the correct capitalization Sometimes there is a simple typo that one overlooks, hopefully it works out
That's frustrating. Sometimes it just doesn't work, only advice I can give is to reset everything and try again with the example files provided into the description
I'm getting this error message on the first cell of the "Now Lets try!" section, could you help please? /bin/bash: -c: line 0: unexpected EOF while looking for matching `"' /bin/bash: -c: line 1: syntax error: unexpected end of file anscombe.json mnist_test.csv california_housing_test.csv mnist_train_small.csv california_housing_train.csv README.md All the names and file directories are correct.
@@WhatMakeArt but how does the ai know about the structure of the insides of the mouth, imagine the character having 4sets of teeth upside and downside unlike human teeth structure the mouth movements it makes , will we need to input a code of how the structure of the mouth is from the inside?
It uses machine learning and then synthesizes the mouth shapes. I did not create the process but you can read more at the creator's github page. github.com/Rudrabha/Wav2Lip
I'm very based on all the GPU use and which type of GPU you're assigned. If it takes a long time I recommend starting a new browser session and trying again
Just like when your internet router malfunctions, best plan is to restart your computer, try a new browser, reload into your Google account or a different Google account and try again Frustrating sometimes...
I encountered an error, how can I get the required torch installed ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. kapre 0.3.7 requires tensorflow>=2.0.0, which is not installed. torchtext 0.12.0 requires torch==1.11.0, but you have torch 1.1.0 which is incompatible. torchaudio 0.11.0+cu113 requires torch==1.11.0, but you have torch 1.1.0 which is incompatible. tables 3.7.0 requires numpy>=1.19.0, but you have numpy 1.17.1 which is incompatible. pywavelets 1.3.0 requires numpy>=1.17.3, but you have numpy 1.17.1 which is incompatible. panel 0.12.1 requires tqdm>=4.48.0, but you have tqdm 4.45.0 which is incompatible. pandas 1.3.5 requires numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you have numpy 1.17.1 which is incompatible. kapre 0.3.7 requires librosa>=0.7.2, but you have librosa 0.7.0 which is incompatible. kapre 0.3.7 requires numpy>=1.18.5, but you have numpy 1.17.1 which is incompatible. jaxlib 0.3.2+cuda11.cudnn805 requires numpy>=1.19, but you have numpy 1.17.1 which is incompatible. jax 0.3.4 requires numpy>=1.19, but you have numpy 1.17.1 which is incompatible. datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible. albumentations 0.1.12 requires imgaug=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Sometimes the environment gets messed up on Google Collab Best thing to do is just reset the browser and start a new session If that doesn't work, then try it with the example video and audio files linked in the description to eliminate variables of things that could be going wrong
Forgive my continuous questions, I have Dyslexia so I want to make sure I follow. Would I need to ask a 3rd party for their lio moving services or would it be someone from your team? The edit itself is a non-profit, never for commercial, fan edit. :)
Make sure you have a face in every frame. Sometimes when you trim a video and a video editing software you can have one blank frame at the end. Even this one frame at the end won't let it work. Also check to see if you have something covering the face in one of the frames.
It can detect faces but the faces have to be clear in the video. Also no frame in the video can be without a face. Also, check that all your file names are correct and there are no typos.
I don't think this works anymore. there's a "WARNING: Skipping tensorflow as it is not installed. WARNING: Skipping tensorflow-gpu as it is not installed." In the first segment of the 'get the prerequisites' section. I'm not having any luck getting it to work
@@WhatMakeArt I'll try again. I got the more recent one to work but it only seems to do 15 second videos. Do you have a resource that will teach me how to do this on my own machine? Have I already overlooked it somehow?
@@WhatMakeArt and Steve Thanks for reaching back! I had this error in the last step of the video (in the "Let's try this!" section): ImportError: cannot import name 'keras_export'. Do you know what may be causing this?
Not sure why that would be happening except for maybe some module not being loaded. Whenever I get a not found error I usually start with a clean collab and start over double checking all file paths.
Gracias, me parece una excelente herramienta para poder hacer llegar el conocimiento de nuestros vídeo tutoriales en UA-cam a otras personas que lo necesitan y no conocen el lenguaje. Esa es la utilización que creo mejor puedo hacer para ayudar. Y también vi, algo de lenguaje de señas de personas virtuales, que también pueden hacer llegar el conocimiento a personas sordas. Beethoven, en sus últimos años, realizó sus mejores composiciones estando sordo. Sus herramientas para recrear la música fueron: su vista, el conocimiento del complejo solfeo donde mentalmente tenía grabada las notas en su mente, y la creatividad. A veces no hace falta oír, sino haber oído y aprendido. Lo demás lo hace la imaginación.
![Artificial Intelligence Can Make Anybody Talk! | Colab Tutorial [Deepfake From Image + Audio Only]](/img/n.gif)
ua-cam.com/video/ca9rcQYTIS0/v-deo.html Trippy Wav2Lip demo of paintings reading Alice in Wonderland - ua-cam.com/video/ca9rcQYTIS0/v-deo.html
Become a member
ua-cam.com/channels/mGXH-jy0o2CuhqtpxbaQgA.htmljoin
👍 Support on Patreon
www.patreon.com/WhatMakeArt
Tip Jar to Support the Channel
paypal.me/pxqx?country.x=US&locale.x=en_US
www.venmo.com/JimmyKuehnle
Paypal: @pxqx
Venmo: @jimmykuehnle
Wow! Great tutorial! I tried running it and it looks like the Google Colab gets tripped up on dependencies. I am getting a ton of red errors near the end right before the "Let's Try It". Even tried their updated version which wasn't any better.
I would check back with the original authors of the Collab and their GitHub page, they have links to some drag and drop examples
Wow this is Mind Blowing 🤯
🦜⛄👍
Oh my goodness. I just watched FIVE more different videos explaining Wav2lip lip-sync that are twice as long as yours and much more confusing. Your video is definitely the best video describing this procedure in a succinct, precise, concise and simple way. I am so thankful that I stumbled on your video first as it is definitely the clearest and simplest explanation that I have seen given. You really should have more subscribers as your explanations (as noted in comments) are the best.
Thanks for the feedback, hopefully you made some fun videos
@@WhatMakeArt I wish there was an even easier way to do all this...like a drag and drop.
There is a drag and drop away. If you look back at the original GitHub repository, someone made a drag and drop way
@@WhatMakeArt what is it titled?
Mona Lisa looks scarier than ever!
The future will be even weirder
Mona lisa saying McDonalds is good
🍔
Yes😖😖😱😨😰
👍
This video is so good, you deserve more subscribers
👍
jeez, took me 30 mins getting all the updated/compatible libs, but it worked in the end, cheers.
Yes, dependency hell can be frustrating, glad you got it to work
Very informative! I'm glad I did stick around to learn what's this about.
Glad you liked it, the researchers sure made an interesting lip sync algorithm
Don't you just love it when technology makes things so easy to use?
Da best
Although this was a very helpful video I have spent over 30 hours trying to get it to work unsuccessfully. Most of my mistakes included things you mentioned such as leaving no spaces in the labels for the audio and video, for example. However there are other problems that I could not easily solve even after watching your video several times in slow motion. These were 1. when I went to paste my google code into the box to authorize me using it, it would not allow me to paste it. When I used the keys control and V it caused an error until I realized I had to press them instantly together for a split second. 2. As I could not get it to work, I downloaded Google Drive thinking that was my problem. I then discovered that Google drive on my Lenovo computer does not make any distinction between small "l" and capital "L" so it indicated they were the same file. I had to uninstall google drive from my computer so that I had the correct files in the cloud. 3. I observed that you moved your files "Kennedy" and "ai" over into the Wave2Lip file but then later I realized you "had to change" the file names as you inputted them as that. I thought this was a later option and realized I needed to change my file names to what they are in the program "input_audio.wav" and "input_vid.mp4" . In spite of all my eventual corrections, I am still not provided a result that I can download even though it appears to have processed it all the way through. So I will keep comparing your video of the computer language you show to figure out where I went wrong. I only mention this stuff in case a complete newbie like me who knows nothing about computer code has similar problems. thanks.
When using control and v to paste. You can always press and hold control as long as you want and then press v to paste. You don't have to do it at the same time The same is true for using control plus c to copy.
Yes, case insensitive file systems can make a problem for making sure something works on a case-sensitive system.
The file naming could work better. If you go to the GitHub site, there are some other versions of Google Collabs that other researchers have made. They may be a bit more user friendly. I recommend going to the original GitHub site linked in the description and exploring.
Appreciate the feedback and hopefully you get a result that works.
brother has u found any succesfull solution??? because i am struggling with the same mistakes i have been here for 3 days and no resultsss
can u try to repeat the tutorial bruh.... maybe something was wrong... plss@@WhatMakeArt
Holy cow dude!
I think The Mona Lisa took the win here.
It seems characters that are farther away from the camera, look a lot more convincing with deepfake technology. Characters closer to the screen, or with larger mouths tend to have more artifacting (if that's what its called?) For example, the presidents chin kept glitching out, and sometimes the lips seemed to fuze together for a frame or two. But overall this is amazing stuff.
And the more this is done, I assume the more refined and better the software will get at doing this.
This is a much older technique now, new methods are much better and convincing
@@WhatMakeArtwhere are the new techniques?
even though u were doing it on mac os and im on windows 10 it was still the exact same and easy to follow unlike some tutorials which are completely different so gg for that :)
Thanks for the feedback, good point, I should probably boot into Windows more often and record tutorials in Windows so people can see the different OS user interfaces
You need to convert a single PNG image to an MP4 and a video editor. You can just stretch out the PNG for as long as you want. There are video converters online as well
@@WhatMakeArt ive been expirmenting with a few different files i have on my computer and ive come across an odd problem. once the process was complete i looked in the results folder and saw tht there wasnt any file there. i looked back over the steps but couldnt find any error codes. do you have any idea what went wrong?
Remember you got to download the output video from the browser
Good stuff! Here cause Corridor Crew mentioned this AI and featured your video 👍🏼
Sweet, thanks for mentioning
Hey! This used to work but I keep getting this fail TypeError: mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given
Any reason known why?
That is usually a comma, semi colon typo, reload the Colab and do a test with the sample files
This is the best tutorial so far. Although it doesn't work.
Haven't tried wav2lip in a while, some of the dependencies may have changed since the original Collab notebook was published by the creators of wav2lip
For everyone thats getting the ^C error at the end of the output then here is what I did to fix the issue. I maxed the length to > 30 seconds and dropped the framerate to 30 from 60. My resolution was still 1080p. After that it actually started rendering.
Thanks for sharing those tips
I still can't get the output and get ^C error
Try the example files linked in the he description
Face not detected? Damnit, I guess the model doesn't work so well for anime characters...
Nice video by the way, easy to understand and follow! I didn't run through any problems, so thanks!
Maybe try shortening the clip. Also make sure when you're edited it, your video editor didn't add a single blank frame at the end of your clip. If there's just one frame the entire video that doesn't have a face then it won't work try it with a short version that you know has faces in every single frame.
when i go to download, no save as box pops up. Basically making the following steps incapable of doing
You just need to download the file on your computer to be able to put it in your Google Drive. Download it how you would save a video from a web page or any file from a web page. It could be under safe page as or just save. You could also try to download via Google Drive.
Thanks for this but my google drive is not giving me a code. The pop up just disappears. Can I pull it from anywhere else?
All the code is available on the original creators' GitHub page linked in the description
What if I want to do another one? For me it just doesn't work.
Make sure you do a test with the sample footage provided in the description. Then make sure that your video has a face in every frame. It should work if you have an MP4 and a wav file
@@WhatMakeArt No worries mate, I'm sorry for the inconvenience. It appears as though my previous session wasn't terminated, my apologies.
Please how to fix this:
TypeError: mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given
Likely missing a, or a; where you changed the code.
Best thing to do is reload the page and then try it with the example audio and video files linked in the description to eliminate variables
I wonder what's the point of downloading the model locally at 2:04 if everything is done on the cloud tho?
You need to have the model in your Google Drive. If everyone linked to the same Google Drive version of the model then it might get timed out.
I think this is a stupid question, but can I input .mp3 files instead of .wav files?
I believe Wav2Lip needs.wav files, you can use the open source audacity to convert any mp3s to wavs
I really appreciate your video but was wondering if you could potentially inform me or help me with an issue I'm having. Is it possible to input 1080p footage and have the program work? Everytime I put in 1080 footage, the "Now lets try!" stops really early, giving me no result. It just spits out the usual stuff, and then "^C". I've done a couple of tests, where I put in 720p footage, it works, then I put in the same exact footage, just scaled to 1080p, and it no longer works. If you have found it to work with 1080p footage, please do tell me your secrets friend, should I be exporting the 1080p footage in some sort of way that the program can read it better? For reference I'm cutting and rendering the footage in Adobe Premiere, but I also have Handbrake to do any modifications if you are familar with that program. I'd appreciate just about any help you could have, I've been working on this on and off for about a week now and can't seem to crack it. Thanks.
Try running it through media encoder to export an H.264
You can also try and coding it with FMPEG
It may be running out of available RAM at the larger size. Since you have it working at 720p you know there is a face in each frame so that shouldn't be the problem.
Try a shorter version of the video to avoid any out of memory problems.
@@WhatMakeArt can you explain how to do that? I'm always have bad resolution every result of video
The algorithm was trained on lower res footage so you might only get it to work with low res footage unless you retrain the model with the info on GitHub
Thank you so much for this tutorial!
Glad it helped, hope you made some fun lip sync videos 👍
@@WhatMakeArt haha yeah you are so good at explaining it. Keep it up!
@@Team_Maguire Hello Maguire
@@TobeysMaguire hi
@stuff lmao
The code worked once and it wouldn't work anymore in the "Now lets try!" section it keeps saying "Using cuda for inference.Reading video frames...^C" even tho every frame has a face and the names are correct please help thanks
That is frustrating when it doesn't work, did you try with the sample video and audio linked in the description?
@@WhatMakeArt thanks for the reply. I think i figured it out. I didn't have enough ram, but the thing is the first time i did it i used a 4 min video and it works
It kind of depends on how much RAM Google lets you use for that session, it can vary depending on how much your account has used the GPUs that month or recently, glad it worked
@@WhatMakeArt thanks!! (subscribed and liked)
It seem it doesn't work anymore. I used this for years and I never had any problem. Now it doesn't work. It always says "cannot stat '/content/gdrive/MyDrive/Wav2Lip/wav2lip_gan.pth': No such file or directory" in STEP 3 of "Get the code and models" section.
It seems something changed in the source code. It says there's no "wav2lip_gan.pth" in "Wav2Lip" folder but it should be in "Wav2lip" (with lower case L) folder.
That's strange, maybe there's a simple typo that's being overlooked or the dependencies changed
@@WhatMakeArt Are you the author of Wav2Lip or can you contact him? I need this to work :(
Not the author at all, their contact information is on the GitHub page linked in the description. On that GitHub page. There are also some online GUIs that are drag and drop and work fine, I recommend trying those
@@WhatMakeArt Ok, Thank you so much.
Thank you. Really easy to repeat. I have a lot of fun with that)
👍
Yup - followed every step and works like a charm. TERRIFYING.
Woot
so does this alter the video or only sync your audio to match lip sync? Im curious if we can simply change the voice on an existing video without altering video, for instance on a cloned audio file. Does the audio file then have to be the exact lenght? how does the machine know where each word goes in sequence with the video? thanks.
It is kind of magical. It just works
@@WhatMakeArt alright, still curious if there are any apps that don't alter the video 🙍
Ah, I understand...
I don't know of any program that swaps voices
If you have a cloned audio file of a voice then just use that and it will update the video to match
Hi. Thanks so much for the tutorial. I'm getting stuck at 2:41 as there is no link there to click. Has this process changed since then??
The processes change the bit, the best thing to do is check the original researchers' GitHub page that is linked in the description
@@WhatMakeArt thanks!
great video! Quick question though; I follow all the steps and I'm careful to make sure everything is done properly, but it only seems to generate a result whenever I try the more lo-res version? When I try the first version, I only get the message:
"Using cuda for inference.
Reading video frames...
^C"
and nothing pops up in the folder. The same happens when I try using more padding. But when I try using resize_factor, then it goes through the entire process to generate a result. I've tried this by inputing different pictures (as .mp4 files), and audio, and this has been the case every time. Any idea what that could be?
It might be running out of RAM on the GPU, try the same MP4 file but just a few seconds long, and make sure there is a face in every frame, if it takes too long to process then the Google collab will time out
@@WhatMakeArt thanks! I’ll try that 😊
@@WhatMakeArt I can confirm that a shorter clip did indeed work, and also clips that were already lo-res, so I think you're correct about the RAM on the GPU
Glad you figured it out, maybe you can stitch together shorter clips to have one big long clip
Hey man! I make motion comics and saw your Wav2lip and wanted to ask if it could possibly work with that? 😊
Wave2lip works on videos with faces. If you have a motion comic that has a video file of the character then you can animate its face with wave2lip.
If you don't have an animated character art face, you could use the first order model to animate it. I'm making a tutorial on how to use the first order model to animate artwork and drawings.
@@WhatMakeArt Yeah i have it as a video but there's no movement. Great! Looking forward to seeing that man!
Yeah it should work even if there's no movement as long as it's a video and it can see a face
@@WhatMakeArt Gotcha! Thanks man!
thank you very much! I am learning about it and I am so happy to find your video!
Great
How long can the video be? I think I'm having a time-out error when I try to use a 2-minute video. Is that possible?
Since there's only a certain amount of free processing time this practice is to break your video up into parts and then link it back together afterwards. You can overlap the cut points so you get rendered video before and after each transition to make it more seamless
I love this work ! Thanks for sharing.
👍
hey it's a great video. by the way, 1) my output video has been rotated 180 degrees. could you gimme some clue?
2) and I've tried with self-record with my camera's phone, eventually the script's output told that something like "can't recognize the face". Why is that?
thanks in advance.
It needs to see a face in every frame. Sometimes when you export a video from an editor it puts an empty frame at the end, double check that.
I have a error with result folder...erro 2 no such file or directory..../content/WavLip/result/voice.mp4
That error is usually a typo, I'm sure you have checked a couple times, but it is easy to miss a capitalization or other file path error
"Using cuda for inference.
Reading video frames...
Number of frames available for inference: 1562
(80, 4161)
Length of mel chunks: 1556
0% 0/13 [00:00
Try with the sample MP4 and wav linked in the description to rule out a bad video file
Stupid question but, I didn't quite understand what I need. I need one picture, and then one mp4 video of someone talking and then sync them? Can the mp4 video be only visual or does it also need an audio of someone speaking? TIA!
You just need an MP4 video of someone's face. There needs to be a face in every frame of the video. The person can be talking or it can be a video of a still image. Then you need a .wav audio file. You upload the video and the audio file and then the video will be lip synced to the audio file.
@@WhatMakeArt Thank you very much for your quick response. You wouldn't believe how helpful you are!
Awesome video and programming!!! Thank you
👍 you're welcome
MAN! BIG THANK YOU!!! THANK YOU SO MUCH! I made some mistakes to begin with but i got it wowrking!!! Thank you thank you thank yoU! I see you responding to everyone's questions. You're a legend!! While I'm here. Any idea on how to execute the tensorflow uninstall [Y enter] command??? I don't have a background in coding. I can't get past that stage in the new updated notebook... I don't know if the notebooks make any difference. i suppose not.
When you get to that point after pressing the play button it will pause, then just press Y on your keyboard and then press the enter or return key
Then you have to wait for a bit
Like always awesome video !
Shame tho that the lips keep moving when the speaker doesn't speak when using a video, but do it correctly when using a picture...
Hope they can enhance that part.
Yes, it works perfectly with a still mouth such as us still image. One way to make it work well is to find footage of someone where they're not talking much and then it'll make their mouth open when the words happen in the wav file. Another option is to time your speaking to when they were originally talking and then the pauses will line up.
Hi thanks for this. I don't get a code when I link my google account and it doesn't display a URL to follow when I click the play button. It just pops up a window and I select my account. It doesn't seem to be linked afterwards as it can't find my Wav2Lip folders. Any thoughts?
Best thing to do is reset the cash on your browser and make a new collab session. Make sure you're logged into the correct Google account.
You can reset the cash by holding shift and then clicking the refresh button on Chrome
You can also clear out all cookies and session IDs to make sure you have a fresh start then it should work
getting error : Could not find a version that satisfies the requirement opencv-python==4.1.0.25
This is an older Google colab, check the original authors' GitHub page for any potential updates
ouch too complex , but great idea , i hope someday we get the software in GUI version as drag and drop, i hope you come up with it soon! since then lets watch some podcast on jefferson!
There is a fui version linked on the GitHub website of Wav2lip
It shows no such file or directory. I followed all your steps. Points by points. What must I do? 😕
That's frustrating, are you using a tablet or mobile device? Sometimes those have issues
If you go to the original researchers' GitHub page they have links to some drag and drop options
@@WhatMakeArt there is no such link and their most recent colab is broken. i believe they no longer support this product
[Errno 2] no such file or directory sample data/ content/sample data...what can i do please
Double check your filepaths for typos
This was really helpful. I wonder: 1) can you save the notebook to G-drive and keep using it without running the initial steps, and 2) is there a way to keep the notebook RAM from getting exceeded. Any idea from your experience?
Yes, you can save a copy of the notebooks to your Google Drive and it'll keep most of your settings, you still have to reinitiate a session, but it can save a lot of time
I think the only way to have more RAM access is to sign up for a pro Google Collab account
@@WhatMakeArt Thanks! Do you by any chance have a similiar video for Lip2Wav?
No but there is updated research on GitHub from the original scientists,
github.com/Rudrabha/Wav2Lip
I'll have to look into making a video about it
Thanks mate !!! Much love from New Zealand mah man... now I can try some Rick & Morty voice overs well that's the plan half way into your video and its very well explained and you made it easy ( tho there's always one person the easier you make it the harder they find it hahaha )
👍 share any fun results
This is great, it’s on a painting witch is very photo real but still a painting. Do you believe this would work on a puppet or a action figure as long as the face is visible
It seems to work on anything that has a face
Thanks. So If this is all run in the cloud, does one still need to DL python, update torch etc?
It pulls the versions it needs, all requirements are listed on the authors' GitHub page
@@WhatMakeArt Great thanks. That's all squared away. Now it's not recognising the file paths I've set up. Arh, tech. :p
Do we have to have a spret audio thing. because my audio is in my mp4 file. and my two files are saved as untitled project (1) and the second one is untitled project (2) . so do i put untitled project (1) in the input video
Your MP4 video file can't have sound in it but that sound won't have an effect on the lip syncing. If your sound is in another video you need to export that sound and save it as a wav file
@@WhatMakeArt it doesnt let me export it as wav
If you use a video editor such as Adobe premiere you can save the file as a wav.
You could also extract the audio with FFMPEG and then edit it in audacity. Both of those programs are free anf open source and have instructions online.
Thank you it worked like a charm , still i don't know if the part where you allowed acces to your drive is safe so can you know how we desactivate this authorization please ?
I believe that access is revoked after the session is terminated
You could also use a separate Google account that doesn't have your personal information to avoid any security issues
@@WhatMakeArt Thank you for your fast response and also for the quality of your content , thank to you I found a funny and easy way to make speak my 3D models =D
👍
Increible! muchas gracias! funcionó perfecto
👍
i`m only getting a read me file in my results folder... any suggestions please
Did you run through all the steps? Did you try the sample audio and video files linked in the description?
how do you make the text reappear in the cell? I tried to fill in my google drive code but as soon as I ran that cell the text and bar disappeared within the cell
You might need to make a new session in Google Collab. go to the manage sessions in the upper right and terminate all sessions. Close your browser tab. Then open up a new Google code app and then you should be able to edit the text as you need to.
@@WhatMakeArt thanks :)
At the end on the last step, it says ../sample_data/hold: No such file or directory. Pls help. Thanks you.
You need to make sure you don't have any typos in your file path
@@WhatMakeArt It still says the same thing.
Try it with the sample files included in the description
@@WhatMakeArt Well I wanted to make one with mines.
That's just to make sure everything is working. If it works with the example files then something's wrong with your files. If it doesn't work with the example files then there's something wrong with the collab page
What if I want to make another video a different day? The first one I made worked just fine, I saved the Google collab file on my drive. The next day when I tried to make another video, I added the files to the folder nad replaced the names on the Google collab file and it didn't work, now it says: "/bin/bash: line 0: cd: Wav2Lip: No such file or directory" on the second line in "Now let's try"
Remember each time that you start a new session you need to reconnect your Google Drive. That's where you click on the link and copy the code and paste it in. Your drive will only stay connected for a short amount of time so you need to reauthorize it. Easiest way is to restart your browser go to the code lab page and start from the beginning and do all the steps
Thanks for this, it was very helpful for me.
Great, post a link of the cool stuff you make
It didnt gave me the link that you say at 2:41 what do i do? Also how do i stop them from having access to may drive?
The way Google mounts the drives has changed a bit, once you close out of the collab session then no access remains
Where on the code is there a place to type in exactly what you want the model or picture to say for you??
You need to record your own wav file with the audio and then it will say that
What was the maximum video length in seconds or minutes that you tried? Thanks
Haven't tested that
can i ask a question, whilst you sink lips to wav or mp3 made files, how can we get celebs to say what we would like them to say?
You can use any video but just use it for fun and laughs, don't try to deceive people
What Make Art Yes understand that wasn’t my intention, just fun with friends etc, but I was still wondering how to get what I would like the video to say if you get what I mean , for example , Tom Cruise wishing my wife a happy birthday, hope your understanding what I’m trying to say, thank you.
Yes, I understand.
You would need to find a video clip of the actor. Sometimes it works best with the actor's mouth not moving sometimes it works best with them talking. You can look at the audio waveform of the clip and time your speaking with when the actor is moving their mouth. You can use audio recorded by you and it will work fine to make the actor's lips move based on the words you say.
If you want the audio in the actor's voice you need to have a large sample of their voice and use a general adversarial network to create an audio print but that is beyond the Wav2Lip technique.
What Make Art Thanks for information, very helpful indeed.
what does ValueError: --face argument must be a valid path to video/image file mean?
You might not have your Google Drive mounted correctly
also, you likely have a typo in the path to the file or the name of the file in the try section
thanks for answering! so can i just refresh the site and do it again, trying to fix it?
Yes, just need to make sure file paths are exact
i did it again but it's stuck at the third step of 'Get the code and models' and it says _cp: overwrite '/content/Wav2Lip/checkpoints/wav2lip_gan.pth'?_
Hard to say, sometimes it's best just to restart the browser and then really double check that all the files are the right folders, and everything is spelled correctly.
Frustrating but it'll eventually work if you follow the instructions exactly
this does not seem to be working anymore. i tried using this google collab to make new content and it will not generate a result anymore. what is going wrong? is this just a me thing? if you look at my past content, i have successfully done this before?
Could be a problem with the new video file, try it with a file that you know worked in the past or with the example file from the description to eliminate all variables
@@WhatMakeArt I just tried it with video files I know have worked in the past, and have used in videos in the past and it sill will not work. It tells me "TypeError: mel() takes 0 positional arguments but 2 positional arguments (and 3 keyword-only arguments) were given."
That sounds like you have a missing; or, in your argument line for the video files or the parameters added
I would start with a completely blank collab page, get the original one from the GitHub page then make sure you have no typos when you add your video files, if you miss a quote or a, or a; then you will have an error
Frustrating when something unknown is causing it not to work
Thats GOLD ! Where do i record my text?
Use your phone or your computer to record the audio file that you want to use
when I run the last cell in "get the code and models" section i get this error: "cp: cannot create regular file '/content/Wav2Lip/checkpoints/': No such file or directory" but I have already connected my drive to the colab and already named the folders in the drive "Wav2lip" and "Wav2Lip", any help?
Did you run all previous cells and put the pre weighted model file in the folder in your Google Drive?
The is likely some step overlooked or a timed out remote file.
It's not much help but I would start a new session with the original collab from GitHub and try again.
You should try this out with Akira and all of its different dubs!
I am sure studios will start using similar technology to overdub video content
is it safe to let something access all your google drive files? is there a way to not use the google collab space?
You can use a separate Google Drive account or you can install the code from the original researcher's available on their GitHub repository
please i made a mistake when trying this exercise on my own, How do i clear it and start all over again..... Please i need help
You can just click up in the top right where it says manage sessions and terminate all sessions
Then close your browser and reopen it, and you can go to the original GitHub repository and open up the collab page again, then everything should be reset and you can start a new session
was super confused for a moment, then i realized I named the folders "Wave2lip"....
It's always simple typos that make things not work, I was confused by that at first as well
@@WhatMakeArt I’m trying to sync a song to a video of a guy talking about something unrelated to song, but when there is silence in the song, the guy continues to talk, any tips?
Thanks man
Only you could help me make a video !Like
Follow the instructions for fun videos 👍
Might be a dumb question, but I'm guessing the .mp4 has to match the .wav length? Or if I put a video shorter than the audio length then the program will expand it?
Don't have to be the same length but you have to have at least enough audio for your video.
I haven't done an experiment recently but I think it truncates your video if the audio audio goes silent.
Is there an updated version of how to do this today? Like a lot of other people trying this in the collab notebook throws tons of errors. If someone has an updated video of the gotchas and how to get around these, would be helpful.
Yes, that would be great, I don't have time to make a new version of the original researcher's colab notebook now, but if a community member doesn't do it first, I will give it a go
on the arrow at 3:11 that says !cp -ri it loads forever even though i have the correct file in that folder
edit: i just started over and it worked huh
Sometimes just starting over is all it takes just like unplugging your router
face argument must be a valid path to video/image file . I got this error?
You probably have a typo in your file name or the path to the video file or it's not a proper video file
Me too. No typos. Mp4 and Wav. :/
Try with the sample video linked in the description
How do I crop the face detection area from a picture so just one of any more faces is detected? Is a parameter there?
I don't think there is a parameter. I would use a video editor to crop out the section you don't want. Then after Wav2Lip you can recombine the footage.
Here is a face detection cropping tool on GitHub - github.com/1adrianb/face-alignment
It's free, sometimes Google does great things but you are limited on a certain amount of GPU time per day based on the usage of the entire system
I’m confused - I need a file named wav2lip_gan.pth. What exactly is that?
It is the weights for the air model that the original researches created, there is more info in their GitHub page
ls: cannot access '/content/gdrive/MyDrive/Wav2Lip': No such file or directory
Check for typos in the folder you created in Google Drive
Shall we make the images talk with wav2lip
You can
Thanks for the tutorial
Post some funny memes
This is wonderful, thank you! They seem to have updated their Colab page and the new version does not match your amazing tutorial. After I paste a link to my uploaded audio, and upload a video, it always says "cannot find custom.mp3". I clicked the "show code" button where I can see this file path, but it's way too complicated for me. I have found their old version and I will try that with your tutorial but maybe you could do an updated video for the other version?
Yeah, the new collab version is in some ways easier and in some ways more confusing. I think overall it is easier
That's a good idea to make a new video. I'll look into it
@@WhatMakeArt Thanks so much for the reply. I thought I was finally getting somewhere with their old colab, but after processing for 41 minutes and reaching 100%, it reported: FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/wav2lip_gan.pth'. Any ideas?
Double check that you have the proper filename and folder name with the correct capitalization
Sometimes there is a simple typo that one overlooks, hopefully it works out
@@WhatMakeArt I have the same issue. Checked all filenames and directories.
That's frustrating. Sometimes it just doesn't work, only advice I can give is to reset everything and try again with the example files provided into the description
If I were to somehow mess this up, would I be able to restart by using another Google account? Thank you for this!
Yes, you can use wave2lip on another Google Collab account
@@WhatMakeArt Thank you!!!
hey i have no error but i am not getting the video in result ???
Test with the demo video to see if it works
I'm getting this error message on the first cell of the "Now Lets try!" section, could you help please?
/bin/bash: -c: line 0: unexpected EOF while looking for matching `"'
/bin/bash: -c: line 1: syntax error: unexpected end of file
anscombe.json mnist_test.csv
california_housing_test.csv mnist_train_small.csv
california_housing_train.csv README.md
All the names and file directories are correct.
Drive mounted? Double check everything run before? Change file name both places? Could restart the session. Try with different mo4
Do this work with cartoons or animated characters?
Yes it does, sometimes with great results, just needs to see a face
@@WhatMakeArt but how does the ai know about the structure of the insides of the mouth, imagine the character having 4sets of teeth upside and downside unlike human teeth structure the mouth movements it makes , will we need to input a code of how the structure of the mouth is from the inside?
It won't know everything if it's not in the original training set, it usually substitutes human shape, mouth for cartoons and things like that
so it doesnt need visual reference for lip sync? it recognize how to move lips on image from audio file? how its possible?
It uses machine learning and then synthesizes the mouth shapes. I did not create the process but you can read more at the creator's github page.
github.com/Rudrabha/Wav2Lip
@@WhatMakeArt thanks!
At 2:37 when you clicked at it how much time did it take to load the while proces where we have to type y mine just keeps loading pls reply fast
I'm very based on all the GPU use and which type of GPU you're assigned. If it takes a long time I recommend starting a new browser session and trying again
@@WhatMakeArt I did it I tried it again and again opening new page pls help me with this I need it
Just like when your internet router malfunctions, best plan is to restart your computer, try a new browser, reload into your Google account or a different Google account and try again
Frustrating sometimes...
@@WhatMakeArt ya really frustrating you explained very well but I am suffering with this problem but will try again
Thanks mate, I will try it tomorrow
Have fun
I encountered an error, how can I get the required torch installed
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
kapre 0.3.7 requires tensorflow>=2.0.0, which is not installed.
torchtext 0.12.0 requires torch==1.11.0, but you have torch 1.1.0 which is incompatible.
torchaudio 0.11.0+cu113 requires torch==1.11.0, but you have torch 1.1.0 which is incompatible.
tables 3.7.0 requires numpy>=1.19.0, but you have numpy 1.17.1 which is incompatible.
pywavelets 1.3.0 requires numpy>=1.17.3, but you have numpy 1.17.1 which is incompatible.
panel 0.12.1 requires tqdm>=4.48.0, but you have tqdm 4.45.0 which is incompatible.
pandas 1.3.5 requires numpy>=1.17.3; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", but you have numpy 1.17.1 which is incompatible.
kapre 0.3.7 requires librosa>=0.7.2, but you have librosa 0.7.0 which is incompatible.
kapre 0.3.7 requires numpy>=1.18.5, but you have numpy 1.17.1 which is incompatible.
jaxlib 0.3.2+cuda11.cudnn805 requires numpy>=1.19, but you have numpy 1.17.1 which is incompatible.
jax 0.3.4 requires numpy>=1.19, but you have numpy 1.17.1 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Sometimes the environment gets messed up on Google Collab
Best thing to do is just reset the browser and start a new session
If that doesn't work, then try it with the example video and audio files linked in the description to eliminate variables of things that could be going wrong
Hey dude, would I be able to hire you for a less than 10 second edit involving manipulated lip movement like what's going on in this video? $$$
Contact info in the description but the data sets used by the researches don't allow commercial use
Forgive my continuous questions, I have Dyslexia so I want to make sure I follow. Would I need to ask a 3rd party for their lio moving services or would it be someone from your team? The edit itself is a non-profit, never for commercial, fan edit. :)
Send an email to the address in description to discuss
can you help me, so basicly i learned how to do wav 2 lip, but i do the deepfake, and my image doesnt move with the video, just the mouth
You need to use a video to make it move
@@WhatMakeArt i did i used an mp4 for the image and mp4 for the video and wav for audio
Did your MP4 have a moving figure in it?
@@WhatMakeArt only thhe video that was supposed to make the other video move, not the video image
tnx man for the video it worked and i love it
Woot
Keep getting: FileNotFoundError: [Errno 2] No such file or directory: 'checkpoints/wav2lip_gan.pth' how do you fix?
Try the example files linked in the description and double check for typos
Frustrating when it doesn't work, double check for typos
@TurtlesSkull yeah I got it but it’s still mad low quality
i get the error in lets try it finishes at ^C and doesnt test the frames
Make sure you have a face in every frame. Sometimes when you trim a video and a video editing software you can have one blank frame at the end. Even this one frame at the end won't let it work. Also check to see if you have something covering the face in one of the frames.
@@WhatMakeArt can it like not recognize a face?
It can detect faces but the faces have to be clear in the video. Also no frame in the video can be without a face.
Also, check that all your file names are correct and there are no typos.
@@WhatMakeArt i changed it and this is what i got Length of mel chunks: 1887
0% 0/15 [00:00
@@WhatMakeArt can it be too long? like if its a minute, is it too long?
Can it be used for live video call?
Not this method but there are examples of the first order motion model running in real time on heavy GPUs - can be found on Google
Thanks, i will! 🙏🤗😌
👍
I don't think this works anymore. there's a "WARNING: Skipping tensorflow as it is not installed.
WARNING: Skipping tensorflow-gpu as it is not installed." In the first segment of the 'get the prerequisites' section. I'm not having any luck getting it to work
You need to press y and press enter and then it'll uninstall the Tensorflow
@@WhatMakeArt I'll try again. I got the more recent one to work but it only seems to do 15 second videos. Do you have a resource that will teach me how to do this on my own machine? Have I already overlooked it somehow?
The creator of Wav2Lip has a GitHub repository with all the source code you need to get up and running on your own machine
github.com/Rudrabha/Wav2Lip
@@WhatMakeArt Hey, I got it to work. Thanks so much for your help.
Thank you for this video sir! I wonder if Wav2Lip also works well with other languages? (brazilian portuguese, more specifically)
I think it works with the visemes so it should work with any language
Yes I have seen a production that has dubbed 'Arabic' onto a video
Thanks for the information, good to know
@@WhatMakeArt and Steve Thanks for reaching back! I had this error in the last step of the video (in the "Let's try this!" section): ImportError: cannot import name 'keras_export'.
Do you know what may be causing this?
Not sure why that would be happening except for maybe some module not being loaded. Whenever I get a not found error I usually start with a clean collab and start over double checking all file paths.
Holy Crap Thanks!
welcome
Gracias, me parece una excelente herramienta para poder hacer llegar el conocimiento de nuestros vídeo tutoriales en UA-cam a otras personas que lo necesitan y no conocen el lenguaje. Esa es la utilización que creo mejor puedo hacer para ayudar. Y también vi, algo de lenguaje de señas de personas virtuales, que también pueden hacer llegar el conocimiento a personas sordas. Beethoven, en sus últimos años, realizó sus mejores composiciones estando sordo. Sus herramientas para recrear la música fueron: su vista, el conocimiento del complejo solfeo donde mentalmente tenía grabada las notas en su mente, y la creatividad. A veces no hace falta oír, sino haber oído y aprendido. Lo demás lo hace la imaginación.
Thank you for commenting and creating, there is such variety in making and art