Finally! I got it working. Thanks for the video! Some parts (especially insightface) were tricky due to my portable ComfyUI installation, and some models must be manually downloaded and put into the correct folders. But heck, it was worth it. The only thing I didn't figure out is why lip and eye retargeting don't work, even though they are enabled. Workflows (now 3 of them) are looking a bit different, maybe there is something else I need to tweak for that to work? Setting them to either 0.1 or 10 using different assets doesn't have any effect at all.
Idk why I insist into looking for something out of this channel just to find what I was looking for in here after I downloaded +20GB of something that I didn't need to download... Thanks a lot Sebastian!
Great, as always. I had to use NOT PORTABLE version of ComfyUI to make it work. It's pretty easy to install. It's amazing how fast it's working even on my GTX 3060.
I use Forge WebUI for the most part. I honestly wish we'd get an extension like this but specifically just for the eyes, even if we could just point them in the direction we want without taking it into photoshop and then back into AI tools.
EXACTLY! Read my comment above where I am raging out over us not having SIMPLE solutions like this to manipulate facial EXPRESSIONS via single image guidance input. If they can do it with video, why not with single images?
This just isn't working for me. I don't have anything in ComfyUI_windows_portable\ComfyUI\models\insightface\models and I'm missing the top nodes in the first and second rows. Manager can't seem to find them. Please, help.
Hi Sebastian, can we combine two or three models output into 1, like if we want to use Lip-sync and some other model with it, can you show us how it's done. How can we route or make nodes in such a way.
I need Live Portrait to be more like Hedra with the body moving, the face just moving at enough no more, but it's the first real alternative to HeyGen, though which is now obsolete.
I had a big memory issues (crashes/freezes when reaching 100%) but it was due to image concatenate multi and video combine nodes that took too much RAM. I just add a save image node at the full images output of the liveportrait node and it solved it (the nodes then took the images on the drive instead of the RAM)
i wish comfyUI can have this feature: setup this kind node test it, it's success, export it as app export will make installer file for windows/Linux / mac/or even for ios or android or arm if it's met requirement. when install the app from installer it also will install all necessary files so the nodes will work as in ComfyUI without any technical installation like going on in this video....
Couldn't get it to work - The following node types were not found: DownloadAndLoadLivePortraitModels LivePortraitProcess Lots of users seem to experience the same issue. This python install madness is so annoying 🤦♂
you are the best, I was really impressed when in your sdxl release video you gave the release date in the beginning without worrying about video counts, thanks
Maybe you should put a disclaimer behind "you can use it for anything". As the licensing for insightface is a mess I guess you can basically just use it for research, right?
Appreciate for the nice tutorial. When I follow the instruction, it generates only 8-second video whatever the length of input is. Can I make output video length same with source video's?
excellent workflow! thank you ... which value is responsible for getting smoother results? most of my tryings are a little bit to much moving around the face ... thx in advance ...
Its currently only img to video and quite frankly a meme still and not useful outside of memes. The true amazing stuff comes when they release the video to video code. Someone has already tweaked the code and uploaded Gladiator scenes with it in action and its honestly amazing. I cant wait to use it to make videos with it. You can parody and entire movie with a whole different script.
Hello! Please tell me how to fix the error. (Error occurred when executing LivePortraitProcess: LivePortraitProcess.process() missing 1 required positional argument: 'crop_info') Thank you
good job！ If I turn the live-action video into AI animation, the AI animation dialogue lip sync is not obvious. So, can I use this technique to enhance the mouth movements of animated characters?
You know what would be coool and awesome?..if such a Top Expert like You would do a review on what PC package with what Graphic card and what ram and etc etc, would be best for Auto1111 , comfyui and even for creating LLMs . i dont think anyone out there has done one in the last 12 months. i see some people doing multiple Graphic cards to create LLMs, which is fascinating,...for which there is no one , who ever having done a video on youtube yet neither.... i mean...how much faster would such a multiple Card setup be for auto1111 and comfyui... if you would consider ever doing such a video, dont forget to make it for 3 different budgets,..super dirt cheap ..versus middle budgets,...versus unlimited budget packages .... all for home.... so it makes sense to viewers here.... i doubt companies or corporation go to youtube to find such packages... wäre echt toll
It depends on how long u can wait. I see generally people recommend RTX 3060. I also remember reading that an RTX 4090 would be 10x speed of RTX 3050 for generations
Love this tool, and a great tutorial. However, only the eyes are animating for me?? I have the both the video and target pic sized at 512x512. I've tried different sources and targets, just the eyes are animating
The issue I'm running into is I get a squash and stretch on the head, at first I thought it was the resizing node that was doing it but even after removing it I still get it even though I'm using your exact settings, my head is not even as smooth as yours, any thoughts? The image I'm using is 512x512, should the image and video be the same size maybe?
After hours, I give up. Every time I give Comfy a chance, this happens. Nothing but installation problems. I followed everything, step by step still nothing but weird errors that I search and can't find answers for. I now think there is something in with my system because even Pinokio installs also fail. Errors like: Prompt outputs failed validation ImageConcatMulti: - Return type mismatch between linked nodes: image_2, LP_OUT != IMAGE ImageResizeKJ: - Return type mismatch between linked nodes: get_image_size, LP_OUT != IMAGE and many more.
Hey guys & gals - this is my entry. I believe our first priority should be to make this 'stuff', the tools of the future, as ACCESSIBLE as possible. To get artists IN we need to be OPEN. And the best way to do that is to flatten the 'learning curve' as fast as humanely possible. * No more gatekeeping! * No more hiding 'workflows' that are out date in a week's time behind paywall! * No more technical jargon!
Hey, that's some valuable info, thanks for sharing! :D Just a question, does the LivePortrait standalone have the same control options that ComfyUI has? More? Less ones?
@@juanjesusligero391 it has options for pasting back into the image or just keeping the square as well as a few more things I haven't checked as of yet.
When loading the graph, the following node types were not found: DownloadAndLoadLivePortraitModels LivePortraitProcess Nodes that have failed to load will show as red on the graph.
Noob here: Can someone tell me were i have to put the json-example to open in it in comfyUI... just installed it with some models and the manager. But now i am not sure in which folder i have to put the example or how to open it. I tried dragndrop it in the broswer window, nothing happened, the same when i click "load" in the manager window.
This turorial was great. Thanks. Question: Is there any tool (like this to run locally) to upload an mp3 voiceover and generate the mouth and eye movements to later use in this process? Thanks!
ipadapter, controlnets, reactor and all of the rest ASIDE....what tools are there that can TRANSFER facial expressions like this, with SINGLE IMAGE input? It seems like it would be so easy. If they can do it with a vid, why not with single images? Fuck a "face swap"! We've got that sorted. Now we want to tweak expressions like with these "driver" videos, but using single images. Fuck text prompting out expressions! Fuck fiddling around with laborious and complicated shit. Just a simple image input where the EXPRESSION, (not the facial characteristics), can affect your gens. What's the hold up, tech bro's?
Dude, you can use exactly this (LivePortrait) for only an image by adding a 'video' containing one frame as a source. I'm not sure if you can load an image sequence as a video, but if you can, that would be it. If the node needs a video, you can use FFmpeg to convert an image into a 1-frame video. You could also make a constructive critique in the comfyUI node repo, asking for the feature nicely. Remember, the people making this tech possible are working hard, even if it's not as user-friendly as we'd like yet.
@@juanjesusligero391 Yeah sorry about the frustrated tone in my comment. :/ I actually did try the solution you suggest, but had sub par results. I will look into it further. I used a video editor and drug a few frames from a sequence down to the time line, and let them play together as a video. Still feels like a "hack" or "work around". What I was getting at was that if LivePortrait can make it so that we can manipulate features on a face without going into face swapping, or using loras, or trying to achieve this by other means such as ipadapter and so on and so forth, and can accomplish this with VIDEO, surely it could be done with single images? There are many guides on how to try and "tease out" expressions in images, but this LivePortrait solution is so "direct" and simple. It's what I have been looking for.
@@juanjesusligero391 I mean think about it. You've done a great gen of a face you like. Now you want this face to smile, or "look cheeky", or distraught or whatever. You wanna try a multitude of different things. Various nuances of emotion. With a single image LivePortrait solution you could do so much easier that via traditional methods.
@@MattLind-oi5kg Yeah, I totally understand :) But all this technology is still soo young... Developers are yet trying to figure things out (and most of them are doing this just out of love, since they don't see a single dollar for their contributions), but in the (probably near) future we'll have a lot more control over things like this. Be patient, my friend! ^^
@@jaywv1981 Good tip. I am faffing around with their own webui version and all their "examples" videos work flawlessly but with others you might use, it's hit or miss. I've had some horrendous "I have no mouth but I must scream" moments, haha.
Patreon subscribers saw this video first as an early access video. Detailed guides and exclusive content on www.patreon.com/sebastiankamph
is there a way to install this with 1 simple button yet? I really don't like installing pythons and etc to get this working.
👋
@@jonrich9675 I made an app for it. Cooraft!
lol... after I spent hours getting the LicePortrait stand alone to work yesterday.....
Thanks a ton!!!!
I can't take anything you say seriously at 7:50 with her in the background doing THAT 😂
Finally! I got it working. Thanks for the video! Some parts (especially insightface) were tricky due to my portable ComfyUI installation, and some models must be manually downloaded and put into the correct folders. But heck, it was worth it.
The only thing I didn't figure out is why lip and eye retargeting don't work, even though they are enabled. Workflows (now 3 of them) are looking a bit different, maybe there is something else I need to tweak for that to work? Setting them to either 0.1 or 10 using different assets doesn't have any effect at all.
Idk why I insist into looking for something out of this channel just to find what I was looking for in here after I downloaded +20GB of something that I didn't need to download... Thanks a lot Sebastian!
Great, as always. I had to use NOT PORTABLE version of ComfyUI to make it work. It's pretty easy to install. It's amazing how fast it's working even on my GTX 3060.
Please make the same workflow guide for v2v!
New tutorial in your face!! 🙌🏼😂 fascinating tool!
I use Forge WebUI for the most part. I honestly wish we'd get an extension like this but specifically just for the eyes, even if we could just point them in the direction we want without taking it into photoshop and then back into AI tools.
EXACTLY! Read my comment above where I am raging out over us not having SIMPLE solutions like this to manipulate facial EXPRESSIONS via single image guidance input. If they can do it with video, why not with single images?
@@MattLind-oi5kgit might have to do with how computer vision/machine learning works ....maybe solution isnt so simple after all
This just isn't working for me. I don't have anything in ComfyUI_windows_portable\ComfyUI\models\insightface\models and I'm missing the top nodes in the first and second rows. Manager can't seem to find them. Please, help.
Thanks. I tried on Mac, and it works with the tweaks for MPS. This is insane tech.
What is the tweak ? I can't get it to work
because insightface : "Please note that insightface license is non-commercial in nature."
the million question....its is possible using it in atomatic1111? thanks
Not unless someone takes the time to port the code...
Hi Sebastian, can we combine two or three models output into 1, like if we want to use Lip-sync and some other model with it, can you show us how it's done.
How can we route or make nodes in such a way.
I need Live Portrait to be more like Hedra with the body moving, the face just moving at enough no more, but it's the first real alternative to HeyGen, though which is now obsolete.
I had a big memory issues (crashes/freezes when reaching 100%) but it was due to image concatenate multi and video combine nodes that took too much RAM. I just add a save image node at the full images output of the liveportrait node and it solved it (the nodes then took the images on the drive instead of the RAM)
i wish comfyUI can have this feature:
setup this kind node test it, it's success, export it as app
export will make installer file for windows/Linux / mac/or even for ios or android or arm if it's met requirement.
when install the app from installer it also will install all necessary files so the nodes will work as in ComfyUI without any technical installation like going on in this video....
Hi Sebastian!
One question: What is the maximum resolution and fps you can get from your output mp4 video?
Couldn't get it to work - The following node types were not found:
DownloadAndLoadLivePortraitModels
LivePortraitProcess
Lots of users seem to experience the same issue. This python install madness is so annoying 🤦♂
you are the best, I was really impressed when in your sdxl release video you gave the release date in the beginning without worrying about video counts, thanks
Maybe you should put a disclaimer behind "you can use it for anything". As the licensing for insightface is a mess I guess you can basically just use it for research, right?
Appreciate for the nice tutorial. When I follow the instruction, it generates only 8-second video whatever the length of input is. Can I make output video length same with source video's?
Got a question, in your example you got settings for eyeretargeting etc in the LIVEPortrait Process, I don't have this setting^^
can this be used with a webcam for live rendering?
crazy cool. best AI stuff on this channel.
excellent workflow! thank you ... which value is responsible for getting smoother results? most of my tryings are a little bit to much moving around the face ... thx in advance ...
This was really helpful thanks. Do you have any idea why it speeds up some of the videos? The output is way faster than the source video.
can't wait until you can use audio file to drive the lip-sync for live portrait !
Is it possible to run live portraits real time with a video feed straight from a camera or is it just a prerecorded video
YAY!!! Monday vidWAIT. Is tuesday... Where is my monday video?
Its currently only img to video and quite frankly a meme still and not useful outside of memes. The true amazing stuff comes when they release the video to video code. Someone has already tweaked the code and uploaded Gladiator scenes with it in action and its honestly amazing. I cant wait to use it to make videos with it. You can parody and entire movie with a whole different script.
there's an early access to the workflow on your patreon, is that correct ??
This is great! Is it only usable with square format?
You can use any ratio
Thank you so much!
Hello! Please tell me how to fix the error.
(Error occurred when executing LivePortraitProcess:
LivePortraitProcess.process() missing 1 required positional argument: 'crop_info')
Thank you
Awesome! Where can we find more example face video assets?
Nice video!
Thank you, but why cant we use a webcam for live recording?
good job！ If I turn the live-action video into AI animation, the AI animation dialogue lip sync is not obvious. So, can I use this technique to enhance the mouth movements of animated characters?
Whats the minimum GPU GB .
you need to run this system in ComfyUI?
3
You know what would be coool and awesome?..if such a Top Expert like You would do a review on what PC package with what Graphic card and what ram and etc etc, would be best for Auto1111 , comfyui and even for creating LLMs . i dont think anyone out there has done one in the last 12 months. i see some people doing multiple Graphic cards to create LLMs, which is fascinating,...for which there is no one , who ever having done a video on youtube yet neither.... i mean...how much faster would such a multiple Card setup be for auto1111 and comfyui... if you would consider ever doing such a video, dont forget to make it for 3 different budgets,..super dirt cheap ..versus middle budgets,...versus unlimited budget packages .... all for home.... so it makes sense to viewers here.... i doubt companies or corporation go to youtube to find such packages... wäre echt toll
It depends on how long u can wait. I see generally people recommend RTX 3060.
I also remember reading that an RTX 4090 would be 10x speed of RTX 3050 for generations
I dont have any of this video nodes and image nodes it looks very different when i launch my run_nvidia_gpu.bat pls help! Thanks!
Love this tool, and a great tutorial. However, only the eyes are animating for me?? I have the both the video and target pic sized at 512x512. I've tried different sources and targets, just the eyes are animating
The issue I'm running into is I get a squash and stretch on the head, at first I thought it was the resizing node that was doing it but even after removing it I still get it even though I'm using your exact settings, my head is not even as smooth as yours, any thoughts? The image I'm using is 512x512, should the image and video be the same size maybe?
Can you tell me how to make a video longer than 8 seconds ? Anyone know in which node I need to set that ?
After hours, I give up. Every time I give Comfy a chance, this happens. Nothing but installation problems. I followed everything, step by step still nothing but weird errors that I search and can't find answers for. I now think there is something in with my system because even Pinokio installs also fail. Errors like: Prompt outputs failed validation
ImageConcatMulti:
- Return type mismatch between linked nodes: image_2, LP_OUT != IMAGE
ImageResizeKJ:
- Return type mismatch between linked nodes: get_image_size, LP_OUT != IMAGE and many more.
How can this be used in the comfyUI version of 'Stability Matrix'?
can you use this on a1111?
Are you going to do the video to video version soon?
Yes, most likely
Anyone else having issues with the video outputs shaking and jittering?
From the name I was kind of hoping that this would be something I could use my webcam as input and an animated picture as a live output.
Hey guys & gals - this is my entry.
I believe our first priority should be to make this 'stuff', the tools of the future, as ACCESSIBLE as possible. To get artists IN we need to be OPEN.
And the best way to do that is to flatten the 'learning curve' as fast as humanely possible.
* No more gatekeeping!
* No more hiding 'workflows' that are out date in a week's time behind paywall!
* No more technical jargon!
for whatever reason, I cannot get the portable version to work lol. So annoying. Nice guide. Thanks for sharing.
as a Chinese born i found this is my wildest dream having big eyes! 👀
boy oh boy I'll be really handsome in camera now!
Don't Be insecure bro Having small eyes is totally ok and don't make you less handsome
@@beyounickvlog5285 thanks but I'll be still use it heheh
Trying to be funny by self discriminating your entire race, good on ya.❤
Doesn't work. Red everywhere. No manager button
is this 100% free to use if done on my local machine?
yes, but I am not sure what are the commercial use terms if that's what you mean
4060 ti, in comfy (portable version) it's taking 15.5 minutes... in LivePortrait stand alone it's taking 3 minutes for a 1 minute video.
Hey, that's some valuable info, thanks for sharing! :D
Just a question, does the LivePortrait standalone have the same control options that ComfyUI has? More? Less ones?
@@juanjesusligero391 it has options for pasting back into the image or just keeping the square as well as a few more things I haven't checked as of yet.
way less@@juanjesusligero391
When loading the graph, the following node types were not found:
DownloadAndLoadLivePortraitModels
LivePortraitProcess
Nodes that have failed to load will show as red on the graph.
Manager fails :
(IMPORT FAILED) ComfyUI-LivePortraitKJ
Try fix
Uninstall
Nodes for LivePortrait, insightface is required
Im on windows10 Portable version of comfyUi
7:25 That's what Zoomers call the Skibidi Toilet mode
why isn't it working on macbook 😥
This is a step forward .. but we need the rest of the body movement
Thanks, can I run on CPU because i don't have GPU
Probably no, need atleast 3gb vram
Noob here: Can someone tell me were i have to put the json-example to open in it in comfyUI... just installed it with some models and the manager. But now i am not sure in which folder i have to put the example or how to open it. I tried dragndrop it in the broswer window, nothing happened, the same when i click "load" in the manager window.
Chatgpt is your friend
Did you RMB and save file? Enter the file and use download arrow on the right hand side instead.
This turorial was great. Thanks. Question: Is there any tool (like this to run locally) to upload an mp3 voiceover and generate the mouth and eye movements to later use in this process? Thanks!
You can use Facefusion lip sync. You can use "source" as mp3 and target as your video
you should work on your lip setting, all your images lips looks glued
great video as always, thank you! I just don't get why I change the resolution to 1024 but it's still output a 512 video?
Please V2V
I hear you. Putting it on the list
ipadapter, controlnets, reactor and all of the rest ASIDE....what tools are there that can TRANSFER facial expressions like this, with SINGLE IMAGE input? It seems like it would be so easy. If they can do it with a vid, why not with single images? Fuck a "face swap"! We've got that sorted. Now we want to tweak expressions like with these "driver" videos, but using single images.
Fuck text prompting out expressions! Fuck fiddling around with laborious and complicated shit. Just a simple image input where the EXPRESSION, (not the facial characteristics), can affect your gens.
What's the hold up, tech bro's?
Dude, you can use exactly this (LivePortrait) for only an image by adding a 'video' containing one frame as a source. I'm not sure if you can load an image sequence as a video, but if you can, that would be it. If the node needs a video, you can use FFmpeg to convert an image into a 1-frame video.
You could also make a constructive critique in the comfyUI node repo, asking for the feature nicely. Remember, the people making this tech possible are working hard, even if it's not as user-friendly as we'd like yet.
@@juanjesusligero391 Yeah sorry about the frustrated tone in my comment. :/
I actually did try the solution you suggest, but had sub par results. I will look into it further. I used a video editor and drug a few frames from a sequence down to the time line, and let them play together as a video. Still feels like a "hack" or "work around".
What I was getting at was that if LivePortrait can make it so that we can manipulate features on a face without going into face swapping, or using loras, or trying to achieve this by other means such as ipadapter and so on and so forth, and can accomplish this with VIDEO, surely it could be done with single images?
There are many guides on how to try and "tease out" expressions in images, but this LivePortrait solution is so "direct" and simple. It's what I have been looking for.
@@juanjesusligero391 I mean think about it. You've done a great gen of a face you like. Now you want this face to smile, or "look cheeky", or distraught or whatever. You wanna try a multitude of different things. Various nuances of emotion. With a single image LivePortrait solution you could do so much easier that via traditional methods.
@@MattLind-oi5kg Yeah, I totally understand :) But all this technology is still soo young... Developers are yet trying to figure things out (and most of them are doing this just out of love, since they don't see a single dollar for their contributions), but in the (probably near) future we'll have a lot more control over things like this. Be patient, my friend! ^^
@@juanjesusligero391 I will take heed. :)
it is not live, it is batch processing... that is misleading.
Lax man its just a brand name
Not Scary At All
I tried it on my own video, the results looks like ass.
I've found it works best when first frame of your video is a neutral expression.
@@jaywv1981 Good tip. I am faffing around with their own webui version and all their "examples" videos work flawlessly but with others you might use, it's hit or miss. I've had some horrendous "I have no mouth but I must scream" moments, haha.
🎵🎹🎶 💃 OpenAI My Sugar Papi by PEACHY da WHUUPi on UA-cam NOW.
Hey competition coming guys !
Used hedra