ComfyUI Multi Person Masking With IPADAPTER Workflow
Вставка
- Опубліковано 13 тра 2024
- In this ComfyUI video, we take 3 sets of images and incorporate them into our final output using IPAdapter and Masking. I am using images of Taylor Swift, Margot Robbie and Jenna Ortega, you can use any image you like. For best results you will need multiple angles of the face. Don't use blurry images or low quality ones for best results.
Workflow: (Copy and paste in to your browser then save as .JSON)
pastebin.com/raw/UBkQ66FX
Anything Everywhere:
github.com/chrisgoringe/cg-us...
IPAdapter:
github.com/cubiq/ComfyUI_IPAd...
GR Prompt Selector:
github.com/GraftingRayman/Com...
Ultimate SD Upscale:
github.com/ssitu/ComfyUI_Ulti...
ComfyRoll:
github.com/Suzie1/ComfyUI_Com...
Inspire Custom Nodes:
github.com/ltdrdata/ComfyUI-I...
Model: Juggernaut Reborn:
civitai.com/models/46422/jugg...
#0013
#ComfyUI #upscale #anthingeverywhere #ipadapter #inpainting #masking #imageenhancement - Наука та технологія
Nice tutorial! I've learned a lot!
Useful, lot of info, easy to understand, thanks.
Glad it was helpful!
Very useful tutorial, thank you so much!
Glad it was helpful!
I've been trying to use this with SDXL and I'm finding the images don't seem to be coming out with faces that look much like my reference pictures. Had 3 quick questions:
1. When you said SDXL wasn't working well for you in the video, did you mean just in general or in the case of this specific workflow in that the faces didn't seem to come out? 2: Do the reference images need to be close ups of the face, or will this work with full body reference photos as well? They can be jpegs right? 3: If I'm only making an image of a single person, it should be okay to use a mask that is just a transparent png image without the black part right?
EDIT: Actually, I think I have it working better now, just needed to adjust some of the models I was using... think I had the wrong IPAdapter model selected hehehe eeer.. I'm not the smartest. Thanks again!
Hi @n3bie, reference images work best if they are head shots. Not been having much luck with multi person generations with sdxl, works fine for single person but soon as I get a 2nd or 3rd person involved, it craps out, but that is using InstantID, works fine with FaceID
@@GraftingRayman Ah I see, I actually only built a single character workflow, but it's nice to have that info for when I try to expand it. I have to say tho, this workflow as a single character generator is working fantastically for me using SDXL. I deepened the reference images to about 20, maybe a third of those are close up portraits as you suggested, but after I was having trouble generating anything but close ups of the face, I threw a bunch more in the folder including full body poses, and I'm having pretty good results generating a lot of different poses that all look like the reference model. If anybody comes across this, I'm using Darker Tanned Skin off of CivitAI for a SDXL compatible LoRA, and it's working quite nicely. Thanks again Rayman!
Great work. I hope people will start noticing your channel. 😀
I hope so too!
I love it, but how do I make the three faces look the same after upscaling? Right now, they still look different
Are you running them through a ksampler with lower noise after the upscale?
Subbed, cool help, thanks
Awesome, thank you!
Awesome video sir. +1 Sub.
Very nice video. Useful tips.
Still got a few questions, if you don't mind.
Is there any reason why you use the nodes [ loraLoader Model Only + IPAdapter Model Loader + clipVision + insightFace ] instead of the IPAdapter Unified loader FaceID?
For the face reference, I see you use a batch load, but do not use a "Prep Image For ClipVision". I've seen this node used in many workflows. But maybe you did prepare manually your reference images. Did you make something special to your dataset, like resizing them or cropping them in the first place?
Anyway, I used to run a second Ksampler at 0.5 denoising. Didn't think about running a third one. I'll try that out, nice idea.
Thanks again, good job.
The unified loader sometimes does not work for me, not sure why, tried a few different workflows and they seem to crap out, swapped for the standard version which works a treat. I use crop face node prior to this workflow to save the faces only.
good done, please how can I make showing laser light for nodes
thats with anything everywhere node
Getting below error,
Error occurred when executing VAEDecode:
'VAE' object has no attribute 'vae_dtype'
File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
how much vram do you have on your gpu?
"When loading the graph, the following node types were not found:
GR Prompt Selector
Nodes that have failed to load will show as red on the graph."
it's inside ComfyUI\custom_nodes but not work
Help me please.
run the following command in your custom_nodes folder or use comfyui manager, "Git clone github.com/GraftingRayman/ComfyUI_GraftingRayman"
@@GraftingRayman "It's inside ComfyUI\custom_nodes but not work" - it's installed, but "Failed to load"
@@GraftingRayman maybe it the same problem:
```
DWPose: Onnxruntime with acceleration providers detected. Caching sessions (might take around half a minute)...
2024-05-20 16:46:37.3574522 [E:onnxruntime:Default, provider_bridge_ort.cc:1534 onnxruntime::TryGetProviderInfo_TensorRT] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\Ai\SD\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"
*************** EP Error ***************
EP Error D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:456 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************
2024-05-20 16:46:38.4698587 [E:onnxruntime:Default, provider_bridge_ort.cc:1534 onnxruntime::TryGetProviderInfo_TensorRT] D:\a\_work\1\s\onnxruntime\core\session\provider_bridge_ort.cc:1209 onnxruntime::ProviderLibrary::Get [ONNXRuntimeError] : 1 : FAIL : LoadLibrary failed with error 126 "" when trying to load "D:\Ai\SD\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnxruntime\capi\onnxruntime_providers_tensorrt.dll"
*************** EP Error ***************
EP Error D:\a\_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:456 onnxruntime::python::RegisterTensorRTPluginsAsCustomOps Please install TensorRT libraries as mentioned in the GPU requirements page, make sure they're in the PATH or LD_LIBRARY_PATH, and that your GPU is supported.
when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
****************************************
```
I have no any D:\a folder at all, by the way
delete the folder for the node and reinstall
Getting below error,
File "C:\Users\Mahesh\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_GraftingRayman
odes.py", line 8, in
from clip import tokenize, model
ModuleNotFoundError: No module named 'clip'
run the following: pip install git+github.com/openai/CLIP.git
Please help, i got message (IMPORT FAILED) GR Prompt Selector in manager
what is the full error?
Dear Sir @@GraftingRayman, here is the log: File "C:\Users\ccc\OneDrive\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_GraftingRayman\__init__.py", line 1, in
from .GRnodes import GRPromptSelector, GRImageResize, GRMaskResize, GRMaskCreate, GRMultiMaskCreate, GRImageSize, GRTileImage, GRPromptSelectorMulti, GRTileFlipImage, GRMaskCreateRandom, GRStackImage, GRResizeImageMethods, GRImageDetailsDisplayer, GRImageDetailsSave
File "C:\Users\ccc\OneDrive\Documents\ComfyUI\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_GraftingRayman\GRnodes.py", line 11, in
from clip import tokenize, model
ModuleNotFoundError: No module named 'clip'
you may try "pip install clip", that should install the clip package
damn bro. whats ur pc specs? FaceID runs so slow on my old ass machine
haha, that speed is done by editing, its still slow as anything
Please make a workflow for SDXL also, with InstaID for multiple people
the results are very crappy with instant id for multiple people, not worth the effort
can you share your mask.png files for us? It's very useful
you can use my mask create node instead, check my github link in the bio
@@GraftingRayman Thank you~
thanks, workflow link wont work
if you right click the link and save the file as a .JSON it will work
@@GraftingRayman No, still not working, apparently is a pastebin issue
@@RodiZai-pk9ty you can download it from my github github.com/GraftingRayman/ComfyUI_GR_PromptSelector/tree/main/Workflows