L2: Cool Text 2 Image Trick in ComfyUI - Comfy Academy
Вставка
- Опубліковано 10 січ 2024
- In this Part of Comfy Academy we build our very first Workflow with simple Text 2 Image. But then I will also show you some cool tricks that use Laten Image Input and also ControlNet to get stunning Results and Variations with the same Image Composition.
Find my Workflow here: openart.ai/workflows/oliviosa...
#### Join and Support me ####
Buy me a Coffee: www.buymeacoffee.com/oliviotu...
Join my Facebook Group: / theairevolution
Joint my Discord Group: / discord
AI Newsletter: oliviotutorials.podia.com/new...
Support me on Patreon: / sarikas - Навчання та стиль
Update: I limited the Workflow to only what is shown in the video, so it is more focused on the Lecture
the ComfyUI can't find AV_ControlNetPreprocessor even after I fully install and update as the Manage tab said.
Same issue! Even on the openart server@@ceelentzpotter2110
@@ceelentzpotter2110 I have the same issue unfortunately🤷♂Solved it by and replacing AV_ControlNetPreprocessor with (right clicking on empty canvas space) Add Node/ControlNet Preprocessors/AIO Aux Preprocessor. Works like a charm👍
@@TheGalacticIndian thank you so much!
same here@@ceelentzpotter2110
your way is simple to follow , thanks for making this easy to understand , your doing a great job , thank you Olivio .
(( 4:24 )) Preview images DO save, but they save to the /temp directory instead of the /output directory.
The saved preview images also still include the Workflow metadata.
They remain there until ComfyUI is restarted, then that folder is deleted with all its contents.
everytime i watch a video with added functionality for comfyui, in this case it was the control net stages, i'm blown away with the potential of these workflows
ohh man, i wish you had shown the entire build flow, the suddenly jumping in progression it's very confusing, especially when you are learning, while you point at a node and explaining what it does my focus is struggling with figure out how we got there, how i can get to that point, where each of the lines goes, what is an VAE, what is a controlnode etc. meaning there's very little attention on what you are trying to teach, if you instead had build it together with us,, isolated the function like explaining controlnet before using it in a larger setting, it would be way easier to follow along and learn... sorry if i am a bit harsh in my critique, i am just a little dissapointed since i looked forward to follow this series, i enjoy your work and have a ton of fun following how you explore the realm of AI :), now i'll watch the video again a few times.
If using aux preprocessors just use aio preprocessor and choose midas or the other depth maps. Olivio here is using default controlnet. They can both do the same thing.
I don't think Node Preprocessor
This video is pure gold, thank you very much Olivio Sarikas.
I'm starting to see the benefit of comfy. Thanks Olivio! Excellent stuff
Nice! Really getting into this now, love how calm and easy your voice is, makes it so easy to follow.
Thanks a Billion Olivio! Helping Artists and Creatives become more efficient in technology
Thank you very much Olivio Sarikas
Olivio, thank you so much for your tutorials!
I like the way you explain it. It's very simple and straightforward. Thanks
Thank you very much for the lesson!
a lot of great tipps and hints, thank you!!
Merci pour ce partage !!!! Vous êtes génial !!!!
Definitely some assumptions left to be made by the people jumping in at this point, but I like learning that way.
Rather than download the workflow, I challenged myself to fill in the gaps and figure it all out on my own and learned quite a bit in the process.
Took me a while to figure out how to get the preprocessors and controlnet models installed - especially given my setup is a little different as I am running ComfyUI via the ModelSurge Auto1111 extension, and I have my models in a non-standard area outside of my WSL2 filesystem; but here I am 3 days later and I got it all working swimmingly, with extras!!!
Thank you for the wonderful tutorials @Olivio. You are appreciated. :)
Please share this apparently secret knowledge - how to get the preprocessors and install ControlNet models?
Thanks for your lesson, it helps a lot, thanks !
Very nice, i learned the basic use pretty quickly.
4:00 - It'd be helpful if you explained WHY one should use a custom VAE instead of a checkpoint's built-in. Actually, NO ONE ever explained this so I am wondering about hat until today 🤔
This. Anyone?
Custom VAE is by choise of course. Some models have baked VAE, so they don't require a custom VAE. But still you can use it. I'm new to Comfyui, but let's say you use Dreamshaper 6.31 VAE baked checkpoint in A1111; you can either use an additional VAE or you can set the VAE to Automatic.
@@coldbeyond That should be clear to everyone - the question, however, is: are there any ADVANTAGES in using different VAE's and why not using the baked-in ones?
@@MikevomMars That's such a great question and my guest is for instance maybe you have a VAE trained specifically on Cyclops from X-men and also the style (realistic, cartoon, 3D etc.) so it would make obvious sense that the custom made specifically for Cyclops would actually be better give better results. Am wondering if that's when the Adetailers (specific in recognizing specific part of an image or removing them) would come in yet I think those maybe called embedding and maybe totally different thing
Super useful, Olivio!
fantastico. obrigado por compartilhar conosco.
Nice vid, If you set the seed to the same fixed value on each sampler and set to progress by one each time. You are going to get more consistency between images.
5:37 I wrote the same prompts and almost got the exact results like you, how cool is that 😄
Thanks for your awesome video, my friend. Just playing around with nodes. Connected the latent output from the first Ksampler to the input latent from a second Ksampler and lowered a bit the denoise from this second Ksampler.Found that gives me some similar results to use contronet adapters. The question is ... using those contronet adapters gives us more consistency making the result look more similar at the original image ?. Thanks for all
great course, thank u so much. i was alright with autmatic 1111 but comfy was not easy to understand
Thank you for the video! It was tricky to understand how to install everything, but I did it
How did you install Controlnet preprocessor and the control_depth-fp16 model?
@@sethmunich4062 to instail model you need to put the file here ...\ComfyUI\models\controlnet and for controlnet preprocessor you need to open manager and find custom node ComfyUI's ControlNet Auxiliary Preprocessors from Fannovel16 Let me know if this doesn't work
No no no! for this level of information I must both like, comment and subscribe
so cool
this video explained perfectly the superiority of comfy ui , and made me give up on vladmandic automatic
Thank you for an understandable and fluent education. How can I preview the recorded images in the interface? They appeared in the section below in some of the trainings I watched.
definitely recommend getting the anything anywhere nodes to help keep workflows clean (you connect a node output to the anything anywhere node and it will automatically connect that output to all inputs of the same type (very useful with assigning models and VAE without having a bazillion lines in complex workflows).
Also recommend coloring the text-encode nodes : Green for Positive and Red for Negative (will make it a bit clearer at a distance which is what)
That's nice for your private workflows, but not good for when you want to share them or teach them, because then people can't see anymore where your cables are going. That's why i'm not going to use it in any of my workflows
@@OlivioSarikas It adds a toggle to show/hide the cables (even animates them depending in the type of node it is transferring data from and to)
like i said it is for the much more complex workflows that have like a massive logic group going on using multiple ksamplers and controlnets etc)
Thank you for these lessons Olivio , they are very clear and quite easy to follow for a beginner. There is a problem with the ControlNet Processor in the workflow, both launched on the clud and run locally. It doesn't seem to be working anymore, so after the first generated image the othes can not follow. can it be reaplced with something else?
Thank you for the video! Where can i get the ControlNet Preprocessor Node ?
hey! i love youre content and channel! Thank you for work and keep going!
How can I add the Controlnet Preprocessor since its not appearing on the list of nodes?
I had the same problem. But I think I solved it - in Manager in Custom Nodes I uninstalled and reinstalled the Art Ventura Custom Node used in this tutorial and all preprocessors appeared after refresh)))
@@Shisgara77 this seemed to work for me as well. instead of double clicking and typing the search field for controlnet preprocessor, i right-clicked art venture,->loaders-.controlnet preprocessor
@@Shisgara77 I had a tough time finding this node. I downloaded Olivio's workflow, expecting to have the node marked as missing, but no, it was there. It's just tricky to find in the AddNode dialog when searching for ControlNet. It's actually called 'AV_ControlNetPreprocessor' (I guess AV is Art Venture)
Update: I also needed to install "ComfyUI's ControlNet Auxiliary Preprocessors" before I could see more than 3 Preprocessors in the drop down list
What is Art Ventura?@@Shisgara77
Oh Art Venture!
I did it yeee!!!
It would be really helpful if you would show how to set up comfyUI properly. Like install the models and comfyUI itself. :)
this is the 2nd lesson, he did all that in the 1st
@OlivioSarikas I followed the first video and I'm trying to follow this video building the node but I don't have any checkpoints. What I'm missing
Where do I get the dreamshaper models?
Can't find them in the manager.
And what models would you suggest today?
There is another instance, where your workflow may be very important. Sometimes seemingly small prompts can change your image in a way you don't want to, because the model was trained that way and has some prejudicess. With your workflow you can circumvent the models prejudices and can approach your imagined outcome step by step.
I love your tutorials, thanks for that ! sooo Useful !
I have a issue and I don't know how to fix it :
When I try to open your graph, I have a error :
When loading the graph, the following node types were not found:
AV_ControlNetPreprocessor
This 3 packages are installed :
ComfyUI's ControlNet Auxiliary Preprocessors
ComfyUI_Comfyroll_CustomNodes
ControlNet Auxiliar
how can i add my own lora to the workflow
Problem i been having is not only to get consistent face but also a body, i guess depth map can help with that 👍
9:37 Apply ControlNet - the strength is tone down to 0.83, any special reason? What if I use 1 or more than 1?
Thank you Olivio for tutorial. What is difference between Batch_size and batch count.
Batch size : generate images in a all in one batch. Use more VRAM but is faster.
Batch count : generate a number of image one after the other. Use the same amount of VRAM but is slower.
You can combine them both.
@@lefourbe5596 thank you
hey Olivio thank you for the amazing tutorials! I am following this tutorial and I keep getting
Error occured when executing CheckpointLoaderSimple:
Error while deserializing header : HeaderTooSmall
how can i install checkpoint? I have a mac
I do not have the manager option visible, how can I fix this?
My KSampler keeps showing the preview. How did you hide it?
I'm following along but running into a speedbump @9:00. I can't figure out which node to load in place of the "Controlnet Preprocessor" Node. That node does not exist in my environment and I instead have a package called Controlnot Auxillary Preprocessors, yet I still can't find any node similar to the Controlnet Preprocessor Node with the CNET_Name output.
I ran into the same problem. With ControlNet Auxiliary Preprocessors installed, search for "MiDaS Depth Map" (or right click the workspace > Add Node > ControlNet Preprocessors > Normal and Depth Estimators > MiDaS Depth Map). This node will accomplish the same result as the tutorial (I just kept the default settings).
I'm not sure what Olivio used to get that ControlNet Preprocessors node. Let me know if you figure it out :)
use MiDas Depth Map as substitute for the ControlNet Preprocessor, the rest is the same
@@sunl3583 and where the output goes?
@@sunl3583 Thank you, this worked for me. Note that you also need to google the control depth fp16 model safetensors and download it to get it to work.
MiDas is not connecting with the VAE Decode@@sunl3583
I pressed control shift V but it just copied out but can't have these wire connections? Tell me why it doesn't work
UPDATE: It started working when I uninstalled some Aux...somethingsomething... preprocessor custom node I had installed some time earlier. Then restarted ComfyUI, then REINSTALLED it, restarted ComfyUI and now it works.
I get an error saying
"Error occurred when executing AV_ControlNetPreprocessor:
'resolution' is not in list"
on the lower-most workflow.
ooof control net preprocessor nodes missing, checked reddit, don't even have the folder, downloaded via pinokio... anyone have a fix
Following along with this and the Controlnet Preprocessor seems to be obsolete. Any suggestions for an updated workflow that achieves the same / similar output?
Same here. Trying to install comfyui-art-venture custom nodes but install is failing. Did you find a solution?
hi! I got an error whe i use the control net " deth-fp-16" 😢😢
How important is the spelling of your keywords when it comes to generating your desired output? I noticed in the video that you misspelled "digital paiting" but it did seem to work anyways?
don't know why the image is strange, eg: the car has more than four lights or wheels. is it the problem for the prompt text? appreciate for your kind reply ...
It says I don't have the AV_ControlNetPreprocessor node when I load the workflow?
Edit: It might be really nice for the beginners for you to note that you are using custom nodes & that openart lists custom nodes and links the repositories. It took me a while to it figure out!
I downloaded your workflow from OpenArt, but the images are coming out very blurry. I didn't change any settings.
Might be a dumb question as just learning...Why would you use a different VAE and not the one within the model? Just for variation?
p.s. great lecture...thank you
I can not find contronet Preproccessor in my comfy UI, can you please explain how that part is done ?
Someone mentioned, install extension by: comfyui-art-venture
And choose: AV_ControlNetPreprocessor
Also required: ComfyUI's ControlNet Auxiliary Preprocessors
It will allow to choose preprocessor models
i can't find the controlnet preprocessor - instead i sent it to MiDas Depth Map directly. The preview image looks similar, but it won't work after that.
you need the preprocessor, otherwise it is useing the color image and not a depth map. if you have the comfyui manager installed, you can install missing nodes with it very easily
@@OlivioSarikas The ControlNet Preprocessor got a red border indicating an issue, which is "depth_midas" is missing. How to fix it please? I run it in the cloud. Thanks and for the great tutorials too !
@@OlivioSarikas thanks! that worked for me!
@@OlivioSarikas I just can't get this to work. It shows RED in the manager.
I had the problem with the preprocessor but solved it via reinstalling art venture. But now, the only preprocessor I am able to select is "tile". I installed the models, also linked my A1111 controlnet-models and also copied them into "models/controlnet". Nothing helps. No preprocessors. Any ideas?
I had the same problem. I solved it ( thanks to the guys in the comments on Olivio's workflow download page).
you have to install, in addition of the art venture node you installed, the 'ComfyUI's ControlNet Auxiliary Preprocessors'
all the controlnet models will show after that.
Worked for me too. this needs more likes!@@lennoyl
so frustrating.
I'm stuck trying to get any ControlNet Preprocessor to work, and to no avail. one error after another, several installations and solutions, and nothing.
Solid tutorial otherwise
Gracias Olivioooooooo
Error occurred when executing AV_ControlNetPreprocessor:
A silly question, How do you download the Dreamshaper 8?
A - Inside comfyUI >>> Install Custom Nodes
B - Inside ComfyUI >>> Install Models
C - By visiting the Civit A.I
D - None of the above
*Please reply honestly 🙏✨
For people who have problem installing "ComfyUI's ControlNet Auxiliary Preprocessors" in comfyUI with this error:
"install failed: ComfyUI's ControlNet Auxiliary Preprocessors"
Solution: I disabled 'comfyui-art-venture' (if installed), then I installed 'ComfyUI's ControlNet Auxiliary Preprocessors' and re-enabled 'comfyui-art-venture'
As a side comment, it's useful to use a different seed for every instance of the Ksampler or you'll get practically identical results.
Hi Olivio, may I ask you to change your UI of choice to Invoke. It has everything, A1111 style UI, workflow UI, Cavas, almost everything.
Is there a real reason why AI youtubers prefer comfyui to invokeai?
I don't know whats different about my workflow (i built it by hand, not downloaded) but the images get different backgrounds, for example, for the asian woman the trees are japanese cherry trees, and for the african woman, the park has dirt paths and african looking trees. Your backgrounds look very similar.
I get better results if I reduce the denoising to 0.7 for the controlnet images. 👍
I have the same result.
I tried that and it made the backgrounds more similar but it made the woman's clothing less similar.@@JavierPortillo1
try changing the strength of the Apply ControlNet node to .83 like he has it in the video@@JavierPortillo1
This is obviously for pros lol. Where do you recommend learning more about SD in general?
I give up with this tutorial, AV_ControlNetPreprocessor missing , how to install, I try ComfyUI's ControlNet Auxiliary Preprocessors and didn't work, why would make video tutorial and skip steps... :/
Restart the server and then refresh your browser like it says to do. Worked for me after installing that and refreshing both.
@@MylesRobinette i think I finally fixed it, comfyui-art-venture was missing.
One bit of feedback, perhaps focus on sdxl (especially when you get to stuff like control net) 1.5 should really be religated to the past.
tbh it should but its not advancing as fast as 1.5 . 1.5 is still getting more support for most things.
@@aivideos322 I find 1.5 prompt crafting to be a nightmare compared to SDXL. Problem is that 1.5 gets mentioned in guides like this more than SDXL and then people learn 1.5 and are reluctant to go to SDXL so it becomes a self fulfilling loop.
i would say that SDXL has already suparssed 1.5 in everyway though
@@GamingDaveUK It has for sure, I love xl and use it A LOT, but still resort to 1.5 for animations, and for the controlnet support(tile in particular)
My best workflows use XL for bases and 1.5 for upscales/enhancing because of TILE controlnet. Somone needs to make this for xl... or it cant compete.
Oops, invitation not valid when try to join Discord!
cannot click ckpt_name nothing is happening 1:07
That means that you don't have any checkpoints in your model folder. you need to download them to your drive to be able to load them.
Are there any instructions where I can get them and is there a library where you can search for specific features for the models?@@OlivioSarikas
Are there any instructions where I can get them and is there a library where you can search for specific features for the models? I am new to the subject and find that this point is not clear. It would be nice to get another tutorial on this. Thank you@@OlivioSarikas
i'm having the same issue @shtogie5972 is having loading the "Controlnet Preprocessor"
Did you get this solved? I'd be happy to help.
I also wonder if you found a solution@@shtogie5972
@@shtogie5972 i did not get this resolved, would appreciate any help.
@@shtogie5972 Not sure about him, but I'm having the same issue. I even tried using Manager to "Install Missing Custom Nodes" and it fails that as well. (Still red on restart.) Did you find a viable workaround?
@@coryreeve1 I had the same issue, went to install custom nodes through manager and installed Fannovel16's "ComfyUI's ControlNet Auxiliary Preprocessors" you'll then get the controlnet preprocessor node option with MiDas Depth Map