Automatic1111 - Depth Library for ControlNet!
Вставка
- Опубліковано 3 бер 2023
- Ever wanted to have a really easy way to generate awesome looking hands from a really easy, pre-made library of hands? Well, this Depth Library extension for ControlNet in Automatic1111 webui is probably something you'll be interested in!
And it doesn't just lend a hand - you can add your own depth maps to the library as well, making this an extension you won't want to miss!
== Links! ==
* Depth Library - github.com/jexom/sd-webui-dep...
* Automatic1111 Web UI - github.com/AUTOMATIC1111/stab...
* INCREDIBLE! ControlNet for Stable Diffusion (SMALL files!) - • How To Install And Use...
* ControlNet Extension - github.com/Mikubill/sd-webui-...
* Original ControlNet - github.com/lllyasviel/ControlNet
* How do I create an animated SD avatar? - • Create your own animat...
* Installing Anaconda for MS Windows Beginners - • Anaconda - Python Inst...
* Stable Diffusion Playlist! - ua-cam.com/play/PLj.html...
* Interested in adding things to your AI Art? Try these!
Dreambooth Playlist - • Stable Diffusion Dream...
* Textual Inversion Playlist - • Stable Diffusion Textu... - Наука та технологія
Sounds quite handy.
Very! 😉
These concise install and usage examples are fantastic. Helpful in building concept collections for reference.
Glad you think so!
Brilliant. Thanks for sharing your knowledge.
Waiting for this tutorial many thanks sir🙏. I was doing photobash and using impainting to fix hand ,tbh requires time but this solution is also effective and less time👌
Great 👍
The best simplest method. Very effective.
when it comes to preserving the exact same image, a *very* effective way that you didn't mention is the guidance start slider. It allows you to delay controlnet until the picture is already partially formed, meaning you can get the same picture, keep a high weight, and still get the hand you want, with very little effort.
But that probably will not be very consistent, as you're just gambling that the model will fix its mess in the last bit of steps. Could be wrong though, have you tried doing this?
So many options with ControlNet ;)
Thank you. I tried your suggestion and it works. I left the weight at 1, set the "guidance start" at 0.3, the "guess" unchecked, and the result is perfect.
I figured out that if you hold the SHFT or ALT key while dragging the handles to resize, you can skew the shape in all different ways, just like you do in Photoshop.
Thank you for always giving a good tutorial~
My pleasure 😊
Very useful, thanks for sharing this
This is a game changer for real
Thanks! 🙌
I also added a second controlnet with Canny to make this work for my illustration. It works!
Nice work!
This is absolutely crazy
Fascinating.
Very nice again...THX alot :)
Thank you too!
Thank you very much!
Very nice
Fantastic :)
Thanks, and save this tutorial in my tutorial playlist. THANKS.
Glad it helped!
thank you!
You're welcome!
Skyclad Squirrel approves this Nerdy Rodent video!
Rodents unite!
Thanks for all the great stuff you do! Can you do a video on training your own ControlNet model? I haven't seen anyone cover that as of yet.
Great suggestion!
game changer
Thanks.
You're welcome!
The negative prompt cracked me up
A funny Easter egg to be sure!
LOL didn't know you could flip them in the webui. I did a horizontal flip batch edit in my image editor for them all already :D.
lmao from your negative prompt: "photo that looks like it's drawn a child wing a bunch of blunt crayons to illustrate twins in an army vehicle from the past."
😉
I see there are some hands that go nicely with your pets, like your cats and tortoises. 😎
Yes they do!
any more hand depthmaps to add to the existing ones?
thanks ❤
You're welcome 😊
How do I bring in a picture I've previously saved to my computer?
with the same setting all I got is a blur where I in paint the hand part
Thanks for this tutorial. 3 questions:
1. What is Automatic1111 and why do "look down" on it (it can't hosted etc)?
2. Can you have ControlNet installed locally with no need for Automatic1111?
3. Is there an AI process/library for working with JUST depth maps without inpainting, so that you start with JUST the hand depth and ask AI to draw a pretty hand?
Links are in the description for 1 & 2. For 3 yes, you can just use the depth map and draw only a hand if you like!
any clue why inpaint in my case change whole image? I can't crush that nut - atm even swapped firefox into chrome, because by second time something require change flags in about:config for firefox. If no clue (anybody who read that message), that perfectly fine :) - just in case, everything 1:1 with AIOverlord and Rodent videos - still everything have, even when mask is drown.
I can't get this working at all. I've followed the instructions here, also remembered to keep the same seed and not changed the prompt at all, but it just doesn't work. I just get the original image again with the mask next to it.
I'm not seeing the ControlNet in Txt2Img or Inpaint. I only see it on nav bar, clicking "send to controlnet" does nothing either.
Edit: I didn't have ControlNet installed, I thought it was all in one. Fixed, thank you ^_^
doesnt' work for anything besides open hands. What am I missing?
Given the Controlnet input image has the same resolution as the output image, as you mask the inpaint area, does it take what exists on the same coordinates as the masked area on the Controlnet image, and use that for the inpainting?
I mean, if you upload just one big hand that covers the entire resolution of the Controlnet image, and you did the same thing, would that ruin it?
Give it a go and let us know what monsters you create!
not really working well for me. Unless hand is big, and has a simple gesture, it doesn't work. e.g. I've tried the "rock on" hand from the depth library templates - it doesn't produce satisfactory results with whatever settings I tried. I can set strength to 1 and indeed, sd will try to fit the hand into the depth mask, but what part of the hand is going to be there, is very random (e.g. I've spent over an hour trying to get fingernail appear in the right location).
Also looks like controlnet only works with 1.5 (it has 2.1 support, but I couldn't get it to work for some reason). I wonder if 2.1 has better results but until then Ill just hide hands.
I bring in the picture to the depth map library as a background, but when clicking on 'send to controlnet' it just sends the hand map
Yes, that’s right.
When I try this, I just get a completely different picture, plus the depth map as a separate picture, I change the weight to 0.15 and everything else is the same, except my ControNet isn't ControlNet 0, it's just ControlNet. Any ideas?
Disable controlnet and generate twice. If the image changes, it’s not controlnet - maybe a random seed or similar.
Legend!
By the way, have you managed to get all the upscalers to work? There arent, afaik, any video tutorials on the upscalers usable within AUTO1111 and how to get them all set up. Everything but RealERGAN2x fails for me, unless its just one of the non-AI models like Latent. I downloaded SwinIR models and they dont work either.
I mostly use latent and pnsr myself
Why do you need to load the preprocessor? I know it's got a depth thing in there.
You need to load a preprocessor in order to process things first. Hope that makes sense!
Anyone know a good place to get additional depth maps for the library?
Any of your 3d software (Maya, Blender, Houdid etc) should be able to provide depth maps of your models
@@NerdyRodent I'm more just looking for a zip file of curated hands that are ready to import, like something supplemental to what comes with the starting library.
For my workflow, it's way easier to just paste a hand into an image that's close to what I want than it is to pose, export, and place in my image
I miss your face in corner, but noticed all those animations and such instead so, something for something :) Please, add your profile thumbnail to video thumbnails - that single element will make your video way more noticeable
¿can you inpaint only with the inpaint model? or can you use anymodel? ty
Any model
You can use any model, but the inpainting model works very well for inpainting ;)
@@NerdyRodent Thanks keep up the great tips, sometimes people dont explain things for also begginers so ty for the answers here.
hello, i dont find the model in controlnet
Hi! You’ll need to download the depth model for controlnet. Check out this video for more - INCREDIBLE! ControlNet for Stable Diffusion (SMALL files!)
ua-cam.com/video/YephV6ptxeQ/v-deo.html
Hi, I don't understand. If I add this hand to the the "text to img", it draws completely different picture and doesnt care about my hand. Because it's text to img, there is every picture is different, why does it draws almost the same picture in your case?
I’m keeping the same seed so the picture is the same one, but with the new hand
@@NerdyRodent I tried to paste seed and pictures still different, just stile the same, face is recognizable and other stuff. But crop different, background, hand positions. etc.
@@IgorDyx same problem here
@@IgorDyx You wanna make sure to keep 100% of the info from the innitial image you generated the same. Seed, prompt, aspect ratio, model, etc.
If you have switched some settings around and dont remember them just drop the image into img2img and proceed with the other method. OR alternatively open the PNG Info tab and drop the image you generated with txt2img before.
That image carries all of the settings you used and once you put it into PNG Info you can click send to txt2img and click generate.
It will generate the exact same image as before since it will automatically adjust all of the settings for you (The only things I am not sure if it changes are the VAE and model).
Once you see that the same image is being generated proceed with adding in the hand with control net. As Nerdy suggested play around with the weight of the controlnet or alternatively try setting the guidence start to something between 0.15 and 0.3.
Goodd luck!
@@Keyaladron Thanks, it helped.
Hands and feet problem solved
Depth Library tab doesn't appear for me...Everything is up to date
Try restarting
@@NerdyRodent tks for answering. I just did, restarted SD, nothing.
@@venytonizidoro time to open an issue in the github repo then, I guess!
It's funny how the Ai has the same problem I have with hands in mu lucid dreams.
again it doesn't work for me. I add the hand to text2img, set the same seed, enable controlnet, but it generates a completely different image, regardless of my hand. Nothing like what's in the video. I am sorry.
No send to control net option, and manually uploading the img with the depth hand doesn't do much of anything. Sadly can't use this, back to Photoshop I suppose.
OMG
O’Brien held up his left hand, its back towards Winston, with the thumb hidden and the four fingers extended.
“How many fingers am I holding up, Winston?’
“Four.”
“And if the Party says that it is not four but five - then how many?”
“Four.”
It's not working very well for me at all. Very much less than perfect results even though it looks like it's referencing Control Depth. Never had great results with ControlNet to any degree.
It didn't work when using text2img, it creates still a hand with 6 fingers like the original.
But to get a perfect result in text2img without changing the original picture, I followed this steps which someone recommended here:
Under ControlNET in the text2img, leave the "weight" at 1, set the "guidance start" at 0.3, leave "guess" unchecked... The result is perfect, no need for inpainting.
Heh, some artists are REALLY not going to like this lol
I’m sure some artists will love it too! Especially once you have loads of depth maps 😉
You mean artisans. Real artists will take advantage of these tools.
DO NOT INSTALL IT! BREAK THE Stable Diffusion!
still, hands not feminine enough
Needs paws too 😉
Then find some images of feminine hands, and use controlnet to generate depth maps so you can add them to the library.
These hands look incredibly bad tbh. Like they're from some kind of cartoon of an alien
That looks terrible! Is there no other solution? Given the amount of work that needs to be done, the artificially created hands are too noticeable!
Cool