Pateron supporter and was one of your followers asking for this for a few months. I finally had time this weekend to explore this, and just wanted to say a huge thanks! I finally have a lora that doesn't look so nasty and overtrained. I had improved results already using your character training from awhile back, but this really nailed it in, and the tip about using a famous artist helped. I actually found using a famous art movement (abstract expressionism) helped it to understand my work. This is massive, thanks so much for all the videos you have posted this year, for stable diffusion that have been game changing for me.
Finally, a video on style LoRA! Very thankful for it. My VRAM isn't so good that I can train/generate stuff on SDXL, but this guide is so general that I can rely on it even for stuff based on SD1.5. I suppose the only difference would be that I'll still be stuck on 512 x 512 resolution and some other minor differences. Thank you very much!
Hey, great video as always! As a video suggestion, could you make a guide to creating abstract concepts with Stable Diffusion? I'm trying to create a Solarpunk model that could work on different styles: photorealism, illustration, anime, oil painting etc.
Also been meaning to ask don't you find when training a character using another celebrity's name that the model is trained on that the model's image generation still picks up a lot of mannerisms and locations that the original celebrity was trained with despite the trained character Lora being used? Like hypothetically if you were a doppelganger for Ben Stiller and lived in the same universe as Zoolander and you decided to train your own face but instead of using Bstiller as the name of the Lora you use Derrik Zoolander because it's a great match and your AI model would certainly know who he was even after training it would seem to be that it would continually show you at modeling runways and dressed fashionably, and doing his poses even though the face would look less dramatic than Zoolander and more like typical Ben Stiller (Edited to add the Zoolander thing)
This is good but What I would really like to see, esp for patreon subscribers. is a tutorial on how to in-paint, and make hyper realistic stuff. or maybe even better is how to use comfy UI and all of its little addons and stuff.
This is great info...but I realized that repeats is nothing more than a way to describe how often you would be theoretically save a checkpoint/LORA because the minimum is one epoch. Also I never experienced that upping your batch size reduces training time. It reduces steps, but iteration time increases as well...so its just a trade off and the training total time stays the same! For Gradient Accumulation to not get weird, just make sure that Batch Size times Gradient Accumulation is not higher that your total images count! 🙂
Yes, the author does not seem to understand the point of repeats. It's there so you can balance your dataset. You have images in folders with the repeat number: woman20, dog10, sword15. You can use it to add more concepts to same LORA and/or balance the importance of the dataset. Want more dog than sword? Then set dog repeats higher.
@@digidope thanks for the info, I did not know that you can create multiple subfolders with different repeats and different subjects... I always thought you'd need another training pass to do so! I will do some testing!! 😁
Everything that is supposed to be a variable has to be named. I.e. you want to change eye colours? Describe the eye colours! But you can also train a bias on purpose. For example you don't like beards but your base model has a lot of beards and you need to use negative prompt to keep the guys shaved? Train the model with shaved guys and don't mention that they are shaved, so the model will start to default to beardless guys more. That way you just need to know what you want to be optional later and base your concept around that. If you are not persistent with these things the model gets confused and produces rubbish.
replaying in background to increase stats :) good to see you posting video once again. Hopefuly you were able to catch up few things when you were waiting for training results :)
Hi. thank you for the tutorial. I get this message at the beggining of the training. The following values were not passed to `accelerate launch` and had defaults used instead: `--num_processes` was set to a value of `1` `--num_machines` was set to a value of `1` `--mixed_precision` was set to a value of `'no'` `--dynamo_backend` was set to a value of `'no'` To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
TBH compressing pngs sounds like a terrible idea. In my graphic design workflow if there's any gradient on the picture, compressed pngs with these online tools won't pass unless it's an unimportant background asset.
All of your trainings seem to just be enhancing things that already exist in the model, what if what you want to train, say a concept (like a skateboard trick performed by a range of people), is not in the base model? Would we be better off training a model refinement instead of a LoRA? How does that differ from LoRA training?
Thank you for another excellent video. One question I still have is how can I change the trigger word(s)? Do they change when you replace the subject in the .txt files? Or is it hard coupled to the instance prompt? In your previous video (the character one) you had to prompt "sascha luss" to create the Milly character, which my brain doesn't really like 🙂
Hey! Thanks for all of your knowledge and support always. I have a quick question. What app do you use for your notes (the app where you save the sample photos of your epochs to compare and choose the best one ?
I used your settings right (SDXL-style.json) with 40 images and 20 reps, I think it overfitted very early on, would you lower the learning rates or change something else?
On the character training tutorial we used 10 epochs and you said it would take about 2 hours. I have a 4090 and it took about 50 minutes. Is this a difference in hardware speed or could it have been incorrect settings?
I cant get SDXL to work at all. In a1111 it just gives errors and can't load the checkpoint. Also didn't work at all with Easy Diff which I much prefer to a1111, where it just gives a weird collage of colors in the generated images. So far all the times I've posted on different videos asking for help, no one has ever replied. I'm close to giving up on all this.
Should I remove character tags when pruning/tagging? I'm training on an anime art style and not sure if I should include tags like "Erza Scarlet" for example.
What if I collect real photos of only 1 person, then train it to create real styles for animated photos? Is this the opposite of your way of doing things? Can it work?
Thank you for File Rename Basic. Windows rename is HORRIBLE. I used to switch over to my mac just to do proper renaming. No I can do it in windows, no problem... =]
I mean the first setting I would change is either resolution or network rank, that's the main 2 parameters that affect a lot the vram usage (with batch size)
I followed the tutorial and started training with 10 images and it takes whopping 12hours for 15 epochs with RTX3090 😮 I have all the same settings and 20 repetitions.
@@joelchamp1949 I did the whole 15 epochs for the 12 hours and when testing the results, the epochs 14 and 15 are the best. The style starts appear around epoch 8 but not as that strong. It depends on dataset and size on dataset. I had only 10 images, but if i have had 200 then less would have been enough.
@@digidope yeah cool. If I'm not mistaken doing 1 epoch would be the equivalent of your 15th epoch, some might say it's overtrained but I've been using it for art style so I want the maximum effect to carry over.
Will you make a tutorial on how to make a Lora for a particular pose / situation ? Btw, do you know how they train "photorealism loras" ? Or "details lora" ? I'm wondering if for each image in the training dataset they put a value for details or photorealism in the captions.
@@Aitrepreneur It's funny as they have always worked the same. For some reason nobody, including you didn't train LORA's first. But isn't the princible still different, so result should also be different? Negative embedings are still TI's not LORAs.
Thanks SO much for the content. Why do you shake the video? like at :53 It's kinda annoying of people trying to read. If you don't want people to read, then I would suggest not adding an image that people would want to read.
So basically... I went to the video in the first place to learn how to properly caption training images after being intimidated by the thought of precisely describing everything for a style... only to be told to suck it up. UGH.
Has any youtuber made a specific video about training objects? I need some advice, tips, and tricks for training objects. I am looking to train a specific style of cake. Can anyone point me in the right direction? I have made about 15 Loras about this specific cake, and none of them are perfect. Thank you for any help you can provide.
Nice tutorial as always but in 40 minutes there's way too long about character training, just for the sake of comparing. Maybe focus more on the essential that's only related to what people come in for : style. ;) Cheers.
![АЛАУДИНОВ у Скабеевой: эти их ВСУ нас НЕ ДОГОНЯТ 😁 [Пародия]](http://i.ytimg.com/vi/an0anqU4WGQ/mqdefault.jpg)
HELLO HUMANS! Thank you for watching & do NOT forget to LIKE and SUBSCRIBE For More Ai Updates. Thx
👋
How about reviewing oobabooga's new TTS extension?
Thank you very much! You have the only video on UA-cam about training LoRa for style, not for character!!! 👍🙏😻
thanks for the great content. you now are the first creator I support on Patreon.
thanks a lot again for this so great tutorial and the patreon you made in conjuction ! Great job Dude !
Pateron supporter and was one of your followers asking for this for a few months. I finally had time this weekend to explore this, and just wanted to say a huge thanks! I finally have a lora that doesn't look so nasty and overtrained. I had improved results already using your character training from awhile back, but this really nailed it in, and the tip about using a famous artist helped. I actually found using a famous art movement (abstract expressionism) helped it to understand my work. This is massive, thanks so much for all the videos you have posted this year, for stable diffusion that have been game changing for me.
Finally, a video on style LoRA! Very thankful for it. My VRAM isn't so good that I can train/generate stuff on SDXL, but this guide is so general that I can rely on it even for stuff based on SD1.5. I suppose the only difference would be that I'll still be stuck on 512 x 512 resolution and some other minor differences. Thank you very much!
what is that tool he's using where he's searching for what prompts sdxl understands? the "something" studio by stability ai
Finally you back ❤
I am ;)
Hey, great video as always!
As a video suggestion, could you make a guide to creating abstract concepts with Stable Diffusion?
I'm trying to create a Solarpunk model that could work on different styles: photorealism, illustration, anime, oil painting etc.
And if anyone has any suggestion on how to do it I'm interested in your guy's thoughts!
Yeah I plan on making more training videos ;)
@@Aitrepreneur Also make a video about how to fix hands.
Thanks for the update
😉
"No effort required whatsoever..."
That pretty much sums it all up.
Can you make one more on character training? The old one is kinda outdated, I followed your guide and my results were pretty good.
I did one a few weeks ago: ua-cam.com/video/N_zhQSx2Q3c/v-deo.html
Also been meaning to ask don't you find when training a character using another celebrity's name that the model is trained on that the model's image generation still picks up a lot of mannerisms and locations that the original celebrity was trained with despite the trained character Lora being used? Like hypothetically if you were a doppelganger for Ben Stiller and lived in the same universe as Zoolander and you decided to train your own face but instead of using Bstiller as the name of the Lora you use Derrik Zoolander because it's a great match and your AI model would certainly know who he was even after training it would seem to be that it would continually show you at modeling runways and dressed fashionably, and doing his poses even though the face would look less dramatic than Zoolander and more like typical Ben Stiller (Edited to add the Zoolander thing)
Please do another simplified video on how to locally install vicuna, previous videos seem to have become outdated.
Stable Diffusion content = interesting
This is good but What I would really like to see, esp for patreon subscribers. is a tutorial on how to in-paint, and make hyper realistic stuff. or maybe even better is how to use comfy UI and all of its little addons and stuff.
Upload into GPT and ask which generated image is best
Is there anyway to enque trainings in khoya? So that you could set a few up and have them train overnight.
What if I want to train a LoRA to generate comic pages with panels and all? What type of images should I use to train it?
What should the regularization images contain for style training?
This is great info...but I realized that repeats is nothing more than a way to describe how often you would be theoretically save a checkpoint/LORA because the minimum is one epoch. Also I never experienced that upping your batch size reduces training time. It reduces steps, but iteration time increases as well...so its just a trade off and the training total time stays the same!
For Gradient Accumulation to not get weird, just make sure that Batch Size times Gradient Accumulation is not higher that your total images count! 🙂
Yes, the author does not seem to understand the point of repeats. It's there so you can balance your dataset. You have images in folders with the repeat number: woman20, dog10, sword15. You can use it to add more concepts to same LORA and/or balance the importance of the dataset. Want more dog than sword? Then set dog repeats higher.
@@digidope thanks for the info, I did not know that you can create multiple subfolders with different repeats and different subjects... I always thought you'd need another training pass to do so! I will do some testing!! 😁
@@swannschilling474 Read the documentation. DOCS folder has series of README documents. There is also examples like: image_folder_structure
10:40 - What's that purple URL website? is it a local instalation of SD?
right? what is that?
When you run a test on all of the models to choose which you like, have you run that a few times to see if your best model stayed the best?
Is there a minimum VRam requiring for this?
18GB
Everything that is supposed to be a variable has to be named. I.e. you want to change eye colours? Describe the eye colours!
But you can also train a bias on purpose. For example you don't like beards but your base model has a lot of beards and you need to use negative prompt to keep the guys shaved? Train the model with shaved guys and don't mention that they are shaved, so the model will start to default to beardless guys more.
That way you just need to know what you want to be optional later and base your concept around that. If you are not persistent with these things the model gets confused and produces rubbish.
replaying in background to increase stats :) good to see you posting video once again. Hopefuly you were able to catch up few things when you were waiting for training results :)
Thanks yeah a little bit ;)
Why not upscale bucket resolution?
Hi. thank you for the tutorial. I get this message at the beggining of the training. The following values were not passed to `accelerate launch` and had defaults used instead:
`--num_processes` was set to a value of `1`
`--num_machines` was set to a value of `1`
`--mixed_precision` was set to a value of `'no'`
`--dynamo_backend` was set to a value of `'no'`
To avoid this warning pass in values for each of the problematic parameters or run `accelerate config`.
Thought you were dead. Welcome back.
Still alive and kicking! 😁
TBH compressing pngs sounds like a terrible idea. In my graphic design workflow if there's any gradient on the picture, compressed pngs with these online tools won't pass unless it's an unimportant background asset.
Whwre do you see that all the styls
Is it available for Radeon yet?
Could you send the link to the Text Replacer app?
The rambling is more than I can take.
All of your trainings seem to just be enhancing things that already exist in the model, what if what you want to train, say a concept (like a skateboard trick performed by a range of people), is not in the base model? Would we be better off training a model refinement instead of a LoRA? How does that differ from LoRA training?
Useful information. Thank you.
Glad it was helpful!
hi overlord
Hey 👋
Thank you for another excellent video. One question I still have is how can I change the trigger word(s)? Do they change when you replace the subject in the .txt files? Or is it hard coupled to the instance prompt? In your previous video (the character one) you had to prompt "sascha luss" to create the Milly character, which my brain doesn't really like 🙂
Hey! Thanks for all of your knowledge and support always. I have a quick question. What app do you use for your notes (the app where you save the sample photos of your epochs to compare and choose the best one ?
I used your settings right (SDXL-style.json) with 40 images and 20 reps, I think it overfitted very early on, would you lower the learning rates or change something else?
Yes you can change the learning to rate to something like 0.0009 instead
Ahoj, dík za návod ale mohl bys prosím zapnout automatické titulky jaksi nejsem zdělán v tvém jazyce. Zdravý odběratel z České Republiky.
Question though - if SD already knew what walking dead style was, why bother making another Style LORA for it?
On the character training tutorial we used 10 epochs and you said it would take about 2 hours. I have a 4090 and it took about 50 minutes. Is this a difference in hardware speed or could it have been incorrect settings?
I have 1660ti laptop, should I quit before I even try installing AI ?
This is great but I'd be interested in training some thing that SDXL doesn't already know about. Would I have to create a model for this?
Can you please make tutorials on anime characters training and anime style? I think such a video would be very helpful for the community 💟
It's the same thing as a normal style, I trained the jojo's bizarre adventure anime style using the same technique
@@Aitrepreneur Thank you very much for the answer. I will also try to create similar styles
I cant get SDXL to work at all. In a1111 it just gives errors and can't load the checkpoint. Also didn't work at all with Easy Diff which I much prefer to a1111, where it just gives a weird collage of colors in the generated images. So far all the times I've posted on different videos asking for help, no one has ever replied. I'm close to giving up on all this.
10:42 what is this website??
I really do need a GPU upgrade soon. Then again.. with 4090s melting im not sure i would be confident having it run overnight xD
the runpod template doesnt work agian pls help im a patreaon supporter
Should I remove character tags when pruning/tagging? I'm training on an anime art style and not sure if I should include tags like "Erza Scarlet" for example.
can you talk about AnimateDiff extension
What if I collect real photos of only 1 person, then train it to create real styles for animated photos?
Is this the opposite of your way of doing things? Can it work?
Thank you for File Rename Basic. Windows rename is HORRIBLE. I used to switch over to my mac just to do proper renaming. No I can do it in windows, no problem... =]
For some reason I can only train with my RTX 4080 if I use the lowvram preset... Is that right?
How much Vram is that 16Gb? You should be able to use a higher resolution at least
@@Aitrepreneur yes, 16gb... It fails because of so little amount of vram...
Idk if some of the settings can help without changing it from 1024px
I mean the first setting I would change is either resolution or network rank, that's the main 2 parameters that affect a lot the vram usage (with batch size)
Does clothing fall under the category of style?
Thank you for all of your hard work. Its mind simply amazing. The time to do this AND make this video must have been crazy tough!
I followed the tutorial and started training with 10 images and it takes whopping 12hours for 15 epochs with RTX3090 😮 I have all the same settings and 20 repetitions.
I do 1 epoch, usually works well enough for what I need.
@@joelchamp1949 I did the whole 15 epochs for the 12 hours and when testing the results, the epochs 14 and 15 are the best. The style starts appear around epoch 8 but not as that strong. It depends on dataset and size on dataset. I had only 10 images, but if i have had 200 then less would have been enough.
@@digidope yeah cool. If I'm not mistaken doing 1 epoch would be the equivalent of your 15th epoch, some might say it's overtrained but I've been using it for art style so I want the maximum effect to carry over.
in Runpod, when i trying to connect to ports, there is message "The port is not up yet", or a "Bad gateway"
in containers log there is - Syncing Kohya_ss to workspace, please wait... but it lasts forever
I love your videos, but wtf is up with the constant renaming of your videos?
Wtf
Probably trying different things to get better CTR
$$$
What happened for the reupload?
youtube demonitized and buried my last video so reup here again, let's hope it works this time...
Will you make a tutorial on how to make a Lora for a particular pose / situation ?
Btw, do you know how they train "photorealism loras" ? Or "details lora" ? I'm wondering if for each image in the training dataset they put a value for details or photorealism in the captions.
Yes more training videos coming ;)
I want to learn and teach people how to train literally ANYTHING!
@@Aitrepreneur Thanks !
Anyone know how can i do it in google colab please ?!
Any work around for running SDXL on RTX 2070 Super 8gb?😢
Use --medvram arguments and --xformers
got it thanks!@@Aitrepreneur
so textual inversion embeddings lost the game?
I mean kinda yeah, no real reason to use them anymore when you have LORA, they work the same way already
@@Aitrepreneur It's funny as they have always worked the same. For some reason nobody, including you didn't train LORA's first. But isn't the princible still different, so result should also be different? Negative embedings are still TI's not LORAs.
reupload?
Yes... youtube being youtube again demonitized and buried my last video...
allah hakbar ragazzuoauoli nuovo stile da rappresentare come intro in hdio
Rah-bee
Thanks SO much for the content.
Why do you shake the video? like at :53
It's kinda annoying of people trying to read. If you don't want people to read, then I would suggest not adding an image that people would want to read.
So basically... I went to the video in the first place to learn how to properly caption training images after being intimidated by the thought of precisely describing everything for a style... only to be told to suck it up. UGH.
How to change face on video ..
Has any youtuber made a specific video about training objects? I need some advice, tips, and tricks for training objects. I am looking to train a specific style of cake. Can anyone point me in the right direction? I have made about 15 Loras about this specific cake, and none of them are perfect. Thank you for any help you can provide.
Second comment 😅
A bit late I'm afraid xD
not a fan of the shaky avatar
Why are you training a style that SDXL already knows?
0:57 from left Volodymyr Zelenskyy - world leader😎❤
Nice tutorial as always but in 40 minutes there's way too long about character training, just for the sake of comparing. Maybe focus more on the essential that's only related to what people come in for : style. ;) Cheers.
Can you teach us how to train a Ai to only make basic pose drawings?
What the hell is up with these giant month long periods between videos now?