This is the most informative Video about LoRA creation on youtube. I learned so much about how a good dataset looks like and how to describe the image in order to let the AI learn. Marvelous work!
Yes great explanation thanks, I've been waiting to create a lora and have had issues with tryint to create the captions and data set. I'm interested in training a style
You can find the Training Dataset and workflow on my patreon for FREE: www.patreon.com/AIKnowledgeCentral Do you struggle with AI-Art? Head over to my Patreon to grab free stuff. No Membership needed! Ultimate Beginners Guide: www.patreon.com/posts/sneak-peek-alert-90799508 Free workflow guide: www.patreon.com/posts/get-your-free-99183367 Style Collection: www.patreon.com/posts/my-collection-of-87325880 I also offer one on one sessions: koalendar.com/e/1hr-1-on-1-stable-diffusion Happy creating Chris
If your outputs start to look too much like specific training images or capture excessive detail that wasn’t intended.Over-trained models might struggle to respond to prompts that deviate from the training data. If the model produces outputs that are too generic, vague, or don’t look like the specific character in your dataset, it needs more training.
Hi you need: images that represent the clothing you want to train on, including different angles, poses, and lighting conditions. If possible, use images with simple or uniform backgrounds to ensure the model focuses on the clothing rather than complex scenery. Remove blurry, low-quality, or overexposed images. Ensure that the dataset includes various body poses, especially if the clothing involves different parts like sleeves, collars, or drapes
hello, i have a small question. Say, I want to make a lora for certain character, and I want to use other's drawings of that character to make the lora. Would that make the lora have problems? I mean, having different art-styles. Because the original art-style of the character is not completely what I like, and I don't want to have a single style for that character
Hi! When creating a LoRA, it’s generally best to stick to one art style. If you mix too many styles, the LoRA might generate images that blend elements of those styles, leading to inconsistent results. The key to a good LoRA is to avoid both overfitting and underfitting: - Overfitting happens when the model learns too much from your dataset, leading it to simply replicate the training images without flexibility. - Underfitting occurs when the model doesn’t learn enough, resulting in outputs that stray too far from the intended subject or style. Ideally, a well-trained LoRA is "flexible" enough to retain its character identity while adapting to different scenarios. If you don’t want to stick to a single style, you can try grouping your dataset by style and training in a balanced way. However, keep in mind that achieving consistency across styles can be tricky, and some trial and error may be needed to fine-tune your dataset and training parameters
Thanks! Let’s say you want to train the beard on George Clooney. Is it possible control the Lora or prompt to use the beard afterwards - without everything looking like George Clooney?
I am not sure if i understand you correctly. If your training images only contain images of George Clooney its pretty hard for the ai to learn that its about the beard and not GC himself. so maybe use inpainting to keep the beard intact while changing the face to another man. Also you should have different camera angles and shot sizes. For example if you have only close Up images the lora will learn that cloesup shots are part of the lora. hope that helps.
Redclaire was your triggerword, i have forgotten mine. Isn't there a way to get to know my trigger word again? Can i also use the civitai lora model for forge flux? And if yes, how? Because for some reason i can't find my LoRA in the UI even after i have putted the file to the LoRA Models folder.
I don't think there is a way to find out the trigger word after. But for flux it seems the Trigger word is less important then the prompt itself. Not all of the Loras from Civitai are compatible with Flux on forge. And there is no way to know for sure apart form testing. For me the NF4 Flux model seem to be the most compatible. usually a Quick refresh should do it. Loras have to be in the models\lora folder.
You can train both in a single lora, this requires a more delicate image selection and the rendering of a lot more images. If the body has to be of nude form, then good luck. You may want to use a textured 3D Model for this and render these Images in a 3D Environment, like Blender or Unreal Engine 5. Hm... actually not a bad idea for a new video... *taking notes*. You also could take real photos, but make sure you have written consent from the person to use them.
Its absolutely great to use ComfyUI for this. I saw a workflow once on civitai that created Multiple different images for Sytle Loras. Maybe I will make a video about XYZ Plot in ComfyUI. Unfortunately its not so straight forward as i thought.
Hi! Can you PLEASE explain, how to use maden Lora on CivitAi generator? I push Create, choose chekpoint, push ADD more - and cant find my Lora, but its exist - i can see it in my profile, and its even have downloads. I dont understand, how to use it in generator
Is there a method to train for facial expressions (sad, mad, glad, etc) where the character is seen crying, screaming, laughing? All of the LoRAs have the same neutral expression.
By including images of facial expressions into the dataset (for sd 1.5 and SDXL there are facial expression loras), you can train your character lora on these. Originally i planned to show this in the video as well and include it, but the video itself is already 19 Minutes long so i got rid of that in the video script.
training on AI images.... so you don't just make bad hands, you make bad hands that are themself based on already bad hands =D ( you filter out the very shi**y ones, but still all hands are at best 90% accurate ) It's a nice shortcut for now, but imagine the 5th generation of AI images that are trained on AI and never saw a real hand. Only hands based on bad hands that are based on bad hands that are based on bad hands..... but still, thanks for the video. Can't quite get Flux training running on my machine, so using civitAI might be the easy route
That was not very informative. Next time please make a guide on how to train LoRA for an original cartoon character unknown for base model, for example some random anime character. Also please use free local trainer like kohya.
Thanks for you feedback. Since most of of my viewers have an RTX 2080 or lower ( i did several polls on this). i decided against local training. I am using the Kohya method in Civitai online trainer. This can be applied 1 - 1 to local kohya and should yield similar results.
Link to my amazing Sponsor Pic Lumen AI: bit.ly/4dLzmoh
This is the most informative Video about LoRA creation on youtube.
I learned so much about how a good dataset looks like and how to describe the image in order to let the AI learn.
Marvelous work!
Glad you enjoyed it!
Yes great explanation thanks, I've been waiting to create a lora and have had issues with tryint to create the captions and data set. I'm interested in training a style
You can find the Training Dataset and workflow on my patreon for FREE:
www.patreon.com/AIKnowledgeCentral
Do you struggle with AI-Art? Head over to my Patreon to grab free stuff. No Membership needed!
Ultimate Beginners Guide: www.patreon.com/posts/sneak-peek-alert-90799508
Free workflow guide: www.patreon.com/posts/get-your-free-99183367
Style Collection: www.patreon.com/posts/my-collection-of-87325880
I also offer one on one sessions:
koalendar.com/e/1hr-1-on-1-stable-diffusion
Happy creating
Chris
Thank you!! I don't know what you are looking at when you said over or under trained.. how do you tell? knowing what to look for would help a lot! : )
If your outputs start to look too much like specific training images or capture excessive detail that wasn’t intended.Over-trained models might struggle to respond to prompts that deviate from the training data.
If the model produces outputs that are too generic, vague, or don’t look like the specific character in your dataset, it needs more training.
could you please give tips for dataset preparation for Flux Lora training for cloth ?
Hi you need: images that represent the clothing you want to train on, including different angles, poses, and lighting conditions. If possible, use images with simple or uniform backgrounds to ensure the model focuses on the clothing rather than complex scenery. Remove blurry, low-quality, or overexposed images. Ensure that the dataset includes various body poses, especially if the clothing involves different parts like sleeves, collars, or drapes
hello, i have a small question. Say, I want to make a lora for certain character, and I want to use other's drawings of that character to make the lora. Would that make the lora have problems? I mean, having different art-styles. Because the original art-style of the character is not completely what I like, and I don't want to have a single style for that character
Hi! When creating a LoRA, it’s generally best to stick to one art style. If you mix too many styles, the LoRA might generate images that blend elements of those styles, leading to inconsistent results.
The key to a good LoRA is to avoid both overfitting and underfitting:
- Overfitting happens when the model learns too much from your dataset, leading it to simply replicate the training images without flexibility.
- Underfitting occurs when the model doesn’t learn enough, resulting in outputs that stray too far from the intended subject or style.
Ideally, a well-trained LoRA is "flexible" enough to retain its character identity while adapting to different scenarios. If you don’t want to stick to a single style, you can try grouping your dataset by style and training in a balanced way. However, keep in mind that achieving consistency across styles can be tricky, and some trial and error may be needed to fine-tune your dataset and training parameters
Thanks! Let’s say you want to train the beard on George Clooney. Is it possible control the Lora or prompt to use the beard afterwards - without everything looking like George Clooney?
I am not sure if i understand you correctly. If your training images only contain images of George Clooney its pretty hard for the ai to learn that its about the beard and not GC himself. so maybe use inpainting to keep the beard intact while changing the face to another man. Also you should have different camera angles and shot sizes. For example if you have only close Up images the lora will learn that cloesup shots are part of the lora. hope that helps.
Redclaire was your triggerword, i have forgotten mine. Isn't there a way to get to know my trigger word again? Can i also use the civitai lora model for forge flux? And if yes, how? Because for some reason i can't find my LoRA in the UI even after i have putted the file to the LoRA Models folder.
I don't think there is a way to find out the trigger word after. But for flux it seems the Trigger word is less important then the prompt itself. Not all of the Loras from Civitai are compatible with Flux on forge. And there is no way to know for sure apart form testing. For me the NF4 Flux model seem to be the most compatible.
usually a Quick refresh should do it. Loras have to be in the models\lora folder.
In automatic1111 you were using SDXL model to generate root model images?
Yes, I used the SDXL model in automatic1111 to generate those base images.
what about the differences between a face-lora and a full-body-shape-lora ?
You can train both in a single lora, this requires a more delicate image selection and the rendering of a lot more images. If the body has to be of nude form, then good luck. You may want to use a textured 3D Model for this and render these Images in a 3D Environment, like Blender or Unreal Engine 5. Hm... actually not a bad idea for a new video... *taking notes*. You also could take real photos, but make sure you have written consent from the person to use them.
@@AIKnowledge2Go that kind if video would be really sick man
What about a xyplot with comfy ui to prepare all the batch images? Thoughts...
Its absolutely great to use ComfyUI for this. I saw a workflow once on civitai that created Multiple different images for Sytle Loras. Maybe I will make a video about XYZ Plot in ComfyUI. Unfortunately its not so straight forward as i thought.
Hi I am a noob at this stuff. Will the LORA's we make also work with Forge UI?
Yes! But you need the according base model. For flux I only tested nf4 flux v2 version of flux and the "normal" flux dev. GGUF is untested.
Hi! Can you PLEASE explain, how to use maden Lora on CivitAi generator? I push Create, choose chekpoint, push ADD more - and cant find my Lora, but its exist - i can see it in my profile, and its even have downloads. I dont understand, how to use it in generator
Good Question really... i hardly use the generator i am afraid. I whished I could help you. Maybe ask in the Civitai Reddit group?
Is there a method to train for facial expressions (sad, mad, glad, etc) where the character is seen crying, screaming, laughing? All of the LoRAs have the same neutral expression.
By including images of facial expressions into the dataset (for sd 1.5 and SDXL there are facial expression loras), you can train your character lora on these. Originally i planned to show this in the video as well and include it, but the video itself is already 19 Minutes long so i got rid of that in the video script.
training on AI images.... so you don't just make bad hands, you make bad hands that are themself based on already bad hands =D ( you filter out the very shi**y ones, but still all hands are at best 90% accurate ) It's a nice shortcut for now, but imagine the 5th generation of AI images that are trained on AI and never saw a real hand. Only hands based on bad hands that are based on bad hands that are based on bad hands..... but still, thanks for the video. Can't quite get Flux training running on my machine, so using civitAI might be the easy route
This seems really dumb to use renders for this.
You are absolutely right. I will make an "advanced" video of this training guide and fixing hands will be on of the talking points.
Youre wrong. Yiu can always generate images of your character using the "perfect hands" lora.
Step 1: Make digital girlfriend
Step 2: There is no step 2
Step 3: Enjoy your virtual dates without the awkward small talk! 😂
german, correct? ;)
I just can’t hide it 😂 You’ve got a good ear!
That was not very informative. Next time please make a guide on how to train LoRA for an original cartoon character unknown for base model, for example some random anime character. Also please use free local trainer like kohya.
Thanks for you feedback. Since most of of my viewers have an RTX 2080 or lower ( i did several polls on this). i decided against local training. I am using the Kohya method in Civitai online trainer. This can be applied 1 - 1 to local kohya and should yield similar results.
First learn how to make your video correctly working on you tube
Thanks for your feedback, if you point me to where the problem lies, then I may be able to improve on it in the future.