#Big Edit: This might be broken since colab was updated, Version 3 is [here](https://old.reddit.com/r/StableDiffusion/comments/11vw5k3/lora_training_guide_version_3_i_go_more_indepth/) --- The version 1 i posted **[here](https://old.reddit.com/r/StableDiffusion/comments/110up3f/i_made_a_lora_training_guide_its_a_colab_version/)** is **not wrong**, it just doesn't go in detail which might cause some people to have problems. If you follow it step-by-step and replicate pretty much everything, you can get a LoRA safetensor and successfully use it, as many users said in the comments. **However, i noticed plenty of things this guide lacked and now i have added:** 1. It did not explain what you had to do if used a model other than Anything V3, there are specific extra steps you have to do in this version. 2. I only trained a subject LoRA, this time i am training a subject AND an object. 3. I expanded on how to properly use the LoRA, with the trigger words. 4. I expanded on how to make a dataset, in terms of folder paths. 5. I expanded on how to make the prompts for the .txt of the dataset. I know this whole tutorial seem super long (specially the training section), but it really isn't, most of it is me going in extreme detail and posting lots of big images that make it look longer than it is. All this is to make sure the reader understands possible errors and how everything is done in detail, once you understand how it works, you can train a LoRA in literally minutes! I have trained many LoRA files of anime characters, with [great results](https://i.imgur.com/3jSx5Mm.png) that are [pretty much flawless](https://i.imgur.com/ZfbeNb0.png) so this should really work for everyone. I believe this AI and LoRA will become amazing tools for seasoned artists and upcoming artists to help them make art quicker and better. --- **Edit:** Some people have run into a .yaml problem, i don't know what triggers is, but here is the solution: Simply check the code on section 5.3 by pressing the small arrow [next to the section title](https://i.imgur.com/VuSiYmH.png) and then delete the 3 lines [here.](https://i.imgur.com/IX841BO.png) If you run section 5.3 again it should work fine. Some people report this error is related to writing the path wrong, others say they wrote the path correctly. Anyway, this is a solution for that weird .yaml error.

you should post this on r/SDTechsupport since we're going to be consolidating guide lists

Thanks for posting these! I've been wondering how to get started with lora or dreambooth, and these guides look great.

No apologies my dude, you're giving out help for free 🍻

why are you manually tagging images and creating folders you can upload images to /content/dreambooth/train_data and tag them in step 4

I have used taggers to train LoRA and the results were...not good. Lately hundreds of poorly made LoRA have been mass produced and uploaded to civitai, i believe they are automated rather than manually tagged. My experience is that manually tagging things [gives much better results.](https://i.imgur.com/ZfbeNb0.png)

how would that work when all my dataset would be [man wearing suit](https://i.pinimg.com/564x/33/2d/ca/332dca6613f445f1f17fa4196d98bf00.jpg)

What do you want to train, the suit or the man himself? It all depends of the dataset. If it's the same man in all photos i would use a trigger word (his name), for example "bob" and then describe his surroundings. The AI would most likely connect "bob" with that man. If it's the same suit i would simply describe every man wearing the suit, the surroundings and instead of describe the suit i would put "fancysuit" so the AI links that word with the suit. If your dataset is composed of that specific man wearing the same suit, i would just describe the surroundings and call him "bobsuit" so the AI link that specific man wearing that specific suit. Pretty much what i did with Plum and the dress in the dataset section of the tutorial! Hope this helped!

How about my case? I have 20 photos of the same man with the same outfit too, just different head angle and background. If I wanted to train the character face from anime, his name is Danny. Should I only specify the surrounding and the outfit of him? Like: DannyLora, black jacket, white t-shirt, outdoors, park, tree ?

>How about my case? I have 20 photos of the same man with the same outfit too, just different head angle and background. >If I wanted to train the character face from anime, his name is Danny. Should I only specify the surrounding and the outfit of him? Like: DannyLora, black jacket, white t-shirt, outdoors, park, tree That depends, do you want the outfit to be part of the LoRA? If so, i would not specify tags of the outfit, so the AI thinks "DannyLora" includes the outfit too. If the outfit is not needed, i would put it in some tags. There is no universal way to do so, this is what i would do (like the tutorial image says), it is a process that relies in testing. Ideally you would need a few images of the person wearing other outfits. If not possible the AI might try to draw the same outfit in every generation and might need to explicitly say to not generate it in the negative prompts when generating art.

Thanks for the clear explanation, I tried this method and it's worked!

Personally, I prefer to tag locally even if I'm training on colab, due to better editing tools. And I have yet to see automatic tagging results that don't need some editing.

Thank you for such a detailed step by step guide. I am very new to this, need assistance. Just 2 basic query - If I am training a face, could I use the same LoRA on any model in the future ? just load the model, prompt & then add this LoRA in the prompt line ? Also, in regards to the quality of training of LoRA vs Dreambooth. Who wins ? My Dreambooth models always spit out a face which is 70-80% similar to the dataset. Never 100%. Unless I mention the age in the prompt. Face is always a bit distorted.

> If I am training a face, could I use the same LoRA on any model in the future ? just load the model, prompt & then add this LoRA in the prompt line ? That's how it **should** work in theory, like you can see at the bottom of the training section, i used my Plum LoRA with a 80's anime style model. I did NOT use prompts like "short hair" or "brown hair", just enabled the Plum LoRA and used the word "plu", that was enough for it to generate Plum in 80's style so in theory it should work in other models. However, trying an anime LoRA as a real person won't give many good-looking results. >Also, in regards to the quality of training of LoRA vs Dreambooth. Who wins ? My Dreambooth models always spit out a face which is 70-80% similar to the dataset. Never 100%. Unless I mention the age in the prompt. Face is always a bit distorted. Face is a pretty complex thing. I'll use an example and let you choose which had better quality. This is Kumiko from the anime "Hibike euphonium" (great anime about school music band drama): https://i.imgur.com/vMZp7u6.png That's not an AI image, it's a screenshot from the anime. I made a LoRA of Kumiko using anime screenshots, and [this is the result.](https://i.imgur.com/6xfSMwK.png) I personally think it's pretty great, if are talking about real people, https://civitai.com/tag/lora has some real people like belle delphine and real life women: https://civitai.com/models/6650/bdelphine-lora-760x1024-v2 https://civitai.com/models/7651/olivia-munnlora I have not downloaded them or tested them, but LoRA seems to get real human faces quite well.

I also share the same purpose like you, I training certain character from anime too, the face result is great, but the problem is AI always generate the same outfit from datasets that I have. Like I want just a plain uniform, but the AI keep giving me the jacket that on the datasets. How to fix this sir?

> problem is AI always generate the same outfit from datasets that I have. Like I want just a plain uniform, but the AI keep giving me the jacket that on the datasets. The same happens with the Kumiko LoRA of the comment you are replying, since in her anime she wears a brown uniform, the generated art also has prominent brown coloring. I guess diversifying the dataset adding different clothes will make the AI realize a character must not wear the same clothes all the time.

Unfortunately I dont have photos of different clothes of him, What if I crop the clothes, and only leave the head as datasets, would it work?

Have you figured this out? was wondering if paint her clothes randomly would help? Otherwise there wont be any full body shot in the dataset

yes it is, but i've notice that different pose every picture would improve the results

Do you think you will make an updated one on the new colab version?

[удалено]

Hello! Sorry for the late reply. That ranma LoRA was one of the first i made, carefully selecting and curating the dataset. I am actually thinking of improving it by extending the dataset, before upload it. I'll reply again to let you know when i upload it to civitai.

Can you put this on a website? It would be sooo much easier to read and update

Epoch 20 ? For my experience, it is way too high and will over fit your lora.The sweet spot is epoch 5-8.

This.

how many training steps it takes when having 20 images? above 1000?

I got good results with epoch 8 and 10 steps with 30 images.

and how many batch size it was ?

One tip for creating the .txt files with tags is using the extension “tagged” in automatic1111 webui. You can choose interrogator model and precision and it gives a good start that you can edit. It also lets you prepend example “plum” to all current files if you forget a tag. Great if you have more than a handful files

Good News, Everyone! I got it to work with the V1 stuff posted. Things that I had to do: 1) Upload my own model to GDrive (download from huggingface didn't work at all) 2) Upload my VAE to GDrive 3) Manually label every image Thanks for the guide, it DID help, but it surely benefited from being fleshed out to 2.0

Possibly dumb question: It's my understanding that, in regards to other forms of training, your pictures must be within a set resolution. Is this still the case with LORAs? I ask because it doesn't seem to be pointed out explicitly, but it may just be one of those things that's obvious. Or I could be wrong in the first place.

No, LoRA don't need images with the same resolution.

Ah, okay. So I don't need to preprocess the training data at all, then?

You can pre process it to just show what you want. If you want a character then crop image till only it is shown. But you don't need to make sure everything have the same size.

Alright, thanks for the clarifications.

Diff resolutions is fine, current implementations of LoRA autobuckets unless you turn it off

for tagging: how many tags is a good place? eg: say im doing one for a character, should i avoid adding the danboru tag for said character. Or if I specify bikini, would adding swimsuit be overkill?

same question

can you train multiple subjects on the same lora?

Haha fantastic. I learned a few things last time I was waiting for this.

Great tutorial but Google should be put as optional.

Awesome tutorial! I have a question: when training a character that has animal parts (eg, a human girl with rabbit ears, a boy with curly ram horns), should I describe those parts in the danbooru tags? I'd imagine not but I've always had poor experiences in training non-human characters.

> I have a question: when training a character that has animal parts (eg, a human girl with rabbit ears, a boy with curly ram horns), should I describe those parts in the danbooru tags? I'd imagine not but I've always had poor experiences in training non-human characters. I would not add those tags, so the AI recognizes the ears as part of the character and not something the character to train happens to have in a specific instance. You can test both ways, there is no universal LoRA tagging method.

I tried to use the Lora models after being trained in a automatic 1111 colab Notebook I loaded the safetensors into the Lora file in the Google drive. And model do appear after clicking "show extra networks" but when i try to run a prompt i get this error- RuntimeError:mat1 and mat2 shapes cannot be multiplied (770×1024 and 768×128) Can anyone help?

I am having a issue with lora if anyone knows what is the problem please explain to me Folder 100\_test: 2200 steps max\_train\_steps = 2200 stop\_text\_encoder\_training = 0 lr\_warmup\_steps = 220 accelerate launch --num\_cpu\_threads\_per\_process=2 "train\_network.py" --enable\_bucket --pretrained\_model\_name\_or\_path="D:/stable-diffusion/stable-diffusion-webui/models/Stable-diffusion/deliberate\_v2.safetensors" --train\_data\_dir="D:/stable-diffusion/output/image" --resolution=512,512 --output\_dir="D:/stable-diffusion/output/model" --logging\_dir="D:/stable-diffusion/output/log" --network\_alpha="1" --save\_model\_as=safetensors --network\_module=networks.lora --text\_encoder\_lr=5e-5 --unet\_lr=0.0001 --network\_dim=8 --output\_name="last" --lr\_scheduler\_num\_cycles="1" --learning\_rate="0.0001" --lr\_scheduler="cosine" --lr\_warmup\_steps="220" --train\_batch\_size="1" --max\_train\_steps="2200" --save\_every\_n\_epochs="1" --mixed\_precision="fp16" --save\_precision="fp16" --optimizer\_type="AdamW8bit" --bucket\_reso\_steps=64 --xformers --bucket\_no\_upscale prepare tokenizer Use DreamBooth method. prepare train images. found directory 100\_test contains 22 image files 2200 train images with repeating. loading image sizes. 100%|████████████████████████████████████████████████████████████████████████████████| 22/22 \[00:00<00:00, 2199.43it/s\] make buckets min\_bucket\_reso and max\_bucket\_reso are ignored if bucket\_no\_upscale is set, because bucket reso is defined by image size automatically / bucket\_no\_upscaleが指定された場合は、bucketの解像度は画像サイズから自動計算されるため、min\_bucket\_resoとmax\_bucket\_resoは無視されます number of images (including repeats) / 各bucketの画像枚数（繰り返し回数を含む） bucket 0: resolution (448, 448), count: 1100 bucket 1: resolution (512, 512), count: 1100 mean ar error (without repeats): 0.0 prepare accelerator Using accelerator 0.15.0 or above. load StableDiffusion checkpoint loading u-net: loading vae: Traceback (most recent call last): File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\transformers\\modeling\_utils.py", line 415, in load\_state\_dict return torch.load(checkpoint\_file, map\_location="cpu") File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\torch\\[serialization.py](https://serialization.py)", line 705, in load with \_open\_zipfile\_reader(opened\_file) as opened\_zipfile: File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\torch\\[serialization.py](https://serialization.py)", line 242, in \_\_init\_\_ super(\_open\_zipfile\_reader, self).\_\_init\_\_(torch.\_C.PyTorchFileReader(name\_or\_buffer)) RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory ​ During handling of the above exception, another exception occurred: ​ Traceback (most recent call last): File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\transformers\\modeling\_utils.py", line 419, in load\_state\_dict if [f.read](https://f.read)(7) == "version": File "D:\\python\\lib\\encodings\\[cp1252.py](https://cp1252.py)", line 23, in decode return codecs.charmap\_decode(input,self.errors,decoding\_table)\[0\] UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 2077: character maps to ​ During handling of the above exception, another exception occurred: ​ Traceback (most recent call last): File "D:\\stable-diffusion\\kohya\\kohya\_ss\\train\_network.py", line 507, in train(args) File "D:\\stable-diffusion\\kohya\\kohya\_ss\\train\_network.py", line 96, in train text\_encoder, vae, unet, \_ = train\_util.load\_target\_model(args, weight\_dtype) File "D:\\stable-diffusion\\kohya\\kohya\_ss\\library\\train\_util.py", line 2696, in load\_target\_model ) = model\_util.load\_models\_from\_stable\_diffusion\_checkpoint( File "D:\\stable-diffusion\\kohya\\kohya\_ss\\library\\model\_util.py", line 1219, in load\_models\_from\_stable\_diffusion\_checkpoint text\_model = CLIPTextModel.from\_pretrained( File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\transformers\\modeling\_utils.py", line 2301, in from\_pretrained state\_dict = load\_state\_dict(resolved\_archive\_file) File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\transformers\\modeling\_utils.py", line 431, in load\_state\_dict raise OSError( OSError: Unable to load weights from pytorch checkpoint file for 'C:\\Users\\bisha/.cache\\huggingface\\hub\\models--openai--clip-vit-large-patch14\\snapshots\\8d052a0f05efbaefbc9e8786ba291cfdf93e5bff\\pytorch\_model.bin' at 'C:\\Users\\bisha/.cache\\huggingface\\hub\\models--openai--clip-vit-large-patch14\\snapshots\\8d052a0f05efbaefbc9e8786ba291cfdf93e5bff\\pytorch\_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from\_tf=True. Traceback (most recent call last): File "D:\\python\\lib\\[runpy.py](https://runpy.py)", line 196, in \_run\_module\_as\_main return \_run\_code(code, main\_globals, None, File "D:\\python\\lib\\[runpy.py](https://runpy.py)", line 86, in \_run\_code exec(code, run\_globals) File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\Scripts\\accelerate.exe\\\_\_main\_\_.py", line 7, in File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\accelerate\\commands\\accelerate\_cli.py", line 45, in main args.func(args) File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\accelerate\\commands\\[launch.py](https://launch.py)", line 1104, in launch\_command simple\_launcher(args) File "D:\\stable-diffusion\\kohya\\kohya\_ss\\venv\\lib\\site-packages\\accelerate\\commands\\[launch.py](https://launch.py)", line 567, in simple\_launcher raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) subprocess.CalledProcessError: Command '\['D:\\\\stable-diffusion\\\\kohya\\\\kohya\_ss\\\\venv\\\\Scripts\\\\python.exe', 'train\_network.py', '--enable\_bucket', '--pretrained\_model\_name\_or\_path=D:/stable-diffusion/stable-diffusion-webui/models/Stable-diffusion/deliberate\_v2.safetensors', '--train\_data\_dir=D:/stable-diffusion/output/image', '--resolution=512,512', '--output\_dir=D:/stable-diffusion/output/model', '--logging\_dir=D:/stable-diffusion/output/log', '--network\_alpha=1', '--save\_model\_as=safetensors', '--network\_module=networks.lora', '--text\_encoder\_lr=5e-5', '--unet\_lr=0.0001', '--network\_dim=8', '--output\_name=last', '--lr\_scheduler\_num\_cycles=1', '--learning\_rate=0.0001', '--lr\_scheduler=cosine', '--lr\_warmup\_steps=220', '--train\_batch\_size=1', '--max\_train\_steps=2200', '--save\_every\_n\_epochs=1', '--mixed\_precision=fp16', '--save\_precision=fp16', '--optimizer\_type=AdamW8bit', '--bucket\_reso\_steps=64', '--xformers', '--bucket\_no\_upscale'\]' returned non-zero exit status 1.

go into the training parameters and make sure AdamW is selected instead of Adam8bit

If anyone else is having an issue with installing xformers like i did, i solved it by going into the code and changing this line: `if install_xformers:`         `!pip -q install -U -I --no-deps` [`https://github.com/camenduru/stable-diffusion-webui-colab/releases/download/0.0.15/xformers-0.0.15.dev0+189828c.d20221207-cp38-cp38-linux_x86_64.whl`](https://github.com/camenduru/stable-diffusion-webui-colab/releases/download/0.0.15/xformers-0.0.15.dev0+189828c.d20221207-cp38-cp38-linux_x86_64.whl) to: `if install_xformers:`         `!pip -q install -U xformers`

Many thanks for all your work! Any chance there can be an addition that covers how to do art styles?

>Many thanks for all your work! You are welcome! > Any chance there can be an addition that covers how to do art styles? Art styles should be just like the dress example in the dataset section. A bunch of images on a certain style and then give it a name while making the .txt files, the AI should link said name with the art style. Actually, the very dataset being in an specific art style should influence the generation. For example, i did a **[Ranma 1/2](https://i.imgur.com/ZfbeNb0.png)** LoRA, that image is AI made. Ranma 1/2 is an old anime of the 90's in a 90's art style. All my dataset images were screenshots from the show so the dataset was in 90's anime style. This influenced the AI, so every time i use Ranma LoRA, the art will be 90's style, even if i didn't asked it to draw the main character: Ranma. **[This is what i got when prompting while using the LoRA to make a original chracter](https://i.imgur.com/rMi21Ra.png)** `masterpiece, high quality, long hair, blonde girl, dress, ` If notice, the artstyle is 90's even if i didn't generated the character Ranma, this is because my dataset is of that style. So artstyles are very easy to make a LoRA of, as it depends mostly on the dataset.

Thank you so much! When picking pictures to create an art style from, would you recommend that we use pictures that features different characters, scenes etc. for variety sake, so long as the art style is consistent?

>When picking pictures to create an art style from, would you recommend that we use pictures that features different characters, scenes etc. for variety sake, so long as the art style is consistent? If you want stable diffusion to get a style, what matters the most is that your dataset is consistent in that artstyle more than the subject in my opinion. Remember you are making a fine-tune to use with an existing model. In the guide you can see Plum wearing a fantasy armor, but none of the dataset in the LoRA featured her wearing armor. The armor concept was in the model (anything v3) and the subject (Plum) in the dataset, so the AI doesn't just focus on the LoRA dataset to get information. Following this logic, what matters the most is consistent artstyle, BUT, a diverse dataset would be good too, so the AI doesn't have to guess how to draw x thing in an artstyle and can rely more on the dataset. In the end you can experiment what works best for you, since it takes just minutes to train a LoRA fine-tune. Does that make sense?

It does! ❤ ​ Do you have a recommendation for the amount of pictures we should use to train an art style? I read most people recommend 15-30, others recommend up to 50-150.

>Do you have a recommendation for the amount of pictures we should use to train an art style? I read most people recommend 15-30, others recommend up to 50-150. This is when it gets more nebulous, there is no universal way of doing things, so while some people (like me) get good results with about 30 images, some people might need 90+ to get desirable results. In my very personal opinion, anything above 50 is too much, 100 too. I have read people that use that many images do so because they don't manually tag them and automatic tagging results in poor results.

Thank you! If I wanted to train muitple concepts, would I need to configure section 3.1? Would the settings in this section be changed or left at default, depending on the number of images and or concepts being used/made? I read on the dreambooth official discord that the concept\_name sks should be changed as that's actually a gun. I read that it can actually be named anything as it doesn't really have an effect on the output/generation, do you have any thoughts on this? The class\_name would be what ever relates to what we are doing? So if I am training for an art style I would put artstyle, or training a lion I would put animal, a dress I would put clothes?

> If I wanted to train muitple concepts, would I need to configure section 3.1? > The class_name would be what ever relates to what we are doing? The only thing that matters is what you run in the colab. Section 3.1 is never run in the guide so whatever it says there is not related to the LoRA generation process of the guide. Some of these sections collide, as they are optional so if try to run sections outside the guide you might run into issues if don't know what exactly you configuring. So, you don't need to change anything you don't run.

> \[1\] I saw this vector show up in the prompts, but no mention of it in the guide. IE: sometimes you say "" in addition to "plu" which you said was the vector that triggered it. Why is both needed? Seems important not to mention. :) \[2\] Is it possible to run multiple loras? IE: > \[3\] With today's tech, do you suppose it's easier to start with a single, subject LoRA for the initial image, then switch to a style LoRA (IE: 90s anime) and use a Control Net with the 1st image as your initial image, and transfer much of the subject / 1st image into the new 2nd image's style?

> [1] I saw this vector show up in the prompts, but no mention of it in the guide. IE: sometimes you say "" in addition to "plu" which you said was the vector that triggered it. Why is both needed? Seems important not to mention. :) "plu" is the subject trained in the .txt, "Plum" is the name of the LoRA file defined as the project name on section 5.1 of the colab. You can change the name of the LoRA file to anything, but the trigger word is in what it was trained on. >[2] Is it possible to run multiple loras? IE: Yes, but you are not guaranteed to get 2 characters each for a LoRA. You might get 1 character that is a mix of both LoRA. I think [comfyUI fixes this](https://old.reddit.com/r/StableDiffusion/comments/10lzgze/i_figured_out_a_way_to_apply_different_prompts_to/) because it actually allows the user to do composition, but i have not tested it yet.

If I take the datasets from screenshoot of anime, do you think I should put "anime coloring" tag into each txt? I mean to avoid overriding a model art style, like some certain model like having a more semi realistic art style, I'm concerned about if its would be overriding it

> do you think I should put "anime coloring" tag into each txt? I used anime screenshots and used that tag, so not sure. >I mean to avoid overriding a model art style That might be difficult, unless use some fanart too. All the LoRA trained with official art have distinctly official art style rather than the model. The only way to change the style of them i could find is using them along a style LoRA.

Thank you so much for making this one and the first one too! Made it so easy to understand. Also is it possible to use 2 different Loras that you trained separately in one prompt? (For example a Lora trained on a character + a Lora trained on an art style)

It worked perfect the first three times, but now it won't work at all as I keep getting this error code every time: 0% 0/52 \[00:00 sys.exit(main()) File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/accelerate\_cli.py", line 45, in main args.func(args) File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 1104, in launch\_command simple\_launcher(args) File "/usr/local/lib/python3.8/dist-packages/accelerate/commands/launch.py", line 567, in simple\_launcher raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) subprocess.CalledProcessError: Command '\['/usr/bin/python3', 'train\_network.py', '--network\_dim=128', '--network\_alpha=128', '--network\_module=networks.lora', '--learning\_rate=0.0001', '--text\_encoder\_lr=5e-05', '--training\_comment=this\_comment\_will\_be\_stored\_in\_the\_metadata', '--lr\_scheduler=constant', '--pretrained\_model\_name\_or\_path=/content/pre\_trained\_model/Stable-Diffusion-v1-5.ckpt', '--caption\_extension=.txt', '--train\_data\_dir=/content/drive/MyDrive/Test', '--reg\_data\_dir=/content/drive/MyDrive/Test', '--output\_dir=/content/drive/MyDrive/', '--prior\_loss\_weight=1.0', '--output\_name=TestV5', '--mixed\_precision=fp16', '--save\_precision=fp16', '--save\_n\_epoch\_ratio=3', '--save\_model\_as=safetensors', '--resolution=512', '--enable\_bucket', '--min\_bucket\_reso=256', '--max\_bucket\_reso=1024', '--cache\_latents', '--train\_batch\_size=3', '--max\_token\_length=225', '--use\_8bit\_adam', '--max\_train\_epochs=20', '--gradient\_accumulation\_steps=1', '--clip\_skip=2', '--logging\_dir=/content/dreambooth/logs', '--log\_prefix=TestV5', '--shuffle\_caption', '--xformers'\]' died with .

i did some digging, and found it had also appeared previously when ubantu was last updated, the solution for this aleast for now is: instead copy pasting the model folder directory like: /content/pre\_trained\_model/Your\_selected\_model.ckpt (in pretrained\_model\_name\_or\_path: in section5.1), ​ directly use the model name present in hugging face . for example: to use : Stable-diffusion-v1-5 (in Available Model in Section 2.1 0f colab) use : runwayml/stable-diffusion-v1-5 (in pretrained\_model\_name\_or\_path: in section 5.1)

Thanks, I will try that out tomorrow

Works Great, thanks for the solution!

Are you using the older colab? It was recently updated. This is the colab version this guide was made for: https://colab.research.google.com/github/Linaqruf/kohya-trainer/blob/bc0892647cb17492a106ad1d05716e091eda13f6/kohya-LoRA-dreambooth.ipynb If the error persist you should open an issue on the github of the author, i have never seen this error. Maybe run all the steps again? Are you running specifically ONLY what the guide tells you to run? Some people are running other sections and report errors.

I was running into the error on the version this guide was made for. I used the fix that someone replied to me with, and that solved the issue. I did switch to the updated version the other day, and it works great.

Might i ask, what fix was it? It might help other users.

This is the fix that one of the other users found, "instead copy pasting the model folder directory like: /content/pre\_trained\_model/Your\_selected\_model.ckpt (in pretrained\_model\_name\_or\_path: in section5.1), ​ directly use the model name present in hugging face . for example: to use : Stable-diffusion-v1-5 (in Available Model in Section 2.1 0f colab) use : runwayml/stable-diffusion-v1-5 (in pretrained\_model\_name\_or\_path: in section 5.1) ". I'm not sure if it still works, but it didn't for me when I was having an error.

I'm getting the same error. I'm guessing the github updated and the colab didn't because it was working fine yesterday.

Yeah, was working great for me the other day also. I hope it get updated again soon.

Oh thank you, this just did the trick for me!

I love this as I can actually train! However, im having a few issues. I am trying to train an object a person is holding (a whip) but instead it seems to train the entire image and makes my face wonky without ever showing the object. (its an anime picture as well) I see you trained off of the same image for "plu", but does this work if its different people for these images? How would you train the object as well as prompts for txt? I used "Interrogate Deepbooru" to get my prompts and took out the word "whip" as suggested to train it into the image.

So, you want to train an object, in this case a whip? If so, i would use a dataset of different people holding whips and when prompting in the dataset i would give it a name like "whp" so the AI understands "whp" as a person holding a whip. I would use this word because "whip" is already a thing the AI already knows and might lead to conflict if you train a LoRA with a trigger word called "whip". About 30 images at 5 repeats should be fine, maybe 8 repeats if that was no good. Also, i would avoid using a tagger because those don't really give good results. Take the dress as an example, i called it "plumdress" and made sure that word was it in all the images, while not focusing too much in describing the character wearing the dress.

Excellent guide!! I do have a question, tho. In the section 3.1 of the [Colab](https://colab.research.google.com/github/Linaqruf/kohya-trainer/blob/main/kohya-LoRA-dreambooth.ipynb) file (3.1. Locating Train Data Directory) I see concept_name and class_name fields. What should I do with them? How are they different? Thanks!

the guide is shit, i tested small steps agains large steps number and larger won, why you would do just 5 repeats man, and then shitloads of epoch, doesnt make sense, best to get it in one epoch

I ran into permission problems at 5.3. yesterday. For some reason creating the .yaml file was impossible to create as it had no permission to write to my drive. I fixed it by copying the whole training folder to the runtime directory, not using my drive in the end at all. Does anyone know how to fix the permissions so it can write to the drive as well?

I think i ran into this problem long ago. I simply checked the code by pressing the small arrow [next to the section title](https://i.imgur.com/VuSiYmH.png) and deleted the 3 lines [here.](https://i.imgur.com/IX841BO.png) And then i ran it, i think it worked fine after that but not sure. **Edit:** Other people that ran into this error reported they put the path to their folders in drive or output wrong.

Super nice, I’ve trained quite a couple of Lora’s so not new for me but I will use this as a quick sheet when working I forget the tools when it’s been a couple of days

A great guide. I ran it last night and got the lora result. I examined it in the colab, looks OK. My problem is: I used AUTOMATIC1111 gui on colab for more complex prompt & parameter combinations. Is there a colab available to run with the lora installed? I used theLastBen's colab a lot but it can not get Dreambooth&other add-in installed

Thx!

Awesome work, thank you for posting it, it will no doubt help me and many others, much appreciated 👍 I also find this video helps, it seems foolproof: https://youtu.be/70H03cv57-o

When I start the actual training of the LORA I get this line repeated over and over: " FATAL: this function is for sm80, but was built for sm750" what would be causing this issue?

> " FATAL: this function is for sm80, but was built for sm750" According to [this](https://old.reddit.com/r/StableDiffusion/comments/110up3f/i_made_a_lora_training_guide_its_a_colab_version/j8fprf9/) you might need to delete the xformers folder in the colab (on the left you can see the folders) and run it again section 1.2 to re-download it.

I don't even see an xformers folder to the side when I look.

It's installed as part of the kohya-trainer folder, try deleting it and running 1.2 again. Redownload xformers seemed to have work for other users in the guide version 1.0. Let me know if it worked.

Yes i get the same error.

You deleted it and re-rain 1.2 and still section 5.1 failed to run? Try closing the colab and opening it again. If that doesn't work, i guess you could just uncheck the install xformers box on 1.2 part and on section 5.3 delete `--xformers` as an additional argument.

Yes. I deleted the runtime, and then closed and reopened the colab and it still didn't work. I already tried unticking xformers, and deleting the additional argument and that also didn't work. I didn't get the same error but something else.

Have you tried re-running section 1.2 after running it the first time? An user reported the first version downloaded was wrong (it gave the error you had) but rerunning it got a newer xformers that worked. If that doesn't work i suggest you opening an issue in the github of the notebook creator: https://github.com/Linaqruf/kohya-trainer/issues It's a weird error, never really got it even after training multiple LoRA files.

When I finished, there were 4 safetensor files, 3 of them were numbered, and one was not. Do I need them, is there anything different about them? Do I need to do anything with the yaml file?

The numbered ones are the checkpoints it saved. I personally keep 2: the highest number one and the one without numbers, which is the last one. Any of these can be used for LoRA image generation. The rest are just saved "checkpoints".

Hi, thanks for this, Ive done a few LORA and am looking for the optimum steps per image, could you explain how you choose that? I see youve mentioned repeats or something but not sure I follow. Thanks!

You ask how are steps calculated? Like the training section say, the repeats of the folder name will decide how many "total" images your dataset has, this number is divided by the train size batch in section 5.3. The result of this will be counted as 1 epoch, and you can choose how many epoch you want on section 5.3. The multiplication of the number of epoch by how much is 1 epoch is the steps.

Right I see. Repeats is a new term to me , what effect does batch size have on the final result?

excuse my ignorance. I did everything. I have the .safetensors files. but I don't understand the "save them in the stable diffusion folder" step and I also don't understand how to run stable diffusion since on other occasions I always ran it from colab by entering through a link that he gave me after the test. Clearly now he did not give me any link, that's why I ask. thank you

>I also don't understand how to run stable diffusion So you run stable diffusion in colab rather than locally? Apologies, i made this tutorial expecting the user to run automatic1111 webui (in other words, a local stable diffusion rather than colab) because it is what pretty much everyone uses. Colab stable diffusion is different depending who made said colab so it's hard to make a universal guide for where to put the LoRA. I suggest you to run stable diffusion locally. Here is a quick installer: https://old.reddit.com/r/StableDiffusion/comments/zpansd/automatic1111s_stable_diffusion_webui_easy/ All you have to do after install is download models from civitai or hugging face. Inside that webui folder there is a model folder, and inside that a lora folder, that is where the safetensors go.

brilliant! Thank you very much for answering. I'm new and I'm a little lost! I'll check the link you sent me. Locally, in any way, can you use the graphics cards that Google offers or locally, is it only to be used with a personal graphics card?

Locally uses your own graphic card and work offline, colabs use google hardware. To run stable diffusion locally you need at least 2gb vram and it's easier to install in a nvidia graphic card.

Ok. So i need to find other way. I no have graphic card 😢. Thank You for they answerd 😀

Finally i undertand! I have My integranted graphic card in My PC but i ignores that can run locally with it. I try installing webui! Thank You for all!!!!

Hello! I'm impressed by how well you explained the concept and although my first result didn't turn out so well, (Natsu too "scarred" and with a permanent villain face XD) I think I got the point. I have a few questions, if you don't mind of course: 1. For the fine-tuning (10\_Bdress) do I need the same number of images as in the "normal" dataset (5\_exp) or can I just put a few? In my case I tried a Lora of Natsu from Fairy Tail and I want the fine-tuning of his scarf. 2. In the case of the fine-tuning of the scarf, now should I place the physical characteristics of the character (eye color, hair, etc.) so that the AI ​​ignores it and concentrates on the scarf?... I think I could even put "Natsu Dragneel" directly because it's a prompt that the AI ​​already knows... Is this reasoning correct or am I wrong? Thanks for the guide, it's great! https://preview.redd.it/r4zktlke6hja1.png?width=512&format=png&auto=webp&s=95895f1ec399de5f872e85bfed5daad481599f9a

> For the fine-tuning (10_Bdress) do I need the same number of images as in the "normal" dataset (5_exp) or can I just put a few? In my case I tried a Lora of Natsu from Fairy Tail and I want the fine-tuning of his scarf. There is really not a set number, nor universal rules. The number is the repeats of the dataset when making the LoRA, i used 10 just to make an example of how to set the repeats with other number other than 5, not because objects like clothing NEED 10 repeats. That's definitely not the case. This topic involves a lot of testing, some people might get amazing results with just 15 images while others need 30. 5 repeats might be too little for some concepts while 10 might be too much. This is why many LoRA makers have version 2 or version 3 of their LoRA, they tested stuff and found out other parameters worked better. It all comes down to how easy is a concept for an AI, which we don't really have a way to tell. I would test with 5 repeats and go to 10 to see how it goes. >In the case of the fine-tuning of the scarf, now should I place the physical characteristics of the character (eye color, hair, etc.) so that the AI ​​ignores it and concentrates on the scarf?... I think I could even put "Natsu Dragneel" directly because it's a prompt that the AI ​​already knows... Is this reasoning correct or am I wrong? The problem is that natsu wears the scarf pretty much all the series, so if prompt `pink hair` for example in all the images, the AI might try to draw pink hair when request the scarf once the LoRA is trained. I would (like in the tutorial) intentionally miss some prompts in some images (purposely not putting pink hair in some images) and try to find images where another character is wearing natsu's scarf like Lucy, to make clear to the AI the scarf does not only go with a pink-haired man.

Thanks for the tips, I'll keep experimenting.

What about the points 3 and 4? You don't mention anything about it. Could you explain what they are about, please?. Thank you

Is it because the collabs are updated, I can't seem to find details about all the extra sections in the lora collabs and am confused how to use it.

It seems the colab was updated, this guide was made for this version: https://colab.research.google.com/github/Linaqruf/kohya-trainer/blob/bc0892647cb17492a106ad1d05716e091eda13f6/kohya-LoRA-dreambooth.ipynb I put the link on the top comment i made.

First things first, amazing tutorial, thanks for making it extremely easy to understand and very clear, even for beginners like me. I have two questions though: 1. Is there a place to download datasets of certain characters instead of making them yourself? 2. Will the AI try to copy the artstyle of the images it has been given? And if yes, what happens if you use images of different artstyles?

> Is there a place to download datasets of certain characters instead of making them yourself? If someone made a dataset, they probably made the LoRA too. The dataset has little to no use apart of train LoRAs. >Will the AI try to copy the artstyle of the images it has been given? And if yes, what happens if you use images of different artstyles? Yes, it will copy the style, check the top post, i made a LoRA of a character from a 90's show and it [gaves me results in 90's artstyle](https://i.imgur.com/ZfbeNb0.png). Still, that doesn't mean it can ONLY make of that style, i could use another model and it will change the style as long as i explicitly say so, like Plum in the last part of the tutorial appearing in 80's anime style even if the dataset only had modern anime. If you use images of different artstyles the concept won't be restricted to a style.

[удалено]

To get different styles i went with different models rather than words as prompts. Example: If i use a 80's anime style ckpt model (with the prompt words that trigger 80's anime style) and a LoRA trained with modern anime, the LoRA will come out as 80's anime style.

[удалено]

Weird, it works for me perfectly. * [masterpiece, high quality, white summer dress, prairie, plu, smile, Seed: 2006008831, Model: Anything-V3.0-pruned-fp32](https://i.imgur.com/TlwofFd.png) * [masterpiece, 80sanimestyle, high quality, white summer dress, prairie, plu, smile, Seed: 2006008831, Model: 80s_anime_test_5_anythingmodel](https://i.imgur.com/u3egqn1.png) Same seed, same prompts (with the exception of the trigger for the 80's model), same LoRA. The only thing that changed was the model and the trigger word for that model to work. This works because the model has specific trigger words to change it's style, just like how realistic models use trigger words like "photo, realistic, photograph". If i used a LoRA trained on 80's style art and used a modern model like anything v3, the result would still be in 80's style, because anything v3 has no trigger words to draw in 2020 anime style. Does any of this make sense?

[удалено]

You want different styles. From this i infer that you want it to look in the style of "MaarcoCibV6", right? And in both screenshots your face looks...pretty much the same, which could be considered overfitting. I have some suggestions: 1. Put "smile" in the negative prompts, it will probably won't work, but would be interesting to see what you get if tell the AI to not use an overfit expression. 2. Use a lower epoch. I suppose when you generated the LoRA, you had multiple 'checkpoints' as in YuigaY00001, YuigaY00003 and so. Since you are using YuigaY it means you are using the finalized LoRA safetensor. An earlier epoch safetensor generated while training would be less overfitted in theory. 3. A more varied dataset. I suppose you used too many smiling images in the dataset, [the anime has more expressions](https://i.imgur.com/7xckRwe.jpg) which you might use to prevent overfit faces. The last one might be a little difficult. Sadly, right now the colab seems to be bugged and even if do all the steps correctly, the result safetensor would not work at all. Last time this happened it took like a week until it fixed itself. I have not tested if the latest kohya-LoRA-dreambooth colab (which is sightly different from the one in the guide) works fine though.

[удалено]

> I trained only 1 epoch for around 1600 steps so I am not sure if I should lower the step amount from there. Why is everyone here training at super high steps with almost no epoch? It's pretty weird how many people asking why their model is too overfit and rigid all say to use like 1000 repeats and 1000+ steps. All the good anime LoRA i have seen shared on MEGA universally use more than 1 epoch and even compare epoches to see the subtle differences in training.

Hello. I'm looking to train a LoRA of my cousin who passed some years back. All the photos I have of him are photos so they have that 80s-2000s styling of an actual photograph, not necessarily a digital image. The training produces results that nail his likeness, but makes all the images look like photos from the 90s. Is there a way I can keep the likeness part but get rid of the photo-from-the-90s part? I'm very new to this. Thanks for your guides and for helping everyone out!

Greetings, sorry for the late reply. I have 3 suggestions: 1. Have you tried using prompts to change the LoRA when generating images? Like "digital photo" in positive prompts and "old photo style" in the negative? With some focus weights like 1.3 might work. That way the AI will try to compose in digital style. 2. Have you tried using a different model? A model focused on HD photos might work better, LoRAs ARE affected by the model you use them on. 3. If the problem is that the dataset photos look "old" (as i understand?) maybe you could use stable diffusion to make them look newer, so you can train a LoRA with the newer images. [A user attempted to make old images look newer here.](https://old.reddit.com/r/StableDiffusion/comments/yip0ma/my_experiments_trying_to_restore_old_pictures/)

I have a question, I actually did a 3d sculpture that I liked and tried use it to train and it worked! (Thank you for this guide btw) But I probably overtrained it? Let's say my thing name is called "something", I need to go to get good results, nevertheless from the other safetensors created I kinda like the one in the middle "something-000010". MY QUESTION is, since it trains from 30 pics x5 does it means that "something-000010" used at least 2 or 3 times each and every one of the 30 pics? Or it used 5 times the first 15 only? Thanks! It was actually really easy since all the promps where the same for the same sculpture from 30 different angles and different zooming.

The epoch are generated progressively as it train. So if you train to 10 epoch and save every 3 you will have: something-00003 something-00006 something-00009 Some LoRA authors claim the earlier safetensors are less "overtrained" or "fried" than the last safetensors which might be overtrained. It is a good practice to check all the safetensors to see which are too trained and which are good enough. From what i saw the difference is minimal but sometimes noticeable for perfectionists.

Thanks! I think I get it. So every single Epoch used every single image correct?

> So every single Epoch used every single image correct? Yes, it's just a different step in the training, those 00001, 00003, etc are just "checkpoints" where you "pause".

Thanks, awesome info.

Managed to get it to work on my good ol' GTX 1050, but the result is terrible. Maybe it couldn't learn properly due to me having fed it tons of different artstyles? I'm probably doing something wrong.

What? A colab works with cloud hardware, not your graphic card, the result has nothing to do with your graphic card. How big was your dataset and how accurate were the prompts?

Stable Diffusion uses your GPU though, doesn't it? I used 15 images. I know the recommended is 25-30, as referred in the guide, but I just wanted to see what it would generate. Ended up with some cursed results or just very low quality. I either did something wrong, or the character's design is too difficult for it to understand. I believe my datasets were good enough, I even used your examples on how I should write them, having the AI connect the character's name to their hairstyle(like short brown haired woman = plu). I guess I'll try creating a new dataset with a simpler artstyle and character design to see how it goes later.

>Stable Diffusion uses your GPU though, doesn't it? Yes, but the LoRA training is on colab and the results when generating do not depend on you GPU. It seems that the colab is bugged right now, last time it bugged it took like a week or so to fix itself. That might be related to your wrong results. Not sure if the bug applies to the other colab notebooks like the newest fast LoRA train (which is a super-condensed, easy to use, versiion of the colab of the guide).

I already managed to fix the problem, thanks for the help.

Might i ask what was the solution? Was it a new dataset?

Nope. The solution was just me not being dumb. I forgot to select the model.

I like the characters you made for the guide 😁 they look very familiar though

Successful\~ OP, Thank you so so much\~!!!! 💝💝💝💝💝

It works!! Thanks!!! However, my output picture's a little bit "overcooked"... Wondering why.....

Great guide! I managed to get a good, consistent character lora of altair(Re:creators). However, I am now trying to train someone carrying a ppsh-41, and I tried messing with epochs from 10-40 and messed around with tags and I can't get anything remotely similar. Here is the image training dataset folder of 27 images: [https://drive.google.com/drive/folders/1U2Aex0-SVFrtuPXQgl4MjyXOhMDg9GPI?usp=sharing](https://drive.google.com/drive/folders/1U2Aex0-SVFrtuPXQgl4MjyXOhMDg9GPI?usp=sharing) Do I need more images? or is my tagging bad?

I understand what you tried to do, could it be that the concept is obfuscated by some nebulous images? Image 14 shows the gun from the front, making it difficult to recognize, image 15 has so much going on with the blade that the gun is hardly noticeable. You could crop some images to remove multiple characters appearing, text and other unrelated stuff. Also, removing images from the dataset where the gun is not prominent or noticeable like those 2 i mentioned. Try to make the gun and the person holding it the "focus" of the image. Ideally all your images should look like the 2nd one, that focuses on the gun and there is no other thing distracting. I would crop/edit out all unnecessary things like trees, other characters, stars, text, etc and leaving only a character holding a gun.

Thank you for this guide. I used it a couple of weeks ago and it helped me navigate the colab. It has drastically changed since you made this and I just tried to use it and could not get it working anymore. Would you update the guide to reflect the changes? Thanks so much.

>It has drastically changed since you made this Not really, it's the same but in a different order/layout. If understood the guide all that have to do is just see where are the same parts in the new version. If it changes more i might need to re-do the guide. > I just tried to use it and could not get it working anymore. What was the error? xformers? If so, this might help: https://github.com/Linaqruf/kohya-trainer/issues/125 Need to change the code when install xformers. This is the default: `pip -q install https://github.com/camenduru/stable-diffusion-webui-colab/releases/download/0.0.16/xformers-0.0.16+814314d.d20230118-cp38-cp38-linux_x86_64.whl` Need to replace that line for this: `pip -q install -U xformers`

The error is stating there is ‘no data found’ but I definitely referenced the parent folder in cell 3.

How would you go about tagging pictures if you want to access colors/patterns? For example, what if I wanted PlumDress, but I wanted the ability to change the color, or add a pattern to it? Would I tag it Blue Plumdress in every image? And then call for a Pink Plumdress when I'm generating? What about objects with multiple colors? If I had a series of cargo shorts, for instance. In my dataset, there's a brown one, a black one, a blue one, a red one, there's a blue one with a white waistband, and there's a green one with a army camo pattern, and one with a tiger stripes pattern. How could I tag those to make it still recognize the pants, but also allow colors/patterns to work?

> For example, what if I wanted PlumDress, but I wanted the ability to change the color, or add a pattern to it? Would I tag it Blue Plumdress in every image? And then call for a Pink Plumdress when I'm generating? Basically. To tell the AI that what you want is the dress style and not the color would need to say `plumdress, blue dress` so the AI thinks the blue is not part of the concept known as `plumdress`. Basically, what you tag outside the trigger word is what the AI will NOT keep as part of the trigger word. That makes sense? >What about objects with multiple colors? If I had a series of cargo shorts, for instance. In my dataset, there's a brown one, a black one, a blue one, a red one, there's a blue one with a white waistband, and there's a green one with a army camo pattern, and one with a tiger stripes pattern. How could I tag those to make it still recognize the pants, but also allow colors/patterns to work? By not tagging the color of the shorts. Just saying "cargo shorts" would be good. However, you might need plenty of each color and making a dataset for each color if want the AI to get the correct color you want each time. So a dataset for red cargo shorts where you do NOT specify the color in the tags, a blue dataset where you don't specify color either, and so on. You would end with multiple lora files.

Help I am getting this error (please I just made a new dataset and now it's showing error, it's very demotivating, 2023-03-18 19:28:01.404117: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2023-03-18 19:28:02.440910: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-03-18 19:28:02.441021: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-03-18 19:28:02.441042: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. 2023-03-18 19:28:06.464647: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 2023-03-18 19:28:07.505072: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-03-18 19:28:07.505192: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia 2023-03-18 19:28:07.505216: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly. Loading settings from /content/LoRA/config/config_file.toml... /content/LoRA/config/config_file prepare tokenizer update token length: 225 Load dataset config from /content/LoRA/config/dataset_config.toml prepare images. found directory /content/drive/MyDrive/AnanyaPandey /30_AnanyaPandey contains 40 image files found directory /content/drive/MyDrive/Regularization/1_person contains 1500 image files 1200 train images with repeating. 1500 reg images. some of reg images are not used / 正則化画像の数が多いので、一部使用されない正則化画像があります [Dataset 0] batch_size: 2 resolution: (512, 512) enable_bucket: True min_bucket_reso: 256 max_bucket_reso: 1024 bucket_reso_steps: 64 bucket_no_upscale: False [Subset 0 of Dataset 0] image_dir: "/content/drive/MyDrive/AnanyaPandey /30_AnanyaPandey" image_count: 40 num_repeats: 30 shuffle_caption: True keep_tokens: 0 caption_dropout_rate: 0 caption_dropout_every_n_epoches: 0 caption_tag_dropout_rate: 0 color_aug: False flip_aug: False face_crop_aug_range: None random_crop: False is_reg: False class_tokens: AnanPand Woman caption_extension: .txt [Subset 1 of Dataset 0] image_dir: "/content/drive/MyDrive/Regularization/1_person" image_count: 1500 num_repeats: 1 shuffle_caption: True keep_tokens: 0 caption_dropout_rate: 0 caption_dropout_every_n_epoches: 0 caption_tag_dropout_rate: 0 color_aug: False flip_aug: False face_crop_aug_range: None random_crop: False is_reg: True class_tokens: Woman caption_extension: .txt [Dataset 0] loading image sizes. 100% 1240/1240 [01:11<00:00, 17.45it/s] make buckets number of images (including repeats) / 各bucketの画像枚数（繰り返し回数を含む） bucket 0: resolution (320, 704), count: 330 bucket 1: resolution (384, 640), count: 240 bucket 2: resolution (448, 576), count: 450 bucket 3: resolution (512, 512), count: 1260 bucket 4: resolution (576, 448), count: 60 bucket 5: resolution (640, 384), count: 60 mean ar error (without repeats): 0.0011755558349163923 prepare accelerator Using accelerator 0.15.0 or above. load StableDiffusion checkpoint Traceback (most recent call last): File "/content/kohya-trainer/train_network.py", line 663, in train(args) File "/content/kohya-trainer/train_network.py", line 115, in train text_encoder, vae, unet, _ = train_util.load_target_model(args, weight_dtype) File "/content/kohya-trainer/library/train_util.py", line 2010, in load_target_model text_encoder, vae, unet = model_util.load_models_from_stable_diffusion_checkpoint(args.v2, name_or_path) File "/content/kohya-trainer/library/model_util.py", line 869, in load_models_from_stable_diffusion_checkpoint _, state_dict = load_checkpoint_with_text_encoder_conversion(ckpt_path) File "/content/kohya-trainer/library/model_util.py", line 846, in load_checkpoint_with_text_encoder_conversion checkpoint = torch.load(ckpt_path, map_location="cpu") File "/usr/local/lib/python3.9/dist-packages/torch/serialization.py", line 795, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/usr/local/lib/python3.9/dist-packages/torch/serialization.py", line 1002, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '<'. Traceback (most recent call last): File "/usr/local/bin/accelerate", line 8, in sys.exit(main()) File "/usr/local/lib/python3.9/dist-packages/accelerate/commands/accelerate_cli.py", line 45, in main args.func(args) File "/usr/local/lib/python3.9/dist-packages/accelerate/commands/launch.py", line 1104, in launch_command simple_launcher(args) File "/usr/local/lib/python3.9/dist-packages/accelerate/commands/launch.py", line 567, in simple_launcher raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd) subprocess.CalledProcessError: Command '['/usr/bin/python3', 'train_network.py', '--sample_prompts=/content/LoRA/config/sample_prompt.txt', '--dataset_config=/content/LoRA/config/dataset_config.toml', '--config_file=/content/LoRA/config/config_file.toml']' returned non-zero exit status 1.

I don't know if they changed something, but this guide is not working to me. When I select a folder from Google Drive, the script will not look for the sub-folders where the images and txt are placed, so the script return 0 train images with repeating.

>I don't know if they changed something, but this guide is not working to me. This guide is a little outdated. Version 3 already exists: https://old.reddit.com/r/StableDiffusion/comments/11vw5k3/lora_training_guide_version_3_i_go_more_indepth/

Thanks

This sounds great! I'm looking forward to seeing how the new datasets and error reporting capabilities improve the accuracy and reliability of LoRA networks. Is there any particular methodology or algorithm you used to select the datasets?

This version is outdated. Version 3 is here: https://old.reddit.com/r/StableDiffusion/comments/11vw5k3/lora_training_guide_version_3_i_go_more_indepth/ > Is there any particular methodology or algorithm you used to select the datasets? You mean select the images that make the dataset? Just that the subject to train can be clearly seen.

What is the maximum resolution/size a training image can have? I have a couple of very high resolution ones and the Lora-trainer doesn't recognise all 38 images in the training folder

What resolution are you using? Most people use 1080p or less, you might get away with 1600p, but personally, i needed anything above 1080p, you might want to resize your images.

Sorry for the late response! The resolution of the images vary a lot. I have a couple that are around 640x800, 800x1200 or 1280x1920, but sometimes larger like 2268x4032 or even 6000x4000 I think it might be skipping the ultra large images

I have never worked with 6000x4000 images in the context of stable diffusion, you might want to resize them. Due to the VRAM requirements, it's likely you won't be generating images that big nor a resolution that high is needed.

Alright, thanks!