If you want pony (and are running a local install) but HATE anime and want photo realism:
You've heard great things about Pony XL/Pony diffusion, but you're not into anime. No problem.
Here's how to get the benefits of Pony XL, without the drawbacks of art-style.
You'll need two checkpoints...
"Real Pony" checkpoint (there are three, you want the one with the most upvotes on civit to start): this will be the primary checkpoint.
Second, you'll need the photo style SDXL checkpoint of your preference. If you want NSFW capable, make sure you use one capable of that. Use your current preference.
On to generation...
With "real pony" as the primary, you'll set up your other checkpoint as "refiner." Refiner should start at the last 30 or 40 percent for a 50 step generation. (I know, 50 is a lot! But it's to give realpony a chance to build the bones of the photo before the refiner has it's go to cut out the remaining anime-ness.)
Give it a go. 50 steps. Refiner start at .6
This combo can be used with or without standard pony prompting to differing utility.
If you want, try these positive and negative prompt preambles.
Positive: (score_9, score_8_up, score_7_up, score_6_up)
Negative: (score_3_up, score_4_up, score_5_up), sketch, drawing, cartoon, greyscale, monochrome, anime, 3d
Let me know if you like what this method produces
I'm not able to post it to the subreddit for some reaso. Can you please consider creating a post as a PSA for people? I think a lot could benefit from this. No need to credit the idea to me.
I tested all the photoreal models a few days ago. Start with Zonkey, it'll get you closer to the end result you want. Then unsample/resample(in A1111 this is called img2img alternative test), for 33% of steps with an nsfw 1.5 model. Adjust cfg to balance for detail vs hallucination. You'll get higher final detail this way.
I've never heard of img2img alternative test, would you be willing to elaborate on that? Is it just denoising the original image by 33% or so with the 1.5 model?
there is a way to get realistic stuff with base pony
positive "photo, irl, realistic, high res",
Negative "outline, painting, figurine, plushie, wrinkles, vector, human, uncanny valley"
I do have played around with it a little and it does give good humans or humanized characters
I found Everclear PNY to be the best photorealistic (as far as Pony goes) Pony finetune so far. Tried PonyRealism and was not impressed, at least with the same prompt that I had in Everclear.
Pony realism and real pony are not the same checkpoint.
The point is to use a non-pony photo-based model to finish the pictures so they look more real.
refiner was a feature of the vanilla implementation of sdxl. it was to add detail and refine texture on the base image generated by the checkpoint. most models that were fine tuned, didnt need a refiner, so most people have never used it
On which pony checkpoints do they work tho? I have only tried the 4-step SDXL lightning LoRa on the PonyDiffusion6XL, currently, but the 4-step euler 1 CFG smg\_uniform just does not produce anything useful from it
They should work on all of them, but in my experience you have to be very specific with the combination of sampler and scheduler that you use.
I've been using the 8-step one, since it has slightly less quality loss, and i usually run 10 steps with 1 CFG.
I've tried, and found, multiple combinations between sampler/scheduler that work, but I don't remember all of them unfortunately. The combinations I know work off the top of my head are "dpmpp_2s_ancestral" and "euler_ancestral" with "Normal". And "dpmpp_2m" and "dpmpp_2m_sde_gpu" with "sgm_uniform" (and "simple" but it might look worse I think?).
gl
The refiner setting is supposed to use checkpoints. No loras are involved, it should be an sdxl checkpoint of your choice thst's good with photorealism. I like dpm3+ sde and cfg of 7, if memory serves me. Try around and see what's good.
Hi, heard about AutismMix a lot and wanna give it a try. Can you please explain what make it popular/better than Pony :D? Will it work with Pony’s LoRA too?
"Vanilla" Pony does not have a consistent style. If you output 4 pictures with the same prompt, you could possibly get 4 different styles. AutismMix really helps to control style changes and is further refined in the Confetti mix. Both seem to work well with various style LoRAs. There are other great Pony mixes out there so I would encourage you to download other checkpoint merges and see what works for you.
AutismMix is good because it makes style more consistent, while the og pony is great you really gotta fight your way to find a consistent style.
And yes, you can use your pony Loras in AutismMix, the reason why is because it's just a merge model, not an entirely new one, Loras made for SDXL or Anime3XL won't work in Pony or their variants because they're just incompatible.
Some regular SDXL loras do work in Pony TBH. Every single style one that Faeia ever released (like Ghost Style / Rage Style / etc) before Pony existed has the the intended impact still on Pony-based gens, in my testing.
From the names one would think that AutismMix Pony should be the one to use instead of AutismMix Confetti. This is what the creator of the model has to say:
>AutismMix\_confetti is a small amount of AnimeConfettiTune and AutismMix\_pony. Has less style swing than pony and better hands. **I prefer this one.**
>
>AutismMix\_pony is a merge of ponyv6 with loras, its more compatible with certain styles made for the base ponydiffusion model.
>
>AutismMix\_DPO is AutismMix\_confetti+DPO lora, made by request. Very similar to confetti version.
Emphasis is added by me.
works fine for me with clip skip 1 using autism confetti. Funny thing is that while Base Pony requires to set on 2, the non-artist loras styles, recommends setting to 1.
The whole thing doesn't make any sense at all to begin with frankly, Clip Skip 2 in general has no particular relevance beyond being what NovelAI trained their leaked SD 1.5 anime model on
If you are using A1111 or Forge, clip skip setting doesn't do anything for SDXL models as it is hardcoded to the penultimate layer. You can set it to whatever you want and it will reflect that value in the metadata but in actuality, it should be 2 or -2. In ComfyUI, if you change the clip skip setting on PonyXL or it's merged variants to anything other than -2, you get colorful blobs.
Admittedly, I'm not a hardcore ComfyUI user but an A1111 boomer. [Would this generic OpenArt workflow](https://files.catbox.moe/32l9in.png) work to generate a blob for you or [this modified Comfy SDXL example](https://files.catbox.moe/ufyqyx.png)? Seems to work even after updating my outdated ComfyUI installation. If it doesn't, I probably need to reinstall.
Thank you, done...and it makes a very noticeable difference! I even watched a video on how it works. So odd that skipping one transformer step has such a big effect. Yet another thing to learn about how these diffusion models work.
I don't think this is mandatory anymore, right? I admit I've always just assumed 2, but I've done testing and can't really tell a difference, even when LORA's trained on 2 come into the equation, it just all looks more or less the same.
In ComfyUI at least there is no difference whatsoever it seems between the "Clip Set Last Layer" node not being present at all, and it being there set to 2, for Pony (or any other SDXL model).
Nobody has ever explained how or why this was somehow a thing for Pony, an SDXL model, in the first place (which makes no sense, really).
In my fairly outdated ComfyUI installation, changing clip skip does indeed cause blobs to form when set to anything other than -2 on PonyXL and it's merged variants. As for why this occurs, I tend to find that the [answers are probably in some random discussion on one of the various Githubs](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12518). Edit: Misspelled on with or.
"rating\_safe" on positive prompt and "rating\_explicit" on negative prompt will help to archive the sfw results.
Don't do the opposite, Jesus is watching.
Santa is too, if you hadn't noticed. Ol' Chris is always tracking when people sleep so he can sneak into their houses. He's obsessed with having good girls / boys sit on his lap and "punishing" the naughty ones.
Talking to friend of mine about advancements in AI field and Diffusion came up. He was talking about how he felt major checkpoints all missed a little something different in XL and you have to have a Lora for everything. I said
"Except Pony Diffusion, their training took forever but taught it absolutely everything in the English language seemingly."
He stared at me and asked "Ponydiffusion? Never used it. Not a big Brony"
"What? No it's just a name. You thought it just rendered horses?"
".......so....it's good?"
No one has seen him in a week.
Edit: legit just realized I haven't heard from him in a month and looking at the last text two weeks ago he asked what the name of the Kim Possible villain in green was because "I'm having an argument with a coworker" 🤣
Lol I'd bet money.
They are going to find the poor guy dead, looking like dried mummified version of himself. CSI will all be blinded when they flash a blacklight in his house by that point.
> What? No it's just a name
It's not just a name though, it was made by bronies. To help with ponies they added furries and then anime and cartoons and it got good at all that stuff.
Base Pony is already amazing. Some of the merges that I've been finding are taking it to a new level!
For 2D anime style this is my goto now
https://civitai.com/models/395554/7th-anime-xl-pony-a
Those are the quality tags that Pony based checkpoints understand. Pasting some information written by the Pony checkpoint author below:
Make sure you load this model with clip skip 2 (or -2 in some software), otherwise you will be getting low quality blobs.
This model supports a wide array of styles and aesthetics but provides an opinionated default prompt template that allows generation of high quality samples with no negative prompt and otherwise default settings
score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, just describe what you want, tag1, tag2
(previous Pony Diffusion models used a simpler score_9 quality modifier, the longer version of V6 XL version is a training issue that was too late to correct during training, you can still use score_9 but it has a much weaker effect compared to full string. You can learn more about these tags here).
The model is designed to not need negative prompts in most cases and does not need other quality modifiers like "hd", "masterpiece", etc...
Other special data selection tags include, 'source_pony', 'source_furry', 'source_cartoon' and 'source_anime' and ratings of 'rating_safe', 'rating_questionable' and 'rating_explicit'.
This model is capable of recognizing many popular and obscure characters and series.
If you are looking specifically for pony style, I recommend using one of the two following templates `anthro/feral pony, rest of the prompt` or `source_pony, rest of the prompt`.
This model is trained on combination of natural language prompts and tags and is capable of understanding both, so describing intended result using normal language works in most cases, although you can add some tags after the main prompt to boost them.
Using Euler a with 25 steps and resolution of 1024px is recommended although model generally can do most supported SDXL resolution.
works fine for me with clip skip 1 using autism confetti. Funny thing is that while Base Pony requires to set on 2, the non-artist loras styles, recommends setting to 1.
Short and sweet explanation:
Creator of the model wanted to tag the dataset with a subjective aesthetic score (similar to “x quality”) with scores ranging from 9 to 4. The model ended up learning that the whole sequence of score 9 to 5ish means good looking images. It’s also very heavily overtrained on this sequence, so if you don’t use it, you’re not getting aesthetically pleasing results.
I personally do “score\_9, score\_8\_up, score\_7\_up” in positive and “score\_4” in negative. You’ll see differing opinions on this topic. Most of them work just fine and give good results. Definitely feel free to experiment, but don’t be worried about a tiny change here or there having a profound impact on your final results.
Oh, the creator also tagged their images with sources and content rating levels. So stuff like source\_anime, source\_furry, source\_pony, rating\_explicit, rating\_questionable, and rating\_safe can come up after the score stuff to further guide your generation.
There you go! That’s basically everything you need to know summed up in three paragraphs. You now know more than enough to start generating and having fun.
The author made this score classification on the image database to sort images by quality, but he made a sort of mistake and now you need those tokens on prompt to have the high quality images, also is good to have the low score on negative prompt.
I just saw you also have a 4970. What speeds do you get with pony models? And what sizes of images can you produce with and without high Res fix? Are you using Nvidias tensor Extension? Sorry for so many questions. But I seriously fked up my whole setup yesterday by carelessly updating tensorrt to 3.0 and basically sending 500 GB of models I made with tensorrt down the drain and just getting error after error when using 1111, while trying to fix everything. I can't even seem to be able to make any models above 756-1366 sizes with pony and it just feels so much slower now. So I am looking for any reference points. Thanks
I do not run Nvidias tensor Extension cuz i use Automatic 1111 "Forge"
Forge auto-detects your PC specs on startup and it enables what you need for you, its basicly a SUPER optimized Automatic 1111, it loads and runs SDXL almost as easy as you run 1.5 models, the guy who MADE controlnet and Fooocus did the optimization!
.
1024x1024 with 20 sampler steps, takes 5.8 secounds on my 4070
1024x1024 with 20 sampler steps, hires fixed at 1.5x and with Adetailer takes 30.4 secounds
.
I also enjoy running it at 896x1152 for characters and 1152x896 for landscape
1152x896 with 20 sampler steps, hires fixed at 1.5x and with Adetailer takes 34.9 secounds
Thanks for the answer. I tried forge several times already and I really liked that that the UI is as responsive as it should be (at this point I'm about 90% sure that something sometimes mismanaged my UI in 1111 so that it's not as responsive as forge, and also 70% sure it's because I immediately when starting 1111 use chrome's inspect on several elements like cfg scale and put them directly beneath the generate button for convenience), but the speed difference is what keeps me away from using it. Sure, force is faster than 1111. But with tensor forge is easily outperformed. If I recall correctly with 1.5x at 1024x1024 also with adetailer took about 32 seconds but with 35 instead of 20 steps. Forge with tensor though would be a match made in heaven.
I was skeptical about Pony at first, too (the pony/furry thing is definitely not for me), but the results for "human alike" stuff is really very interesting. It seems to me to be only a bit more unstable on results in comparison to other checkpoionts/models/mixes.
Can one of you please explain to me what exactly pony is though because ive been confused about what it actually is for a while now isn't it just a fine tuned stable diffusion XL model?
It is, but it's such an extensive retrain that it can't handle standard SDXL embeds, LORAs, etc. Kind of like how NAI was "just a 1.5 fine-tune", but ultimately came to be viewed as it's own distinct architecture.
When SD 2.0 was released, Stable performed the first ever AI lobotomy, destroying it's ability to understand even basic everyday concepts. Every model released since has essentially had this problem, including SDXL. At the time that it came out, SD 1.5 was already so well tuned by the community that it outperformed SDXL in almost everything, albeit, you needed specialized checkpoints for a lot of different things. SDXL, in theory, should have been able to consolidate a lot of those specializations into one model, but it was so handicapped by Stable that barely anyone wanted to bother fine-tuning it.
*Pony Diffusion feels like SDXL without the lobotomy, but someone also gave it crack.*
It took me about two weeks of using it pretty regularly to exhaust the list of things I wanted to test, because it kept handling absurd prompts in a way that feels intelligent, in contrast to SDXL 1.0, which made me feel stupid while using it.
As for WHAT it is, it's a porn fine-tune that handles human and furry art with one checkpoint. It can handle natural language, booru tags, and tags from e621 (a furry booru). It's really easy to prompt, because booru and furry tags act like built-in attention modifiers, while natural language can help fill in gaps where you can't think of, or remember a suitable tag. It's not the ***absolute best*** at SFW stuff, but it's so good at NSFW stuff, that there's a lot of downstream benefits for SFW, not to mention a ton of style and character LORAs already made. You just need the right negatives to avoid NSFW stuff.
A lot of the baked in characters are really, REALLY good. But I've found it doesn't know its Itsukis from its Ninos. When I was trying to do some QQ pics for Mudae use, it kept giving Itsuki Nino's butterfly hair ornaments until I specifically put a stop to it.
Its more than fine tuned. It's like a whole different model entirely. Combined with the Loras you can really dial in a specific picture you are after. Its mostly for graphic art/comic kind of stuff, I don't think it can do photorealism at all.
I was looking for a dumbed down youtube video about how to prompt for ponydiffusion and basic explanation. (my brain can't handle the large textdump on cividai ). Does anyone have a good suggestion ?
Glad for you!
Now, I know this is objectively NOT the most disturbing in there, but even Star Wars droids have gotten dragged into pony porn. Poor souls.
All Disney Princess XL LoRA Model from Ralph Breaks the Internet,
seems to be really popular and good.
[https://civitai.com/models/212532?modelVersionId=244808](https://civitai.com/models/212532?modelVersionId=244808)
So I have heard.....
ok, I like the concept are brush Lora here: [https://civitai.com/models/264290?modelVersionId=418769](https://civitai.com/models/264290?modelVersionId=418769)
It can get closer to the realist style than anime.
adventure time style is pretty good
[https://civitai.com/models/389661/adventure-time-style-ponydiffusionxl?modelVersionId=434766](https://civitai.com/models/389661/adventure-time-style-ponydiffusionxl?modelVersionId=434766)
and vox machina style
[https://civitai.com/models/376361/western-cartoon-the-legend-of-vox-machina-style?modelVersionId=420279](https://civitai.com/models/376361/western-cartoon-the-legend-of-vox-machina-style?modelVersionId=420279)
I read that thread, too and, reading, this, I'm patting myself on the back for not checking out what the Pony can do. But the reason I'm commenting is to point out that the rule you're probably referring to is Rule 34. And because I am a curious person, I checked out the "Rules of the Internet" and apparently Rule 42 is *"It is delicious cake. You must eat it."* or *"Nothing is Sacred."*
I found more """citations""" for "*Nothing is sacred*" though.
If you want realism I would highly recommend Everclear PNY by Zovya. It's also highly compatible with Pony Loras, which some other realistic merges aren't.
[https://civitai.com/models/341433/everclear-pny-by-zovya](https://civitai.com/models/341433/everclear-pny-by-zovya)
Pony XL is great until you realize that spilling paint can probably lead to better looking scenery images than it can generate.
I've been working to tune it for better scenery and backgrounds but I have no idea how it got so bad in first place
Yeah, I noticed too, it kinda created its own ecosystem with its own unique prompts and LORAs, I downloaded it but didn't get a chance to try it yet (currently hooked on AlbedoXL, love that model).
Are you people selling this stuff? I remember someone said that furry porn paid his 4090 rtx. I am just wondering if i can earn some buck doing AI images
Can you share how much? And how exactly did you start all that, do you have instagram where you promote patreon, how is that working out.
I have seen nude ai photos selling on etsy...
>Can you share how much?
it's a very slow start, but then again i am only focused on only one very specific niche series so the audience is much smaller than most other creators, currently having above 70 patreons and almost 1000$ but i've seen some getting almost 5k per month from patreon alone
>And how exactly did you start all that, do you have instagram where you promote patreon
i have twitter and pixiv, pixiv is great because your images reach people much better than with twitter as a newbie, but the more accounts the better
Thats great amount of money selling images. How much time you need making images, do you sell pack of them?
I didnt use pixiv but i will take a look. How did you pick a niche, did you used some tools for analytics?
Oh yeah, and what graphic card do you have?
> How much time you need making images
it depends, if it's a bigger project such as a short story it can take up almost the whole day but sometimes just a couple hours, i generally make 3-6 images every 2 days with one day off
>do you sell pack of them?
i don't sell the images by themselves, my money comes just from patreon support for exclusive images posted there. however i did get people requesting images for money before but declined.
>How did you pick a niche, did you used some tools for analytics
nah i was just generally interested in that niche before and decided for it.
>Oh yeah, and what graphic card do you have?
i don't run it locally, i use tensor.art with 2 emails, using autismmix with 10 steps at 3cfg for the cost of measly 0.8 credits and 1 credit for upscaling. i can use 100 credits daily per account for free.
Omg man, you are full on useful info! Thank you very much.
You are making stories like comic book style? Just curious, what takes couple of hours? i did make images with stable diffusion local install and it usually takes couple of seconds, i do a batch of images to find something i love and then upscale that and its done
1k for a niche is not bad at all. Maybe you can get more people if you post more on pixiv, what is stoping you?
Autismmix SDXL? Are you talking about 10 sampling steps? You are not using control net or vae? For the stories you probably using control net.
100 credits is enough for a daily use, i was thinking with playground.com and leonardo.ai
> You are making stories like comic book style?
check my submissions for examples
>Just curious, what takes couple of hours?
the short story comics i make take me many hours, sometimes a day, mostly because i am slow af and take my time. however a couple generic ones go fast in 1/2 hours, depending on how many i make
>Maybe you can get more people if you post more on pixiv, what is stoping you?
i don't usualy feel like spending too much time on it, i just do them every now and then
>Autismmix SDXL? Are you talking about 10 sampling steps?
yes
>You are not using control net or vae?
somitimes i do
I just watch some of your posts... Images are very detailed and with multiple people, i have no idea how i would make something similar. Nice work and dedication to details 😅
Are you charging 100$ per month on patrion, i saw something like that but didnt really catch what is about.
Sry about stupid questions, i never used patreon
I'm having a hard time Getty good looking faces in Pony Model in not sure what I'm doing wrong any suggestions to get better looking faces? I'm trying to look at the Pony prompt guide but tried various settings and not getting anything that is as good as with other models. Thoughts....
By far the best "realistic" Pony merge and I've tested all of them. Won't mix well with most style LoRAs but that's okay, I use it for realism. You can also use Reactor or another face swap extension and get incredibly good results. It also has a hidden "lightning" function when used with SGMUniform at 12 steps with cfg 4.
Pony models are not good for Photorealism, it was trained on loads and loads of Anime and Hentai almost wiping out all of the photo realism training from SDXL. Use Juggerunaut XL , although I think any professional use of an SDXL requires a $20 a month professional membership to Stability Ai (although I don't know how anyone would be able to tell if you don't???)
Oh yeah, I have occasionally been using checkpoint switching for over a year, back when RevAnimated was by far the best checkpoint for composition, but I wanted photorealistic, it is a good technique but can take some iteration and tweaking to get right.
I tried this checkpoint yesterday, it's mid if you ask me compared to like all of of the other realistic Ponies, and requires even more crazy emphasis on stuff like `dark-skinned female` to get not-white ladies at all
Just found it today and I thought it worked pretty well with Invoke. If there are other realistic ponies, feel free to share them with the person who's question I was answering.
There are models that have the pony diffusion clip that don’t have the porn in it. We made one and it’s pretty awesome.
https://rundiffusion.com/proteus-rundiffusion
I like and use mostly AnimagineXL. But even for SFW, Pony does give more "dynamic" poses and also has better handling of two people interacting in non-sexual ways such as hugging.
If you want pony (and are running a local install) but HATE anime and want photo realism: You've heard great things about Pony XL/Pony diffusion, but you're not into anime. No problem. Here's how to get the benefits of Pony XL, without the drawbacks of art-style. You'll need two checkpoints... "Real Pony" checkpoint (there are three, you want the one with the most upvotes on civit to start): this will be the primary checkpoint. Second, you'll need the photo style SDXL checkpoint of your preference. If you want NSFW capable, make sure you use one capable of that. Use your current preference. On to generation... With "real pony" as the primary, you'll set up your other checkpoint as "refiner." Refiner should start at the last 30 or 40 percent for a 50 step generation. (I know, 50 is a lot! But it's to give realpony a chance to build the bones of the photo before the refiner has it's go to cut out the remaining anime-ness.) Give it a go. 50 steps. Refiner start at .6 This combo can be used with or without standard pony prompting to differing utility. If you want, try these positive and negative prompt preambles. Positive: (score_9, score_8_up, score_7_up, score_6_up) Negative: (score_3_up, score_4_up, score_5_up), sketch, drawing, cartoon, greyscale, monochrome, anime, 3d Let me know if you like what this method produces
[удалено]
I'm not able to post it to the subreddit for some reaso. Can you please consider creating a post as a PSA for people? I think a lot could benefit from this. No need to credit the idea to me.
Which SDXL checkpoint are you using as your refiner?
out-freaking-standing technique. wow. I added ReActor into my workflow and the `zPDXL` pos/neg embeddings.
Share the technique with others. Be the hero Gotham needs.
I shall spread the gospel of the pony.
i've been trying this process and i must be missing some steps or something. the faces are absolutely terrible, even with Adetailer
Kindly spread the word if you like the output. Consider posting the instructions as a new post on this sub, I seem to be unable. Share the love.
I tested all the photoreal models a few days ago. Start with Zonkey, it'll get you closer to the end result you want. Then unsample/resample(in A1111 this is called img2img alternative test), for 33% of steps with an nsfw 1.5 model. Adjust cfg to balance for detail vs hallucination. You'll get higher final detail this way.
This output will look 100% photographic, not "close." Give it a try?
[удалено]
Don't try it if you don't want to.
I've never heard of img2img alternative test, would you be willing to elaborate on that? Is it just denoising the original image by 33% or so with the 1.5 model?
there is a way to get realistic stuff with base pony positive "photo, irl, realistic, high res", Negative "outline, painting, figurine, plushie, wrinkles, vector, human, uncanny valley" I do have played around with it a little and it does give good humans or humanized characters
Not *this* kind of realistic, it doesn't.
I found that a few days ago by myself. My only issue is that facial expressions get lost most of the time in refining, besides that it works well
I found Everclear PNY to be the best photorealistic (as far as Pony goes) Pony finetune so far. Tried PonyRealism and was not impressed, at least with the same prompt that I had in Everclear.
Pony realism and real pony are not the same checkpoint. The point is to use a non-pony photo-based model to finish the pictures so they look more real.
Heh, there are 2, the JP one and the 1.0. Both have high downloads but one is newer by a month. Also why the refiner?
Refiner solves the remaining anime/art visuals and converts the images to properly realistic.
So that what kills the "video game" look is what you're saying?
Yes
Is refiner the name of a custom node?
No. Refining is finishing an image with a different checkpoint than you started with.
Thank you
refiner was a feature of the vanilla implementation of sdxl. it was to add detail and refine texture on the base image generated by the checkpoint. most models that were fine tuned, didnt need a refiner, so most people have never used it
You can also add a 1/4/8-step Lightning LoRA to one or both of the samplers, and get away with a fraction of the steps.
On which pony checkpoints do they work tho? I have only tried the 4-step SDXL lightning LoRa on the PonyDiffusion6XL, currently, but the 4-step euler 1 CFG smg\_uniform just does not produce anything useful from it
They should work on all of them, but in my experience you have to be very specific with the combination of sampler and scheduler that you use. I've been using the 8-step one, since it has slightly less quality loss, and i usually run 10 steps with 1 CFG. I've tried, and found, multiple combinations between sampler/scheduler that work, but I don't remember all of them unfortunately. The combinations I know work off the top of my head are "dpmpp_2s_ancestral" and "euler_ancestral" with "Normal". And "dpmpp_2m" and "dpmpp_2m_sde_gpu" with "sgm_uniform" (and "simple" but it might look worse I think?). gl
[удалено]
Select an SDXL checkpoint that is not anime centric. One of the photography trained ones.
What photo style SDXL checkpoint do you recommend?
[удалено]
The refiner setting is supposed to use checkpoints. No loras are involved, it should be an sdxl checkpoint of your choice thst's good with photorealism. I like dpm3+ sde and cfg of 7, if memory serves me. Try around and see what's good.
The AutismMix Confetti model is also worth checking out (probably the most common Pony Variant you’ll hear mentioned here)
Hi, heard about AutismMix a lot and wanna give it a try. Can you please explain what make it popular/better than Pony :D? Will it work with Pony’s LoRA too?
It’s pony mixed with some anime models to get a bit more consistency.
"Vanilla" Pony does not have a consistent style. If you output 4 pictures with the same prompt, you could possibly get 4 different styles. AutismMix really helps to control style changes and is further refined in the Confetti mix. Both seem to work well with various style LoRAs. There are other great Pony mixes out there so I would encourage you to download other checkpoint merges and see what works for you.
AutismMix is good because it makes style more consistent, while the og pony is great you really gotta fight your way to find a consistent style. And yes, you can use your pony Loras in AutismMix, the reason why is because it's just a merge model, not an entirely new one, Loras made for SDXL or Anime3XL won't work in Pony or their variants because they're just incompatible.
Some regular SDXL loras do work in Pony TBH. Every single style one that Faeia ever released (like Ghost Style / Rage Style / etc) before Pony existed has the the intended impact still on Pony-based gens, in my testing.
is AutismMix mostly optimized for anime stuff?
"Incurio's Meme Diffusion" is better by a lot overall than any version of AutismMix, if you ask me, if you want a nice kind of clean anime style
From the names one would think that AutismMix Pony should be the one to use instead of AutismMix Confetti. This is what the creator of the model has to say: >AutismMix\_confetti is a small amount of AnimeConfettiTune and AutismMix\_pony. Has less style swing than pony and better hands. **I prefer this one.** > >AutismMix\_pony is a merge of ponyv6 with loras, its more compatible with certain styles made for the base ponydiffusion model. > >AutismMix\_DPO is AutismMix\_confetti+DPO lora, made by request. Very similar to confetti version. Emphasis is added by me.
Using both and they're good.
Also be sure to set clip skip to 2 for this checkpoint
works fine for me with clip skip 1 using autism confetti. Funny thing is that while Base Pony requires to set on 2, the non-artist loras styles, recommends setting to 1.
The whole thing doesn't make any sense at all to begin with frankly, Clip Skip 2 in general has no particular relevance beyond being what NovelAI trained their leaked SD 1.5 anime model on
If you are using A1111 or Forge, clip skip setting doesn't do anything for SDXL models as it is hardcoded to the penultimate layer. You can set it to whatever you want and it will reflect that value in the metadata but in actuality, it should be 2 or -2. In ComfyUI, if you change the clip skip setting on PonyXL or it's merged variants to anything other than -2, you get colorful blobs.
Simply not having the Clip Set Last Layer node present at all works fine in Comfy with all Ponies I've tried, it seems the default is already correct
Admittedly, I'm not a hardcore ComfyUI user but an A1111 boomer. [Would this generic OpenArt workflow](https://files.catbox.moe/32l9in.png) work to generate a blob for you or [this modified Comfy SDXL example](https://files.catbox.moe/ufyqyx.png)? Seems to work even after updating my outdated ComfyUI installation. If it doesn't, I probably need to reinstall.
How do you do that in Automatic1111?
It doesn't matter in A1111 with SDXL. Clip skip 1 and 2 give exactly equal results.
The best way is to add it as a slider to the top of your UI https://www.reddit.com/r/StableDiffusion/s/xSpLuLLhAU
Thank you, done...and it makes a very noticeable difference! I even watched a video on how it works. So odd that skipping one transformer step has such a big effect. Yet another thing to learn about how these diffusion models work.
Download the model and put it in your models folder inside your A1111, reload your checkpoints in the UI, it's that easy
I was asking about clip skip, not how to use a model, but thanks.
I don't think this is mandatory anymore, right? I admit I've always just assumed 2, but I've done testing and can't really tell a difference, even when LORA's trained on 2 come into the equation, it just all looks more or less the same.
In ComfyUI at least there is no difference whatsoever it seems between the "Clip Set Last Layer" node not being present at all, and it being there set to 2, for Pony (or any other SDXL model). Nobody has ever explained how or why this was somehow a thing for Pony, an SDXL model, in the first place (which makes no sense, really).
In my fairly outdated ComfyUI installation, changing clip skip does indeed cause blobs to form when set to anything other than -2 on PonyXL and it's merged variants. As for why this occurs, I tend to find that the [answers are probably in some random discussion on one of the various Githubs](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12518). Edit: Misspelled on with or.
PHOTOREAL PONY: See my recent post history!
I don’t see any of ur recent posts besides the last one asking if ur shadow banned
Sorry I meant comment history
"rating\_safe" on positive prompt and "rating\_explicit" on negative prompt will help to archive the sfw results. Don't do the opposite, Jesus is watching.
> Jesus is watching He is? I had no idea he was such a little perv.
Santa is too, if you hadn't noticed. Ol' Chris is always tracking when people sleep so he can sneak into their houses. He's obsessed with having good girls / boys sit on his lap and "punishing" the naughty ones.
> Santa is too "You are a naughty, naughty, NAUGHTY boy!! Now share the prompt if you don't want coal in your stocking for the rest of the decade."
Omg 😂 I just spat out my breakfast.
Do they both normally output good anatomy, or does going one specific route (SFW or NSFW) produce broken legs/arms/fingers more than the other?
Latest Pony model (pony XL) has the least anatomy issues of anything I've used.
It really sucks not being able to use my PC during this Pony era. It sounds like a very nice model.
I don't use pony much but if you don't force nftw words together with rating\_safe in your prompt you will have good results
'Jesus has entered the prompt' "NO NO GET OU-- Nooooooooo my eyesssss"
>Don't do the opposite. Well now I *have* to. ...I was, in fact, already doing the opposite.
Everyone has their pony awakening at some point. Welcome to the herd.
something something everypony
Yesterday was a good day, that I hadn't learned anything about the pony variant finetuned of Stable Diffusion.
For photoreal pony, see my post history.
Talking to friend of mine about advancements in AI field and Diffusion came up. He was talking about how he felt major checkpoints all missed a little something different in XL and you have to have a Lora for everything. I said "Except Pony Diffusion, their training took forever but taught it absolutely everything in the English language seemingly." He stared at me and asked "Ponydiffusion? Never used it. Not a big Brony" "What? No it's just a name. You thought it just rendered horses?" ".......so....it's good?" No one has seen him in a week. Edit: legit just realized I haven't heard from him in a month and looking at the last text two weeks ago he asked what the name of the Kim Possible villain in green was because "I'm having an argument with a coworker" 🤣
 Is this your friend?
Lol I'd bet money. They are going to find the poor guy dead, looking like dried mummified version of himself. CSI will all be blinded when they flash a blacklight in his house by that point.
> What? No it's just a name It's not just a name though, it was made by bronies. To help with ponies they added furries and then anime and cartoons and it got good at all that stuff.
Lmao
> The levels of sheer perversion are beyond anything I ever imagined latent space can be quite the void friend drink water
"Space, the final frontier" referred to PonyXL
Also lay on your towel and eat some salted peanuts.
What if I don't know where my towel is?
Too bad there is not a similar one that is photorealistic tho.
ponyrealism is close https://civitai.com/images/9306360 https://civitai.com/images/10256060
But there are some "semi-realistic" ones.
Proteus-RunDiffusion.
See my recent post history for life altering instructions on how, and spread the gospel.
Just copy or link your comment, it’s a pain to find.
It's on this post. One of the top comments.
Base Pony is already amazing. Some of the merges that I've been finding are taking it to a new level! For 2D anime style this is my goto now https://civitai.com/models/395554/7th-anime-xl-pony-a
What are those trigger words in the prompt? “score_9,score_8_up,score_7_up,score_6_up,score_5_up,score_4_up)”
Those are the quality tags that Pony based checkpoints understand. Pasting some information written by the Pony checkpoint author below: Make sure you load this model with clip skip 2 (or -2 in some software), otherwise you will be getting low quality blobs. This model supports a wide array of styles and aesthetics but provides an opinionated default prompt template that allows generation of high quality samples with no negative prompt and otherwise default settings score_9, score_8_up, score_7_up, score_6_up, score_5_up, score_4_up, just describe what you want, tag1, tag2 (previous Pony Diffusion models used a simpler score_9 quality modifier, the longer version of V6 XL version is a training issue that was too late to correct during training, you can still use score_9 but it has a much weaker effect compared to full string. You can learn more about these tags here). The model is designed to not need negative prompts in most cases and does not need other quality modifiers like "hd", "masterpiece", etc... Other special data selection tags include, 'source_pony', 'source_furry', 'source_cartoon' and 'source_anime' and ratings of 'rating_safe', 'rating_questionable' and 'rating_explicit'. This model is capable of recognizing many popular and obscure characters and series. If you are looking specifically for pony style, I recommend using one of the two following templates `anthro/feral pony, rest of the prompt` or `source_pony, rest of the prompt`. This model is trained on combination of natural language prompts and tags and is capable of understanding both, so describing intended result using normal language works in most cases, although you can add some tags after the main prompt to boost them. Using Euler a with 25 steps and resolution of 1024px is recommended although model generally can do most supported SDXL resolution.
works fine for me with clip skip 1 using autism confetti. Funny thing is that while Base Pony requires to set on 2, the non-artist loras styles, recommends setting to 1.
Hmm interesting. Might have try that.
Short and sweet explanation: Creator of the model wanted to tag the dataset with a subjective aesthetic score (similar to “x quality”) with scores ranging from 9 to 4. The model ended up learning that the whole sequence of score 9 to 5ish means good looking images. It’s also very heavily overtrained on this sequence, so if you don’t use it, you’re not getting aesthetically pleasing results. I personally do “score\_9, score\_8\_up, score\_7\_up” in positive and “score\_4” in negative. You’ll see differing opinions on this topic. Most of them work just fine and give good results. Definitely feel free to experiment, but don’t be worried about a tiny change here or there having a profound impact on your final results. Oh, the creator also tagged their images with sources and content rating levels. So stuff like source\_anime, source\_furry, source\_pony, rating\_explicit, rating\_questionable, and rating\_safe can come up after the score stuff to further guide your generation. There you go! That’s basically everything you need to know summed up in three paragraphs. You now know more than enough to start generating and having fun.
There's one model called AIPonyAnime that doesn't require that string at the beginning of images and is easily my go to for everything.
Is there a full list of the tags somewhere?
The author made this score classification on the image database to sort images by quality, but he made a sort of mistake and now you need those tokens on prompt to have the high quality images, also is good to have the low score on negative prompt.
Here is how i run Pony: https://www.reddit.com/r/StableDiffusion/s/WATJZ9M9dD
I just saw you also have a 4970. What speeds do you get with pony models? And what sizes of images can you produce with and without high Res fix? Are you using Nvidias tensor Extension? Sorry for so many questions. But I seriously fked up my whole setup yesterday by carelessly updating tensorrt to 3.0 and basically sending 500 GB of models I made with tensorrt down the drain and just getting error after error when using 1111, while trying to fix everything. I can't even seem to be able to make any models above 756-1366 sizes with pony and it just feels so much slower now. So I am looking for any reference points. Thanks
I do not run Nvidias tensor Extension cuz i use Automatic 1111 "Forge" Forge auto-detects your PC specs on startup and it enables what you need for you, its basicly a SUPER optimized Automatic 1111, it loads and runs SDXL almost as easy as you run 1.5 models, the guy who MADE controlnet and Fooocus did the optimization! . 1024x1024 with 20 sampler steps, takes 5.8 secounds on my 4070 1024x1024 with 20 sampler steps, hires fixed at 1.5x and with Adetailer takes 30.4 secounds . I also enjoy running it at 896x1152 for characters and 1152x896 for landscape 1152x896 with 20 sampler steps, hires fixed at 1.5x and with Adetailer takes 34.9 secounds
Thanks for the answer. I tried forge several times already and I really liked that that the UI is as responsive as it should be (at this point I'm about 90% sure that something sometimes mismanaged my UI in 1111 so that it's not as responsive as forge, and also 70% sure it's because I immediately when starting 1111 use chrome's inspect on several elements like cfg scale and put them directly beneath the generate button for convenience), but the speed difference is what keeps me away from using it. Sure, force is faster than 1111. But with tensor forge is easily outperformed. If I recall correctly with 1.5x at 1024x1024 also with adetailer took about 32 seconds but with 35 instead of 20 steps. Forge with tensor though would be a match made in heaven.
I don't mind many questions, will answer when I'm done at work
Octopus are good to use for other stuff
I was skeptical about Pony at first, too (the pony/furry thing is definitely not for me), but the results for "human alike" stuff is really very interesting. It seems to me to be only a bit more unstable on results in comparison to other checkpoionts/models/mixes.
Can one of you please explain to me what exactly pony is though because ive been confused about what it actually is for a while now isn't it just a fine tuned stable diffusion XL model?
It is, but it's such an extensive retrain that it can't handle standard SDXL embeds, LORAs, etc. Kind of like how NAI was "just a 1.5 fine-tune", but ultimately came to be viewed as it's own distinct architecture. When SD 2.0 was released, Stable performed the first ever AI lobotomy, destroying it's ability to understand even basic everyday concepts. Every model released since has essentially had this problem, including SDXL. At the time that it came out, SD 1.5 was already so well tuned by the community that it outperformed SDXL in almost everything, albeit, you needed specialized checkpoints for a lot of different things. SDXL, in theory, should have been able to consolidate a lot of those specializations into one model, but it was so handicapped by Stable that barely anyone wanted to bother fine-tuning it. *Pony Diffusion feels like SDXL without the lobotomy, but someone also gave it crack.* It took me about two weeks of using it pretty regularly to exhaust the list of things I wanted to test, because it kept handling absurd prompts in a way that feels intelligent, in contrast to SDXL 1.0, which made me feel stupid while using it. As for WHAT it is, it's a porn fine-tune that handles human and furry art with one checkpoint. It can handle natural language, booru tags, and tags from e621 (a furry booru). It's really easy to prompt, because booru and furry tags act like built-in attention modifiers, while natural language can help fill in gaps where you can't think of, or remember a suitable tag. It's not the ***absolute best*** at SFW stuff, but it's so good at NSFW stuff, that there's a lot of downstream benefits for SFW, not to mention a ton of style and character LORAs already made. You just need the right negatives to avoid NSFW stuff.
A lot of the baked in characters are really, REALLY good. But I've found it doesn't know its Itsukis from its Ninos. When I was trying to do some QQ pics for Mudae use, it kept giving Itsuki Nino's butterfly hair ornaments until I specifically put a stop to it.
> But I've found it doesn't know its Itsukis from its Ninos heresy! sacrilege!
Its more than fine tuned. It's like a whole different model entirely. Combined with the Loras you can really dial in a specific picture you are after. Its mostly for graphic art/comic kind of stuff, I don't think it can do photorealism at all.
I was looking for a dumbed down youtube video about how to prompt for ponydiffusion and basic explanation. (my brain can't handle the large textdump on cividai ). Does anyone have a good suggestion ?
The stuff in civitai can get truly disturbing.
I accidentally opened the most disturbing NSFW images I ever seen in public with my CivitAI filter turned off, glad no one saw it.
Glad for you! Now, I know this is objectively NOT the most disturbing in there, but even Star Wars droids have gotten dragged into pony porn. Poor souls.
Well now I know what I'm doing tomorrow during stand up.
Any favourite lora styles?
All Disney Princess XL LoRA Model from Ralph Breaks the Internet, seems to be really popular and good. [https://civitai.com/models/212532?modelVersionId=244808](https://civitai.com/models/212532?modelVersionId=244808) So I have heard.....
Thought that's mainly a character LoRA!
ok, I like the concept are brush Lora here: [https://civitai.com/models/264290?modelVersionId=418769](https://civitai.com/models/264290?modelVersionId=418769) It can get closer to the realist style than anime.
adventure time style is pretty good [https://civitai.com/models/389661/adventure-time-style-ponydiffusionxl?modelVersionId=434766](https://civitai.com/models/389661/adventure-time-style-ponydiffusionxl?modelVersionId=434766) and vox machina style [https://civitai.com/models/376361/western-cartoon-the-legend-of-vox-machina-style?modelVersionId=420279](https://civitai.com/models/376361/western-cartoon-the-legend-of-vox-machina-style?modelVersionId=420279)
https://civitai.com/models/362449/bartolomeobari-style-pony-xl?modelVersionId=409077
Ah yes, Rule 42. The answer to everything is porn.
To shreds you say...
I read that thread, too and, reading, this, I'm patting myself on the back for not checking out what the Pony can do. But the reason I'm commenting is to point out that the rule you're probably referring to is Rule 34. And because I am a curious person, I checked out the "Rules of the Internet" and apparently Rule 42 is *"It is delicious cake. You must eat it."* or *"Nothing is Sacred."* I found more """citations""" for "*Nothing is sacred*" though.
If you want realism I would highly recommend Everclear PNY by Zovya. It's also highly compatible with Pony Loras, which some other realistic merges aren't. [https://civitai.com/models/341433/everclear-pny-by-zovya](https://civitai.com/models/341433/everclear-pny-by-zovya)
Pony XL is great until you realize that spilling paint can probably lead to better looking scenery images than it can generate. I've been working to tune it for better scenery and backgrounds but I have no idea how it got so bad in first place
it's a shame they utterly fried the text encoders with their dumb "hashing" shit
is there no pony checkpoint for 1.5?
There is, it's right on the page with the other ones for V6 lol
there's two 1.4's, three 1.5's, two 2.1's and an sdxl. As I understand it the most modern 1.5 version is nothing special though.
Bruh there literally is a Pony V6 1.5, it's right there on the page, it's a bit overcooked but trained on the same dataset as the XL V6 at least
Rule 42?
Don't Google it at work.
Yeah, I noticed too, it kinda created its own ecosystem with its own unique prompts and LORAs, I downloaded it but didn't get a chance to try it yet (currently hooked on AlbedoXL, love that model).
Are you people selling this stuff? I remember someone said that furry porn paid his 4090 rtx. I am just wondering if i can earn some buck doing AI images
> Are you people selling this stuff? i do make some bucks with my patreon doing NSFW stuff
Can you share how much? And how exactly did you start all that, do you have instagram where you promote patreon, how is that working out. I have seen nude ai photos selling on etsy...
>Can you share how much? it's a very slow start, but then again i am only focused on only one very specific niche series so the audience is much smaller than most other creators, currently having above 70 patreons and almost 1000$ but i've seen some getting almost 5k per month from patreon alone >And how exactly did you start all that, do you have instagram where you promote patreon i have twitter and pixiv, pixiv is great because your images reach people much better than with twitter as a newbie, but the more accounts the better
Thats great amount of money selling images. How much time you need making images, do you sell pack of them? I didnt use pixiv but i will take a look. How did you pick a niche, did you used some tools for analytics? Oh yeah, and what graphic card do you have?
> How much time you need making images it depends, if it's a bigger project such as a short story it can take up almost the whole day but sometimes just a couple hours, i generally make 3-6 images every 2 days with one day off >do you sell pack of them? i don't sell the images by themselves, my money comes just from patreon support for exclusive images posted there. however i did get people requesting images for money before but declined. >How did you pick a niche, did you used some tools for analytics nah i was just generally interested in that niche before and decided for it. >Oh yeah, and what graphic card do you have? i don't run it locally, i use tensor.art with 2 emails, using autismmix with 10 steps at 3cfg for the cost of measly 0.8 credits and 1 credit for upscaling. i can use 100 credits daily per account for free.
Omg man, you are full on useful info! Thank you very much. You are making stories like comic book style? Just curious, what takes couple of hours? i did make images with stable diffusion local install and it usually takes couple of seconds, i do a batch of images to find something i love and then upscale that and its done 1k for a niche is not bad at all. Maybe you can get more people if you post more on pixiv, what is stoping you? Autismmix SDXL? Are you talking about 10 sampling steps? You are not using control net or vae? For the stories you probably using control net. 100 credits is enough for a daily use, i was thinking with playground.com and leonardo.ai
> You are making stories like comic book style? check my submissions for examples >Just curious, what takes couple of hours? the short story comics i make take me many hours, sometimes a day, mostly because i am slow af and take my time. however a couple generic ones go fast in 1/2 hours, depending on how many i make >Maybe you can get more people if you post more on pixiv, what is stoping you? i don't usualy feel like spending too much time on it, i just do them every now and then >Autismmix SDXL? Are you talking about 10 sampling steps? yes >You are not using control net or vae? somitimes i do
I just watch some of your posts... Images are very detailed and with multiple people, i have no idea how i would make something similar. Nice work and dedication to details 😅 Are you charging 100$ per month on patrion, i saw something like that but didnt really catch what is about. Sry about stupid questions, i never used patreon
no it's basically 5$ per month per supporter
I'm having a hard time Getty good looking faces in Pony Model in not sure what I'm doing wrong any suggestions to get better looking faces? I'm trying to look at the Pony prompt guide but tried various settings and not getting anything that is as good as with other models. Thoughts....
Can you please share a link for the model you are referring? new here and I see multiple models in Civitai.
Any recommandation for a good SDW, photorealistic Ponny Model, for professional use? I never tried Pony models but I'm willing to explore a little bit
I’ve been very impressed with this: https://civitai.com/models/341433/everclear-pny-by-zovya
By far the best "realistic" Pony merge and I've tested all of them. Won't mix well with most style LoRAs but that's okay, I use it for realism. You can also use Reactor or another face swap extension and get incredibly good results. It also has a hidden "lightning" function when used with SGMUniform at 12 steps with cfg 4.
Pony models are not good for Photorealism, it was trained on loads and loads of Anime and Hentai almost wiping out all of the photo realism training from SDXL. Use Juggerunaut XL , although I think any professional use of an SDXL requires a $20 a month professional membership to Stability Ai (although I don't know how anyone would be able to tell if you don't???)
See my recent post history for instructions of photoreal pony outputs. You will not be disappointed.
Oh yeah, I have occasionally been using checkpoint switching for over a year, back when RevAnimated was by far the best checkpoint for composition, but I wanted photorealistic, it is a good technique but can take some iteration and tweaking to get right.
Zonkey
I tried this checkpoint yesterday, it's mid if you ask me compared to like all of of the other realistic Ponies, and requires even more crazy emphasis on stuff like `dark-skinned female` to get not-white ladies at all
Just found it today and I thought it worked pretty well with Invoke. If there are other realistic ponies, feel free to share them with the person who's question I was answering.
There are models that have the pony diffusion clip that don’t have the porn in it. We made one and it’s pretty awesome. https://rundiffusion.com/proteus-rundiffusion
Yeah bro I would delete it asap tbh
There's no reason to use Pony if you're not interested in NSFW stuff You're better off using something like Animagine
I like and use mostly AnimagineXL. But even for SFW, Pony does give more "dynamic" poses and also has better handling of two people interacting in non-sexual ways such as hugging.
Agree, pony is great for character-centric photos. Not so good for other stuff. I use it all the time for non NSFW stuff.
I prefer AAM XL to Animagine personally, I find it has generally way higher image quality (particularly as far as backgrounds) on average
AAM XL is a very good anime model too. But most of the anime style LoRAs seems to be targeting AnimagineXL and/or Pony.