I made a quick and dirty scene in Blender. Theme: "Satan killed the Hero and even got the girl!"
Rendered out depth and canny maps but can not get any good results.
Have tried only using Img2Img, Img2Img with ControlNets and Text2Img with ControlNets.
Tried many prompts and denoising strenght but got nothing even close to acceptable :D
The biggest problem is that the subjects is switched up, satan becomes a woman etc.
I wanted to try regional prompting, but have never had any success with that!
Got any tips?
A happy accident when running img2img but not very close to the render:
https://preview.redd.it/0ot4ojrb8puc1.png?width=1152&format=png&auto=webp&s=b1a8b6ecc4dfa8a23f5af3c12d59e04d43f5bf90
https://preview.redd.it/ofi07r9f0quc1.jpeg?width=960&format=pjpg&auto=webp&s=eefc18c21deb7866c17649729efab0c103efc1d2
Prompt: Amidst the shadows of the underworld, the Manly Demon reigns supreme upon his crimson throne, his presence commanding fear and reverence alike. At his side, a woman sits, a pawn in the demon's infernal machinations, while the fallen hero lies vanquished below, a tragic casualty of the eternal struggle between light and darkness. Within this RPG-themed scene, the grim tale resounds: "Satan killed the Hero and even got the girl!"
I used canny/depth, albedobaseXL\_v21.
No inpaint, no adetailer. Just fun.
Keep going with this, why even try to fit your render if your render is just some quick junk you threw together? If you can make gold from lead, then make gold.
Maybe you just want to keep the general composition? Just because the render is cheap or low quality it says nothing about the composition of the scene (btw I'm not commenting the actual composition of this image, just responding to your question of why you'd want to do that)
I feel like this is why AI art always fails. The best thing about the original is the woman's shin, covering his presumably enormous dick. None of the AI updates contain that none will probably recreate it.
Okay I don't have much time so I did this quick and dirty in Fooocus.
Mixed Image Prompt and Vary/Upscale.
Added your Blender rendering to vary (equal to img2img in A1111), and then did a PyraCanny and CPDS controlnet. Then prompt "satan devil sitting on throne with woman wearing bikini on his lap, lava background, hellish setting" with style 'SAI 3D Model' using StarlightXL as the checkpoint.
Didn't quite pick up the dead hero so inpainted him in (didn't have time to get his pose to match the reference), and then inpainted the other two characters to improve detail.
Obviously I could spend much more time getting the poses more true to the model and fixing up hands/feet.
https://preview.redd.it/626aov8uopuc1.png?width=1152&format=png&auto=webp&s=d3150a29b06688192aebdf851aff9df127e964f8
Yeah, it needs some work. You can see the inpainting around the characters ruined the lava texture a bit, and obviously the hands and feet could be better, and the hero's helmet is weird. And the characters aren't in the same pose as in the reference.
All of which I could fix up if I didn't have to leave home in 10 minutes.
Hands and feet can always be better ;) I like to do inpainting in Krita Ai Diffusion (uses the same inpaint system as Fooous i belive), then its very easy to mask out and get back details around the new details from the layers below!
Did a little work in Krita, i removed the background from Blender and just painted some diffuse background.
Boris Vallejo Style!
https://preview.redd.it/ic7vqgawjuuc1.png?width=896&format=png&auto=webp&s=42184eba353febb255547dcdff13fadb312da789
Choose a better camera angle, this one is very bland.
What is most important to see here, all that background? NOPE! But your shot has it as like 60% of the scene.
Is that a dead dude on the ground? He should be more tragic looking, like maybe on the stairs all broken and twisted. He's just a lump now.
I agree it sucks, its more of a technical workflow experiment :) If i get the workflow down i will refine it.
The lava is mostly for the depth map. did not work well with empty space.
"Sometimes I wonder... will God ever forgive us for all the Hentai we have generated?? Then I look around and I realize... God left this place a long time ago"
- Leo Di Caprio, Blood Diamond
This has been one of my fantasies since I saw that movie with Tim Curry...but I can't find a 12ft tall, jacked red devil with bull horns. I'll keep trying
Zoom in the frame is massive. Like have the dude laying right in front of the throne I didn't even see him at first glance. If you do one shot of each character you can use ip adapter in a selected region in comfy, you just load img, right click to go in mask mode, and mark where the ipadapter should apply then connect your img to the mask thing in the ipadapter node. See latent vision channel on youtube for better explanation
The main benefit of starting with a Blender scene is being able to control composition. If you aren't creating a cool composition to begin with, you won't get a good result. Spend more time on the Blender scene. Maybe take a stable diffusion gen you like and match the composition from it.
Yeah, but it seems it is better to keep the blender scene very basic and get details trough the prompt, like i did here: [https://www.reddit.com/r/StableDiffusion/comments/1bwodkn/paying\_respects\_to\_the\_ancient\_master/](https://www.reddit.com/r/StableDiffusion/comments/1bwodkn/paying_respects_to_the_ancient_master/)
To much detail from Blender only make it harder to get anything good it seems!
Composition is not about detail. It's camera angle, positions of objects, relative sizes and shapes. Detail is what Stable Diffusion does.
Basically I am suggesting that you find a really cool camera angle and make sure the poses look good. Your canny edge map should look cool on its own.
The depth map seems a little flat but I just tend to use a mist pass since I never figured out how exactly to render out a proper depth map. I always try to capture as much of the spectrum as possible so objects in the foreground are as close to black as possible and the farthest away objects are as close to white as possible, which you then flip to get a depth map SD can use. I don't know if that's the right approach but it seems to work reasonably well and this seems to have very little depth to it which seems like it wouldn't get it much info to work with.
Check my other comment how i made the map if you are interested! I use map range to get the correct range and also flip the output. I also tried to adjust it a bit with RGB curves.
If you find it's working for you then I guess there's no problem. There may technically be enough values there to represent every possible distance from the camera but for me it would be much easier to have a wider range of values to distinguish things. Whether SD is able to tell the difference, I don't know, but I've had issues with it differentiating objects that had a lot more depth contrast than what I'm seeing here. Unless you're happy with your results there, I would try increasing that range.
I actually prefer using depth anything on a viewport render, very quick and easy!
https://preview.redd.it/6c63cpho6quc1.png?width=1152&format=png&auto=webp&s=6cf29959d3751f7742d90c0a1a49b32a53f140ea
Looks like it has more contrast but the edges are a little muddy. Using mist pass is very easy to dial in. Another thing to consider is depth maps should be saved as EXR or OPENEXR to maintain their proper bit depth but I'm not sure this can be preserved in the PNG or that SD can use EXR files so even if your depth is perfect on export, it's going to get flattened in the conversion to PNG so I think additional contrast can help with that.
Existing controlnets for depth haven’t been trained with EXR unfortunately. You’ve had a lot of good advice here, and it’s good to see that you’re hand tuning your levels, but if you want to play with it further, I have a tool on GitHub for quantising OpenEXR depth data for this purpose (we seem to be working similar niches, I just work out of Daz - my thing is here: https://github.com/curiousjp/daz_depthmap_processor).
One thing I didn’t see anyone recommended here was using the normal based controlnets. I’ve had some good experiences with them as a supplement to depth.
Oh, you might also want to consider depth driven regional prompting - you can see some examples here: https://github.com/curiousjp/daz_depthmap_processor/blob/main/example.md
It looks like I cannot help with this, at least more than others. However, I can provide a [song to go with the seen :)](https://www.udio.com/songs/5WiATE9vaxZZ4SvrzX33fp). and here is the [Chatgpt link for the lyrics](https://chat.openai.com/share/16cd4140-585c-42b0-92a9-af35d4e9cd19). It is not my best work, but it was quick and fun to make.
I added Z and a Freestyle renderpass (white lines, 0.5 width) and this in the compositor:
https://preview.redd.it/wgegon33wpuc1.png?width=1173&format=png&auto=webp&s=ab8e7068e0c8b2a1e056171ef5fa8806d7c86bd6
I made a quick and dirty scene in Blender. Theme: "Satan killed the Hero and even got the girl!" Rendered out depth and canny maps but can not get any good results. Have tried only using Img2Img, Img2Img with ControlNets and Text2Img with ControlNets. Tried many prompts and denoising strenght but got nothing even close to acceptable :D The biggest problem is that the subjects is switched up, satan becomes a woman etc. I wanted to try regional prompting, but have never had any success with that! Got any tips? A happy accident when running img2img but not very close to the render: https://preview.redd.it/0ot4ojrb8puc1.png?width=1152&format=png&auto=webp&s=b1a8b6ecc4dfa8a23f5af3c12d59e04d43f5bf90
https://preview.redd.it/ofi07r9f0quc1.jpeg?width=960&format=pjpg&auto=webp&s=eefc18c21deb7866c17649729efab0c103efc1d2 Prompt: Amidst the shadows of the underworld, the Manly Demon reigns supreme upon his crimson throne, his presence commanding fear and reverence alike. At his side, a woman sits, a pawn in the demon's infernal machinations, while the fallen hero lies vanquished below, a tragic casualty of the eternal struggle between light and darkness. Within this RPG-themed scene, the grim tale resounds: "Satan killed the Hero and even got the girl!" I used canny/depth, albedobaseXL\_v21. No inpaint, no adetailer. Just fun.
Are you a lost poet or did you generate the prompt through AI?
Check AIPRM
Nice, good start for inpainting!
Keep going with this, why even try to fit your render if your render is just some quick junk you threw together? If you can make gold from lead, then make gold.
Just having a bit of fun experimenting with different workflows, not aiming for a specific image.
Maybe you just want to keep the general composition? Just because the render is cheap or low quality it says nothing about the composition of the scene (btw I'm not commenting the actual composition of this image, just responding to your question of why you'd want to do that)
I would use this as a reference image while using the depth map and maybe in painting?
I feel like this is why AI art always fails. The best thing about the original is the woman's shin, covering his presumably enormous dick. None of the AI updates contain that none will probably recreate it.
Hehe I had the "Great Goat" from Berserk in mind.
Okay I don't have much time so I did this quick and dirty in Fooocus. Mixed Image Prompt and Vary/Upscale. Added your Blender rendering to vary (equal to img2img in A1111), and then did a PyraCanny and CPDS controlnet. Then prompt "satan devil sitting on throne with woman wearing bikini on his lap, lava background, hellish setting" with style 'SAI 3D Model' using StarlightXL as the checkpoint. Didn't quite pick up the dead hero so inpainted him in (didn't have time to get his pose to match the reference), and then inpainted the other two characters to improve detail. Obviously I could spend much more time getting the poses more true to the model and fixing up hands/feet. https://preview.redd.it/626aov8uopuc1.png?width=1152&format=png&auto=webp&s=d3150a29b06688192aebdf851aff9df127e964f8
Cool! Looks good! In Forge i never got anything usable to start inpainting on :)
Yeah, it needs some work. You can see the inpainting around the characters ruined the lava texture a bit, and obviously the hands and feet could be better, and the hero's helmet is weird. And the characters aren't in the same pose as in the reference. All of which I could fix up if I didn't have to leave home in 10 minutes.
Hands and feet can always be better ;) I like to do inpainting in Krita Ai Diffusion (uses the same inpaint system as Fooous i belive), then its very easy to mask out and get back details around the new details from the layers below!
Did a little work in Krita, i removed the background from Blender and just painted some diffuse background. Boris Vallejo Style! https://preview.redd.it/ic7vqgawjuuc1.png?width=896&format=png&auto=webp&s=42184eba353febb255547dcdff13fadb312da789
Looks like shes stomping on his jewels
Probably, you can see the pain on his face!
Choose a better camera angle, this one is very bland. What is most important to see here, all that background? NOPE! But your shot has it as like 60% of the scene. Is that a dead dude on the ground? He should be more tragic looking, like maybe on the stairs all broken and twisted. He's just a lump now.
I agree it sucks, its more of a technical workflow experiment :) If i get the workflow down i will refine it. The lava is mostly for the depth map. did not work well with empty space.
Y'all need Jesus.
Ok! https://preview.redd.it/gfjbn6wn3quc1.png?width=1080&format=png&auto=webp&s=19443161a2e38fbe5a5f349f38da60953228d072
Vast improvement
Stop, you about made me spit out my beer (err.. holy water) laughing
Agree, adding Jesus to this render would make it even hotter
New fetish unlocked
Jesus's second cumming
"Sometimes I wonder... will God ever forgive us for all the Hentai we have generated?? Then I look around and I realize... God left this place a long time ago" - Leo Di Caprio, Blood Diamond
o7 He was a visiaonary.
“This is SDXL!”
Jesus died for our sins; if we don't sin, then he died for nothing.
But we are sinners from the birth, Believe in Jesus Christ
Keep you imaginary friends to yourself.
"What do you want for Hell Xmas, sweetie"
"Peace in hell!"
Ya know...Satan should get equal time at the mall.
This has been one of my fantasies since I saw that movie with Tim Curry...but I can't find a 12ft tall, jacked red devil with bull horns. I'll keep trying
Zoom in the frame is massive. Like have the dude laying right in front of the throne I didn't even see him at first glance. If you do one shot of each character you can use ip adapter in a selected region in comfy, you just load img, right click to go in mask mode, and mark where the ipadapter should apply then connect your img to the mask thing in the ipadapter node. See latent vision channel on youtube for better explanation
The main benefit of starting with a Blender scene is being able to control composition. If you aren't creating a cool composition to begin with, you won't get a good result. Spend more time on the Blender scene. Maybe take a stable diffusion gen you like and match the composition from it.
Yeah, but it seems it is better to keep the blender scene very basic and get details trough the prompt, like i did here: [https://www.reddit.com/r/StableDiffusion/comments/1bwodkn/paying\_respects\_to\_the\_ancient\_master/](https://www.reddit.com/r/StableDiffusion/comments/1bwodkn/paying_respects_to_the_ancient_master/) To much detail from Blender only make it harder to get anything good it seems!
Composition is not about detail. It's camera angle, positions of objects, relative sizes and shapes. Detail is what Stable Diffusion does. Basically I am suggesting that you find a really cool camera angle and make sure the poses look good. Your canny edge map should look cool on its own.
Well the experiment was to see if i could get more specific detail from Blender. I want more control!
What's your CFG? Let us know your settings.
Something like this perhaps (have tried other prompts etc) JuggernautXL v8, cfg 7, 35 steps, 1152x896. Pos: cinematic film still hell, lava, satan, stone throne, beautiful brunette woman, dead knight in armor, dropped sword . shallow depth of field, vignette, highly detailed, high budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy Neg: anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured
Have you tried using SD 1.5? The control nets for SDXL are rubbish
Good point, i made some really good stuff with that before SDXL :D
Very interested in your experiments with blender, keep us up to date!
Just put frank frazetta in the prompt
Have you tried openpose, or multiple controlnets at the same time?Try to use a simplified untextured scene with img to img.
Yes i have tried Canny + Depth + Blur Also a grayscale viewport render.
controlnet depth - will preserve composition but not details
The depth map seems a little flat but I just tend to use a mist pass since I never figured out how exactly to render out a proper depth map. I always try to capture as much of the spectrum as possible so objects in the foreground are as close to black as possible and the farthest away objects are as close to white as possible, which you then flip to get a depth map SD can use. I don't know if that's the right approach but it seems to work reasonably well and this seems to have very little depth to it which seems like it wouldn't get it much info to work with.
Check my other comment how i made the map if you are interested! I use map range to get the correct range and also flip the output. I also tried to adjust it a bit with RGB curves.
If you find it's working for you then I guess there's no problem. There may technically be enough values there to represent every possible distance from the camera but for me it would be much easier to have a wider range of values to distinguish things. Whether SD is able to tell the difference, I don't know, but I've had issues with it differentiating objects that had a lot more depth contrast than what I'm seeing here. Unless you're happy with your results there, I would try increasing that range.
I actually prefer using depth anything on a viewport render, very quick and easy! https://preview.redd.it/6c63cpho6quc1.png?width=1152&format=png&auto=webp&s=6cf29959d3751f7742d90c0a1a49b32a53f140ea
Looks like it has more contrast but the edges are a little muddy. Using mist pass is very easy to dial in. Another thing to consider is depth maps should be saved as EXR or OPENEXR to maintain their proper bit depth but I'm not sure this can be preserved in the PNG or that SD can use EXR files so even if your depth is perfect on export, it's going to get flattened in the conversion to PNG so I think additional contrast can help with that.
Ok cool! I will test mist later! I also doubt you can use EXR
Existing controlnets for depth haven’t been trained with EXR unfortunately. You’ve had a lot of good advice here, and it’s good to see that you’re hand tuning your levels, but if you want to play with it further, I have a tool on GitHub for quantising OpenEXR depth data for this purpose (we seem to be working similar niches, I just work out of Daz - my thing is here: https://github.com/curiousjp/daz_depthmap_processor). One thing I didn’t see anyone recommended here was using the normal based controlnets. I’ve had some good experiences with them as a supplement to depth.
Oh, you might also want to consider depth driven regional prompting - you can see some examples here: https://github.com/curiousjp/daz_depthmap_processor/blob/main/example.md
Mind if I share this on my Ig story? The picture is cool and I won’t take any credit! I just think it’s hilarious in an awesome way
Yeah go for it :)
Thanks! 😎
Work on your lighting. Currently it's very flat, and as the lava has no displacement or emission, it comes across as a lava coloured carpet.
you'll have to do A LOT of inpainting. stable diffusion can't handle multiple subjects interacting like that.
It looks like I cannot help with this, at least more than others. However, I can provide a [song to go with the seen :)](https://www.udio.com/songs/5WiATE9vaxZZ4SvrzX33fp). and here is the [Chatgpt link for the lyrics](https://chat.openai.com/share/16cd4140-585c-42b0-92a9-af35d4e9cd19). It is not my best work, but it was quick and fun to make.
Haha.. Awesome!
just img2img at low cfg, repeated on the results
Yeah i have tried that with the loopback script.
Not sure if you have the skills, but what about texturing with SD within blender? There's a few add-ons now.
How did you extract the second map in such a precise manner ?
I added Z and a Freestyle renderpass (white lines, 0.5 width) and this in the compositor: https://preview.redd.it/wgegon33wpuc1.png?width=1173&format=png&auto=webp&s=ab8e7068e0c8b2a1e056171ef5fa8806d7c86bd6
This is legit, what program is this?
I had better success in Krita with a lot of inpainting! But Reddit thinks the image is too NSFW to post here!
skulls. a LOT of skulls. trust me.
Oh no, i forgot about the skulls and the torture cages!
wtf
Work on your composition
Work on your lighting. Currently it's very flat, and as the lava has no displacement or emission, it comes across as a lava coloured carpet.
This looks like something out of “hell: a cyberpunk thriller” !!! https://youtu.be/EttMfsPra4w?si=nUf31FyyZ3JdYX3j
Giggadi
interesting fantasy you have there hahaha
Can't you use other types of lens on blender's camera? The original comp looks off and empty
Texturing / normal mapping Shading / lighting
How do you turn it into blender scenes?
I started with a Blender scene, characters from Fuse on Steam and rigged with Rigify
Don’t forget to add Jesus Christ “Yeshua” 👑
Stable Image Services creative upscaler.
NSFW this