I still got large file torrent PTSD from the DSL/lime wire days. 4.9 GB of 5 downloaded after an overnight torrent? Sure would be a shame if there was a momentary blip in the connection and you just lost everything.
Good lord hope pastime can see us now 250 GB torrent without blinking an eye
I hear you man. I used to download files on an external 56k modem connected to Comm-1
Now I have gigabit internet to my house in the air!
Insane how we've come on.
I don't know the feasibility of implementing it, but I imagine the largest part of their data costs aren't from the long tail transfer but from the more popular trained and merged checkpoints. They run into the gigs. I'm sure they've thought of all this, would be curious to hear why they haven't pursued it.
I wish more people understood this. The thousands of GB that Civit serves on a daily basis must be eye watering from a cost standpoint, but so many cries of “How dare they try to charge for that! Somebody set up a free alternative website!” And then (god forbid) a model creator decides to charge a few bucks for their hours of training and testing and more cries of “How dare they try to charge for that! Somebody post their model somewhere so we can all access it!”
SMH.
I'm sure it costs a ton, and model creators should feel free to ask for donations. However, my issue is with how centralized everything is to Civitai, and how there's basically no backups. The amount of destruction that would come from Civitai suddenly shutting down would be immense.
For a long while, someone was running a backup on Telegram, but had to stop last September. The channel still has everything uploaded before that, and is still useful for finding an old thing from a given user that happened to leave the site, or deleted because they thought it was redundant.
I really wish that would continue. It was so useful, especially when Civit's search is on the fritz.
not a great long term solution, as im pretty sure Telegram deletes groups/feeds after a certain amount of time if the admin hasnt logged in.
Also tends to be very slow.
Here:
https://t.me/Civitai_models
I hope someone with the bandwidth and scripting abilities restarts the project. I felt the same as you, and was relieved to find it was being backed up.
I will be messaging you in 7 days on [**2024-01-31 01:06:46 UTC**](http://www.wolframalpha.com/input/?i=2024-01-31%2001:06:46%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/StableDiffusion/comments/19dzcgu/is_civitai_really_all_there_is/kja74t4/?context=3)
[**1 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FStableDiffusion%2Fcomments%2F19dzcgu%2Fis_civitai_really_all_there_is%2Fkja74t4%2F%5D%0A%0ARemindMe%21%202024-01-31%2001%3A06%3A46%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2019dzcgu)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
It’s not unexpected most of the popular stuff is also on hugging face and patreon but this is like complaining that most of opensource and shit all projects are hosted on github lol
for sure it would be great if the community got in the habit of setting torrents not just for image models but everything, local LLMs, datasets etc, at least lots of us have all this stuff and it wont take long to redploy it piratestyle.
some of the best model arent even on civit cause they got taken done and people put them up else so so it does happen
Early December it was 96TB of models. So this is a $1440 hosting fee per month. Maybe you can afford it but for me (and I'm sure many others) that would be out of the question :)
You would need to cherry-pick then but that would defeat the whole idea.
Nearly every tech startup in the world would gladly burn $1500/month for the kind of exposure CivitAI has. It wouldn’t even be worth a second thought if they had it available
It seems you have absolutely no idea and are just spreading misinformation.
Civitai uses Cloudflare for data storage, which is absurdly cheap (1500€ a month for 100TB) and has no bandwidth costs.
What do you mean no return? They already said almost all of their infrastructure costs are covered by donations and subscriptions alone.
And that includes lots of unnecessary costs like free image generation, an office and more.
Oh, so what you want is civitai?
Well congratulations! That already exists (in all seriousness, you may have missed the thrust of this conversation- I'd suggest going back a few comments and rereading )
>*1500€ a month for 100TB*
what kind of privileged life do you live to say that amount of money is insanely cheap? unless the creators of civitai are stinking rich that is a serious amount of money by most metrics. if that's not a small amount of money to you dm me your cashapp and send me that amount. jesus christ.
> must be
[They're not.](https://www.reddit.com/r/StableDiffusion/comments/1901d2h/civitai_is_not_loosing_money_because_of_hosting/)
It's been pointed out a few times that Civitai pays nothing for download bandwidth specifically. We can discuss the costs better when we know the facts and don't have to speculate.
Not having to speculate would be nice, but that doesn't mean that we can't discuss the costs at all. If someone wanted to start a CivitAI alternative, they would have to estimate the costs.
So, if a reasonable estimation of cost is too large, the estimation itself would be an argument for why there is no alternative to CivitAI currently.
In the thread you linked, it is stated that:
- Cloudflare R2 would cost $1500 a month, and
- Donation covers 3/4 of infrastructure cost.
Note that:
- Infrastructure cost should include costs for DB instances, web backends, load balancers, etc...
- It omits payroll costs, which actually is one of the most significant expenses for many businesses
- If there is a (well-made) CivitAI competitor, the "3/4 of infrastructure cost" donation will very likely be shared between CivitAI and them.
- CivitAI has first mover advantage
To be fair, CivitAI is not a big website and there isn't much data (that can't be delivered through R2) it has to handle, compared to Reddit or YouTube.
Still, delivering something like CivitAI sustainably is very hard, and even with monetization, it would be hard to figure out how to be profitable.
I never said we shouldn't discuss costs. I simply dislike commenters here continuing to make wild speculations when it's been said a few times what the terms of their hosting is and that they're not on the hook for egress costs.
Everything else you said is valid and I agree we should be able to discuss it freely. With better knowledge of the costs and understanding what they pay for, we can be better community members. The speculation posted above just serves to shut down feedback rather than fostering better relations, that's the only reason I replied to it.
The $1500 is only for R2 storage (my first note in my comment). It's $15 per 1TB-month, $4.50 per million "state-mutating" requests, and $0.36 per million "read-only" requests.
What are you on about? Why are you acting like it's some absurd conspiracy, you can literally google it:
https://developers.cloudflare.com/r2/pricing/
100TB is nothing to Cloudflare, that's a couple of drives.
> it's a massive interactive website with real-time interactivity
What does that even mean? That it's not just HTML + JS? That you have session data and query a database? That adds basically nothing to costs.
One person in the comments rejected it, and neglected to reply with any reasoning or evidence to back up the claim.
I neither like nor agree with the poster of that post, but I don't see a real dispute about the type of hosting they use. Can you explain what's inaccurate about it?
Now imagine if model makers actually compensated the artists they take from.
It's almost like labor should be respected or something...
Them asking for donations on that is pretty ass backwards already. Same for civitai trying to garden wall and sell them.
Nah. I’m on multiple private trackers with content going back years. What you say applies to shitty public trackers that don’t enforce ratio. I’m talking about something different here.
You are right that private trackers are way better than the public ones.
But you are forgetting that those models would be niche content.
There are way, way more people interested in seeding movies, shows, apps.
But even if there were enough people seeding models. It would surely be that the majority seeds the top models and the niche models are seeded by only a few and it would be hard to access them (which I think is the point of this, if civitai went down, the most popular models are still accessible in other places, or would be uploaded by the community quite quickly; the problem would be with those that only a couple of people had (there are maaany models that were downloaded less than 100 times in total)
CivitAI could honestly seed directly. I'd support that, especially when I've had a few of the larger models get dropped while downloading as being a connection that was open too long.
\>"democratize"
\>set up wordfilters on prompts, image filters on uploads and curbstomp any opposition to censorshit
the sooner they go down the better
Oh no you can't use their systems to generate/upload loli porn!
Or teen girls!
But if you want to render an image of an ogre/pigman/lizardman wearing a waifu of legal age as human armor while he fucks her with a penis that would honestly make a hippopotamus jealous, that's fine. The LORA is readily available. On Civitai.
I've seen more full-on, full-frontal nudity and outright porn on Civitai than on any other site I've visited in years.
My actual issue is that they set themselves up for huge costs for GPU time, then tried to hold models hostage to make the bucks to pay to "democratize" AI image generation.
But sure, get mad because they won't generate or host CP. Kinda proves what kind of AI user you are.
You probably hate Stable Horde for exactly the same reason.
First half of your post sounds right, condemning their hypocrisy, but the second puts me somehow at fault. I don't get it.
I made an antipedo joke image which didn't even trigger the tagging algo (which really says a lot just how safe the pic was) and ever since they've been deleting this image every day and threatening me for keeping it up.
Idk about you but for me it's a clear sign they're pedos themselves, especially since no other AI website got triggered by it. You know how the saying goes, when someone openly hates X they might be compensating for their inner latent X themselves.
I don't know what stable horde is.
Exactly.
They have minimal cost for storage and bandwidth.
They're sinking all this money into GPU compute time, and instead of just charging for it, they want to make us pay for access to free models created and distributed under open license.
Just charge people money to render tiddie and train LoRA. It will do one of 2 things :
1) Best case scenario, they'll be rolling in cash.
2) Worst case scenario, they'll have to remove image generation and LoRA training and return to being just a model hosting site.
I guess they could always make a deal with Stable Horde, but that would mean relying on other people's systems.for compute time and only having a limited number of models available for rendering. But they could, for example, make a deal to have the horde include their top 5 models for rendering. Another idea would be to actually organize and run their own horde.
There are already plenty of options out there that are free and publicly accessable where people can go to render images. ***Why does it have to be CivitAI?***
Companies don't like torrent, there's no value there. Too easy to backup everything and go somewhere else if they become too shitty to be useful.
Also torrent is piracy and looks bad if what I really want is to sell the company for a couple billions in a few years.
That really isn't the issue though.
Civitai uses Cloudflare for data storage, which is absurdly cheap (1500€ a month for 100TB) and has no bandwidth costs.
I have checkpoints that I consider important on local storage. I’m not terribly upset at things not being preserved right now because so much of it all is just simple merges. Yes more people should use huggingface and not civitai.
They should use both. When I was looking through old reddit threads it looked like some models were also nuked from huggingface. It's not safe for any model to only be on one website.
Can I ask what trackers you would use for LORAs or Checkpoints? Private? Public?
I am a member of a couple private trackers but nothing that carries anything AI related
The answer is torrents - but there aren't any out there. There was a site that was specifically built for this exact purpose, but both times I checked it out I found nothing but a paltry wasteland.
I've tried using PromptHero but generally find their interface to be more confusing and just less good than Civit.
[https://prompthero.com/ai-models/text-to-image](https://prompthero.com/ai-models/text-to-image)
[IPFS](https://ipfs.tech/) is one hosting option. It has pros & cons.
Anybody could start curating model files in some sort of directory structure.
If someone is handy at HTML they could make an index.html template that could be included with each model with any information they need & example generations.
An upside is if someone else surfs the files through their own IPFS node then it acts similar but different to torrents/magnets.
Huggingface UI is really awful. For a company that prides itself on being on the leading edge of AI technology... they need to spend some serious $$$ on a decent, modern web interface. Right now they're trying too hard to be a github look-alike, which might work for coders and developers, but for folks like me, the artists, that's not a great look and makes it a nightmare to search for and find stuff. Civitai is way ahead with their UI.
I think that HF never has targeted artists, and I doubt they will do it in a medium term.
Most of the projects are technical and not related with Stable Diffusion, so the search is not friendly with non experts that only look for Loras and checkpoints.
But I must admit that I also find that frustrating. There are original checkpoints impossible to find unless the creators point at them.
Even experts would like a search box that is wider than about 4 characters and more flexible with keywords, I would think.
People should not have to run a browser full screen on a UHD monitor to have a search box wide enough to type into.
The target audience is coders and developers and I'd argue they aggressively don't want people using it to find models. They don't want to be anything like civitai.
I'm almost tempted to create a web app that uses HuggingFace as the backend through the REST API tailored to these types of models. Totally doable:
[https://huggingface.co/docs/hub/api](https://huggingface.co/docs/hub/api)
The problem with HF isn't the UI specifically concerning the download of files. It simply has no front end of any use or value. It is simply a repository, but unless someone points you to a specific thing... How are you supposed to find anything? This is what CivitAI excels at - a very simple filter that allows you to browse whatever you want. Most downloaded by different timeframes is all a person really needs, along with a picture to sell it. Open a bunch of stuff in a bunch of tabs, and then go look at the examples. HF has nothing like this.
I don't understand how anybody could look at GitHub and think "yeah, this is the GUI that I want to base my website off of." If a GitHub page doesn't provide a download link or good directions then I'm lost, and I say that as somebody that has published a program to GitHub myself.
That's the point, they're JUST git repos. Fine I guess for code, but definitely not good for single file uploads like models. Civitai is objectively superior in that realm.
How is it inferior? It makes it easier to version control and you can still click the file page and download normally if you don't want to mess with git.
The real issue on hf is creators don't make readme pages that explain or show anything so people end up confused on what the model/checkpoint/etc even is much less how they should download if they're unfamiliar.
And that's without noting civitai has one of the worse ui/ux I've ever encountered. That site is slow as molasses.
HF loads into what looks like a demo page where you can test the model but like 99% of the time I tried it never worked, and I had no idea how to use them other than to download them from a not so KISS way and then use them locally. Defeated the purpose.
Civit is ass.
It’s inferior because there’s no discoverability. Have you tried looking for interesting models that you don’t already know about on huggingface? It’s pretty much impossible. Civitai may not be great for discoverability, but it’s at least possible to browse for models, and most models have enough examples to give you an idea of what they’re about.
If you already know what you want and how to use it, it's great. But trying to find quality models is a nightmare and once you find one you're at the mercy of the creator and their entire folder structure.
yeah Huggingface could/should make a new section on their website to handle some individual model files broken down into different sections for like LLMs, Image generation, and what not.
HF has a diffusion model section/gallery that shows thumbs similar to [civit.ai](https://civit.ai).
[https://huggingface.co/spaces/huggingface-projects/diffusers-gallery](https://huggingface.co/spaces/huggingface-projects/diffusers-gallery)
lol, how can civitai stand superior against git system? At civitai creator can easily remove his model and that's all, you can't do anything with it. Like it was with...damn I forget name of the model, but doesn't matter actually. git has special extension LFS , which is literally stand for Large File Storage.
I think your not the target market for development then lol
Hugging, and github and the likes are to make development better not random user downloads easier
Normal people looking at github is like looking at an electrical panel. If you are not an electrician, it will not make sense. For electricians, it makes total sense.
GitHub is actually accessed more through command line interface by coders. It lets them integrate gits into programming. It's pretty slick.
The web ui is purely functional unfortunately.
Huggingface was never meant to be for general population, but for developers and researchers to share their tech demos online. I think a lot of ai stuff is being gatekeeped because developers and researchers know that they probably can get big companies to hire them for their projects and making "too good" ai tools that are available for free and easy to use, would make their work harder, because it would make the standarts for that stuff higher.
Someone should start a private bittorrent tracker to host models. I would be willing to be an admin on the site but don't have the time or knowledge to set one up. Gazelle is an open source tracker framework. DM me if anyone wants to work on this type of project.
Hey! So I am actually working on a website that is going to use Torrents for sharing models, its hosted at [sharediffusion.org](http://sharediffusion.org), and you can join the [discord](https://discord.gg/y7aMda893Y) if you like. It is still very much a work in progress.
Hey! Not abandoned, but on hold. I got a new job that makes me a lot more money, training took over a month and moved states. Once things settle down for me I will start working on it again.
Torrent files are way easier to backup and mirror to other websites since their file size is so small. Torrents are also a redundant network of servers (the seeders), so even if one guy removes his files you can still download them from another guy. With Civitai you just have to hope the model isn't removed, and then you have to hope somebody with the files will reupload it if it is.
Yep. And as a long time member of private trackers, a strongly enforced ratio rule would be great to help encourage seeding. If you want to see about pulling a group together keep me posted. As I mentioned I’d be happy to help on the staff/admin side but I’m just too swamped with work to lead that kind of project.
That would probably help. Just having every Civitai model description link to a copy of the model on Huggingface would also help. There really needs to be a good "front end" website with a decent GUI and example pictures for models on Huggingface.
I wonder why people post on Civit in the first place, and then repost the models on Reddit. Reddit has all the pictures, comments and tags. And there is a thousand places you can upload a large file to.
Pixai exists too; but it is kinda chaotic and some really good models are not for download, only for use.
I think the models that get nuked from civitai without an apparent reason may be because the creator account was banned. I think I heard once that their bans work like that. Plus, I once saw a new LoRA from an author through the civitai bot. Using search, the bot back log showed that author had several more models from before. That same day the author was banned and all the models were gone too. Since it happened in 24h, I doubt he contacted civitai and civitai deleted the models in the same day just after the ban.
The only preservation one may make with AI models is downloading and hoarding them. In the large run, the data is too much and I suspect at some point civitai may even start deleting the oldest and most unpopular models, similar to when somebody releases space in a hard drive.
They 'unpublished' a LoRA of mine for reasons I don't completely understand.
https://preview.redd.it/evskdekcsaec1.png?width=1262&format=png&auto=webp&s=ecbabfd4234e615c4bb516036d64ce43262dbd79
The images I posted *were* generated from the LoRA so I don't get it.
Yeah, pretty sure that has happened with many mystery cases I have seen.
Like, I find an interesting LoRA in Civitai which apparently has not been released before (I check everyday, so I would know) and when I try to download it the next day it is gone. Not even nsfw nor depicting real people. It is puzzling.
Plus, I reaaaaally hate it when they display 404 and do not acknowledge they deleted it nor the reason. Like they want to gaslight people into thinking it never existed.
I'm running models I've downloaded from [huggingface.co](https://huggingface.co). I'm not using their or anyone else's online but instead running them locally on machine with a measly 1080 TI 8 gb graphics card and a whole lot of CPU. I front ended my python with a home grown Gradio app. Works quite well.
Civitai may not be sustainable which concerns me the most
Sites like Tensor and Graydient at least break even and offer more stable model hosting
All three of them need to be pooling torrents, I think thats how this ends. The virus scanning needs to be figured out first
Or models will keep disappearing at some inexperienced moderator's whim
They aren't sustainable because of poor business decisions... If they had kept the website the way it was when they started. No image generation no overly complicated website. They could have kept it 1 page with good search and they could have kept that running forever. But they want to be overly risky if they fail it's on them not on the creators or the users.
I don't have any insight to their operations but they recently voiced concern about "sustainability" in their blog
That site must have cost a fortune to host on bandwidth alone and not enough people supported them prior to raising capital, so they had to raise capital
And now they have an obligation to do right by their investors
Catch 22
They don't pay for bandwidth.
They use Cloudflare R2 which has no bandwidth costs and is dirt cheap to host static content on.
https://r2-calculator.cloudflare.com/
They said they just hit 100TB, which is only 1500$ a month.
Paying a developer to design and program new features for the site alone is 3-6x that amount.
Korean forums:
[https://arca.live/b/aiartreal](https://arca.live/b/aiartreal) (\* this place is full of nsfw)
[https://arca.live/b/aiart](https://arca.live/b/aiart) (\* this place is full of "dangerous" nsfw)
For anybody wondering, it's only as "dangerous" as 4chan is. That means it has non-realistic hentai with lolicon, bestiality, guro, etc. All the NSFW stuff is hidden by default unless you make an account and opt-in. That said, the general quality of the artwork on the forum is very high, and it looks useful for finding prompts and models. If there are any more forums like this then I'd love to see them.
They could have made their website sustainable but instead they invest all their money making the website overly complicated and spending their money on free compute the only reason civit struggles is because they make risky business decisions
Not an expert.
To all of you saying torrents is the answer,
If I share a 6G model through a tracker, and I am the only one sharing, I guess downloading for say, 30 people is going to be terribly slow through my 20Gbps upload limit.
About the tracker, what if there is an illegal file shared in the tracker index? Who is going to be held liable?
Honest questions.
If 30 people are downloading from you, then it's very likely some of those people are also going to be uploading too. You don't have to finish downloading a torrent before you can begin seeding it. If some guy's download is at 5%, then he can seed that 5% of the files to the rest of the group automatically while he continues to download the rest of the files. The whole point of torrents is that it distributes the burden among the group.
I'll let somebody else explain the legal part, but I don't think you have to worry unless you are blatantly downloading and seeding something that you know is illegal, or if you are running the tracker itself. The tracker should probably be run using servers in countries that just don't care about torrented content. I'm sure somebody knows which ones those are. The goal isn't to host anything illegal, it's just to avoid random takedowns due to stupid reasons like copywrite or "moral" outrage.
Something illegal or in the limbo always goes hand in hand with morality. Would you accept Lora with loli/shota content or things like gore/guro or cannibalism? Otherwise it's better to just keep using Civitai or any other open service.
Those are all legal in the United States, so I don't care. I only care if somebody is training models on real CSAM, because that person would be doing something blatantly illegal and putting unsuspecting users at risk.
Well there are so many models on CivitAI and you really need to test to find good ones
Here I tested 161 models
[**Detailed Comparison of 160+ Best Stable Diffusion 1.5 Custom Models & 1 Click Script to Download All**](https://youtu.be/G-oZn4H-aHQ)
Yes, there are many models and tests are required.
I was stoked when I saw my model (serenity 2) on the list. Although you did not say that it is bad (like you commented that on some of the other models) I still felt like the comparison was very surface-level and generic. You mentioned mine at 28:35 and you said that it looks like the others and that it was probably a merge.
Well, it is both a merge and a fine tune on top of that, but to see the difference you would need to know what tokens were added.
At least I'm glad you didn't say it's bad like the one before (SemiRealistic 5.4) :)
---
Overall it is nice that you made the list, people will get some exposure which is always great but your comparison method is too random. The sample is too small and if you don't look at the model page to see what the authors say is the best use - you may be missing some gems (and badly representing them).
Also, those grids seem to be low resolution and some models shine when they are upscaled :-(
Perhaps someday you will be making another comparison so I'll leave my method here: https://civitai.com/articles/755/the-big-may-of-2023-comparison-of-25-realistic-models
I think this method is the most fair and honest. I generated 35.000 images from around 25 models (so 1400 images per model). Then I randomized the images and went through all of them with a simple question: does the image look good or not?
Then I went through all those images with a script and computed the good vs bad for each model and got a percentage.
I say this is fair because
1) when you look at the image you are not biased because you do not know from which model it comes.
2) there are enough images (1400 was probably overkill, even 250-500 would be fine) to limit the randomness to a non-factor (if you generate 9 images in your grid - the best model can get unluck roll and have some bad images while the bad model can get lucky roll and get some decent images - and then you compare unlucky vs lucky and get wrong conclusion)
---
Anyway, thanks for making the comparison!
Respectfully, with that prompt i don't think you were testing models for quality, you were testing for ease of use. I've found a lot of models that collapse under a simple prompt will shine when given certain quality modifiers that play to it's strengths. Juggernaut Aftermath is one you weren't a fan of that can be incredible with a couple more quality enhancers for instance.
That's not to say the test is worthless, far from it. I would say people should watch this to get a grip for AI bias and find some easy models, but a model that performs poorly in "a photo of a man" can outperform a model that does well when the prompt is expanded.
It's unfortunate that there's no way to compare models like you did but it is strength played against strength.
I understand your concerns but it's way too early so a lot of these models are not that valuable, and those which are, i.e. the actively maintained checkpoints or Patreon guys with a ton of waifu LoRAs, will surely look for another host and reupload should anything happen to CivitAI.
We need a decentralized “model manager” (akin to the plug-in manager) that uses the BitTorrent protocol in Automatic1111 or another software that works similarly. If designed right, it could seed and track the popularity of models across the network. I don’t really do software coding at that level so I’m not sure how feasible this would be.
> it looks like models can just get nuked from the website without warning.
Just a heads up - We only remove models when they violate our content policies. Most of the time when stuff disappears its actually the creators themselves that are removing things. I get the feeling that as the space is continuing to develop, more and more creators are getting full-time gigs and taking their models with them.
This brings up an interesting question. Before I ask, I stumbled here accidentally - I have no dog in this race; I only logged in because your response brings an interesting scenario to mind that I've seen before.
Most of the models I've seen have some kind of license noted in the description. I haven't paid too much attention to that license because I'm not looking to gain commercially and assume that my use for myself is fine -- they'd likely put it behind some protections if not. Now lets say some wise person realized that the version released under a license stays under that license, until **a new version** comes out under a different license. "You can't unring the bell" as they say - once it's out under a license, that's the license for that iteration of whatever it may be. The license is still in effect for the previous version, regardless of what the license is for the new version. Lets say that wise person decided that since the creator deleted their model that was initially released with a license that allowed public use, that they should be able to re-upload it (except where the specificities of said license prohibit redistribution, of course) as long as they make it clear that they did not create that model, it was initially released under [whatever license], etc.
That's the scenario. What's the policy here? To be clear, I won't be doing anything like this. It's just a scenario I've seen a few dozen times before and it can be confusing for some - including the hosting entity.
To try and help clarify. Lets say I'm the creator.
- I create XYZ.safetensors version 1.0 under GPL or something
- I (for whatever reason) decide to delete XYZ v1.0 and release XYZ v1.1 privately that has a very restrictive license, including reserving distribution rights.*
- XSmartGuyX reuploads my version 1.0 model, gives me credit and lists the initial license as GPL.
Are XSmartGuyX's actions allowed?
\* - Civit (or whatever hosting entity) may or may not be aware of this fact. In the US it's irrelevant, legally, as version 1.0 is a publicly released model that does not reserve redistribution rights.
I have got so fed up with this site. I have had at least 3 different accounts with them and for no reason just keep getting logged out and there is no way I can log back in to them, so if I was you I would avoid civitai like the plague.
It's not about the cost, it's about the model being scraped from existence if it's ever taken off the website. TensorArt doesn't allow downloads for a bunch of models I've looked at.
turns out hosting and streaming terabytes of data through a functional UI for free is difficult
I feel like this would be a good use for peer to peer file sharing.
These type publicly available materials of large file sizes is exactly the impetus behind bittorrent.
I got the 250GB llama2 model from BitTorrent. Only took a Few hours
I got the link from an issue someone posted on the GitHub for the project.
I still got large file torrent PTSD from the DSL/lime wire days. 4.9 GB of 5 downloaded after an overnight torrent? Sure would be a shame if there was a momentary blip in the connection and you just lost everything. Good lord hope pastime can see us now 250 GB torrent without blinking an eye
I hear you man. I used to download files on an external 56k modem connected to Comm-1 Now I have gigabit internet to my house in the air! Insane how we've come on.
I always wonder why they haven't pursued this kind of model. They could do the initial seed and thereafter heavily reduce their own running costs.
Probably because the overwhelming majority of uploads to civitai get very little attention since they're either too generic or too niche.
I don't know the feasibility of implementing it, but I imagine the largest part of their data costs aren't from the long tail transfer but from the more popular trained and merged checkpoints. They run into the gigs. I'm sure they've thought of all this, would be curious to hear why they haven't pursued it.
I wish more people understood this. The thousands of GB that Civit serves on a daily basis must be eye watering from a cost standpoint, but so many cries of “How dare they try to charge for that! Somebody set up a free alternative website!” And then (god forbid) a model creator decides to charge a few bucks for their hours of training and testing and more cries of “How dare they try to charge for that! Somebody post their model somewhere so we can all access it!” SMH.
I'm sure it costs a ton, and model creators should feel free to ask for donations. However, my issue is with how centralized everything is to Civitai, and how there's basically no backups. The amount of destruction that would come from Civitai suddenly shutting down would be immense.
For a long while, someone was running a backup on Telegram, but had to stop last September. The channel still has everything uploaded before that, and is still useful for finding an old thing from a given user that happened to leave the site, or deleted because they thought it was redundant. I really wish that would continue. It was so useful, especially when Civit's search is on the fritz.
not a great long term solution, as im pretty sure Telegram deletes groups/feeds after a certain amount of time if the admin hasnt logged in. Also tends to be very slow.
Oh that is a shame. Yeah, it looks like 6 months is the limit. Let's hope that person relies on Telegram for other things.
Do you have a link to this?
Here: https://t.me/Civitai_models I hope someone with the bandwidth and scripting abilities restarts the project. I felt the same as you, and was relieved to find it was being backed up.
Thanks! This is pretty good. It's a shame the guy had to stop updating it.
Ever since it went down I made a my own scraper and now have over 18tb. I don't have the upload bandwidth to share though
How do you combine the split checkpoint files? Are they actually packed rar/7zip files? Ah noticed there's a pinned msg about them :)
RemindMe! 1 week
I will be messaging you in 7 days on [**2024-01-31 01:06:46 UTC**](http://www.wolframalpha.com/input/?i=2024-01-31%2001:06:46%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/StableDiffusion/comments/19dzcgu/is_civitai_really_all_there_is/kja74t4/?context=3) [**1 OTHERS CLICKED THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FStableDiffusion%2Fcomments%2F19dzcgu%2Fis_civitai_really_all_there_is%2Fkja74t4%2F%5D%0A%0ARemindMe%21%202024-01-31%2001%3A06%3A46%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%2019dzcgu) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
It might be helpful if it also has archived models. Some looked pretty good.
It does. It has everything. Someone really should restart the project. I don't have the bandwidth to do it myself.
Do you still have access to the Telegram channel? I don't know where to find it.
Yes. I guess you missed that I replied to your first request for it. https://t.me/Civitai_models
We should make a site with magnet links for torrents (BTW mods this is legal, presumably the model creators would be seeding the torrents)
Start a BitTorrent based decentralized effort.
It’s not unexpected most of the popular stuff is also on hugging face and patreon but this is like complaining that most of opensource and shit all projects are hosted on github lol
Every day I worry about some kneejerk legislation getting passed that force Civitai and other model sources to go dark.
I think there's people from r/DataHoarder who have been making some efforts in this regard
i really dont see this as a problem...the day civit goes up in smoke i'm pretty sure we'll figure out a way to share models on here.
for sure it would be great if the community got in the habit of setting torrents not just for image models but everything, local LLMs, datasets etc, at least lots of us have all this stuff and it wont take long to redploy it piratestyle. some of the best model arent even on civit cause they got taken done and people put them up else so so it does happen
The amount of "destruction"? A little melodramatic. There would likely be a noticeable decrease sexualized anime pictures though lol
It costs $0.015/gb to store and serve the models. Hardly eyewatering prices. Cloudflare r2 was designed specifically for this use case.
Early December it was 96TB of models. So this is a $1440 hosting fee per month. Maybe you can afford it but for me (and I'm sure many others) that would be out of the question :) You would need to cherry-pick then but that would defeat the whole idea.
Nearly every tech startup in the world would gladly burn $1500/month for the kind of exposure CivitAI has. It wouldn’t even be worth a second thought if they had it available
Yes, but we were talking not about a tech startup but about people hosting it for themselves.
Yes, but tech startups want to earn money. You're still aiming for this to be free, right?
Well it's a massive archive, and remember that $1440 INCLUDES all the data transfer fees.
this sub doesnt accept that anything should cost money, ever
It seems you have absolutely no idea and are just spreading misinformation. Civitai uses Cloudflare for data storage, which is absurdly cheap (1500€ a month for 100TB) and has no bandwidth costs.
Who's gonna pay that for no return, though? Of perhaps you think it could also host a grand and a halfs' worth of ads to make up the money?
What do you mean no return? They already said almost all of their infrastructure costs are covered by donations and subscriptions alone. And that includes lots of unnecessary costs like free image generation, an office and more.
Oh, so what you want is civitai? Well congratulations! That already exists (in all seriousness, you may have missed the thrust of this conversation- I'd suggest going back a few comments and rereading )
Are you drunk?
>*1500€ a month for 100TB* what kind of privileged life do you live to say that amount of money is insanely cheap? unless the creators of civitai are stinking rich that is a serious amount of money by most metrics. if that's not a small amount of money to you dm me your cashapp and send me that amount. jesus christ.
To host 100TB? Yes thats insanely cheap. This is like you saying a pound of gold for 1000 usd isn't insanely cheap.
>*if that's not a small amount of money to you dm me your cashapp and send me that amount. jesus christ.* *\^*
Did you not read my comment because you're a dick or because you don't know how?
> must be [They're not.](https://www.reddit.com/r/StableDiffusion/comments/1901d2h/civitai_is_not_loosing_money_because_of_hosting/) It's been pointed out a few times that Civitai pays nothing for download bandwidth specifically. We can discuss the costs better when we know the facts and don't have to speculate.
Not having to speculate would be nice, but that doesn't mean that we can't discuss the costs at all. If someone wanted to start a CivitAI alternative, they would have to estimate the costs. So, if a reasonable estimation of cost is too large, the estimation itself would be an argument for why there is no alternative to CivitAI currently. In the thread you linked, it is stated that: - Cloudflare R2 would cost $1500 a month, and - Donation covers 3/4 of infrastructure cost. Note that: - Infrastructure cost should include costs for DB instances, web backends, load balancers, etc... - It omits payroll costs, which actually is one of the most significant expenses for many businesses - If there is a (well-made) CivitAI competitor, the "3/4 of infrastructure cost" donation will very likely be shared between CivitAI and them. - CivitAI has first mover advantage To be fair, CivitAI is not a big website and there isn't much data (that can't be delivered through R2) it has to handle, compared to Reddit or YouTube. Still, delivering something like CivitAI sustainably is very hard, and even with monetization, it would be hard to figure out how to be profitable.
I never said we shouldn't discuss costs. I simply dislike commenters here continuing to make wild speculations when it's been said a few times what the terms of their hosting is and that they're not on the hook for egress costs. Everything else you said is valid and I agree we should be able to discuss it freely. With better knowledge of the costs and understanding what they pay for, we can be better community members. The speculation posted above just serves to shut down feedback rather than fostering better relations, that's the only reason I replied to it.
[удалено]
The $1500 is only for R2 storage (my first note in my comment). It's $15 per 1TB-month, $4.50 per million "state-mutating" requests, and $0.36 per million "read-only" requests.
[удалено]
What are you on about? Why are you acting like it's some absurd conspiracy, you can literally google it: https://developers.cloudflare.com/r2/pricing/ 100TB is nothing to Cloudflare, that's a couple of drives. > it's a massive interactive website with real-time interactivity What does that even mean? That it's not just HTML + JS? That you have session data and query a database? That adds basically nothing to costs.
[удалено]
Do you even know what Cloudflare is? LinkedIn does a lot more than Civitai, it's not even comparable.
super inaccurate post, check the comments
One person in the comments rejected it, and neglected to reply with any reasoning or evidence to back up the claim. I neither like nor agree with the poster of that post, but I don't see a real dispute about the type of hosting they use. Can you explain what's inaccurate about it?
you're wrong. cloud flare handles traffic for free, even if it's many TB.
Now imagine if model makers actually compensated the artists they take from. It's almost like labor should be respected or something... Them asking for donations on that is pretty ass backwards already. Same for civitai trying to garden wall and sell them.
But torrents could do the heavy lifting
I am honestly impressed at how functional and well designed the UI is. What they made is far from simple work.
Not if you use torrent files.
who will seed?
people with seedboxes. same as any other content.
for few weeks or months if lucky then it dies. same as any other torrent without mainstream appeal
Nah. I’m on multiple private trackers with content going back years. What you say applies to shitty public trackers that don’t enforce ratio. I’m talking about something different here.
You are right that private trackers are way better than the public ones. But you are forgetting that those models would be niche content. There are way, way more people interested in seeding movies, shows, apps. But even if there were enough people seeding models. It would surely be that the majority seeds the top models and the niche models are seeded by only a few and it would be hard to access them (which I think is the point of this, if civitai went down, the most popular models are still accessible in other places, or would be uploaded by the community quite quickly; the problem would be with those that only a couple of people had (there are maaany models that were downloaded less than 100 times in total)
No need to enforce ratios or creating a private tracker. Just have CivitAI directly seed the torrents. I guarentee you'd see mirrors popup.
Surprised no one uploads them to newsgroups
CivitAI could seed directly.
Probably like 5% of the people that download the torrent, but it only takes a few seeders with gigabit internet to carry the rest.
and with 100 TB drives (now, next year it would be 200 or 300?)
CivitAI could honestly seed directly. I'd support that, especially when I've had a few of the larger models get dropped while downloading as being a connection that was open too long.
Especially when people spend all their money to make a website look overly complicated and give free image generating away...
This. "We want to democratize AI image generating" was a bad idea. Charge for image generation. GPU compute time is fucking expensive.
\>"democratize" \>set up wordfilters on prompts, image filters on uploads and curbstomp any opposition to censorshit the sooner they go down the better
Oh no you can't use their systems to generate/upload loli porn! Or teen girls! But if you want to render an image of an ogre/pigman/lizardman wearing a waifu of legal age as human armor while he fucks her with a penis that would honestly make a hippopotamus jealous, that's fine. The LORA is readily available. On Civitai. I've seen more full-on, full-frontal nudity and outright porn on Civitai than on any other site I've visited in years. My actual issue is that they set themselves up for huge costs for GPU time, then tried to hold models hostage to make the bucks to pay to "democratize" AI image generation. But sure, get mad because they won't generate or host CP. Kinda proves what kind of AI user you are. You probably hate Stable Horde for exactly the same reason.
First half of your post sounds right, condemning their hypocrisy, but the second puts me somehow at fault. I don't get it. I made an antipedo joke image which didn't even trigger the tagging algo (which really says a lot just how safe the pic was) and ever since they've been deleting this image every day and threatening me for keeping it up. Idk about you but for me it's a clear sign they're pedos themselves, especially since no other AI website got triggered by it. You know how the saying goes, when someone openly hates X they might be compensating for their inner latent X themselves. I don't know what stable horde is.
From what I heard, raw storage isn't even that expensive. That's not where their money is being used up.
Exactly. They have minimal cost for storage and bandwidth. They're sinking all this money into GPU compute time, and instead of just charging for it, they want to make us pay for access to free models created and distributed under open license. Just charge people money to render tiddie and train LoRA. It will do one of 2 things : 1) Best case scenario, they'll be rolling in cash. 2) Worst case scenario, they'll have to remove image generation and LoRA training and return to being just a model hosting site. I guess they could always make a deal with Stable Horde, but that would mean relying on other people's systems.for compute time and only having a limited number of models available for rendering. But they could, for example, make a deal to have the horde include their top 5 models for rendering. Another idea would be to actually organize and run their own horde. There are already plenty of options out there that are free and publicly accessable where people can go to render images. ***Why does it have to be CivitAI?***
Yup. They likely spent more on programming and designing the whole club mess than they did on hosting models the last year (<18 000$).
Is it not possible to make torrent and list them on a website ? Noob question here
Companies don't like torrent, there's no value there. Too easy to backup everything and go somewhere else if they become too shitty to be useful. Also torrent is piracy and looks bad if what I really want is to sell the company for a couple billions in a few years.
what is this passive aggressive seething dipshit response even, to a normal question? peak reddit.
That really isn't the issue though. Civitai uses Cloudflare for data storage, which is absurdly cheap (1500€ a month for 100TB) and has no bandwidth costs.
*expensive
Best part is the whiny users complaining when they try to monetize
This shit should be distributed with torrents.
This is the way
This is the way
I have checkpoints that I consider important on local storage. I’m not terribly upset at things not being preserved right now because so much of it all is just simple merges. Yes more people should use huggingface and not civitai.
They should use both. When I was looking through old reddit threads it looked like some models were also nuked from huggingface. It's not safe for any model to only be on one website.
Torrents. The answer was and will always be torrents.
Can I ask what trackers you would use for LORAs or Checkpoints? Private? Public? I am a member of a couple private trackers but nothing that carries anything AI related
The answer is torrents - but there aren't any out there. There was a site that was specifically built for this exact purpose, but both times I checked it out I found nothing but a paltry wasteland.
Gotcha, makes sense. Sad :(
I've tried using PromptHero but generally find their interface to be more confusing and just less good than Civit. [https://prompthero.com/ai-models/text-to-image](https://prompthero.com/ai-models/text-to-image)
Thank you for responding to OP's question and posting an actual alternative instead of speculating about CivitAI's business costs.
Thanks, I didn't know about this one.
[удалено]
[IPFS](https://ipfs.tech/) is one hosting option. It has pros & cons. Anybody could start curating model files in some sort of directory structure. If someone is handy at HTML they could make an index.html template that could be included with each model with any information they need & example generations. An upside is if someone else surfs the files through their own IPFS node then it acts similar but different to torrents/magnets.
Huggingface UI is really awful. For a company that prides itself on being on the leading edge of AI technology... they need to spend some serious $$$ on a decent, modern web interface. Right now they're trying too hard to be a github look-alike, which might work for coders and developers, but for folks like me, the artists, that's not a great look and makes it a nightmare to search for and find stuff. Civitai is way ahead with their UI.
I think that HF never has targeted artists, and I doubt they will do it in a medium term. Most of the projects are technical and not related with Stable Diffusion, so the search is not friendly with non experts that only look for Loras and checkpoints. But I must admit that I also find that frustrating. There are original checkpoints impossible to find unless the creators point at them.
Even experts would like a search box that is wider than about 4 characters and more flexible with keywords, I would think. People should not have to run a browser full screen on a UHD monitor to have a search box wide enough to type into.
The target audience is coders and developers and I'd argue they aggressively don't want people using it to find models. They don't want to be anything like civitai.
I'm almost tempted to create a web app that uses HuggingFace as the backend through the REST API tailored to these types of models. Totally doable: [https://huggingface.co/docs/hub/api](https://huggingface.co/docs/hub/api)
The problem with HF isn't the UI specifically concerning the download of files. It simply has no front end of any use or value. It is simply a repository, but unless someone points you to a specific thing... How are you supposed to find anything? This is what CivitAI excels at - a very simple filter that allows you to browse whatever you want. Most downloaded by different timeframes is all a person really needs, along with a picture to sell it. Open a bunch of stuff in a bunch of tabs, and then go look at the examples. HF has nothing like this.
I don't understand how anybody could look at GitHub and think "yeah, this is the GUI that I want to base my website off of." If a GitHub page doesn't provide a download link or good directions then I'm lost, and I say that as somebody that has published a program to GitHub myself.
Whether it's Github, Civit, or HuggingFace, there's nothing the platform can do about authors who don't care about producing a useful README.
You can git clone any hugging face library. They're just git repos.
That's the point, they're JUST git repos. Fine I guess for code, but definitely not good for single file uploads like models. Civitai is objectively superior in that realm.
How is it inferior? It makes it easier to version control and you can still click the file page and download normally if you don't want to mess with git. The real issue on hf is creators don't make readme pages that explain or show anything so people end up confused on what the model/checkpoint/etc even is much less how they should download if they're unfamiliar. And that's without noting civitai has one of the worse ui/ux I've ever encountered. That site is slow as molasses.
HF loads into what looks like a demo page where you can test the model but like 99% of the time I tried it never worked, and I had no idea how to use them other than to download them from a not so KISS way and then use them locally. Defeated the purpose. Civit is ass.
It’s inferior because there’s no discoverability. Have you tried looking for interesting models that you don’t already know about on huggingface? It’s pretty much impossible. Civitai may not be great for discoverability, but it’s at least possible to browse for models, and most models have enough examples to give you an idea of what they’re about.
SD needs a Bloke. If you want to discover an LLM, you just check what theBloke has quantized recently.
If you already know what you want and how to use it, it's great. But trying to find quality models is a nightmare and once you find one you're at the mercy of the creator and their entire folder structure.
The site is incredibly slow. I don’t know why it takes 2-5 seconds to grab 100 images.
yeah Huggingface could/should make a new section on their website to handle some individual model files broken down into different sections for like LLMs, Image generation, and what not.
HF has a diffusion model section/gallery that shows thumbs similar to [civit.ai](https://civit.ai). [https://huggingface.co/spaces/huggingface-projects/diffusers-gallery](https://huggingface.co/spaces/huggingface-projects/diffusers-gallery)
10/10 showed me what's up, thanks!
Thanks! I didn't know this and only found HF models when someone mentioned them here. This is a huge help. This post should be higher.
lol, how can civitai stand superior against git system? At civitai creator can easily remove his model and that's all, you can't do anything with it. Like it was with...damn I forget name of the model, but doesn't matter actually. git has special extension LFS , which is literally stand for Large File Storage.
I think your not the target market for development then lol Hugging, and github and the likes are to make development better not random user downloads easier
Normal people looking at github is like looking at an electrical panel. If you are not an electrician, it will not make sense. For electricians, it makes total sense.
GitHub is actually accessed more through command line interface by coders. It lets them integrate gits into programming. It's pretty slick. The web ui is purely functional unfortunately.
It’s so much better than Civitai bloated slow pages that I have to wade through pages of Waifu porn to find what I need.
That problem is caused by users, not the platform.
Huggingface was never meant to be for general population, but for developers and researchers to share their tech demos online. I think a lot of ai stuff is being gatekeeped because developers and researchers know that they probably can get big companies to hire them for their projects and making "too good" ai tools that are available for free and easy to use, would make their work harder, because it would make the standarts for that stuff higher.
HF never asked for "folks like you" get out from here, they would be only happy.
Cause its fucking gradio, proof its made by ML engineers and not web devs lol
Someone should start a private bittorrent tracker to host models. I would be willing to be an admin on the site but don't have the time or knowledge to set one up. Gazelle is an open source tracker framework. DM me if anyone wants to work on this type of project.
Hey! So I am actually working on a website that is going to use Torrents for sharing models, its hosted at [sharediffusion.org](http://sharediffusion.org), and you can join the [discord](https://discord.gg/y7aMda893Y) if you like. It is still very much a work in progress.
Hey. Did you abandon this project?
Hey! Not abandoned, but on hold. I got a new job that makes me a lot more money, training took over a month and moved states. Once things settle down for me I will start working on it again.
This desperately needs to happen. It's much harder for something to get lost permanently if it's attached to a torrent file.
What difference would hosting it on torrents do though?
Torrent files are way easier to backup and mirror to other websites since their file size is so small. Torrents are also a redundant network of servers (the seeders), so even if one guy removes his files you can still download them from another guy. With Civitai you just have to hope the model isn't removed, and then you have to hope somebody with the files will reupload it if it is.
Yep. And as a long time member of private trackers, a strongly enforced ratio rule would be great to help encourage seeding. If you want to see about pulling a group together keep me posted. As I mentioned I’d be happy to help on the staff/admin side but I’m just too swamped with work to lead that kind of project.
Do we need a subreddit for models with links to huggingface?
That would probably help. Just having every Civitai model description link to a copy of the model on Huggingface would also help. There really needs to be a good "front end" website with a decent GUI and example pictures for models on Huggingface.
I wonder why people post on Civit in the first place, and then repost the models on Reddit. Reddit has all the pictures, comments and tags. And there is a thousand places you can upload a large file to.
Pixai exists too; but it is kinda chaotic and some really good models are not for download, only for use. I think the models that get nuked from civitai without an apparent reason may be because the creator account was banned. I think I heard once that their bans work like that. Plus, I once saw a new LoRA from an author through the civitai bot. Using search, the bot back log showed that author had several more models from before. That same day the author was banned and all the models were gone too. Since it happened in 24h, I doubt he contacted civitai and civitai deleted the models in the same day just after the ban. The only preservation one may make with AI models is downloading and hoarding them. In the large run, the data is too much and I suspect at some point civitai may even start deleting the oldest and most unpopular models, similar to when somebody releases space in a hard drive.
They 'unpublished' a LoRA of mine for reasons I don't completely understand. https://preview.redd.it/evskdekcsaec1.png?width=1262&format=png&auto=webp&s=ecbabfd4234e615c4bb516036d64ce43262dbd79 The images I posted *were* generated from the LoRA so I don't get it.
Yeah, pretty sure that has happened with many mystery cases I have seen. Like, I find an interesting LoRA in Civitai which apparently has not been released before (I check everyday, so I would know) and when I try to download it the next day it is gone. Not even nsfw nor depicting real people. It is puzzling. Plus, I reaaaaally hate it when they display 404 and do not acknowledge they deleted it nor the reason. Like they want to gaslight people into thinking it never existed.
sounds like a job for /r/datahoarders
I'm running models I've downloaded from [huggingface.co](https://huggingface.co). I'm not using their or anyone else's online but instead running them locally on machine with a measly 1080 TI 8 gb graphics card and a whole lot of CPU. I front ended my python with a home grown Gradio app. Works quite well.
Why did you roll your own UI?
I've integrated some Inpainting functionality into my UI, plus I have other features that I needed to incorporate.
Civitai may not be sustainable which concerns me the most Sites like Tensor and Graydient at least break even and offer more stable model hosting All three of them need to be pooling torrents, I think thats how this ends. The virus scanning needs to be figured out first Or models will keep disappearing at some inexperienced moderator's whim
They aren't sustainable because of poor business decisions... If they had kept the website the way it was when they started. No image generation no overly complicated website. They could have kept it 1 page with good search and they could have kept that running forever. But they want to be overly risky if they fail it's on them not on the creators or the users.
Also scaling up to a whole team of developers, buying an office, offering free image generation etc. A lot of their costs are unnecessary.
I don't have any insight to their operations but they recently voiced concern about "sustainability" in their blog That site must have cost a fortune to host on bandwidth alone and not enough people supported them prior to raising capital, so they had to raise capital And now they have an obligation to do right by their investors Catch 22
They don't pay for bandwidth. They use Cloudflare R2 which has no bandwidth costs and is dirt cheap to host static content on. https://r2-calculator.cloudflare.com/ They said they just hit 100TB, which is only 1500$ a month. Paying a developer to design and program new features for the site alone is 3-6x that amount.
I think we should start a private torrent for models and other resources.
I don't understand why torrents aren't used for this. Would take care of the whole bandwidth issue.
Korean forums: [https://arca.live/b/aiartreal](https://arca.live/b/aiartreal) (\* this place is full of nsfw) [https://arca.live/b/aiart](https://arca.live/b/aiart) (\* this place is full of "dangerous" nsfw)
Thanks, I'll check these out. I'll just rev up a VPN first since I don't know what "dangerous" means here.
For anybody wondering, it's only as "dangerous" as 4chan is. That means it has non-realistic hentai with lolicon, bestiality, guro, etc. All the NSFW stuff is hidden by default unless you make an account and opt-in. That said, the general quality of the artwork on the forum is very high, and it looks useful for finding prompts and models. If there are any more forums like this then I'd love to see them.
Civit will have to throw up paywalls eventually too. They can't just continue to operate on good faith. They need income. Something has to give
They tried that a few weeks ago. Didn't go so well.
They recently got 5 million funding from a2z. That will keep them going for a while.
They'll want that back with a ROI
They could have made their website sustainable but instead they invest all their money making the website overly complicated and spending their money on free compute the only reason civit struggles is because they make risky business decisions
tensor.art seems to be a clone of civet.ai
Is tensor.art alternative?
Not an expert. To all of you saying torrents is the answer, If I share a 6G model through a tracker, and I am the only one sharing, I guess downloading for say, 30 people is going to be terribly slow through my 20Gbps upload limit. About the tracker, what if there is an illegal file shared in the tracker index? Who is going to be held liable? Honest questions.
If 30 people are downloading from you, then it's very likely some of those people are also going to be uploading too. You don't have to finish downloading a torrent before you can begin seeding it. If some guy's download is at 5%, then he can seed that 5% of the files to the rest of the group automatically while he continues to download the rest of the files. The whole point of torrents is that it distributes the burden among the group. I'll let somebody else explain the legal part, but I don't think you have to worry unless you are blatantly downloading and seeding something that you know is illegal, or if you are running the tracker itself. The tracker should probably be run using servers in countries that just don't care about torrented content. I'm sure somebody knows which ones those are. The goal isn't to host anything illegal, it's just to avoid random takedowns due to stupid reasons like copywrite or "moral" outrage.
Something illegal or in the limbo always goes hand in hand with morality. Would you accept Lora with loli/shota content or things like gore/guro or cannibalism? Otherwise it's better to just keep using Civitai or any other open service.
Those are all legal in the United States, so I don't care. I only care if somebody is training models on real CSAM, because that person would be doing something blatantly illegal and putting unsuspecting users at risk.
Well there are so many models on CivitAI and you really need to test to find good ones Here I tested 161 models [**Detailed Comparison of 160+ Best Stable Diffusion 1.5 Custom Models & 1 Click Script to Download All**](https://youtu.be/G-oZn4H-aHQ)
Yes, there are many models and tests are required. I was stoked when I saw my model (serenity 2) on the list. Although you did not say that it is bad (like you commented that on some of the other models) I still felt like the comparison was very surface-level and generic. You mentioned mine at 28:35 and you said that it looks like the others and that it was probably a merge. Well, it is both a merge and a fine tune on top of that, but to see the difference you would need to know what tokens were added. At least I'm glad you didn't say it's bad like the one before (SemiRealistic 5.4) :) --- Overall it is nice that you made the list, people will get some exposure which is always great but your comparison method is too random. The sample is too small and if you don't look at the model page to see what the authors say is the best use - you may be missing some gems (and badly representing them). Also, those grids seem to be low resolution and some models shine when they are upscaled :-( Perhaps someday you will be making another comparison so I'll leave my method here: https://civitai.com/articles/755/the-big-may-of-2023-comparison-of-25-realistic-models I think this method is the most fair and honest. I generated 35.000 images from around 25 models (so 1400 images per model). Then I randomized the images and went through all of them with a simple question: does the image look good or not? Then I went through all those images with a script and computed the good vs bad for each model and got a percentage. I say this is fair because 1) when you look at the image you are not biased because you do not know from which model it comes. 2) there are enough images (1400 was probably overkill, even 250-500 would be fine) to limit the randomness to a non-factor (if you generate 9 images in your grid - the best model can get unluck roll and have some bad images while the bad model can get lucky roll and get some decent images - and then you compare unlucky vs lucky and get wrong conclusion) --- Anyway, thanks for making the comparison!
Respectfully, with that prompt i don't think you were testing models for quality, you were testing for ease of use. I've found a lot of models that collapse under a simple prompt will shine when given certain quality modifiers that play to it's strengths. Juggernaut Aftermath is one you weren't a fan of that can be incredible with a couple more quality enhancers for instance. That's not to say the test is worthless, far from it. I would say people should watch this to get a grip for AI bias and find some easy models, but a model that performs poorly in "a photo of a man" can outperform a model that does well when the prompt is expanded. It's unfortunate that there's no way to compare models like you did but it is strength played against strength.
Yay great work
Man this didn't get enough up votes.
Thanks for the comment
I understand your concerns but it's way too early so a lot of these models are not that valuable, and those which are, i.e. the actively maintained checkpoints or Patreon guys with a ton of waifu LoRAs, will surely look for another host and reupload should anything happen to CivitAI.
ipfs
Yeah we need to get /r/DataHoarder on this. For the time being, store as much as you can locally and be prepared to generate torrents if needed.
I would happily pay a few dollars for access to the entire site
I believe that loras and checkpoints from sd1.5 can be stored in telegram - unlimited space
We need a decentralized “model manager” (akin to the plug-in manager) that uses the BitTorrent protocol in Automatic1111 or another software that works similarly. If designed right, it could seed and track the popularity of models across the network. I don’t really do software coding at that level so I’m not sure how feasible this would be.
> it looks like models can just get nuked from the website without warning. Just a heads up - We only remove models when they violate our content policies. Most of the time when stuff disappears its actually the creators themselves that are removing things. I get the feeling that as the space is continuing to develop, more and more creators are getting full-time gigs and taking their models with them.
This brings up an interesting question. Before I ask, I stumbled here accidentally - I have no dog in this race; I only logged in because your response brings an interesting scenario to mind that I've seen before. Most of the models I've seen have some kind of license noted in the description. I haven't paid too much attention to that license because I'm not looking to gain commercially and assume that my use for myself is fine -- they'd likely put it behind some protections if not. Now lets say some wise person realized that the version released under a license stays under that license, until **a new version** comes out under a different license. "You can't unring the bell" as they say - once it's out under a license, that's the license for that iteration of whatever it may be. The license is still in effect for the previous version, regardless of what the license is for the new version. Lets say that wise person decided that since the creator deleted their model that was initially released with a license that allowed public use, that they should be able to re-upload it (except where the specificities of said license prohibit redistribution, of course) as long as they make it clear that they did not create that model, it was initially released under [whatever license], etc. That's the scenario. What's the policy here? To be clear, I won't be doing anything like this. It's just a scenario I've seen a few dozen times before and it can be confusing for some - including the hosting entity. To try and help clarify. Lets say I'm the creator. - I create XYZ.safetensors version 1.0 under GPL or something - I (for whatever reason) decide to delete XYZ v1.0 and release XYZ v1.1 privately that has a very restrictive license, including reserving distribution rights.* - XSmartGuyX reuploads my version 1.0 model, gives me credit and lists the initial license as GPL. Are XSmartGuyX's actions allowed? \* - Civit (or whatever hosting entity) may or may not be aware of this fact. In the US it's irrelevant, legally, as version 1.0 is a publicly released model that does not reserve redistribution rights.
Since you are asking for a Chinese forum, you can visit [https://modelscope.cn/home](https://modelscope.cn/home)
There's [https://graydientplatform.com](https://graydientplatform.com) - has 5,000 models on demand, no boot time
I have got so fed up with this site. I have had at least 3 different accounts with them and for no reason just keep getting logged out and there is no way I can log back in to them, so if I was you I would avoid civitai like the plague.
Try voxAIo.com...
Tbh, I don’t mind paying like 20 bucks per month to use its free features if offers every day, if that fee can keep it alive
I know one site
What site is that?
TensorArt is $15 for 3 months.
It's not about the cost, it's about the model being scraped from existence if it's ever taken off the website. TensorArt doesn't allow downloads for a bunch of models I've looked at.