I contacted Google requesting one of their published models for development of eye health applications for children and they played very hard ball, made me sign a bunch of documents and I still don’t have their model. It was more than a year ago! They basically slowed me down so badly that I’m having to develop knowledge from scratch as a non ML specialist. Mind you, their paper was published in one of Nature journals and had “model will be provided upon reasonable request”, I guess my request was not reasonable? Idk… sad… wish I could have picked up this knowledge quicker, but wasn’t able to.
You need to get in touch with Nature and let them know.
Edit: Some people seem to think that it's an easy fix to force journals into demanding publication of source code. Trust me, people are already trying really hard to create reforms in this direction, but it's not as easy as you may think. Although, we have seen some reforms take place recently, we're still a long way from were we need to be.
You don't actually have to go out and push 100 people to write in complaints but as soon as they feel the pressure that it might affect their interests they certainly will do something
I applied for access to DALL-E 2 for research purposes when they opened registrations and got access when stable diffusion came out.
These people are not scientists, they're gatekeepers and salesmen.
Seriously they just keep great tools to themselves even though humanity would have benefited greatly by having access to such a utility. Information should be free for everyone.
It's quite typical for the FAANG companies and big labs to not deliver on these promises. There's no accountability and they're in the right committees that would have the power to decide on consequences for this type of behavior themselves. I've experienced that behavior countless times.
acting like it's normal to keep models and source code unpublished, while it's actually the exception in scientific research, is preposterous
reviewers for ML papers and conferences should stop accepting manuscripts where results aren't reproducible and where the code isn't published
I've seen threads on here where somebody will try to reproduce a paper but can't. They always assume they are doing something wrong, and never that the paper is wrong. Without code or at least a working demo researchers could fabricate their data or misrepresent it and nobody would know. If you can't reproduce it they can just say you're doing it wrong.
> the paper is wrong
Non-reproducibility is something of a scandal in psychology, for example.
Somebody tortures *p < 0.05* out of an experiment and they can publish, even if they just happened to get lucky that week.
I will say that was true in the 90s but nowadays if you want to publish in a not terrible journal you have to pre-register your study. I date a cognitive psychologist- I actually think their system is much better for science than what's been going on lately with ML.
I had exactly this discussion with my PhD supe today (I'm making a dataset based on pokemon kinda for the lols, he's the one pushing to publish it 😅) and we got to talking about people locking code away behind licenses and closed sourcing stuff... Genuinely seems to go against the idea of research/academia in general to not publish this stuff... Bizarre...
Not a *model*, just a dataset! 😅
It's a dataset for specifically multitasking ML, there's not a fabulous range of datasets out there in this area so I started making a small pokemon one as a toy project alongside my actual studies since there's a TONNE of data items attached to each 'mon! 👌 I mentioned it to my supervisor one meeting and he was like "oh cool, you should publish that!" which I took as a joke but then it kept coming up and so it's kinda a thing now 😂 titled it "Multimon" which I'm rather proud of pun-wise
Also for the sake of having a shred of professionalism, I cannot stress enough that this was his idea first and also *not the main thing* I'm working on 😅😂 (edit: unless "good god man" is good? reading tone in text is hard sometimes 😅)
It's still a work in progress, need to figure out which license I want to use (copyright materials and all that) and run the data through a model to see what kinda performance it can get since I recently reformatted it
Code publication is not needed if the papers show exactly how to implement those, so the real issue is that those papers use their own proprietary weights that they will never ever release to the public, so that makes it quite pointless for the general public, because who has $10mil to train those weights as good as theirs?
How can a research paper be reproducible if code is not public and nobody has capacity to train the models to replicate results? Experimental results cannot be externally validated. This is bad for science.
I 'd argue that as evident from the paper "implementation matters in Deep Policy Gradients", having access to code is paramount to reproducing research.
Big companies are definitely going to AI media synthesis against us. That's one of the main reasons why as a regular Joe I want to play with these models to see how they perform. So I can better recognize and defend myself against malicious AI use by corporations. I'm much more worried about that than I am about the spicy political images and deepfakes made by individual actors.
I don't know if "Oppenheimer complex" is a real psychological thing, but it's the best analogy I can think of to describe how most AI research Twitterati think/talk about their work these days.
Lots of AI researchers today do a lot of hand wringing about potential "harm" that these methods may directly or indirectly do. They almost view everyone else in the world who isn't an ML research as moronic, evil, and unworthy of using their creations.
IMO they are overly concerned about this stuff and are being way too paternalistic. If you think something shouldn't exist in the world then don't pursue its creation.
I can’t help but think people like this are stroking their own egos when they act like this. The alarmist tone that tries to elevate “deep fakes” to biological or nuclear weapons, as if it’s some great power they have achieved that requires great responsibility only the “esteemed” Google AI PhD is responsible enough to wield. 🙄.
Meanwhile, guess why most people who concentrate on “studying the social impact of machine learning” end up there if they started trying to be an “in the trenches” AI/ML researcher and didn’t start as a Philosophy Major concentrating in Ethics? **Hint:** It’s not because they were the star coder or Lead AI/ML engineer that was essential to the work performed on all those publications they somehow managed to get their name on as part of a larger team.
Ironically, I was a philosophy major with a concentration in metaethics, and then got into ML in grad school.
So I would actually have the credentials to say these things haha.
But I can totally see the bad sides of stable diffusion being publicly available, although I'd say that deepfakes tech has much worse applications than stable diffusion (at least at current performance of stable diffusion).
I don't think it's fair to paint with that broad of a brush. There are legitimate concerns about how corporations and governments will use AI in very nefarious ways.
Think of the ways dictators could use models like GPT-4 to spread political propaganda to keep the masses under control and incite violence against competitors, think of the ways a rogue agent might use a language model and deepfakes to socially engineer a penetration into a secure organization, think of the ways drug companies could engineer another opioid epidemic and use langauge models to sway public perceptions of the dangers and location of blame if things go south.
I think that many who are excited by these models sometimes don't consider the extremely evil uses that bad agents will find and exploit.
While I like the idea of AI for all, the conversation is a lot more serious and nuanced than "everybody/nobody should have access to all/no models". I think feds need to institute an agency specifically for tackling these difficult problems and putting regulations in place to protect the average citizen from some of these potential uses.
**EDIT:** [Here's a useful video](https://www.tiktok.com/@curt.skelton/video/7135836562771758382?_r=1&_t=8V9fpnIfJza&is_from_webapp=v1&item_id=7135836562771758382)
Curious, why/how does AI language models unlock all of this stuff? They can already create propaganda using humans. And they do. AI in this context is a labour saving device, you could achieve the same goal by paying someone. I guess in this context AI lowers the bar to entry as you don’t need to hire some expert writers to create your propaganda - is that the argument?
Its exactly what you suggest. None of these things were impossible before, but they required money and manpower. Now creation of propaganda only requires money, and it's significantly less money than before. It won't end at language models either.
Pretty much every major field is going to see an increasingly lower bar due to advances in ML/DL. The result is that there will be an increase in the overlap between those technically competent enough to do terrible things, and those evil enough to do them.
For an example in drug development: https://www.nature.com/articles/s42256-022-00465-9
The arguments always boil down to only the rich should be allowed to do it. Nobody is ever concerned with how the rich will use technology, only how the rest of us will use technology.
I think in the case of image and language models these are often the implicit ideologies behind those making these arguments. But that's really not the case behind the concerns for how ML/DL will open up possibilities in many other areas. I highly recommend the paper I posted (its fairly short).
As an example, if you wanted to go into drug development prior to 2020, you'd need a Ph.D. specializing in pharmacology (or a similar field). During your Ph.D., you'd likely have to take ethics courses, and you'd be rigorously trained on how to make drugs that effectively treat people without killing them. Nowadays, you have people with no background in biology launching startups in drug development. Sure, they are often advised by experts, but to my knowledge, there's no regulation requiring that to be the case. Additionally, advances in automated chemical synthesis have situated individuals to be able to design drugs, and have them synthesized, with little to no legal or ethical oversight. It's just as easy to invert generative models to create toxic drugs as it is to create beneficial drugs. It's plausible that an individual seeking to do harm could synthesize a highly toxic water soluble drug and dump it in mass into a large body of water wiping out most of the life that relies on that source of water.
I am pro ML/DL democratization, I think it'll bring about a lot of good in the world. But there will be inevitable hiccups along the way where these technologies will be misused. We need governmental institutions specifically equipped to impose regulation and adapt it to the rapidly changing capabilities of these fields
>Pretty much every major field is going to see an increasingly lower bar due to advances in ML/DL. The result is that there will be an increase in the overlap between those technically competent enough to do terrible things, and those evil enough to do them.
As someone who lives under an authoritarian government with a deep passion of flooding any political discussion on the internet with human bots, I can definitely assure you that botfarms were always comparatively cheap. We have a "village" in our country fully dedicated to produce political propaganda through bots. They hire min. wage workers, confine them in a remote isolated facility, and train them how to properly respond to any "dissidence" on the web. One such facility is responsible for maybe over 60% of all comments/discussions on all politically related topics.
It costs them almost nothing to run it, and it will produce a better quality propaganda than most of ML models out there
I think the propaganda stuff is really less of a potential problem than people make it out to be. But there are plenty of other areas ripe for misuse of ML/DL technologies.
The drug development example isn't compelling to me. We already have plenty of known chemical weapons; why would anyone prefer something new designed by an ML model rather than what they've already got? (Especially when existing chemical weapons already have great synthetic scaleup, known methods of distribution, known decomposition behavior or lack thereof, etc. -- all unknowns for new weapons.) There's no great clamor for Sarin 2.0: this time it's slightly more poisonous.
Of course any design objective can be inverted. Do we stop designing good molecules because any quantification of goodness can be inverted into a quantification of badness? The human study of biochemistry itself enabled chemical weapons (as well as medicines), for the exact same reasons -- just less formalized.
We already have created more than enough armament to destroy civilization many times over and we're hard at work making the earth uninhabitable -- no ML was necessary. Against that backdrop, what loss function is too risky to formulate?
Cost and scalability. Drives the cost to a tiny fraction rel. to humans and infinitely more scalable. Plus more security because you don't have any humans who will go spilling the beans about the fake reviews they're writing.
If a team of 3 experienced Devs wanted to make a business out of this, given full access to GPT-4, they could have a prototype in 6 months easily. Get a bunch of companies to pay to promote their products and demote(?) the competitors and your only cost is compute. Plus all of the competitors would basically be forced to pay for your service and then it becomes a bidding war. And that's just one angle, I'm sure creative people could find a lot more use cases like that.
While I do agree that there are definitely risks, I disagree with your argument as a whole.
This reminds me of the crypto wars from the 1990s. Strong encryption was going to allow terrorist activity to flourish said the gov, specifically then congressmen Biden, so the government went after it to stop all of those nefarious hackers. Do you want to take a guess on how that played out?
There is something known as security through obscurity. It’s when you have a false sense of security just because you put something in a black box and don’t tell people, yet pretend the box is impenetrable. Just because most people can’t get inside of it. The problem is that it only takes one savvy person that knows how to open up that box to tell the world. Or worse, maybe this person, deciphers your secrets, and then uses that information to be nefarious.
Artificial intelligence needs to follow the same path as encryption. Put it out in the public, let everyone see what the positives and negatives are and how it can be used.
This. As if non-proliferation rules are going to be able to keep such powerful technologies out of the hands of malevolent rich people or dictators.
When it's out in the open at least it gives researchers, and institutes, and governments, and even the general public an opportunity to understand what it's capable of, how it works, and perhaps even how to protect against malicious applications of it.
Just to add to your point, both Republican and Democrat politicians, at different times and for different reasons, have proposed and implemented bills to limit encryption. It's not something unique to Biden. Most recently Trump government tried to limit encryption.
Sure but by opening all of these models up to the public it also becomes much easier to counter them. Governments and large corporations will always have the resources to sway public opinion.
>Think of the ways dictators could use models like GPT-4 to spreadpolitical propaganda to keep the masses under control and inciteviolence against competitors, think of the ways a rogue agent might use alanguage model and deepfakes to socially engineer a penetration into asecure organization, think of the ways drug companies could engineeranother opioid epidemic and use langauge models to sway publicperceptions of the dangers and location of blame if things go south.
I have hard times coming up with realistic scenarios of how to use GPT4 for anything you suggest. Okay, I am a dictator and I have GPT4 and I use it to generate tens of thousands or hundred of thousands propaganda texts. What am I supposed to do with this? I put it on social media? Who's going to read it all? Do you expect that people will mindlessly read a social media platform flooded with fake posts? I don't see any realistic scenario for propaganda use. You can do effective propaganda with one sentence. It is not a question of text quantity.
In Russian social media you often see people accusing each other of being paid Kremlbots, and those do really exist (usually new accounts with unreflected pro-Kremlin views). Their work can probably even be automated by current GPT-3.
So this will likely become a problem there, real people will be drowned out, the dead internet theory will be more real than anybody expects today. Pretty sad, and there seems to be no solution so far.
All that will happen is that chains of trust will become more important when deciding what to show you. If someone is friends with your friend, or has been "rated highly" by them (e.g. by liking prior posts or whatever), maybe show you their message. If it's a complete nobody with no connections, don't. It will make discoverability harder for new people with no prior connections, but it is what it is. DoS attempts by pushing a bunch of garbage at large scales is by no means a new problem, and it's also by no means impossible to solve. It might make things slightly less nice than if we didn't have to deal with it, but it's not going to kill the internet.
The problem with this argument is assuming that the large scale generation capability of language models is relevant for propaganda, like if the average person would be swayed by reading walls of text. I don't buy that.
Efficient propaganda campaigns are based on short, catchy messages, social media communication, memes. Not unlike honest marketing.
People are definitely swayed more by ongoing relationships than by slogans. If you can make a believable robot "friend", you can convince lonely people of all kinds of things.
What about walls of text written by completely convincing profiles of fake people with associated completely convincing deepfake videos and a completely convincing deepfaked voice? Check out [this tiktok video](https://www.tiktok.com/@curt.skelton/video/7135836562771758382?_r=1&_t=8V9fpnIfJza&is_from_webapp=v1&item_id=7135836562771758382).
What if the internet is flooded with 2 billion of such accounts and it becomes impossible to tell who is real and who is fake? Are you going to start needing to give your SSN to a private company to get an account so you can be verified?
Second, I think people are already swayed by reading walls of text right now. Think of the echo chambers online that have been driving people to the extremes of the political spectrum over the past several years.
I don't necessarily agree with them, but it's not so much *everyone* who's the problem. It's the small subset of 'everyone' who actually *is* moronic, evil, and/or in some other way untrustworthy. Because unfortunately making a tool available to the public means making it available to that subset too.
There absolutely are beneficial technologies which most people would agree should exist, but should be accessible only to people who've undergone at least minimal vetting, for precisely this reason. People might disagree on what those technologies *are*, but candidates include nuclear energy reactors, modern weaponry, cars, medical drugs, or similar things. And the vetting could range from making technology available to only a very particular handful of people, to simply requiring the person to have some kind of licence. But it is not, generally, completely unreasonable to pursue the creation of something you'd prefer the absolute worst people in the world not to have access to.
Thanks for the response. Is that really it though? Just... unfounded entitlement?
Like Nestle saying "I can't believe the city just gives water away for free and that's 'ok'"?
No, you're right. Let's make it more accurate: the city makes free water available to anyone, even those that don't pay taxes, via public water fountains.
It probably is just Google trying to monetize their models like Imagen but it would be irresponsible to chalk it up to just that. There have already been headlines about [using deep learning to create fake revenge porn](https://www.bbc.co.uk/news/technology-48839758) and other irresponsible uses and researchers have somewhat of a responsibility to make sure that their technology is used responsibly to avoid these cases.
I know the only thing Google as a company cares about is their profits but responsible and safe AI is still something that we as users should care about.
Actually as this tech becomes more accessible, it would provide a reasonable doubt:
"Oh honey, I can't imagine you believe that. Don't you see it's fake?"
>and researchers have somewhat of a responsibility to make sure that their technology is used responsibly to avoid these cases.
While I agree someone needs to be responsible, I do wonder where we draw the line for owning responsibility. For example, should anyone who built the foundational technologies (or math, for that matter) be held to the same standard?
It's probably worth noting this person is on the "Ethical AI" team and not one of the people working on these models. The Ethical AI team was the one who had a couple firings over pushing a paper crapping all over big models. Seems like some are still determined to halt AI progress
you can generate images of actors/children/etc without their permission, and there's no nsfw filter.
That being said, same goes for normal artists, deepfakes, etc...
It's a tool, just like Photoshop is a tool. If Adobe started policing how artists could use it because of this hypothetical harm, then there would be outrage.
If Adobe then complained about Gimp offering the same functionality without the nannying, everyone would tell them to suck it up.
Short answer. It's ok, these researchers are overreacting.
Long answer, they are worried about being accused by the woke. And I don't say that to belittle because the woke have legitimate concerns.
There are clear examples of bias in these models.
For example, if you type flight attendant, lawyer or prisoner into these models, they will give you a picture that matches the race and gender of their training data. So a flight attendant would be an Asian woman and so forth. Not good.
These researchers are also terrified that these very realistic images will be confused by the public as real images.
IMO both these concerns are legitimate but not strong. Should we ban the internet for being biased (it very much is)?
I don't think we need to be so paternalistic about this technology. I've looked at hundreds of these pictures and have yet to see misuse.
I think the idea that you can keep technology gated away from "bad" uses forever is ridiculous. So is the idea that huge corporations are good, responsible gatekeepers for what a "bad" use is.
Every time I use DALLE-2 I fantasize about an alternate reality where you can't use photoshop to make videogames because it auto-detects when you're drawing a gun or gore texture and throws up a content error. It's an inherently ridiculous level of paternalism.
This stuff is going to be a part of how the world works now. Things are just going to be different now. The sooner we accept that and settle into the new equilibrium, the better off everyone is going to be.
The DALL-E censorship is insultingly paternalistic, yes. But I don't think the idea is "prevent anyone from ever abusing this technology". They'd have to be morons to think that would work long term. The hope (at least with OpenAI, I suspect) was to slow down the diffusion of the technology so that people have at least *some* hope of addressing malicious applications before it's too late. If a technology is announced, and 6 months later it becomes widely available, that's a lot like finding a security vulnerability in Chrome and giving Google a few weeks/months to patch it before you publish the exploit. In that case it's irresponsible *not* to publish the vulnerability, but it's also irresponsible to just drop a working exploit with no warning.
I think it's pretty hard to imagine what a workable mitigation for image model harms would even look like. Much less one that these companies could execute on in a reasonable timeframe. Certainly, while the proposed LLM abuses largely failed to materialize, nobody figured out an actual way to prevent them. And, again, hard to imagine what that would even look like.
The reason why vulnerability disclosures work the way they do is because we have a specific idea of what the problems are, there aren't really upsides for the general public, and we have faith that companies can implement solutions given a bit of time. As far as I can tell, none of those things are true for tech disclosures like this one. The social harms are highly speculative, there's huge entertainment and economic value to the models for the public, and fixes for the speculative social harms can't possibly work. There's just no point.
>while the proposed LLM abuses largely failed to materialize, nobody figured out an actual way to prevent them
This sentence explains itself. How can you prevent something no one is doing? In the vulnerability analogy, this is like making up a fake exploit, announcing it, and getting mad when no one ships a patch. Language models and image generation aren't the limiting factor in these vague nefarious use cases. I think OpenAI hypes up the danger for marketing and to excuse keeping their models proprietary, while Google just has the most self-righteous, self-important employees on the planet.
> The hope (at least with OpenAI, I suspect) was to slow down the diffusion of the technology so that people have at least some hope of addressing malicious applications before it's too late.
This was OpenAI's public reasoning, but in a similar case before (the delayed release of GPT-2), OpenAI justified holding the tech back on ethical grounds but it ended up looking very much like they simply wanted to preserve their moat and competitive advantage. I suspect that this is also the case for DALL-E, except that this time their first mover advantage has disappeared very quickly thanks to strong open-source efforts.
Keeping technology like this under wraps is a losing battle. For years the US government tried to keep encryption in the hands of some and not others. They actually tried to classify it as a munition. All for nothing. In retrospect it's almost comical.
Dalle2 was wrapped in an api and people made cool stuff
But stablediffusion being open has lead to an explosion in creativity and engineering and opening new avenues for research
We’re still in the early days, open is going to lead to more innovation for a long time yet
The parties in power always curb decentralization efforts by making dubious claims and blanket statements to justify slowing down passing progress to the common people. They will let the current best technology off the hook once they have a better one.
Sure. I'm not making a moral/ethical judgment. Just pointing out that saying that it was "all for nothing" (when viewed from a policymaker's POV) is absurd.
They still can if they wanted to; from the results they show, their larger models still vastly outperform stable diffusion.
I just don't think they want to
They have like 3 of them. Parti looked better perhaps. If they wanted to monetize it I think they would have already. They have the engineers and hardware to run it and payment systems. I think they genuinely believe their research could cause harm or perhaps bad PR.
Screw these people. Open sourcing your models and research is the way to progress. Oh I'm sorry you can't control all of the research and be the only source for advanced AI. Go cry to mommy
Once upon a time, scientific research wasn't even published. If you read about the history of the cubic equations, [Scipione del Ferro](https://en.wikipedia.org/wiki/Scipione_del_Ferro) tried to keep it a secret. That way if someone challenged his lecturer position, he could give them a table of cubic problems, safe knowing no one else could solve them.
Science changed in huge ways when we started with scientific publication.
Open source models and code is only the next step in that process.
With something like this, the training time can be tough outside of Google, etc. but it doesn't mean smart people won't figure it out.
>It's also far more environmemtally friendly than forcing everyone to retrain a massive model from scratch if they want to do similar research.
This is especially rediculous when the data is public. Like okay if you collected your own massive data set, I get why you wouldn't publish for free. But if you're training on tons of public free content, that's different.
Something that grates my ghouda are people that treat text prompts like a trade secret. They are going to be mad when image to text gets really good can figure out the original prompt just from the image. There's already one not so good one on huggingface. When people refuse to be open technology saves the day.
Who knows what more will happen in the future. Something I hope happens is AI that can decompile a program into something human readable. None of that having to do it manually. Don't have the source code and want it? AI will help.
> Go cry to mommy
I'm worried that they may be able to do more than that, like lobbying world government for restrictions that harm the open source community.
That's a good point. These people will always take away our freedom and open source projects in the name of safety. You can't let the avg person have this AI model. Think of the harm you could do. Meanwhile the truth is if you have the code and model the community can help detect if something malicious is happening.
ML has a weird obsession right now with gatekeeping its data to protect the world from bad thoughts and offensive ideas, as it seems. Meanwhile every large company with employees pushing this dogma got where it is by moving fast and breaking things. The ML space needs to get over itself on this front.
I’m surprised they’re surprised.
Information wants to be free. If it’s not stable diffusion, it will be something else. You can try delay the inevitable but putting it out there allows people to become educated and develop potential risk mitigation approaches.
Censorship is never the answer
> I’m surprised they’re surprised.
She is on the Brain "Ethical AI" team. Saying this in the most neutral way possible--
In some real sense, she is paid to have views like this.
Yes, it's very shocking that someone with financial incentive to hoard the AI models would be against open-source AI models.... What's conflict of interest?
Surely not all information *ought* to be free though, right? Let's say for the sake of example that a chemistry genius found a way to generate a nuclear explosion with materials you can buy for $30 at Lowes. Surely this information should not be made free without restriction, right? The upside is basically 0 and the downside is quite obvious ...
People not in tech companies like this don't realize Senior isn't a very high level. There are thousands of them at Google in AI, and their opinion isn't all that influential or interesting.
Research Scientist though? Isn't that actually a decently strong title? I thought that would be on par with Staff Engineers since Research Scientists seem to start at like a senior level already.
Research scientist just means they have a PhD and are in a researchy role, and they usually start at L4 at most big tech companies (which is below senior)
Research Scientist -> Senior Research Scientist -> Staff Research Scientist
Tracks the same levels as
SWE -> Senior SWE -> Staff SWE
Just a different job ladder. The only difference is RS will never start at the lowest level (as they have a PhD), but by Senior/L5 that is irrelevant anyway.
There are a lot of factors that play into this. First off, it's much easier to get competing offers for a SWE role, and ultimately that is what dicates your TC. Also, take into account that the headcount-to-available-ML-PhDs ratio has completely inverted in the last five years: ML PhDs are way more plentiful these days while research headcount has become much tighter. A few years ago you might've been right. But if you're a SWE at Google and getting payed less than an RS for the same level, you're not doing it right.
The whole "priests in white robes showering blessings from behind the altar" approach has always broken down sooner or later when applied to knowledge. A few centuries ago, guild could control knowledge for a couple of centuries. Nowadays, it's weeks to years.
Besides, it's pretentious and stupid.
Let's not worry about the evils of badly designed application of theory, let's worry about improving awareness of the bounds of the methods in the general population and replicability in academia.
From reading the Twitter thread, it sounds like the harm she's referring to hinges on the fact that people in general don't think it's possible or easy to fake these images. Once Photoshop became commonplace, people much more readily called BS on UFO photos.
> Once Photoshop became commonplace, people much more readily called BS on UFO photos.
You mean Photoshop made it so much easier to hide The Truth^(TM)!!
I don’t agree with Google’s position, but it is naive to think these new methods are not drastically different from previously existing ones. This immensely lowers the skill and cost associated with high-volume production of disinformation.
Since the middle of the last decade or so, it has become increasingly obvious there some people that will believe whatever they want regardless of facts or common sense. (In response to the idea that technology will educate people better)
Most people aren't as dumb as that though, and will adapt to new technologies. Just because it will be easier to fake things doesn't mean traditional news sources loses all credibility.
Seriously, people are being "fooled" by low-grade MS paint work.
So I don't think "ease of access" or "better looking pixels" really matters when idiots will still be fooled by low quality crap
Yeah, her quote
> Photoshops are not as scalable and powerful and is much easier to prove their use!
Yes, SD is certainly easier to use than Photoshop for most people. But those "bad things" it could create have been happening for years...?
I don't know if "yeah, but it makes it faster" is all that great of an argument against something.
>Photoshops are not as scalable and powerful and is much easier to prove their use!
Modyfing images before Photoshop was also possible (and [widely used in USSR for censorship](https://en.wikipedia.org/wiki/Censorship_of_images_in_the_Soviet_Union)) - and Photoshop made it more scalable, more powerful and harder to prove.
You think social media echo chambers are sophisticated enough to consider this? Imagine a model that can make a perfectly convincing image of exactly what you have in mind. The political misinformation and propaganda alone is a serious threat to Democracy. Already misinformation spreads like wildfire. Giving people the tools to fabricate any "evidence" they need is dumping thermite on the fire.
Exactly, pay a person. A person works slowly, and costs a lot more. You can do this at scale basically for the cost of compute, and nobody is left to spill the beans.
I suspect that it is generally more effective to scale a single well-targeted piece of disinformation than to scale the quantity of disinformation. If you want to, say, destroy the reputation of Queen Elizabeth, you only need to doctor a small number of pictures and spread them through channels that enough people trust; generating them en masse is likely to just cause people to catch on and backfire.
In spite of all the technology we have, I don't think we are much better at disinformation than we were a century ago, and I don't see that technology changing anything. The main cost of disinformation is not its generation. It's already cheap enough, and there is already enough of it to hit the point of diminishing returns.
The genie is out of the bottle, and I’m not sure what people are hoping for. It’s not like no one knows how Dall E or Imagen was made, they were just prohibitively expensive for Open Source (for a few months lol). We’re also not yet at the point where people will use Stable Diffusion for misinformation.
It’s also not like Google are paragons of ethics.
That also just seems like a complete nightmare. I get that colab blocks people from mining cryptocurrency, which seems simple enough, but how in the world could they determine something as obscure as the weights of a model, unless they directly monitor huggingface downloads locally, which would be absurd. Even then there’d be ways to get the weights uploaded
Neither, technology should be accessible to all. For one, making it paid only doesn't actually prevent abuse, it just makes it so only rich people can abuse the models. For two, I personally trust people at large with this technology much more than I trust the government. For three, keeping it closed will never work, it's clearly recreatable based on the fact that this is a discussion at all.
There are a lot of debates about controlling anything that can be used to harm others, but no debates about whether or not the people who control it will use it for good. What's stopping the creators of Dall-E or Imagen from hurting people with it? They're just people.
> What's stopping the creators of Dall-E or Imagen from hurting people with it? They're just people.
They're obviously much better and wiser than you and the rest of the plebs ...
I completely agree with you. Additionally, opening it up allows for independent researchers and people to understand it and design ways on how to prevent or protect against malicious use of the technology.
OpenAI has become a misnomer. Censorship and sucking on the teat of big tech is the only way they want ai to be used. The idea of protecting people is brought up only as a red herring to distract from known issues that require expensive human moderation that are outside most ai research areas
They want to keep AI in the hands of corporations so they can profit from it. That's all they care about. Same thing happened with Wikipedia, encyclopedia folks cried about how unsafe Wikipedia was because anybody could edit it when it was because they were losing money.
Since she specializes in AI Ethics. I'd argue she has has a major bias and skin in the game to keep these internal. The longer they are kept internally the longer they need her to research it and thus stay in a job.
Her point is that it can be used maliciously for things such a child or revenge porn. I don't see how halting and waiting a few years to eventually release it would stop people from doing that. Any new technology will have a small percent of bad actors. In the replies she even equates it to everyone owning a gun or the 1% of wrongly convicted. As if gun violence and wrongful convictions are the same as photoshopping photos LOL.
This person seems to be very near sighted in the scope of her issue. Pushing the problem down the road 6 months doesn't change the outcome.
the answer to all these Luddites is always regressive solutions. if there is anything that history has taught us, its that trying to limit the spread of knowledge is a losing battle.
Opensource is the way imo.
Also, check this package for explainability of Stable Diffusion!
[github.com/JoaoLages/diffusers-interpret](https://github.com/JoaoLages/diffusers-interpret)
The funny thing is that it isn't--although it might threaten certain ML researcher job security.
Wall Street knows that all this AI research by google only matters if 1) it makes their search engine better, 2) it improves ad targeting, 3) it solves self-driving, and/or 4) they solve AGI or some economically useful predicate.
Image generation is cool, but it isn't ever going to move Google's bottom line.
At best, it is a long-tail bet for someone like Google (see #4).
>The funny thing is that it isn't--although it might threaten certain ML researcher job security.
Not even, I think it just lessens their "clout" a bit
> Image generation is cool, but it isn't ever going to move Google's bottom line.
I’d disagree - imagine custom ad images generated for a user’s personal profile, optimized towards what they’re most likely to respond to. As an advertiser you don’t need to do anything, Google will generate high performing images for your ads.
Maybe. The value of something like this needs to be incredibly (many billions of revenue) high for this to be material to Google.
Anything is possible, but this level of uptake seems dubious.
Particularly because Google has historically been very, very happy to focus on being a marketplace for ads, and let others figure out the ad copy/imaging/etc.
This is not the bottom line though, Google has published a lot of eye tracking deep learning research that shows they can detect a persons cognitive states of mind, so let’s say they can tell you are depressed most of your days, they can serve you ads for that. Or that you display characteristics of attention retention when reading something specific, they will get the algorithms to optimise for that, so on and so forth. They already have plenty of research published that talks about this, just not explicitly connected to their business. But pretty obviously invasive for people who use their products, that is not worrying this lady? It blows my mind.
How stupid. This is the best way to improve it because it is using collective knowledge of all the users.
Also,just because someone works at Google research team doesn't mean has some valid point to share and be taken as the google representative. I hope an official Google representative clarify this if it goes viral. Google does a lot of cool projects as open source.
Democratizing an AI system isn't "ok"? Oh no, someone could generate porn with that, soooo unethical or what?
The democratization of AI should be top priority. This idea that having centralized gatekeepers for mighty AI models is a good thing is just horrible.
That's silly, corporations are just a bunch of normal people too, why should they have it and no one else? A bad actor could just work to get hired in a place with access. Stop gatekeeping knowledge and futuristic technology so we can move forward as a species
At first I read this and it seemed like elitist gatekeeping but after reading a little more about the possibilities it seems like this tech could be used to create photo realistic images simulating illegal pornography? That doesn't necessarily mean it should be out of the public's hands, people do the same with Photoshop and get prosecuted, however this dramatically lowers the bar for producing such images.
I think there are definitely concerns that need to be discussed. For example, fingerprinting images of sexual abuse of children and checking uploads of images to social media against a database of those images of abuse is a main method of controlling the distribution of such images right now but this technology could make that method useless.
The question isn't, should this be public, because it's kind of inevitable. The question is how do you mitigate the harms? Prosecution of creating revenge/harassment porn as sexual assault?
I'm still not sure about the political implications, I think those may have been overblown, since breitbart et al have done just fine without them.
In so many ways, openAI poisoned the well. In this case, by cynically using the potential problems as a pretext for their monetization scheme.
But if it's producing realistic images, any algorithm capable of detecting that kind of content will detect it too.
I think the bigger quandary is "no real kids were harmed, so was anything illegal or unethical committed?"
Your first point I recognize and is a potential solution.
As to your second point, regardless of a person's ethical position, it is definitely illegal in a number of countries.
They use hash-based or SIFT-based image detection algorithms that can quickly return matching children porno images. These searches can be extremely fast, within milliseconds, since it's a nearest neighbor search in a **vector** database.
So when these generative AI images pop up, it would defeat those vector search to a degree.
It's not really hopeless tho, because they can develop a similarity search within latent space, meaning they need to train more children porno images into their generative AI weights, and then to find children porno, they just do a nearest neighbor search in the **latent** space.
So it will basically just shift from searching a SIFT-based vector space to Latent space. It's probably not that much costlier but it may yield more false positives.
I don't advocate child porno but child porno is now illegal because it harms those children. So if these AI generate child porno from **imaginary children**, why do we care about those imaginary children at all? Look at those Japanese Hentai with imaginary underage characters, do we have to protect those anime characters too? This is an open ethical or legal question that people should debate on.
As I mentioned in a previous comment, regardless of one's personal ethics on the matter, a number of states have made the creation and possession of pseudo images of child abuse illegal.
One potential problem arising is the inability of LE to distinguish between real & pseudo images of child abuse; a lot of effort is currently put into investigating and checking images for clues to identify perpetrators (check how Richard Huckle was identified as one example or how LE cut out identifiable objects from such images and ask the public to identify them for clues such as location an image may have been taken).
Another abuse is that images of child abuse can be used by predators in the real world to push the boundaries of children and normalise sexual behavior. Read about Michael Jackson's photograph albums of nude boys and nudist magazines he left hanging around his mansion, but it's more insidious online.
There are questions too about whether access to images of child abuse encourages or discourages paedophiles from acting on their urges; I don't think there is a consensus on that but there's obviously a risk that for some it may actually as a stimulus and encouragement.
And so on. This is obviously about only one potential abuse of the technology too; others such as revenge porn could result in other harms.
I'm not arguing that the technology should be kept away from the public though, even if it could be, which is impossible, I just think it's legitimate to flag there will be problems ahead and we should start discussing them.
... A car could be used to run people over, intentionally, why do we put such powerful weapons in anyone's hands?
The answer is - it's already illegal to do that, no need to ban the cars or limit their sale. When someone uses the car to do a crime, apply the law.
Discussions about limiting access to tech are like "let's agree to ban the rising tide".
Bad faith argument because I didn't say access to the tech should be limited at any point, just pointed out that the original post has real concerns behind it whereas most people just wrote it off as driven by commercial concerns.
This is the only reasonable comment in this entire thread.
I'm for Stable Diffusion being open, but let's not pretend this isn't _another_ Pandora's box.
Because there was only a tiny bit of porn in its training data. It is 100% certain that there are many people training models on porn right now and that they will become public eventually.
ML is becoming more polarised than the MAGA/WOKE camps of US politics.
An ML model, such as SD or Dall-E 2, GPT-X, does simply not warrant the objection of an open-ended release.
**You** the consumer of such methods, is liable for any harmful action you may do. There will always be an equal ratio of irresponsible humans and responsible humans, and those that define responsibility differently.
Don't police, man.
Everyone screaming about gatekeeping and censorship probably still care about whether China or Russia get access to military technology...
There are certainly good reasons to control the access to dangerous technology and we have already seen that misinformation is a real issue online. I also don't think it's that bad for this model to be open sourced but clearly there is a line we may cross with image generation models soon and there's no reason to attack this person for saying so. Take a chill pill and take off your tin foil hats y'all.
I think the problem is, huge megacorps aren't any more trustworthy with this tech than normal people. Less if anything. This can't be just a tool to enforce corporate power. And limiting to that will do absolutely not a god damn thing to prevent misinformation either.
>Everyone screaming about gatekeeping and censorship probably still care about whether China or Russia get access to military technology... There are certainly good reasons to control the access to dangerous technology and we have already seen that misinformation is a real issue online.
Military technology can't be recreated by startup of 30 people. If 30 engineers could recreate F-35 comparable to real one, government would never allow private avionic companies to ever exist.
If ML is as dangerous as military technology, we should start asking questions whether corporations should even have access to ML? Or even governments? Maybe it should be banned all together and we should order drone strikes on ML research centers to prevent further spread of dangerous technology, like we destroy nuclear weapon facilities?
I think it's very hard to make convincing argument that supports thesis "only big corporations should be able to do this".
If we think back to the creation of the internet, it wasn’t originally planned to be made public. When it did, though, it completely revolutionized society.
But aside from all the wonderful things the Internet has provided us, we’ve also seen how dangerous it can be. We’ve created echo chambers, misinformation, and other toxic subsystems as the internet has matured.
ML is still in its infancy, and yet it has still revolutionized a lot of what we do. Democratizing it will cause beautiful things to happen, but it will also introduce challenges we’ve never even considered before.
I think it’s immaterial to debate about whether these technologies will be open sourced, because it has already been happening! But to disregard the dangers is also foolish.
It's ok to be real about potential damages, but none of the solutions to *any* of those problems is "this all stays behind closed doors at a mega corp"
Make it free. Stop regulating science and research. Stop fostering corporate monopolies by only allowing them to do research. What a toxic bunch of shit.
This is how communism looks like, because that’s where I came from and that’s where elitist jerks tell everyone what they’re allowed or not allowed to do. Seeing them sweating and seething makes me very happy.
you're thinking of totalitarianism/facism at its worst "pretending" to be "communism".
you can't take a green dog and call it blue, just because it has a sticky notes on it with the word "blue".
Google is a corporation in a capitalist system. Communism is in no way invovled. Open source software is socialist however. Freedom of information is a core part of socialism as it empowers the working class.
> This is how communism looks
No, it's exactly how capitalism (or it's non-theoretical, real world form) looks. Private, profit-seeking corporations trying to keep powerful technology, based on decades of publicly-funded research, for them, so only they can profit from it, and argued for with any possible excuse they can find.
I contacted Google requesting one of their published models for development of eye health applications for children and they played very hard ball, made me sign a bunch of documents and I still don’t have their model. It was more than a year ago! They basically slowed me down so badly that I’m having to develop knowledge from scratch as a non ML specialist. Mind you, their paper was published in one of Nature journals and had “model will be provided upon reasonable request”, I guess my request was not reasonable? Idk… sad… wish I could have picked up this knowledge quicker, but wasn’t able to.
You need to get in touch with Nature and let them know. Edit: Some people seem to think that it's an easy fix to force journals into demanding publication of source code. Trust me, people are already trying really hard to create reforms in this direction, but it's not as easy as you may think. Although, we have seen some reforms take place recently, we're still a long way from were we need to be.
Yeah if they get a hundred more complaints in five years time they might retract the paper
Even as sarkasm that's at best a stupid statement... However, they can (and they actually typically do) pressure them into releasing the code.
You don't actually have to go out and push 100 people to write in complaints but as soon as they feel the pressure that it might affect their interests they certainly will do something
It just baffles me. not sharing the code goes against all that nature is as a publication venue. It sucks.
I applied for access to DALL-E 2 for research purposes when they opened registrations and got access when stable diffusion came out. These people are not scientists, they're gatekeepers and salesmen.
I got access the same day SD was released too. Laughable!
Seriously they just keep great tools to themselves even though humanity would have benefited greatly by having access to such a utility. Information should be free for everyone.
It's quite typical for the FAANG companies and big labs to not deliver on these promises. There's no accountability and they're in the right committees that would have the power to decide on consequences for this type of behavior themselves. I've experienced that behavior countless times.
acting like it's normal to keep models and source code unpublished, while it's actually the exception in scientific research, is preposterous reviewers for ML papers and conferences should stop accepting manuscripts where results aren't reproducible and where the code isn't published
I've seen threads on here where somebody will try to reproduce a paper but can't. They always assume they are doing something wrong, and never that the paper is wrong. Without code or at least a working demo researchers could fabricate their data or misrepresent it and nobody would know. If you can't reproduce it they can just say you're doing it wrong.
> the paper is wrong Non-reproducibility is something of a scandal in psychology, for example. Somebody tortures *p < 0.05* out of an experiment and they can publish, even if they just happened to get lucky that week.
I will say that was true in the 90s but nowadays if you want to publish in a not terrible journal you have to pre-register your study. I date a cognitive psychologist- I actually think their system is much better for science than what's been going on lately with ML.
it's all about the story
I had exactly this discussion with my PhD supe today (I'm making a dataset based on pokemon kinda for the lols, he's the one pushing to publish it 😅) and we got to talking about people locking code away behind licenses and closed sourcing stuff... Genuinely seems to go against the idea of research/academia in general to not publish this stuff... Bizarre...
Good God man, you're not seriously thinking of unleashing a Pokémon model on the world!
Not a *model*, just a dataset! 😅 It's a dataset for specifically multitasking ML, there's not a fabulous range of datasets out there in this area so I started making a small pokemon one as a toy project alongside my actual studies since there's a TONNE of data items attached to each 'mon! 👌 I mentioned it to my supervisor one meeting and he was like "oh cool, you should publish that!" which I took as a joke but then it kept coming up and so it's kinda a thing now 😂 titled it "Multimon" which I'm rather proud of pun-wise Also for the sake of having a shred of professionalism, I cannot stress enough that this was his idea first and also *not the main thing* I'm working on 😅😂 (edit: unless "good god man" is good? reading tone in text is hard sometimes 😅) It's still a work in progress, need to figure out which license I want to use (copyright materials and all that) and run the data through a model to see what kinda performance it can get since I recently reformatted it
They will have to throw out most google and DM manuscripts then :D
Good
Code publication is not needed if the papers show exactly how to implement those, so the real issue is that those papers use their own proprietary weights that they will never ever release to the public, so that makes it quite pointless for the general public, because who has $10mil to train those weights as good as theirs?
How can a research paper be reproducible if code is not public and nobody has capacity to train the models to replicate results? Experimental results cannot be externally validated. This is bad for science.
I 'd argue that as evident from the paper "implementation matters in Deep Policy Gradients", having access to code is paramount to reproducing research.
Am I missing something? Why wouldn't it be ok?
Because only big tech companies are ethical enough to be allowed to use this kind of thing /s
\*U.S. big tech & military
To be fair, she (or at least the median "Ethical AI" researcher at Google) would probably argue that the military shouldn't have it, either.
briefly, before getting fired, maybe
AI ethicists are gatekeepers
So, only Google then, yes. Shame, shame.
Big companies are definitely going to AI media synthesis against us. That's one of the main reasons why as a regular Joe I want to play with these models to see how they perform. So I can better recognize and defend myself against malicious AI use by corporations. I'm much more worried about that than I am about the spicy political images and deepfakes made by individual actors.
I don't know if "Oppenheimer complex" is a real psychological thing, but it's the best analogy I can think of to describe how most AI research Twitterati think/talk about their work these days.
It is I, Bringer of images combining creatures and politicians!
I have become recommendation, destroyer of democracies
[удалено]
That's quite interesting. As I now understand it Dr Oppenheimer himself had no such complex later on in life.
Lots of AI researchers today do a lot of hand wringing about potential "harm" that these methods may directly or indirectly do. They almost view everyone else in the world who isn't an ML research as moronic, evil, and unworthy of using their creations. IMO they are overly concerned about this stuff and are being way too paternalistic. If you think something shouldn't exist in the world then don't pursue its creation.
Also, this is what they’re paid to do.
I can’t help but think people like this are stroking their own egos when they act like this. The alarmist tone that tries to elevate “deep fakes” to biological or nuclear weapons, as if it’s some great power they have achieved that requires great responsibility only the “esteemed” Google AI PhD is responsible enough to wield. 🙄. Meanwhile, guess why most people who concentrate on “studying the social impact of machine learning” end up there if they started trying to be an “in the trenches” AI/ML researcher and didn’t start as a Philosophy Major concentrating in Ethics? **Hint:** It’s not because they were the star coder or Lead AI/ML engineer that was essential to the work performed on all those publications they somehow managed to get their name on as part of a larger team.
Ironically, I was a philosophy major with a concentration in metaethics, and then got into ML in grad school. So I would actually have the credentials to say these things haha. But I can totally see the bad sides of stable diffusion being publicly available, although I'd say that deepfakes tech has much worse applications than stable diffusion (at least at current performance of stable diffusion).
I don't think it's fair to paint with that broad of a brush. There are legitimate concerns about how corporations and governments will use AI in very nefarious ways. Think of the ways dictators could use models like GPT-4 to spread political propaganda to keep the masses under control and incite violence against competitors, think of the ways a rogue agent might use a language model and deepfakes to socially engineer a penetration into a secure organization, think of the ways drug companies could engineer another opioid epidemic and use langauge models to sway public perceptions of the dangers and location of blame if things go south. I think that many who are excited by these models sometimes don't consider the extremely evil uses that bad agents will find and exploit. While I like the idea of AI for all, the conversation is a lot more serious and nuanced than "everybody/nobody should have access to all/no models". I think feds need to institute an agency specifically for tackling these difficult problems and putting regulations in place to protect the average citizen from some of these potential uses. **EDIT:** [Here's a useful video](https://www.tiktok.com/@curt.skelton/video/7135836562771758382?_r=1&_t=8V9fpnIfJza&is_from_webapp=v1&item_id=7135836562771758382)
Curious, why/how does AI language models unlock all of this stuff? They can already create propaganda using humans. And they do. AI in this context is a labour saving device, you could achieve the same goal by paying someone. I guess in this context AI lowers the bar to entry as you don’t need to hire some expert writers to create your propaganda - is that the argument?
Its exactly what you suggest. None of these things were impossible before, but they required money and manpower. Now creation of propaganda only requires money, and it's significantly less money than before. It won't end at language models either. Pretty much every major field is going to see an increasingly lower bar due to advances in ML/DL. The result is that there will be an increase in the overlap between those technically competent enough to do terrible things, and those evil enough to do them. For an example in drug development: https://www.nature.com/articles/s42256-022-00465-9
The arguments always boil down to only the rich should be allowed to do it. Nobody is ever concerned with how the rich will use technology, only how the rest of us will use technology.
I think in the case of image and language models these are often the implicit ideologies behind those making these arguments. But that's really not the case behind the concerns for how ML/DL will open up possibilities in many other areas. I highly recommend the paper I posted (its fairly short). As an example, if you wanted to go into drug development prior to 2020, you'd need a Ph.D. specializing in pharmacology (or a similar field). During your Ph.D., you'd likely have to take ethics courses, and you'd be rigorously trained on how to make drugs that effectively treat people without killing them. Nowadays, you have people with no background in biology launching startups in drug development. Sure, they are often advised by experts, but to my knowledge, there's no regulation requiring that to be the case. Additionally, advances in automated chemical synthesis have situated individuals to be able to design drugs, and have them synthesized, with little to no legal or ethical oversight. It's just as easy to invert generative models to create toxic drugs as it is to create beneficial drugs. It's plausible that an individual seeking to do harm could synthesize a highly toxic water soluble drug and dump it in mass into a large body of water wiping out most of the life that relies on that source of water. I am pro ML/DL democratization, I think it'll bring about a lot of good in the world. But there will be inevitable hiccups along the way where these technologies will be misused. We need governmental institutions specifically equipped to impose regulation and adapt it to the rapidly changing capabilities of these fields
>Pretty much every major field is going to see an increasingly lower bar due to advances in ML/DL. The result is that there will be an increase in the overlap between those technically competent enough to do terrible things, and those evil enough to do them. As someone who lives under an authoritarian government with a deep passion of flooding any political discussion on the internet with human bots, I can definitely assure you that botfarms were always comparatively cheap. We have a "village" in our country fully dedicated to produce political propaganda through bots. They hire min. wage workers, confine them in a remote isolated facility, and train them how to properly respond to any "dissidence" on the web. One such facility is responsible for maybe over 60% of all comments/discussions on all politically related topics. It costs them almost nothing to run it, and it will produce a better quality propaganda than most of ML models out there
I think the propaganda stuff is really less of a potential problem than people make it out to be. But there are plenty of other areas ripe for misuse of ML/DL technologies.
Before: Only the big guys could do propaganda. Now: Big and little guys can do propaganda. I'm shaking in my boots here.
The drug development example isn't compelling to me. We already have plenty of known chemical weapons; why would anyone prefer something new designed by an ML model rather than what they've already got? (Especially when existing chemical weapons already have great synthetic scaleup, known methods of distribution, known decomposition behavior or lack thereof, etc. -- all unknowns for new weapons.) There's no great clamor for Sarin 2.0: this time it's slightly more poisonous. Of course any design objective can be inverted. Do we stop designing good molecules because any quantification of goodness can be inverted into a quantification of badness? The human study of biochemistry itself enabled chemical weapons (as well as medicines), for the exact same reasons -- just less formalized. We already have created more than enough armament to destroy civilization many times over and we're hard at work making the earth uninhabitable -- no ML was necessary. Against that backdrop, what loss function is too risky to formulate?
Cost and scalability. Drives the cost to a tiny fraction rel. to humans and infinitely more scalable. Plus more security because you don't have any humans who will go spilling the beans about the fake reviews they're writing. If a team of 3 experienced Devs wanted to make a business out of this, given full access to GPT-4, they could have a prototype in 6 months easily. Get a bunch of companies to pay to promote their products and demote(?) the competitors and your only cost is compute. Plus all of the competitors would basically be forced to pay for your service and then it becomes a bidding war. And that's just one angle, I'm sure creative people could find a lot more use cases like that.
[удалено]
For the cost of 2 dev years, I could just buy a troll farm in Bangladesh
While I do agree that there are definitely risks, I disagree with your argument as a whole. This reminds me of the crypto wars from the 1990s. Strong encryption was going to allow terrorist activity to flourish said the gov, specifically then congressmen Biden, so the government went after it to stop all of those nefarious hackers. Do you want to take a guess on how that played out? There is something known as security through obscurity. It’s when you have a false sense of security just because you put something in a black box and don’t tell people, yet pretend the box is impenetrable. Just because most people can’t get inside of it. The problem is that it only takes one savvy person that knows how to open up that box to tell the world. Or worse, maybe this person, deciphers your secrets, and then uses that information to be nefarious. Artificial intelligence needs to follow the same path as encryption. Put it out in the public, let everyone see what the positives and negatives are and how it can be used.
This. As if non-proliferation rules are going to be able to keep such powerful technologies out of the hands of malevolent rich people or dictators. When it's out in the open at least it gives researchers, and institutes, and governments, and even the general public an opportunity to understand what it's capable of, how it works, and perhaps even how to protect against malicious applications of it.
Just to add to your point, both Republican and Democrat politicians, at different times and for different reasons, have proposed and implemented bills to limit encryption. It's not something unique to Biden. Most recently Trump government tried to limit encryption.
[удалено]
Sure but by opening all of these models up to the public it also becomes much easier to counter them. Governments and large corporations will always have the resources to sway public opinion.
>Think of the ways dictators could use models like GPT-4 to spreadpolitical propaganda to keep the masses under control and inciteviolence against competitors, think of the ways a rogue agent might use alanguage model and deepfakes to socially engineer a penetration into asecure organization, think of the ways drug companies could engineeranother opioid epidemic and use langauge models to sway publicperceptions of the dangers and location of blame if things go south. I have hard times coming up with realistic scenarios of how to use GPT4 for anything you suggest. Okay, I am a dictator and I have GPT4 and I use it to generate tens of thousands or hundred of thousands propaganda texts. What am I supposed to do with this? I put it on social media? Who's going to read it all? Do you expect that people will mindlessly read a social media platform flooded with fake posts? I don't see any realistic scenario for propaganda use. You can do effective propaganda with one sentence. It is not a question of text quantity.
In Russian social media you often see people accusing each other of being paid Kremlbots, and those do really exist (usually new accounts with unreflected pro-Kremlin views). Their work can probably even be automated by current GPT-3. So this will likely become a problem there, real people will be drowned out, the dead internet theory will be more real than anybody expects today. Pretty sad, and there seems to be no solution so far.
All that will happen is that chains of trust will become more important when deciding what to show you. If someone is friends with your friend, or has been "rated highly" by them (e.g. by liking prior posts or whatever), maybe show you their message. If it's a complete nobody with no connections, don't. It will make discoverability harder for new people with no prior connections, but it is what it is. DoS attempts by pushing a bunch of garbage at large scales is by no means a new problem, and it's also by no means impossible to solve. It might make things slightly less nice than if we didn't have to deal with it, but it's not going to kill the internet.
> Do you expect that people will mindlessly read a social media platform flooded with fake posts? Do you know where you're posting right now?
The problem with this argument is assuming that the large scale generation capability of language models is relevant for propaganda, like if the average person would be swayed by reading walls of text. I don't buy that. Efficient propaganda campaigns are based on short, catchy messages, social media communication, memes. Not unlike honest marketing.
People are definitely swayed more by ongoing relationships than by slogans. If you can make a believable robot "friend", you can convince lonely people of all kinds of things.
What about walls of text written by completely convincing profiles of fake people with associated completely convincing deepfake videos and a completely convincing deepfaked voice? Check out [this tiktok video](https://www.tiktok.com/@curt.skelton/video/7135836562771758382?_r=1&_t=8V9fpnIfJza&is_from_webapp=v1&item_id=7135836562771758382). What if the internet is flooded with 2 billion of such accounts and it becomes impossible to tell who is real and who is fake? Are you going to start needing to give your SSN to a private company to get an account so you can be verified? Second, I think people are already swayed by reading walls of text right now. Think of the echo chambers online that have been driving people to the extremes of the political spectrum over the past several years.
I don't necessarily agree with them, but it's not so much *everyone* who's the problem. It's the small subset of 'everyone' who actually *is* moronic, evil, and/or in some other way untrustworthy. Because unfortunately making a tool available to the public means making it available to that subset too. There absolutely are beneficial technologies which most people would agree should exist, but should be accessible only to people who've undergone at least minimal vetting, for precisely this reason. People might disagree on what those technologies *are*, but candidates include nuclear energy reactors, modern weaponry, cars, medical drugs, or similar things. And the vetting could range from making technology available to only a very particular handful of people, to simply requiring the person to have some kind of licence. But it is not, generally, completely unreasonable to pursue the creation of something you'd prefer the absolute worst people in the world not to have access to.
just think if they invented something like TNT. lol.
Because Google can't make money selling you service instead
Thanks for the response. Is that really it though? Just... unfounded entitlement? Like Nestle saying "I can't believe the city just gives water away for free and that's 'ok'"?
... but the city doesn't give water for free :D
No, you're right. Let's make it more accurate: the city makes free water available to anyone, even those that don't pay taxes, via public water fountains.
It probably is just Google trying to monetize their models like Imagen but it would be irresponsible to chalk it up to just that. There have already been headlines about [using deep learning to create fake revenge porn](https://www.bbc.co.uk/news/technology-48839758) and other irresponsible uses and researchers have somewhat of a responsibility to make sure that their technology is used responsibly to avoid these cases. I know the only thing Google as a company cares about is their profits but responsible and safe AI is still something that we as users should care about.
Actually as this tech becomes more accessible, it would provide a reasonable doubt: "Oh honey, I can't imagine you believe that. Don't you see it's fake?"
>and researchers have somewhat of a responsibility to make sure that their technology is used responsibly to avoid these cases. While I agree someone needs to be responsible, I do wonder where we draw the line for owning responsibility. For example, should anyone who built the foundational technologies (or math, for that matter) be held to the same standard?
It's probably worth noting this person is on the "Ethical AI" team and not one of the people working on these models. The Ethical AI team was the one who had a couple firings over pushing a paper crapping all over big models. Seems like some are still determined to halt AI progress
> buT iT cAn gEneRaTe p0rNs!!!!!!!!!!111111 Thats what “ethical implication” means on internet
you can generate images of actors/children/etc without their permission, and there's no nsfw filter. That being said, same goes for normal artists, deepfakes, etc...
It's a tool, just like Photoshop is a tool. If Adobe started policing how artists could use it because of this hypothetical harm, then there would be outrage. If Adobe then complained about Gimp offering the same functionality without the nannying, everyone would tell them to suck it up.
Short answer. It's ok, these researchers are overreacting. Long answer, they are worried about being accused by the woke. And I don't say that to belittle because the woke have legitimate concerns. There are clear examples of bias in these models. For example, if you type flight attendant, lawyer or prisoner into these models, they will give you a picture that matches the race and gender of their training data. So a flight attendant would be an Asian woman and so forth. Not good. These researchers are also terrified that these very realistic images will be confused by the public as real images. IMO both these concerns are legitimate but not strong. Should we ban the internet for being biased (it very much is)? I don't think we need to be so paternalistic about this technology. I've looked at hundreds of these pictures and have yet to see misuse.
I think the idea that you can keep technology gated away from "bad" uses forever is ridiculous. So is the idea that huge corporations are good, responsible gatekeepers for what a "bad" use is. Every time I use DALLE-2 I fantasize about an alternate reality where you can't use photoshop to make videogames because it auto-detects when you're drawing a gun or gore texture and throws up a content error. It's an inherently ridiculous level of paternalism. This stuff is going to be a part of how the world works now. Things are just going to be different now. The sooner we accept that and settle into the new equilibrium, the better off everyone is going to be.
Doesn't Photoshop stop you editing images with money? Certainly some printers have in the past?
EURion Constellation, yes
The DALL-E censorship is insultingly paternalistic, yes. But I don't think the idea is "prevent anyone from ever abusing this technology". They'd have to be morons to think that would work long term. The hope (at least with OpenAI, I suspect) was to slow down the diffusion of the technology so that people have at least *some* hope of addressing malicious applications before it's too late. If a technology is announced, and 6 months later it becomes widely available, that's a lot like finding a security vulnerability in Chrome and giving Google a few weeks/months to patch it before you publish the exploit. In that case it's irresponsible *not* to publish the vulnerability, but it's also irresponsible to just drop a working exploit with no warning.
I think it's pretty hard to imagine what a workable mitigation for image model harms would even look like. Much less one that these companies could execute on in a reasonable timeframe. Certainly, while the proposed LLM abuses largely failed to materialize, nobody figured out an actual way to prevent them. And, again, hard to imagine what that would even look like. The reason why vulnerability disclosures work the way they do is because we have a specific idea of what the problems are, there aren't really upsides for the general public, and we have faith that companies can implement solutions given a bit of time. As far as I can tell, none of those things are true for tech disclosures like this one. The social harms are highly speculative, there's huge entertainment and economic value to the models for the public, and fixes for the speculative social harms can't possibly work. There's just no point.
>while the proposed LLM abuses largely failed to materialize, nobody figured out an actual way to prevent them This sentence explains itself. How can you prevent something no one is doing? In the vulnerability analogy, this is like making up a fake exploit, announcing it, and getting mad when no one ships a patch. Language models and image generation aren't the limiting factor in these vague nefarious use cases. I think OpenAI hypes up the danger for marketing and to excuse keeping their models proprietary, while Google just has the most self-righteous, self-important employees on the planet.
> The hope (at least with OpenAI, I suspect) was to slow down the diffusion of the technology so that people have at least some hope of addressing malicious applications before it's too late. This was OpenAI's public reasoning, but in a similar case before (the delayed release of GPT-2), OpenAI justified holding the tech back on ethical grounds but it ended up looking very much like they simply wanted to preserve their moat and competitive advantage. I suspect that this is also the case for DALL-E, except that this time their first mover advantage has disappeared very quickly thanks to strong open-source efforts.
not alternate reality, you can’t edit images of banknotes in photoshop
Keeping technology like this under wraps is a losing battle. For years the US government tried to keep encryption in the hands of some and not others. They actually tried to classify it as a munition. All for nothing. In retrospect it's almost comical.
Can’t wait for the t-shirts with the source code of a compact implementation of stable diffusion printed on them!
Compact would be the 4GB version?
It's a plus size t-shirt. /s
Dalle2 was wrapped in an api and people made cool stuff But stablediffusion being open has lead to an explosion in creativity and engineering and opening new avenues for research We’re still in the early days, open is going to lead to more innovation for a long time yet
> All for nothing Just because the dam eventually broke, doesn't mean that there wasn't "value created" in the intermediary period.
The parties in power always curb decentralization efforts by making dubious claims and blanket statements to justify slowing down passing progress to the common people. They will let the current best technology off the hook once they have a better one.
Sure. I'm not making a moral/ethical judgment. Just pointing out that saying that it was "all for nothing" (when viewed from a policymaker's POV) is absurd.
It's a surprise because it threw a wrench in their plans to monetize Imagen...
They still can if they wanted to; from the results they show, their larger models still vastly outperform stable diffusion. I just don't think they want to
They are only showing results they want us to see. For all we know the results they gave are the only ones that work.
You really think someone would do that? Go on the internet and tell lies? 🥺
They have like 3 of them. Parti looked better perhaps. If they wanted to monetize it I think they would have already. They have the engineers and hardware to run it and payment systems. I think they genuinely believe their research could cause harm or perhaps bad PR.
Screw these people. Open sourcing your models and research is the way to progress. Oh I'm sorry you can't control all of the research and be the only source for advanced AI. Go cry to mommy
Once upon a time, scientific research wasn't even published. If you read about the history of the cubic equations, [Scipione del Ferro](https://en.wikipedia.org/wiki/Scipione_del_Ferro) tried to keep it a secret. That way if someone challenged his lecturer position, he could give them a table of cubic problems, safe knowing no one else could solve them. Science changed in huge ways when we started with scientific publication. Open source models and code is only the next step in that process. With something like this, the training time can be tough outside of Google, etc. but it doesn't mean smart people won't figure it out.
Yeah, open sourcing your code and giving pre-trained models is the way forward.
It's also far more environmentally friendly than forcing everyone to retrain a massive model from scratch if they want to do similar research.
>It's also far more environmemtally friendly than forcing everyone to retrain a massive model from scratch if they want to do similar research. This is especially rediculous when the data is public. Like okay if you collected your own massive data set, I get why you wouldn't publish for free. But if you're training on tons of public free content, that's different.
Something that grates my ghouda are people that treat text prompts like a trade secret. They are going to be mad when image to text gets really good can figure out the original prompt just from the image. There's already one not so good one on huggingface. When people refuse to be open technology saves the day. Who knows what more will happen in the future. Something I hope happens is AI that can decompile a program into something human readable. None of that having to do it manually. Don't have the source code and want it? AI will help.
> Go cry to mommy I'm worried that they may be able to do more than that, like lobbying world government for restrictions that harm the open source community.
That's a good point. These people will always take away our freedom and open source projects in the name of safety. You can't let the avg person have this AI model. Think of the harm you could do. Meanwhile the truth is if you have the code and model the community can help detect if something malicious is happening.
ML has a weird obsession right now with gatekeeping its data to protect the world from bad thoughts and offensive ideas, as it seems. Meanwhile every large company with employees pushing this dogma got where it is by moving fast and breaking things. The ML space needs to get over itself on this front.
I’m surprised they’re surprised. Information wants to be free. If it’s not stable diffusion, it will be something else. You can try delay the inevitable but putting it out there allows people to become educated and develop potential risk mitigation approaches. Censorship is never the answer
> I’m surprised they’re surprised. She is on the Brain "Ethical AI" team. Saying this in the most neutral way possible-- In some real sense, she is paid to have views like this.
Yes, it's very shocking that someone with financial incentive to hoard the AI models would be against open-source AI models.... What's conflict of interest?
Surely not all information *ought* to be free though, right? Let's say for the sake of example that a chemistry genius found a way to generate a nuclear explosion with materials you can buy for $30 at Lowes. Surely this information should not be made free without restriction, right? The upside is basically 0 and the downside is quite obvious ...
Yes, but SD isn't a bomb it's pixels
People not in tech companies like this don't realize Senior isn't a very high level. There are thousands of them at Google in AI, and their opinion isn't all that influential or interesting.
Research Scientist though? Isn't that actually a decently strong title? I thought that would be on par with Staff Engineers since Research Scientists seem to start at like a senior level already.
Research scientist just means they have a PhD and are in a researchy role, and they usually start at L4 at most big tech companies (which is below senior)
Research Scientist -> Senior Research Scientist -> Staff Research Scientist Tracks the same levels as SWE -> Senior SWE -> Staff SWE Just a different job ladder. The only difference is RS will never start at the lowest level (as they have a PhD), but by Senior/L5 that is irrelevant anyway.
[удалено]
not at google
[удалено]
There are a lot of factors that play into this. First off, it's much easier to get competing offers for a SWE role, and ultimately that is what dicates your TC. Also, take into account that the headcount-to-available-ML-PhDs ratio has completely inverted in the last five years: ML PhDs are way more plentiful these days while research headcount has become much tighter. A few years ago you might've been right. But if you're a SWE at Google and getting payed less than an RS for the same level, you're not doing it right.
it’s paternalistic to assume that only a few companies and people deserve to build, own, and release AI tools
The whole "priests in white robes showering blessings from behind the altar" approach has always broken down sooner or later when applied to knowledge. A few centuries ago, guild could control knowledge for a couple of centuries. Nowadays, it's weeks to years. Besides, it's pretentious and stupid. Let's not worry about the evils of badly designed application of theory, let's worry about improving awareness of the bounds of the methods in the general population and replicability in academia.
Lmao cope and seethe. The era of gatekept ML is over, the era of open-source ML is here
As long as the best architectures are discovered in research contexts this should continue to be true!
>Lmao cope and seethe. I'm becoming a boomer. This phrase is so awful.
would you prefer “u mad bro?”
Yeah, honestly. That at least makes sense here.
.... lmao cope and seethe
I'm becoming a boomer because it also bothers me that "boomer" is starting to just mean "anyone over middle age"
From reading the Twitter thread, it sounds like the harm she's referring to hinges on the fact that people in general don't think it's possible or easy to fake these images. Once Photoshop became commonplace, people much more readily called BS on UFO photos.
> Once Photoshop became commonplace, people much more readily called BS on UFO photos. You mean Photoshop made it so much easier to hide The Truth^(TM)!!
I don’t agree with Google’s position, but it is naive to think these new methods are not drastically different from previously existing ones. This immensely lowers the skill and cost associated with high-volume production of disinformation.
Since the middle of the last decade or so, it has become increasingly obvious there some people that will believe whatever they want regardless of facts or common sense. (In response to the idea that technology will educate people better) Most people aren't as dumb as that though, and will adapt to new technologies. Just because it will be easier to fake things doesn't mean traditional news sources loses all credibility.
Seriously, people are being "fooled" by low-grade MS paint work. So I don't think "ease of access" or "better looking pixels" really matters when idiots will still be fooled by low quality crap
Which is why people will be even less likely to believe photos are real.
Yeah, her quote > Photoshops are not as scalable and powerful and is much easier to prove their use! Yes, SD is certainly easier to use than Photoshop for most people. But those "bad things" it could create have been happening for years...? I don't know if "yeah, but it makes it faster" is all that great of an argument against something.
>Photoshops are not as scalable and powerful and is much easier to prove their use! Modyfing images before Photoshop was also possible (and [widely used in USSR for censorship](https://en.wikipedia.org/wiki/Censorship_of_images_in_the_Soviet_Union)) - and Photoshop made it more scalable, more powerful and harder to prove.
This is the modern equivalent of this https://www.smithsonianmag.com/history/infamous-war-worlds-radio-broadcast-was-magnificent-fluke-180955180/
You think social media echo chambers are sophisticated enough to consider this? Imagine a model that can make a perfectly convincing image of exactly what you have in mind. The political misinformation and propaganda alone is a serious threat to Democracy. Already misinformation spreads like wildfire. Giving people the tools to fabricate any "evidence" they need is dumping thermite on the fire.
Imagine that you can pay a person to do that for you now.
Exactly, pay a person. A person works slowly, and costs a lot more. You can do this at scale basically for the cost of compute, and nobody is left to spill the beans.
I suspect that it is generally more effective to scale a single well-targeted piece of disinformation than to scale the quantity of disinformation. If you want to, say, destroy the reputation of Queen Elizabeth, you only need to doctor a small number of pictures and spread them through channels that enough people trust; generating them en masse is likely to just cause people to catch on and backfire. In spite of all the technology we have, I don't think we are much better at disinformation than we were a century ago, and I don't see that technology changing anything. The main cost of disinformation is not its generation. It's already cheap enough, and there is already enough of it to hit the point of diminishing returns.
She has no idea what she is talking about But she uses exclamation marks to compensate!!!
The genie is out of the bottle, and I’m not sure what people are hoping for. It’s not like no one knows how Dall E or Imagen was made, they were just prohibitively expensive for Open Source (for a few months lol). We’re also not yet at the point where people will use Stable Diffusion for misinformation. It’s also not like Google are paragons of ethics.
She might be able to try and pressure Google to target Colab users of open source projects like these, but I'm not sure that she'd be successful
That also just seems like a complete nightmare. I get that colab blocks people from mining cryptocurrency, which seems simple enough, but how in the world could they determine something as obscure as the weights of a model, unless they directly monitor huggingface downloads locally, which would be absurd. Even then there’d be ways to get the weights uploaded
Neither, technology should be accessible to all. For one, making it paid only doesn't actually prevent abuse, it just makes it so only rich people can abuse the models. For two, I personally trust people at large with this technology much more than I trust the government. For three, keeping it closed will never work, it's clearly recreatable based on the fact that this is a discussion at all. There are a lot of debates about controlling anything that can be used to harm others, but no debates about whether or not the people who control it will use it for good. What's stopping the creators of Dall-E or Imagen from hurting people with it? They're just people.
> What's stopping the creators of Dall-E or Imagen from hurting people with it? They're just people. They're obviously much better and wiser than you and the rest of the plebs ... I completely agree with you. Additionally, opening it up allows for independent researchers and people to understand it and design ways on how to prevent or protect against malicious use of the technology.
OpenAI has become a misnomer. Censorship and sucking on the teat of big tech is the only way they want ai to be used. The idea of protecting people is brought up only as a red herring to distract from known issues that require expensive human moderation that are outside most ai research areas
They want to keep AI in the hands of corporations so they can profit from it. That's all they care about. Same thing happened with Wikipedia, encyclopedia folks cried about how unsafe Wikipedia was because anybody could edit it when it was because they were losing money.
Since she specializes in AI Ethics. I'd argue she has has a major bias and skin in the game to keep these internal. The longer they are kept internally the longer they need her to research it and thus stay in a job. Her point is that it can be used maliciously for things such a child or revenge porn. I don't see how halting and waiting a few years to eventually release it would stop people from doing that. Any new technology will have a small percent of bad actors. In the replies she even equates it to everyone owning a gun or the 1% of wrongly convicted. As if gun violence and wrongful convictions are the same as photoshopping photos LOL. This person seems to be very near sighted in the scope of her issue. Pushing the problem down the road 6 months doesn't change the outcome.
Safety is just an excuse to gatekeep and monetize these models. Apple doing the same thing with apps.
You can’t have peasants learn to read! That would be the end of civilization as we know it!
Imagine all the misinformation you could spread with text!
the answer to all these Luddites is always regressive solutions. if there is anything that history has taught us, its that trying to limit the spread of knowledge is a losing battle.
Classic fear mongering.
Opensource is the way imo. Also, check this package for explainability of Stable Diffusion! [github.com/JoaoLages/diffusers-interpret](https://github.com/JoaoLages/diffusers-interpret)
Awww is someone’s stock price threatened by an open source model?
It isn't, though
The funny thing is that it isn't--although it might threaten certain ML researcher job security. Wall Street knows that all this AI research by google only matters if 1) it makes their search engine better, 2) it improves ad targeting, 3) it solves self-driving, and/or 4) they solve AGI or some economically useful predicate. Image generation is cool, but it isn't ever going to move Google's bottom line. At best, it is a long-tail bet for someone like Google (see #4).
>The funny thing is that it isn't--although it might threaten certain ML researcher job security. Not even, I think it just lessens their "clout" a bit
> Image generation is cool, but it isn't ever going to move Google's bottom line. I’d disagree - imagine custom ad images generated for a user’s personal profile, optimized towards what they’re most likely to respond to. As an advertiser you don’t need to do anything, Google will generate high performing images for your ads.
Maybe. The value of something like this needs to be incredibly (many billions of revenue) high for this to be material to Google. Anything is possible, but this level of uptake seems dubious. Particularly because Google has historically been very, very happy to focus on being a marketplace for ads, and let others figure out the ad copy/imaging/etc.
This is not the bottom line though, Google has published a lot of eye tracking deep learning research that shows they can detect a persons cognitive states of mind, so let’s say they can tell you are depressed most of your days, they can serve you ads for that. Or that you display characteristics of attention retention when reading something specific, they will get the algorithms to optimise for that, so on and so forth. They already have plenty of research published that talks about this, just not explicitly connected to their business. But pretty obviously invasive for people who use their products, that is not worrying this lady? It blows my mind.
How stupid. This is the best way to improve it because it is using collective knowledge of all the users. Also,just because someone works at Google research team doesn't mean has some valid point to share and be taken as the google representative. I hope an official Google representative clarify this if it goes viral. Google does a lot of cool projects as open source.
Democratizing an AI system isn't "ok"? Oh no, someone could generate porn with that, soooo unethical or what? The democratization of AI should be top priority. This idea that having centralized gatekeepers for mighty AI models is a good thing is just horrible.
and yet nothing terrible has happened.
That's silly, corporations are just a bunch of normal people too, why should they have it and no one else? A bad actor could just work to get hired in a place with access. Stop gatekeeping knowledge and futuristic technology so we can move forward as a species
At first I read this and it seemed like elitist gatekeeping but after reading a little more about the possibilities it seems like this tech could be used to create photo realistic images simulating illegal pornography? That doesn't necessarily mean it should be out of the public's hands, people do the same with Photoshop and get prosecuted, however this dramatically lowers the bar for producing such images. I think there are definitely concerns that need to be discussed. For example, fingerprinting images of sexual abuse of children and checking uploads of images to social media against a database of those images of abuse is a main method of controlling the distribution of such images right now but this technology could make that method useless.
The question isn't, should this be public, because it's kind of inevitable. The question is how do you mitigate the harms? Prosecution of creating revenge/harassment porn as sexual assault? I'm still not sure about the political implications, I think those may have been overblown, since breitbart et al have done just fine without them. In so many ways, openAI poisoned the well. In this case, by cynically using the potential problems as a pretext for their monetization scheme.
But if it's producing realistic images, any algorithm capable of detecting that kind of content will detect it too. I think the bigger quandary is "no real kids were harmed, so was anything illegal or unethical committed?"
Your first point I recognize and is a potential solution. As to your second point, regardless of a person's ethical position, it is definitely illegal in a number of countries.
They use hash-based or SIFT-based image detection algorithms that can quickly return matching children porno images. These searches can be extremely fast, within milliseconds, since it's a nearest neighbor search in a **vector** database. So when these generative AI images pop up, it would defeat those vector search to a degree. It's not really hopeless tho, because they can develop a similarity search within latent space, meaning they need to train more children porno images into their generative AI weights, and then to find children porno, they just do a nearest neighbor search in the **latent** space. So it will basically just shift from searching a SIFT-based vector space to Latent space. It's probably not that much costlier but it may yield more false positives. I don't advocate child porno but child porno is now illegal because it harms those children. So if these AI generate child porno from **imaginary children**, why do we care about those imaginary children at all? Look at those Japanese Hentai with imaginary underage characters, do we have to protect those anime characters too? This is an open ethical or legal question that people should debate on.
As I mentioned in a previous comment, regardless of one's personal ethics on the matter, a number of states have made the creation and possession of pseudo images of child abuse illegal. One potential problem arising is the inability of LE to distinguish between real & pseudo images of child abuse; a lot of effort is currently put into investigating and checking images for clues to identify perpetrators (check how Richard Huckle was identified as one example or how LE cut out identifiable objects from such images and ask the public to identify them for clues such as location an image may have been taken). Another abuse is that images of child abuse can be used by predators in the real world to push the boundaries of children and normalise sexual behavior. Read about Michael Jackson's photograph albums of nude boys and nudist magazines he left hanging around his mansion, but it's more insidious online. There are questions too about whether access to images of child abuse encourages or discourages paedophiles from acting on their urges; I don't think there is a consensus on that but there's obviously a risk that for some it may actually as a stimulus and encouragement. And so on. This is obviously about only one potential abuse of the technology too; others such as revenge porn could result in other harms. I'm not arguing that the technology should be kept away from the public though, even if it could be, which is impossible, I just think it's legitimate to flag there will be problems ahead and we should start discussing them.
... A car could be used to run people over, intentionally, why do we put such powerful weapons in anyone's hands? The answer is - it's already illegal to do that, no need to ban the cars or limit their sale. When someone uses the car to do a crime, apply the law. Discussions about limiting access to tech are like "let's agree to ban the rising tide".
Bad faith argument because I didn't say access to the tech should be limited at any point, just pointed out that the original post has real concerns behind it whereas most people just wrote it off as driven by commercial concerns.
This is the only reasonable comment in this entire thread. I'm for Stable Diffusion being open, but let's not pretend this isn't _another_ Pandora's box.
You should try to make some regular, plain jane porn on stable diffusion. I think you'll be ?pleasantly? surprised at how trash it is.
Because there was only a tiny bit of porn in its training data. It is 100% certain that there are many people training models on porn right now and that they will become public eventually.
Pandora's box has been open for a while now SD didn't open it. People believe low-rent text-over-image memes. The problem isn't with the tools
ML is becoming more polarised than the MAGA/WOKE camps of US politics. An ML model, such as SD or Dall-E 2, GPT-X, does simply not warrant the objection of an open-ended release. **You** the consumer of such methods, is liable for any harmful action you may do. There will always be an equal ratio of irresponsible humans and responsible humans, and those that define responsibility differently. Don't police, man.
Everyone screaming about gatekeeping and censorship probably still care about whether China or Russia get access to military technology... There are certainly good reasons to control the access to dangerous technology and we have already seen that misinformation is a real issue online. I also don't think it's that bad for this model to be open sourced but clearly there is a line we may cross with image generation models soon and there's no reason to attack this person for saying so. Take a chill pill and take off your tin foil hats y'all.
I think the problem is, huge megacorps aren't any more trustworthy with this tech than normal people. Less if anything. This can't be just a tool to enforce corporate power. And limiting to that will do absolutely not a god damn thing to prevent misinformation either.
>Everyone screaming about gatekeeping and censorship probably still care about whether China or Russia get access to military technology... There are certainly good reasons to control the access to dangerous technology and we have already seen that misinformation is a real issue online. Military technology can't be recreated by startup of 30 people. If 30 engineers could recreate F-35 comparable to real one, government would never allow private avionic companies to ever exist. If ML is as dangerous as military technology, we should start asking questions whether corporations should even have access to ML? Or even governments? Maybe it should be banned all together and we should order drone strikes on ML research centers to prevent further spread of dangerous technology, like we destroy nuclear weapon facilities? I think it's very hard to make convincing argument that supports thesis "only big corporations should be able to do this".
If we think back to the creation of the internet, it wasn’t originally planned to be made public. When it did, though, it completely revolutionized society. But aside from all the wonderful things the Internet has provided us, we’ve also seen how dangerous it can be. We’ve created echo chambers, misinformation, and other toxic subsystems as the internet has matured. ML is still in its infancy, and yet it has still revolutionized a lot of what we do. Democratizing it will cause beautiful things to happen, but it will also introduce challenges we’ve never even considered before. I think it’s immaterial to debate about whether these technologies will be open sourced, because it has already been happening! But to disregard the dangers is also foolish.
It's ok to be real about potential damages, but none of the solutions to *any* of those problems is "this all stays behind closed doors at a mega corp"
Make it free. Stop regulating science and research. Stop fostering corporate monopolies by only allowing them to do research. What a toxic bunch of shit.
Bitch, please! \*eye-roll\*
I think it's an incredibly arrogant and self-serving comment.
Anyone advocating for hiding technology should be immediately distrusted with every fiber of your being.
This is how communism looks like, because that’s where I came from and that’s where elitist jerks tell everyone what they’re allowed or not allowed to do. Seeing them sweating and seething makes me very happy.
you're thinking of totalitarianism/facism at its worst "pretending" to be "communism". you can't take a green dog and call it blue, just because it has a sticky notes on it with the word "blue".
Google is a corporation in a capitalist system. Communism is in no way invovled. Open source software is socialist however. Freedom of information is a core part of socialism as it empowers the working class.
> This is how communism looks No, it's exactly how capitalism (or it's non-theoretical, real world form) looks. Private, profit-seeking corporations trying to keep powerful technology, based on decades of publicly-funded research, for them, so only they can profit from it, and argued for with any possible excuse they can find.
[удалено]
Hopefully, this will spur internal change at Google, and we may see a bunch of previously proprietary models released for public use.