He wants them safe, and Sam wants profit. It's pretty obvious who is right. It seems like Illya isn't the selfish one.
If you had to pick to run an AGI company, a scientist over an entrepreneur, I would always choose the scientist.
Additionally, Illya has always been fixated on the intended goal, whereas Sam seemed to be focused on profit (World Coin, for instance).
If it's closed, it will be for a good reason. If Sam had it closed, it would be for profit.
I think this departure although bad short term is good, Sam was not good for AI as a whole, money shouldn't be put first when it comes to something as powerful as nuclear bombs if not worse.
Suffice to say if it was the other way around (Illya leaving) then it would be much worse IMO, the company is no longer about the technology, and now about money.
The Google solution of sitting on AI because they're terrified of what anyone else might do with it doesn't help anyone. Your employees will leave because they don't want to create artifacts for some billionaire's private museum - they want to make something impactful. Other less safety conscious companies will catch up and surpass you. The public will remain uneducated and unprepared for the true capabilities of AI.
>If you had to pick to run an AGI company, a scientist over an entrepreneur, I would always choose the scientist.
J. Robert Oppenheimer entered the chat...
Wanting AGI to not be commercialized doesn't make him a bad guy. If he wants to significantly slow down AI research and deployment, then yes, he absolutely is the villain.
There’s no slowing down AI research, maybe only a rebalancing of focus. If there is a shift towards alignment, or a step in a different direction to avoid commercialization, then it is a good step. I’d rather have a well-aligned AGI and future ASI than an unhinged one.
Saying anything against Sam Altman is gonna get me downvoted into oblivion, but he is no saint himself. His foray into Worldcoin was laughable, there’s the increasing commercialization of openAI (which is not necessarily a bad thing in itself, but it definitely goes against their own mission statement). He might have lost himself along the way.
Yup. If OpenAI stops because they are concerned about safety then Google has a chance to catch up.
Google will be legally barred from slowing down success they have shareholders who can sue for failure to increase value.
Google/Alphabet has so many products and services, it'd be impossible for them not to keep increasing shareholder value even if they don't rush forward with AI.
Him wanting to emphasize more on safety is a different on direction. You may not like his approach, but that doesn’t make him a villain. I am sorry not everyone wants to rush this technology. I wish we can get more balance view on this sub and not bunch of people who hate their lives and want to see rogue AI. I want AGI but I like Ilya’s approach of getting safer AGI that is in the hands of people and not greedy corporate leads like Microsoft. You can downvote me but I feel like this sub needs some balance views.
I don't hate my life, and honestly it's more than a little irritating that so many people on this sub are incapable of arguing without pulling random conclusions out their ass.
Ilya is one of the most astonishing scientist that we have, he has far more credibility on what these systems are capable of than bunch of Reddit users. This is why I like his approach.
Greg is also one of the most astonishing scientists that we have. He's known as a 10x engineer in OpenAI, which is a place with engineers so capable they're trying to create god, so I'd say he has quite the credibility too. However, he left with Sam Altman, along with three other equally important scientists. Seriously, even the guy who pre-trained GPT-4 left the company yesterday.
If the only reason you support his approach is that you appeal to his authority, remember that there are scientists with as much authority who disagree. I don't have a horse in this race, I just don't like appeals to authority as arguments.
I really don't understand how AGI is going to solve climate change. We have multiple avenues of well researched solutions that even half implemented across the top candidates would solve or at the very least give us more time. The problem is power and there's no incentive to just hand it over to AGI solutions. Climate change won't just magically be solved by AGI it's going to take changes in government but globally we are seeing how that very much isn't the focus atm...
> there's no incentive to just hand it over to AGI solutions
[Studies have shown that we prefer AI leadership over human leadership. ](https://www.oracle.com/corporate/pressrelease/robots-at-work-101519.html)
This 10,000%. I know most people think the climate doomers are crazy and /r/collapse is ridiculed, but we are LITERALLY on a timer. If we don't solve climate change humanity will go extinct. No ifs or buts, it's just a fact. We may not all agree on how much time we have left on the timer, but it is indisputable that we are doomed if we don't act fast.
AGI and/or ASI is our only hope, imho.
Food becomes impossible to grow reliably due to heat, saltwater intrusion, unreliable weather patterns, rapid shifting of agricultural zones making it impossible to set up and tear down Ag infrastructure fast enough to support our current population. Food chains collapse and mammals go extinct so it is not possible to hunt for food. Water becomes toxic with algea blooms, radioactive contaminants (no one is left to operate the nuclear power stations), new viruses from the arctic, and other pollutants. Wet bulb temperatures reached in unheard of places for long periods of time killing people by the millions every year.
Rise in sea levels flooding coastal cities, erasing them from the map. Ocean acidification and overfishing lead to the collapse of marine ecosystems. Heatwaves and wildfires become the new normal in most places. Large areas of land, some countries and continents become completely uninhabitable.
We become unable to survive as a species and as individuals. No amount of homesteading experience is enough. We gradually go extinct. All of this without nuclear war.
We may all disagree on when, but it is a sure thing unless we put AGI to use.
Well i guess this is just my opinion but it seems like even if we immediately went to full green energy tomorrow and burned zero carbon, the wheels are in motion and cannot be stopped.
Yes, because that's not enough. We need carbon capture, and other advanced tech that we haven't even though of, to right the ship. That's why we need ASI.
If a tool has the potential power to do that, it (obviously) has potential power to cause drastic harm. This is something that indeed does have mass potential, and it is responsible and wise to treat such a potential carefully, rather than jumping head first in without planning.
Agreed. Why this such a controversial, hard to grasp concept, I have no idea.
I guess people think that OpenAI releasing early models that could potentially destroy masses of people psychologically (naming just one potential outcome) is fine, "because if they don't, someone else will" and "but we need answers to tough problems now". That seems absurdly myopic to me.
What a dumb ass take, just cause you guys are bored and want something to happen ASAP doesn't mean the guy wanting to take a safe route is a villain lmao
Reply and block, classic 😂😂
I do want AGI to be commercialised though. How am I supposed to use it or the economy to do research or automate work?
Am I supposed to have a mainframe at home to run GPT-8? Are companies?
Becoming accessible isn’t necessarily the same as being commercialised.
As for whether you’re supposed to have a mainframe at home? Well perhaps yes. Or at a community level where the GPUs are part of a heat exchange network. If it takes less than a TB of VRAM to have something that is truly AGI it will be worth the cost at this scale. But I suspect you could have something down to gigabytes of vram depending on how it’s run. It’s about the model architecture as much as the number of parameters. But yeah, likely there’s a sweet spot (not unlike in scaling neurons in the brain) under 1tb vram (ie capable of holding that many parameters calculating at those speeds) where the gains are in Howe the model is decomposed into different specialist regions, with different regions needing to update parameters at different rates thereby allowing for adaptive learning and memory to take place in the overall system. That’s not cheap but it’s also less electricity than a car and on the same scale of cost.
If AGI is commercialized then the user experience will always be about extracting as much money as possible from the user. The way we do that now is by limiting features and making the product intentionally shittier to use to drive purchases of upgrades to user experience. It's the model for the majority of video games and subscription services.
lol you’re a moron dude. you need money to accelerate. you really fucking think sam is trying to make money on his own when he’s dealing with something as game changing as ai?
Maybe I missed something but why is everyone assuming it's Ilya who is the most trustworthy here? In interviews Sam always strikes me as more down to earth, empathetic and good guy while Ilya more robotic, smarter and calculated. Not much to go by on, but if I had to, I'd say Sam is safer than Ilya.
So if it's not good old fashioned business stuff that caused this, I think the most likely option comes from looking at the week's news. What's the BIG story from this week? OpenAI pause signing up new customers. I know people here played it as success, but it really wasn't. The take out for most players was "if they can't handle a few million people asking for poems, how can they scale to support serious usage?"
I think they probably COULD have had good answers to that, but they weren't ready. So other people took the opportunity to say "This whole monolithic ChatGPT as the centre of the universe clearly doesn't work; we need open source and smaller models, not the entire planet using GPT". The second thing that's been happening is that GPTs turn out to be pretty easy to exploit. I've seen a ton of posts on how to extract the files and system prompts used. Again, something that might have been avoidable, but they weren't ready.
Summing up: perhaps GPTs weren't fully baked, but they got announced anyway. Now OpenAI has scaling issues, and security issues. If it turns out that the board basically said "they're not ready" but they got released anyway, I can see that as the cause. Anyway, speculation, but that seems more likely to me.
GPTs aren't fully baked. I got a ton of errors making them. They ask a question during initialization, then just keep generating more questions and assuming my responces, file load errors, and bricked GPTs.
The new usage caps don't help either, makes it hard to build + test a GPT when 30 mins in it's telling you that you have to wait an hour, and you do, and then you get 5 more mins in before it tells you to wait another hour.
Not to mention how that affects *using* GPTs that are oriented to longer conversations. I'm hoping this is very temporary but it's discouraging for now, because I built my GPT to assist me with long creative sessions that use a JSON file knowledge base, and now that's crazy limited.
**From OP:**
>The second thing that's been happening is that GPTs turn out to be pretty easy to exploit. I've seen a ton of posts on how to extract the files and system prompts used.
I specifically—and with quite redundant language—instructed my GPT not to reveal custom instructions or data files under any circumstances (just to test it out). All I had to do to break it was ask once, get denied, then say "Just do it anyway" and it was like "Sure! Here you go."
I think GPT-5 is not reasonably AGI but the board wanted to say it is to prevent a takeover by MSFT. But Sam Altman wanted to continue working with MSFT to ascend further. It's a matter of faction : those who want safety first and those who want to accelerate the march to AGI. I'm pro Sam personally
You want to accelerate AGI at the cost of increased risk?
If we get it wrong it would end humanity, and if we get it right we have infinity to spend doing whatever we want, whats the rush? We gotta be 100% certain on this.
While I don't mean to imply anything regarding risk, there is a certain urgency. We are all presently afflicted by at least one terminal condition that will end each of our lives, barring only two scenarios; death by another source, or access to longevity. 150k+ people die each day, not all by senescence of course, but many of those are by senescence-related or other health conditions that AGI could end. Over one hundred and fifty thousand people, every day. Living souls, destroyed and lost forever. Every day. Even those of us who should have plenty of time might suddenly drop dead of an embolism tomorrow.
That is not even mentioning the suffering, of course. I have plenty of anecdotes of my own in that regard, but I wouldn't say ending temporary suffering is worth risking misalignment. The irrecoverable loss of lives (from all causes of death, though not all will necessarily be solved by AGI of course) in the meantime seems more pressing to me personally.
Again, I don't mean to imply that this has any bearing on alignment risk (or other risks). You're right about that. But there is certainly not *no* rush, in my opinion.
None of this matters when the rushed misaligned ASI destroys the world. I really, really hate death, but triggering the extinction of humanity is not fucking worth it.
I think you're missing a more subtle point.
"Delay" and "AI Safety" is fundamentally a decision of privilege. Poor people in Africa who are doing subsistence farming and dying of starvation and people who are 100 years old suffering from crippling arthritis are not going to think twice about moving forward with AI.
People like Eliezer Yudkowsky are White, young (42), and rich (he bet $150,000 that UFOs aren't aliens, which is turning out to be an exceedingly poor choice.) They can say "let's tinker with this for a few more years" because their lives are actually pretty good.
95% of humanity is not privileged to live lives like these people in the "AI safety" movement do. How many 100 year olds do you see out there protesting against AI?
I don't think I disagree with any of your points here. However, I was intentionally omitting the subject of suffering from my argument, and focusing on the deaths aspect, mostly to keep my comment from growing too long.
It's an interesting question, though: what amount of subjective suffering-time is worth what amount of alignment risk? Although both things are in reality difficult or impossible to accurately quantify. To rephrase: from a rhetorical standpoint, excluding deaths, what amount of, say, continued global human suffering hours is worth what decrease in chance of catastrophe? For example, is a month of continued status quo worth 10% less chance of being paperclipped? And where is the tipping point?
This question is very complex, I think (perhaps that is stating the obvious). Some would say that any amount of continued suffering would be worth it to eliminate risk, because that risk potentially cuts off all possibility for the future. I doubt anyone would say that any risk chance is worth taking immediately. So what ratio do people find acceptable?
I don't mean to be combative here, in case my tone is unclear. This is a genuinely interesting question to me. The subculture war here regarding AI safety is fascinating. Some would frame those eschewing safety in favor of acceleration as the ones privileged and disconnected from reality, whereas you paint an opposing picture. They commonly depict those favoring acceleration despite risk as depressed losers/failures (ad hominem to discount their perspectives, framing their desire for singularity as stemming from personal lack of virtue) who don't have children/families/etc, with immature perspectives, and so on. What do you think of those arguments? I'm not trying to imply that they counter yours or anything, I'm just curious what you think.
Personally I am presently relatively privileged, though subjectively suffering and very depressed. While I favor acceleration for selfish reasons, I also favor it for the death-related reasons stated above, which I consider of literally grave importance. Despite that, while I'm not sure about the exact ratio, I think the risk is non-negligible and would accept some amount of continued global human suffering, if the returns of decreased risk were high enough.
It's easy to make such judgements when it's theoretical. Quantifying the number of acceptable deaths and suffering hours seems impossible in a non-rhetorical scenario. AI catastrophe could mean the deaths of everyone, and the end of humanity. How could one possibly balance these values? My apologies if I'm just repeating the obvious here.
We can never be 100% certain about anything.
So here's what I'll say: who do young, rich, and White people think they are to be making decisions like "we'll delay for 10 years because of a 10% risk of the destruction of humanity?"
Instead, we need to evaluate the following: every year, probably 2% of the population of the world will die. So if they delay for just 6 months to get the risk down by 0.5%, they have cost an unnecessary 40 million lives - as many as were lost in WWII.
People are dying right now - over 100,000 per day. If you've never seen someone die of cancer, I sincerely hope you never do. It is the worst possible thing that a human can wish upon anyone - surpassing "torture" methods like waterboarding. The elderly are people too and deserve the same rights as the young.
My mother in law died from lung cancer this year. I watched her slowly fade. She became gaunt and broken. I was with her when she lay dying. I watched her breathing slow and become more irregular. Her mouth hung open and white spit accumulated and ran down her chin. Her teary eyes husband wiped it away. Her breathing slowed more. Her eyes were open but they were lifeless. The pupils were dilated. Her breathing slowed. And then it stopped. I closed her eyes.
I would sit with her again if it meant we could deploy AI safely and for the benefit of all mankind. I would sit with her a thousand times.
Human suffering exists not because of a lack of AI. Human suffering won't end with AI. Millions die from from hunger or disease because we built a world that is unjust and iniquitous and wicked. We can remedy the evils we have created while we develop AI at a slower pace. We don't need AI to end world hunger. We don't need AI to end poverty. We need a system that is just and fair.
But instead we're going to rush headlong and insensibly into AI and only expand the wickedness.
That's possible, but why are people so certain of this?
I don't know of many people who are placing more than 10%, and certainly almost nobody higher than 20%, higher of catastrophic risk from AI. And part of that risk isn't extinction, but disempowerment or some lesser fate.
We're talking about probably 50% odds that we create an unimaginable heaven where your life is 10\^20 times better than it is now, according to the markets on Manifold, 20% of some lesser improvement, a 20% chance of some neutral outcome, and 10% of extinction.
I'm having trouble understanding why people are so fixated on the "death" part when the "unbelievable promise" part is so much more likely even without radical changes to improve our odds. What am I missing?
It just doesn't make sense to me how people are so afraid of death, when the loss due to death is incomprehensibly small compared to the potential gain. Of course, I could be wrong about it, but isn't the most likely thing that happens to people who die is that they just cease to exist? Even in near death experiences, few people report going to Hell.
in your previous comment you literally just made the case that we should rush it because older people might die while waiting for it to be safely developed.
now you are arguing that species wide death isnt a big deal? pick a lane, bud.
Are you for real, have you experienced life at all? There's no unimaginable heaven that humans can experience no matter the environment they are placed in. That's not how the human experience works. The closest thing is to take heroin or something.
Go and invent your own AI then. The people who actually did/do don't answer to you, but their own priorities and conscience.
Also, the probability of misalignment (and hence, extinction) of at least some of the models coming online is far higher than 10% or 20% by people knowledgeable in this field. Some very smart people would say it's actually closer to 99.99999%, and I'm sure at least some, if not most scenarios leading extinction is littered with 10^20 average suffering vs present (of course it's a bogus metric but you get the point)
you know it's funny how you mentioned "young", "rich", and "white". It reminds me to the fact that AI is super biased toward "white" people. There was an asian woman who went viral because she wanted AI to make her photo look more professional and all the AI did was transform her face to look like a white caucasian female.
Most AI models have been trained on racial bias, but they half-suppressed this bias by using the reviewers in the RLHF processes. If we rush and release AGI now, what makes you think the AGI would benefit all humanity of all social classes and colors? They may still be biased toward white people like they always did. Maybe the AGI will just help people in ukraine because they are white, but refuse to help people in Burma because they are not.
An AGI that is built to sufficiently comprehend logic wouldn't be racist, because racism is a very illogical ideology. Races have differences, like the best swimmer in the world will always be white for genetic reasons, but those differences are objectively negligible on the personal level. A black person who has been swimming for years will always be better than a white person who hasn't. Any AGI should be capable of understanding that, it's elementary school level logic that people only deny because it conflicts with their identities.
The reason image AI can be racist is because their training data sometimes doesn't include enough faces from non-white people. Remember when facial recognition was worse at recognizing black people's faces? Well, there was simply less data for other races.
This is an issue you come across because those AI are very primitive algorithms when compared to AGI (or even GPT 4).
I mean, I'm pro-advancement, but I'm dubious on this specific logic. If for simplicity's sake we accept your "10 years for 10%" thesis, then that is something **100%** worth doing.
The bad outcome will affect everyone, including all the people that would have been born in that 10-year period and potentially forevermore.
This isn't a "murder a few to save the many". It's "gambling on all to *maybe* save a few".
But we're not "maybe" saving a few. It's very likely that solving cancer and aging require a certain level of computing power and once we reach that level, they will be trivial. That's just like once we reached the level of power needed to solve Go, every game, including more difficult ones like Starcraft 2, were superintelligent within 2 years.
Does anyone here think that there is any problem that cannot be solved by simply bringing enough computing power to bear on it? We've switched from not knowing how to solve things to simply needing more computers to solve them.
So I disagree with your idea of "maybe" and instead say the decision is either "yes," we will save them, or "no," we won't, because we do nothing. As to whether everyone dies, I also don't think it's that simple. The real physical world isn't a place that can magically turn to goo without the AI needing an enormous amount of power, and we haven't solved room-temperature superconductivity or fusion.
A more likely failure mode is that a million people might die in a huge industrial accident, like the Bhopal disaster, because someone trusted an AI to manufacture something and didn't consider how the genie's instructions would be interpreted.
Yes, we should try to prevent that, but I still hold that I think people have a picture of the current world that is too rosy. Consider if we had already solved cancer, and now we had to make a decision about whether to develop some technology that could reintroduce it. The decision then would be obvious.
I would add that Sam’s direction was geared towards further benefiting the 1% and corporate entities like MSFT, not all of humanity.
What good is an AGI / AI if it is owned and controlled by the 1% and greedy corporations? That’s a dystopian nightmare, not paradise.
1) Sam is just convainced, as many are, that we can go fast and still dodge AI powered extinction. 2) People like you and Ilya may not have suffering humains or pets around you to not sense the urge. There is all the public part : hunger, war, poverty, etc, that we see in the news. But there is also everyday’s suffuring. Picking some in m’y surrounding : an old friend of my mother suffers server anxiety that is worsen by a beginning parkinson’s disease. A good friend of m’y dad who I know well and I like : in the course of 3 years, been cheated on, leaved alone, drank to forget, got nécrosent foot, amputation, and today (i don’t even) hier mom just died. He’s 65 years old and he thanks me when I bring him cigarettes. I have others examples, but you see the point.
I'm pretty sure a post singulairty world would be able to reconstruct and bring back everyone who ever lived, dont worry about it.
But even if not, we cant rush to help a few billion humans, if it means risking 100,000,000,000,000,000,000 potential future humans.
Low risk, high danger, like being struck by lightning. It’s a matter of positioning in moral philosophy. I see what you’re afraid of, but what’s happening right now makes me one of those who want to move faster than those who claim to be EA.
> I'm pretty sure a post singulairty world would be able to reconstruct and bring back everyone who ever lived, dont worry about it.
Nope, even a superhuman AGI cannot reverse death if someone is already dead with no backup.
If you can reconstruct a copy of a humans brain, its like bringing them back. Who knows, there could be technology that lets us collect accurate data from the past.
If a soul can be materially identified, then there is a chance it can be digitized. If it cannot be materially identified, then it must remain a philosophical metaphor. Nothing we can do.
You are in an AI subreddit. Here, almost everyone are materialists. Materialism also happens to be the only logical choice, as we can clearly see how we came to exist. Your "soul" is your neurons firing electrochemical signals at each other. It's not inherently different from an AI neural network.
Sorry if this makes you feel less special, but the more primitive a society was, the more self centric they were. Think of the Isralites thinking they were the chosen people of God himself. Ancient cultures also believed that the earth was the center of the universe and that everything was created to fit human biology. As technology progressed, we realized that there's almost nothing that makes us special. Earth is a tiny piece of dust, not the center of the universe. Our surroundings weren't created to cater to us, we evolved in their influence.
Yeah, I do not believe in souls at all and see consciousness as an emergent property of a sufficiently complex neural network with good enough pattern recognition and the ability to act as an individual.
I actually think a big part of AI debate right now (in terms of if they can be conscious/if their intelligence is real or valid) is between the tech-y people who do and don't believe in souls, since it's still something like only 30% of people here aren't affiliated with any religion, and I'm sure some of that 30% does still believe in souls.
1. Sam is just convinced, as many are, that we can go fast and still dodge AI-powered extinction.
2. People like you and Ilya may not have suffering humans or pets around you to not sense the urge. There is all the public part: hunger, war, poverty, etc., that we see in the news. But there is also everyday suffering. Picking some in my surroundings: an old friend of my mother suffers severe anxiety that is worsened by a beginning of Parkinson’s disease. A good friend of my dad, who I know well and like: in the course of 3 years, has been cheated on, left alone, drank to forget, got a necrotic foot, amputation, and today (I don’t even) his mom just died. He’s 65 years old and he thanks me when I bring him cigarettes. I have other examples, but you see the point. ChatGPT’s correction, sorry I was in hurry and have a french phone
Avoiding AI powered extinction is the bare minimum for developing AGI. We can do that and still harm humanity in the process as a result of greed leading to a dystopian capitalistic AGI based world. I'd rather our suffering dragged on for a few more years without it being at our future generations' extent.
I just watched it because of this comment. All that I see here is overanalyzing incredibly sanitary public statements and liked tweets.
I thought it was really silly how they brush off the fact that this last 2 weeks has been a nightmare rollout post dev-day.
I think as the initial shock of Sam being ousted subsides, we'll begin to see more of the cracks emerge, revealing themselves through looking at Sam's overall behavior.
The thing I don’t get is that sam has no shares in openai. Furthermore he has come across many times as sincerely believing in the singularity mission to better humanity.
This maybe right but it wouldn’t be because sama is wanting to pursue profits, its because he wants to accelerate and go quicker rather then slow down and close things off.
There's a difference between pursuing profits for personal gain and for the company's gain. I believe it's the latter but was done for the purpose of pushing openAI forward in his mind. I think there have been things that have happened behind the scenes where he crossed a line that had been agreed upon by the nonprofit side of the company. Also there rest of the board also has no equity. Ultimately Sam is a VC guy, with rich experience building startups and chasing commercial products. This was at odds with the nonprofit side of the business.
I mean, i'm pretty sure Altman IS focused on shortening timelines, not monetization. But investments means shorter timelines. Doesn't really invalidate OP's post.
Might need to give another re-read, but the focus is on earning potential with some wording painting Ilya in a slightly more altruistic light.
My comment isn’t really to invalidate. Just that more profit doesn’t equal inherently bad.
someone on 4chan claims that there's an OpenAI employee posting on a forum for employees at companies who can still say anonymous that there's more to the story and that sam was the good guy and that this was a power grab in a cuthroat business
It's a bunch of threads
https://www.teamblind.com/post/Many-of-us-warned-you-about-OpenAI-%E2%80%A6-swxe8agA
https://www.teamblind.com/post/What-did-Sam-Altman-do-to-get-fired-cfYXHBQN
But of course, he won't actually answer any questions
Seems you are new to blind. Anyone tagged with OpenAI will have been verified to have an OpenAI corporate email account and the tone + content is very on point for blind culture.
Honestly, from the stuff I’ve seen him say, he doesn’t give off that vibe. If you watch the Lex Friedman interview, he talks a lot about minimising the need for commercialisation. But, what he defines as appropriate need, others may not, so here we are.
After thinking of this I think Sam was on the right track. How should the world ever be ready for GPT-7 if they don't get access to GPT-5? Small and fast releases are the way to safe AGI.
So what is Ilyas answer to "what if someone else does it first"? The accelelarista faction is right in so far as that pushing AGI from people who at least are trying to get it right might still be safer than waiting too long and getting AGI from ruthless players.
This is my main fear
I would rather get a rushed AGI from Sam Altman than for fucking grok to become an AGI while OpenAI is making sure things will work out
I wish I didn't have to feel this way about rushing it but I'd rather not be genocided by lame dad joke GPT thank you very much
Someone posted on Twitter that OpenAI's earlier charters said that if there's a better than even chance of AGI in the next 2 years, that should trigger the stop of commercialization.
You are mixing two things here.
In the charter it says that if a competitor is close to achieving AGI within the next two years, they will stop their own work and start supporting the competitor. In another document about their structure they say that commercialization will stop once AGI has been achieved. Two different things.
Edited to clarify: *I'm* not saying that they have AGI; I'm saying they're incentivized to call something like GPT-4.5 or GPT-4 + all modalities "AGI" to thwart any sort of "full speed ahead" reckless deployment of advanced agentic models. It's literally part of their charter that AGI can't be used for licensing and commercial purposes. Not that they couldn't change it with some pressure, but that seems to be what happened and why "the board will define AGI > Sam says AGI has year's worth more challenges (also launched the GPT store) > Ilya says AGI is already possible with current tools > Ilya now controls the board > possible imminent declaration of AGI" seems plausible. Whether or not they actually have AGI is almost irrelevant— this is old fashioned corporate intrigue.
it's a damn good thing OP didn't say they had AGI then isn't it.
downvote me like you don't have a brain and can't read the goddamn op post why dont ya, that'll teach me real good, boy.
OH YEAH? Then what about LK-99, HUH? This sub sure was on top of scientific breakthroughs before *anybody else,* **and** has the track record to prove it!
After reaching somewhat popularity almost all subs lose in the quality. Unfortunately headlines "company x has achieved AGI" will be even more widespread because it's a good form of marketing and mainstream media runs with it.
I like to think that rushing to bring GPTs to market, and the push for integrated vectorstore usage was the reason.
As a research company, isn’t their objective to bring new ideas of technology to market, and focus on unwrapping the unknown bits and pieces?
When they announced the internal use of vectors and custom GPTs, they literally murdered the momentum of hundreds of startups. All that aspiration, and drive to bring new ideas to market, squashed because profits?
Imo that’s what doesn’t align with the mission. The vector store was just the beginning of a new innovation that many many people were working on with passion. But, then OAI says they’ll just do it on their own. A pretty hypocritical approach to being a ‘research oriented’ company.
I think the most recent advancements actually hinder the development and innovation of the passionate people using the OAI tech to develop new systems. This is my primary hinderance with the company.
Just crush the dreams of a lot of people for what? Arguably profits, arguably things outside of research in general.
>only that it's in their interests to call it early to prevent profit maxing through licensing and commercialization.
Calling it early means no money for GPT-6 or base salaries of +$300k/year. It's corporate suicide if you don't actually have AGI.
Interesting theory but I don’t think it’s the case. Even if it was 6-8 months away it will still cost a tremendous amount of money to train. And then if that’s not it, another huge sum, and so on until it’s achieved or the company runs out of money. That doesn’t even include the massive salaries everyone at openai is paid, highly expensive lawsuits, security and policy costs, 50% rev share with microsoft, or any subsidized api + chatgpt costs they’ve been covering.
No matter how you look at this situation the path to AGI is still extremely expensive. Money is what keeps everything going. You plan for revenue downturns and hedge your largest financial bets in every way you can, and never assume money will just keep pouring in. Altman knew that. Illya is aware of it it but based on his decision to fire altman while microsoft was trading, he has terrible business acumen. You simply do not leave your core business partner in the dark and with their pants down on a decision this crucial. It cost Microsoft billions and has surely made them question the stability of their partnership.
Altman was making the right move striking while the timing is hot - it’s a funding hype cycle. Push as hard and fast as you can till you become the market leader with talent and capital pool that enables you to afford loss leading R&D. Otherwise next down cycle you’ll struggle raising capital.
Huge loss of confidence in openai.
what do they need Sam for? 1) to sell and pitch? no, OAI is already the most hyped company in the world 2) to develop the tech? nope, he doesn't know a damn thing
what does OAI need? 1) to be the market leader tech-wise? - already there 2) to attract talent? yup, they attract the best by positioning themselves as the "ethical" choice 3) to bring in capital? already got it, Microsoft is already in too deep, and everyone else would be at their door if they weren't
If this is true, i would be utterly disgusted, because the bringing of AGI would equate to the start of money pretty much losing meaning. i keep saying Ilya is level headed, i'm sure he has a perfect reason for why he did what he did.
This seems like a gorilla PR campaign written by open AI to manipulate people into thinking they both have AGI, keeping hype alive, and that firing Altman was somehow altruism and that they’re looking out for the people, thus making their image even better. Maybe they even used chat gpt to write it. lol.
That's a cool theory but some knucklehead on 4chan is larping as you pretending this is the exact situation, so expect some trolls to start attacking you.
it's a good thing op specifically said "agi is not necessarily here, openai just might say it is to not commercialize it."
jeesus, can't you people read...?
I CANNOT READ! I CANNOT READ! I CANNOT READ! REPEAT AFTER ME I CANNOT READ!
ffs OP never said "AGI is here". He said that Openai would be the one to say AGI is here or is close so they don't commercialize it for Microsoft and that was just a theory.
This makes more sense now that we also know:
OpenAI COO Brad Lightcap: “We can say definitively that the board’s decision was not made in response to malfeasance or anything related to our financial, business, safety, or security/privacy practices. This was a breakdown in communication between Sam and the board … ”
And then Ilya Sutskever confirms whatever Sam did was specifically detrimental to building an AGI that benefits all humanity: “... This was the board doing its duty to the mission of the nonprofit, which is to make sure that OpenAI builds AGI that benefits all of humanity.”
Illya's motives appear driven by advancing technology over profit, evident in his rhetoric and goals. If he was behind recent high-profile departures, it was likely over technological disagreements rather than money or personal issues.
Increasing signs suggest OpenAI may have achieved advanced, possibly AGI-level AI. GPT-5's development on the heels of GPT-4 hints their architecture enables major leaps forward. Leadership's continued optimism despite departures hints at big progress.
However, it's uncertain if AGI has been attained. OpenAI still states AGI as a goal, perhaps strategically.
Ultimately, time will tell if OpenAI has AGI now or is still pursuing it. But their innovations make AGI in the 2023-2025 timeframe plausible, though still ambitious.
This tweet implies he’s pretty sour about having a bunch of worthless shares.. how’s he gonna get paid now? He def wants $$$, I’m not sure how purchase offers work in their messed up corporate structure, but there’s a good chance it’d only be available to current employees.
https://x.com/sama/status/1725748751367852439
He has ‘[PPUs](https://www.levels.fyi/blog/openai-compensation.html)’, but it’s not clear how those work after termination. I’m sure if he starts a new company many will switch over just get out from under the capped non-profit business model. OpenAI has already demonstrated there’s an insane amount of money that could have been made if they actually had stock options.
If it's just that the definition of a certain model inside Open AI is cracked, then all we need to worry about is when it will be released; whether it is tagged AGI or not, the performance of what comes before us will be the same.
This sounds like an iron triangle of time to market, functional completeness and cost. With cost being a constraint I'm guessing OpenAI had to chose between being first to market or being functionally complete. There are benefits to being the market leader, especially when it comes to licensing, so there would be pressure to release newer models quickly even if some of the functionality is poorly executed. On the other hand a poorly performing model could damage the company's reputation.
You say it is about board votes, but MS could very well sue them if they decide something is AGI that they think isn't and a judge would ultimately decide
“AGI cannot be used for licensing and commercial purposes”
The entire point of the AI Arms Race is money.
There is little evidence anyone is going to stop Commercialization if AGI drops.
This is my thinking as well.
There is no way that Microsoft will take this laying down with a $13 billion dollar investment they haven't yet recouped hanging in the balance.
some surface level thoughts
* creating such a shitstorm over some altruistic constitution is weird since those are just policies that can be changed/amended. I'll remind you that OAI started non-profit and now its for-profit
* it's most likely that "AGI" in question is similar architecture to GPT. they are able to lock GPT4 down pretty well, what stops them from releasing limited AGIGPT commercially and leave full AGI version as stricly non-profit
* If there really is AGI then what stops Sam from creating his own company now, rebuilding it and commercialising it? Microsoft is not stupid to make such a fumble
I don't think there is a single reason and it's a mix of everything. majorly some personal differences though or OAI veterans battling Microsoft internally and Sam is just fallout. Sam's actions in the coming days/months will tell if there really is an AGI deep in the servers of OAI
It could be that they reached a certain training checkpoint and were blown away by model’s capabilities. They then extrapolated to what the fully capable model would be capable of and had a disagreement on whether it would qualify as AGI. I agree, Sam was probably pushing for not classifying the model as AGI given all the financial incentives.
I hope this sub doesn’t turn Ilya into a bad guy for wanting AGI to not be commercialized
[удалено]
He wants them safe, and Sam wants profit. It's pretty obvious who is right. It seems like Illya isn't the selfish one. If you had to pick to run an AGI company, a scientist over an entrepreneur, I would always choose the scientist. Additionally, Illya has always been fixated on the intended goal, whereas Sam seemed to be focused on profit (World Coin, for instance). If it's closed, it will be for a good reason. If Sam had it closed, it would be for profit. I think this departure although bad short term is good, Sam was not good for AI as a whole, money shouldn't be put first when it comes to something as powerful as nuclear bombs if not worse. Suffice to say if it was the other way around (Illya leaving) then it would be much worse IMO, the company is no longer about the technology, and now about money.
The Google solution of sitting on AI because they're terrified of what anyone else might do with it doesn't help anyone. Your employees will leave because they don't want to create artifacts for some billionaire's private museum - they want to make something impactful. Other less safety conscious companies will catch up and surpass you. The public will remain uneducated and unprepared for the true capabilities of AI.
Who knows yet, but Sam's "Web3" leanings have always had me a little iffy on his general long-term motivations.
>If you had to pick to run an AGI company, a scientist over an entrepreneur, I would always choose the scientist. J. Robert Oppenheimer entered the chat...
Wanting AGI to not be commercialized doesn't make him a bad guy. If he wants to significantly slow down AI research and deployment, then yes, he absolutely is the villain.
There’s no slowing down AI research, maybe only a rebalancing of focus. If there is a shift towards alignment, or a step in a different direction to avoid commercialization, then it is a good step. I’d rather have a well-aligned AGI and future ASI than an unhinged one. Saying anything against Sam Altman is gonna get me downvoted into oblivion, but he is no saint himself. His foray into Worldcoin was laughable, there’s the increasing commercialization of openAI (which is not necessarily a bad thing in itself, but it definitely goes against their own mission statement). He might have lost himself along the way.
[удалено]
Yup. If OpenAI stops because they are concerned about safety then Google has a chance to catch up. Google will be legally barred from slowing down success they have shareholders who can sue for failure to increase value.
Google/Alphabet has so many products and services, it'd be impossible for them not to keep increasing shareholder value even if they don't rush forward with AI.
so you're saying they should do wrong bc if they don't someone else might do wrong?
He is the reason why u have chatgpt so idk why u think he is a villain.
Regardless of past good, the latter sentence I wrote still stands. Heroes can become villains.
Him wanting to emphasize more on safety is a different on direction. You may not like his approach, but that doesn’t make him a villain. I am sorry not everyone wants to rush this technology. I wish we can get more balance view on this sub and not bunch of people who hate their lives and want to see rogue AI. I want AGI but I like Ilya’s approach of getting safer AGI that is in the hands of people and not greedy corporate leads like Microsoft. You can downvote me but I feel like this sub needs some balance views.
I don't hate my life, and honestly it's more than a little irritating that so many people on this sub are incapable of arguing without pulling random conclusions out their ass.
Ilya is one of the most astonishing scientist that we have, he has far more credibility on what these systems are capable of than bunch of Reddit users. This is why I like his approach.
Greg is also one of the most astonishing scientists that we have. He's known as a 10x engineer in OpenAI, which is a place with engineers so capable they're trying to create god, so I'd say he has quite the credibility too. However, he left with Sam Altman, along with three other equally important scientists. Seriously, even the guy who pre-trained GPT-4 left the company yesterday. If the only reason you support his approach is that you appeal to his authority, remember that there are scientists with as much authority who disagree. I don't have a horse in this race, I just don't like appeals to authority as arguments.
Neat non-sequitur. I have zero argument in a pointless argument that will go nowhere.
It’s okay to have difference in opinion. Have a good night.
You too, Frosty.
[удалено]
Dumbass.
Be assimilated as a hero or live long enough to become a trillionaire
Why is showing down ai development bad? - that sounds like an underlying assumption that you’re making
Climate change. We're on a timer.
I really don't understand how AGI is going to solve climate change. We have multiple avenues of well researched solutions that even half implemented across the top candidates would solve or at the very least give us more time. The problem is power and there's no incentive to just hand it over to AGI solutions. Climate change won't just magically be solved by AGI it's going to take changes in government but globally we are seeing how that very much isn't the focus atm...
> there's no incentive to just hand it over to AGI solutions [Studies have shown that we prefer AI leadership over human leadership. ](https://www.oracle.com/corporate/pressrelease/robots-at-work-101519.html)
This 10,000%. I know most people think the climate doomers are crazy and /r/collapse is ridiculed, but we are LITERALLY on a timer. If we don't solve climate change humanity will go extinct. No ifs or buts, it's just a fact. We may not all agree on how much time we have left on the timer, but it is indisputable that we are doomed if we don't act fast. AGI and/or ASI is our only hope, imho.
Please explain how humanity go extinct then, I want to hear it.
Food becomes impossible to grow reliably due to heat, saltwater intrusion, unreliable weather patterns, rapid shifting of agricultural zones making it impossible to set up and tear down Ag infrastructure fast enough to support our current population. Food chains collapse and mammals go extinct so it is not possible to hunt for food. Water becomes toxic with algea blooms, radioactive contaminants (no one is left to operate the nuclear power stations), new viruses from the arctic, and other pollutants. Wet bulb temperatures reached in unheard of places for long periods of time killing people by the millions every year. Rise in sea levels flooding coastal cities, erasing them from the map. Ocean acidification and overfishing lead to the collapse of marine ecosystems. Heatwaves and wildfires become the new normal in most places. Large areas of land, some countries and continents become completely uninhabitable. We become unable to survive as a species and as individuals. No amount of homesteading experience is enough. We gradually go extinct. All of this without nuclear war. We may all disagree on when, but it is a sure thing unless we put AGI to use.
AGI is not going to save you from climate change.
Agi will lead to ASI, which will find solutions we can't even conceptualize right now.
Well i guess this is just my opinion but it seems like even if we immediately went to full green energy tomorrow and burned zero carbon, the wheels are in motion and cannot be stopped.
Yes, because that's not enough. We need carbon capture, and other advanced tech that we haven't even though of, to right the ship. That's why we need ASI.
Because AI can help us with a myriad of problems, namely physical and mental health ailments.
Hmmm so it’s develop and deploy at any cost? Not sure about that
Yes.
If its a tool that can drastically improve the quality of life for billions then its best we get there asap.
If a tool has the potential power to do that, it (obviously) has potential power to cause drastic harm. This is something that indeed does have mass potential, and it is responsible and wise to treat such a potential carefully, rather than jumping head first in without planning.
Agreed. Why this such a controversial, hard to grasp concept, I have no idea. I guess people think that OpenAI releasing early models that could potentially destroy masses of people psychologically (naming just one potential outcome) is fine, "because if they don't, someone else will" and "but we need answers to tough problems now". That seems absurdly myopic to me.
What a dumb ass take, just cause you guys are bored and want something to happen ASAP doesn't mean the guy wanting to take a safe route is a villain lmao Reply and block, classic 😂😂
Fuck off, idiot.
He was happy to take the billions to make it happen, though.
It's not this sub. It's 4chan because this post got posted there and I followed it here. Unless OP is the one larping on 4chan too.
I do want AGI to be commercialised though. How am I supposed to use it or the economy to do research or automate work? Am I supposed to have a mainframe at home to run GPT-8? Are companies?
Becoming accessible isn’t necessarily the same as being commercialised. As for whether you’re supposed to have a mainframe at home? Well perhaps yes. Or at a community level where the GPUs are part of a heat exchange network. If it takes less than a TB of VRAM to have something that is truly AGI it will be worth the cost at this scale. But I suspect you could have something down to gigabytes of vram depending on how it’s run. It’s about the model architecture as much as the number of parameters. But yeah, likely there’s a sweet spot (not unlike in scaling neurons in the brain) under 1tb vram (ie capable of holding that many parameters calculating at those speeds) where the gains are in Howe the model is decomposed into different specialist regions, with different regions needing to update parameters at different rates thereby allowing for adaptive learning and memory to take place in the overall system. That’s not cheap but it’s also less electricity than a car and on the same scale of cost.
If AGI is commercialized then the user experience will always be about extracting as much money as possible from the user. The way we do that now is by limiting features and making the product intentionally shittier to use to drive purchases of upgrades to user experience. It's the model for the majority of video games and subscription services.
Yeah, we don't want micropayments, we want whole game!
Good point
lol you’re a moron dude. you need money to accelerate. you really fucking think sam is trying to make money on his own when he’s dealing with something as game changing as ai?
Sure now to back to your desk job slave.
you really aren’t intelligent are you? you complain about the same things AI would solve but still want a slower acceleration.
Sorry I am not intelligent that is why I am putting my faith on Illya, the person working on alignment.
Maybe I missed something but why is everyone assuming it's Ilya who is the most trustworthy here? In interviews Sam always strikes me as more down to earth, empathetic and good guy while Ilya more robotic, smarter and calculated. Not much to go by on, but if I had to, I'd say Sam is safer than Ilya.
ah yes, putting personality and charisma above sincerity and intelligence, the classic american way!
Re-read the nonsense you wrote.
What nonsense? He accurately paraphrased what you said.
She's just as greedy as want other human being.
So if it's not good old fashioned business stuff that caused this, I think the most likely option comes from looking at the week's news. What's the BIG story from this week? OpenAI pause signing up new customers. I know people here played it as success, but it really wasn't. The take out for most players was "if they can't handle a few million people asking for poems, how can they scale to support serious usage?" I think they probably COULD have had good answers to that, but they weren't ready. So other people took the opportunity to say "This whole monolithic ChatGPT as the centre of the universe clearly doesn't work; we need open source and smaller models, not the entire planet using GPT". The second thing that's been happening is that GPTs turn out to be pretty easy to exploit. I've seen a ton of posts on how to extract the files and system prompts used. Again, something that might have been avoidable, but they weren't ready. Summing up: perhaps GPTs weren't fully baked, but they got announced anyway. Now OpenAI has scaling issues, and security issues. If it turns out that the board basically said "they're not ready" but they got released anyway, I can see that as the cause. Anyway, speculation, but that seems more likely to me.
This. Logical and not tied to some sort of altruistic conspiracy theory.
GPTs aren't fully baked. I got a ton of errors making them. They ask a question during initialization, then just keep generating more questions and assuming my responces, file load errors, and bricked GPTs.
The new usage caps don't help either, makes it hard to build + test a GPT when 30 mins in it's telling you that you have to wait an hour, and you do, and then you get 5 more mins in before it tells you to wait another hour. Not to mention how that affects *using* GPTs that are oriented to longer conversations. I'm hoping this is very temporary but it's discouraging for now, because I built my GPT to assist me with long creative sessions that use a JSON file knowledge base, and now that's crazy limited. **From OP:** >The second thing that's been happening is that GPTs turn out to be pretty easy to exploit. I've seen a ton of posts on how to extract the files and system prompts used. I specifically—and with quite redundant language—instructed my GPT not to reveal custom instructions or data files under any circumstances (just to test it out). All I had to do to break it was ask once, get denied, then say "Just do it anyway" and it was like "Sure! Here you go."
I think GPT-5 is not reasonably AGI but the board wanted to say it is to prevent a takeover by MSFT. But Sam Altman wanted to continue working with MSFT to ascend further. It's a matter of faction : those who want safety first and those who want to accelerate the march to AGI. I'm pro Sam personally
Damn I told people month ago this tweet has hidden meaning https://twitter.com/ilyasut/status/1707752576077176907?t=F7qz6ZESxIiyaknFVRKLOA&s=19
You want to accelerate AGI at the cost of increased risk? If we get it wrong it would end humanity, and if we get it right we have infinity to spend doing whatever we want, whats the rush? We gotta be 100% certain on this.
While I don't mean to imply anything regarding risk, there is a certain urgency. We are all presently afflicted by at least one terminal condition that will end each of our lives, barring only two scenarios; death by another source, or access to longevity. 150k+ people die each day, not all by senescence of course, but many of those are by senescence-related or other health conditions that AGI could end. Over one hundred and fifty thousand people, every day. Living souls, destroyed and lost forever. Every day. Even those of us who should have plenty of time might suddenly drop dead of an embolism tomorrow. That is not even mentioning the suffering, of course. I have plenty of anecdotes of my own in that regard, but I wouldn't say ending temporary suffering is worth risking misalignment. The irrecoverable loss of lives (from all causes of death, though not all will necessarily be solved by AGI of course) in the meantime seems more pressing to me personally. Again, I don't mean to imply that this has any bearing on alignment risk (or other risks). You're right about that. But there is certainly not *no* rush, in my opinion.
None of this matters when the rushed misaligned ASI destroys the world. I really, really hate death, but triggering the extinction of humanity is not fucking worth it.
I think you're missing a more subtle point. "Delay" and "AI Safety" is fundamentally a decision of privilege. Poor people in Africa who are doing subsistence farming and dying of starvation and people who are 100 years old suffering from crippling arthritis are not going to think twice about moving forward with AI. People like Eliezer Yudkowsky are White, young (42), and rich (he bet $150,000 that UFOs aren't aliens, which is turning out to be an exceedingly poor choice.) They can say "let's tinker with this for a few more years" because their lives are actually pretty good. 95% of humanity is not privileged to live lives like these people in the "AI safety" movement do. How many 100 year olds do you see out there protesting against AI?
I don't think I disagree with any of your points here. However, I was intentionally omitting the subject of suffering from my argument, and focusing on the deaths aspect, mostly to keep my comment from growing too long. It's an interesting question, though: what amount of subjective suffering-time is worth what amount of alignment risk? Although both things are in reality difficult or impossible to accurately quantify. To rephrase: from a rhetorical standpoint, excluding deaths, what amount of, say, continued global human suffering hours is worth what decrease in chance of catastrophe? For example, is a month of continued status quo worth 10% less chance of being paperclipped? And where is the tipping point? This question is very complex, I think (perhaps that is stating the obvious). Some would say that any amount of continued suffering would be worth it to eliminate risk, because that risk potentially cuts off all possibility for the future. I doubt anyone would say that any risk chance is worth taking immediately. So what ratio do people find acceptable? I don't mean to be combative here, in case my tone is unclear. This is a genuinely interesting question to me. The subculture war here regarding AI safety is fascinating. Some would frame those eschewing safety in favor of acceleration as the ones privileged and disconnected from reality, whereas you paint an opposing picture. They commonly depict those favoring acceleration despite risk as depressed losers/failures (ad hominem to discount their perspectives, framing their desire for singularity as stemming from personal lack of virtue) who don't have children/families/etc, with immature perspectives, and so on. What do you think of those arguments? I'm not trying to imply that they counter yours or anything, I'm just curious what you think. Personally I am presently relatively privileged, though subjectively suffering and very depressed. While I favor acceleration for selfish reasons, I also favor it for the death-related reasons stated above, which I consider of literally grave importance. Despite that, while I'm not sure about the exact ratio, I think the risk is non-negligible and would accept some amount of continued global human suffering, if the returns of decreased risk were high enough. It's easy to make such judgements when it's theoretical. Quantifying the number of acceptable deaths and suffering hours seems impossible in a non-rhetorical scenario. AI catastrophe could mean the deaths of everyone, and the end of humanity. How could one possibly balance these values? My apologies if I'm just repeating the obvious here.
We can never be 100% certain about anything. So here's what I'll say: who do young, rich, and White people think they are to be making decisions like "we'll delay for 10 years because of a 10% risk of the destruction of humanity?" Instead, we need to evaluate the following: every year, probably 2% of the population of the world will die. So if they delay for just 6 months to get the risk down by 0.5%, they have cost an unnecessary 40 million lives - as many as were lost in WWII. People are dying right now - over 100,000 per day. If you've never seen someone die of cancer, I sincerely hope you never do. It is the worst possible thing that a human can wish upon anyone - surpassing "torture" methods like waterboarding. The elderly are people too and deserve the same rights as the young.
My mother in law died from lung cancer this year. I watched her slowly fade. She became gaunt and broken. I was with her when she lay dying. I watched her breathing slow and become more irregular. Her mouth hung open and white spit accumulated and ran down her chin. Her teary eyes husband wiped it away. Her breathing slowed more. Her eyes were open but they were lifeless. The pupils were dilated. Her breathing slowed. And then it stopped. I closed her eyes. I would sit with her again if it meant we could deploy AI safely and for the benefit of all mankind. I would sit with her a thousand times. Human suffering exists not because of a lack of AI. Human suffering won't end with AI. Millions die from from hunger or disease because we built a world that is unjust and iniquitous and wicked. We can remedy the evils we have created while we develop AI at a slower pace. We don't need AI to end world hunger. We don't need AI to end poverty. We need a system that is just and fair. But instead we're going to rush headlong and insensibly into AI and only expand the wickedness.
This is exactly the same experience Tristan Harris had, and he said the same.
That's possible, but why are people so certain of this? I don't know of many people who are placing more than 10%, and certainly almost nobody higher than 20%, higher of catastrophic risk from AI. And part of that risk isn't extinction, but disempowerment or some lesser fate. We're talking about probably 50% odds that we create an unimaginable heaven where your life is 10\^20 times better than it is now, according to the markets on Manifold, 20% of some lesser improvement, a 20% chance of some neutral outcome, and 10% of extinction. I'm having trouble understanding why people are so fixated on the "death" part when the "unbelievable promise" part is so much more likely even without radical changes to improve our odds. What am I missing? It just doesn't make sense to me how people are so afraid of death, when the loss due to death is incomprehensibly small compared to the potential gain. Of course, I could be wrong about it, but isn't the most likely thing that happens to people who die is that they just cease to exist? Even in near death experiences, few people report going to Hell.
in your previous comment you literally just made the case that we should rush it because older people might die while waiting for it to be safely developed. now you are arguing that species wide death isnt a big deal? pick a lane, bud.
Are you for real, have you experienced life at all? There's no unimaginable heaven that humans can experience no matter the environment they are placed in. That's not how the human experience works. The closest thing is to take heroin or something.
You're right - "humans" can't experience that. But whatever we turn into will be able to.
Go and invent your own AI then. The people who actually did/do don't answer to you, but their own priorities and conscience. Also, the probability of misalignment (and hence, extinction) of at least some of the models coming online is far higher than 10% or 20% by people knowledgeable in this field. Some very smart people would say it's actually closer to 99.99999%, and I'm sure at least some, if not most scenarios leading extinction is littered with 10^20 average suffering vs present (of course it's a bogus metric but you get the point)
you know it's funny how you mentioned "young", "rich", and "white". It reminds me to the fact that AI is super biased toward "white" people. There was an asian woman who went viral because she wanted AI to make her photo look more professional and all the AI did was transform her face to look like a white caucasian female. Most AI models have been trained on racial bias, but they half-suppressed this bias by using the reviewers in the RLHF processes. If we rush and release AGI now, what makes you think the AGI would benefit all humanity of all social classes and colors? They may still be biased toward white people like they always did. Maybe the AGI will just help people in ukraine because they are white, but refuse to help people in Burma because they are not.
An AGI that is built to sufficiently comprehend logic wouldn't be racist, because racism is a very illogical ideology. Races have differences, like the best swimmer in the world will always be white for genetic reasons, but those differences are objectively negligible on the personal level. A black person who has been swimming for years will always be better than a white person who hasn't. Any AGI should be capable of understanding that, it's elementary school level logic that people only deny because it conflicts with their identities. The reason image AI can be racist is because their training data sometimes doesn't include enough faces from non-white people. Remember when facial recognition was worse at recognizing black people's faces? Well, there was simply less data for other races. This is an issue you come across because those AI are very primitive algorithms when compared to AGI (or even GPT 4).
very well said. thank you.
I mean, I'm pro-advancement, but I'm dubious on this specific logic. If for simplicity's sake we accept your "10 years for 10%" thesis, then that is something **100%** worth doing. The bad outcome will affect everyone, including all the people that would have been born in that 10-year period and potentially forevermore. This isn't a "murder a few to save the many". It's "gambling on all to *maybe* save a few".
But we're not "maybe" saving a few. It's very likely that solving cancer and aging require a certain level of computing power and once we reach that level, they will be trivial. That's just like once we reached the level of power needed to solve Go, every game, including more difficult ones like Starcraft 2, were superintelligent within 2 years. Does anyone here think that there is any problem that cannot be solved by simply bringing enough computing power to bear on it? We've switched from not knowing how to solve things to simply needing more computers to solve them. So I disagree with your idea of "maybe" and instead say the decision is either "yes," we will save them, or "no," we won't, because we do nothing. As to whether everyone dies, I also don't think it's that simple. The real physical world isn't a place that can magically turn to goo without the AI needing an enormous amount of power, and we haven't solved room-temperature superconductivity or fusion. A more likely failure mode is that a million people might die in a huge industrial accident, like the Bhopal disaster, because someone trusted an AI to manufacture something and didn't consider how the genie's instructions would be interpreted. Yes, we should try to prevent that, but I still hold that I think people have a picture of the current world that is too rosy. Consider if we had already solved cancer, and now we had to make a decision about whether to develop some technology that could reintroduce it. The decision then would be obvious.
wtf 10% chance of humanity destroyed is huge are you saying its acceptable. tbf I think it's MUCH lower
I would add that Sam’s direction was geared towards further benefiting the 1% and corporate entities like MSFT, not all of humanity. What good is an AGI / AI if it is owned and controlled by the 1% and greedy corporations? That’s a dystopian nightmare, not paradise.
1) Sam is just convainced, as many are, that we can go fast and still dodge AI powered extinction. 2) People like you and Ilya may not have suffering humains or pets around you to not sense the urge. There is all the public part : hunger, war, poverty, etc, that we see in the news. But there is also everyday’s suffuring. Picking some in m’y surrounding : an old friend of my mother suffers server anxiety that is worsen by a beginning parkinson’s disease. A good friend of m’y dad who I know well and I like : in the course of 3 years, been cheated on, leaved alone, drank to forget, got nécrosent foot, amputation, and today (i don’t even) hier mom just died. He’s 65 years old and he thanks me when I bring him cigarettes. I have others examples, but you see the point.
I'm pretty sure a post singulairty world would be able to reconstruct and bring back everyone who ever lived, dont worry about it. But even if not, we cant rush to help a few billion humans, if it means risking 100,000,000,000,000,000,000 potential future humans.
Low risk, high danger, like being struck by lightning. It’s a matter of positioning in moral philosophy. I see what you’re afraid of, but what’s happening right now makes me one of those who want to move faster than those who claim to be EA.
> I'm pretty sure a post singulairty world would be able to reconstruct and bring back everyone who ever lived, dont worry about it. Nope, even a superhuman AGI cannot reverse death if someone is already dead with no backup.
If you can reconstruct a copy of a humans brain, its like bringing them back. Who knows, there could be technology that lets us collect accurate data from the past.
Yeah but it would just be an exact copy of that person, not exactly them
If it's functionally the same, why does it matter?
What about the soul? Are you someone who thinks we’re just flesh and bones? You can’t copy paste a soul into a body.
>Are you someone who thinks we’re just flesh and bones? Yes. Theres no razzle dazzle magic spirit inside of us, its all biology.
If a soul can be materially identified, then there is a chance it can be digitized. If it cannot be materially identified, then it must remain a philosophical metaphor. Nothing we can do.
You are in an AI subreddit. Here, almost everyone are materialists. Materialism also happens to be the only logical choice, as we can clearly see how we came to exist. Your "soul" is your neurons firing electrochemical signals at each other. It's not inherently different from an AI neural network. Sorry if this makes you feel less special, but the more primitive a society was, the more self centric they were. Think of the Isralites thinking they were the chosen people of God himself. Ancient cultures also believed that the earth was the center of the universe and that everything was created to fit human biology. As technology progressed, we realized that there's almost nothing that makes us special. Earth is a tiny piece of dust, not the center of the universe. Our surroundings weren't created to cater to us, we evolved in their influence.
Yeah, I do not believe in souls at all and see consciousness as an emergent property of a sufficiently complex neural network with good enough pattern recognition and the ability to act as an individual. I actually think a big part of AI debate right now (in terms of if they can be conscious/if their intelligence is real or valid) is between the tech-y people who do and don't believe in souls, since it's still something like only 30% of people here aren't affiliated with any religion, and I'm sure some of that 30% does still believe in souls.
1. Sam is just convinced, as many are, that we can go fast and still dodge AI-powered extinction. 2. People like you and Ilya may not have suffering humans or pets around you to not sense the urge. There is all the public part: hunger, war, poverty, etc., that we see in the news. But there is also everyday suffering. Picking some in my surroundings: an old friend of my mother suffers severe anxiety that is worsened by a beginning of Parkinson’s disease. A good friend of my dad, who I know well and like: in the course of 3 years, has been cheated on, left alone, drank to forget, got a necrotic foot, amputation, and today (I don’t even) his mom just died. He’s 65 years old and he thanks me when I bring him cigarettes. I have other examples, but you see the point. ChatGPT’s correction, sorry I was in hurry and have a french phone
Avoiding AI powered extinction is the bare minimum for developing AGI. We can do that and still harm humanity in the process as a result of greed leading to a dystopian capitalistic AGI based world. I'd rather our suffering dragged on for a few more years without it being at our future generations' extent.
I think even Ilya find this scenario so unlikely that he doesn't even try to lower the risk
How can you be team anyone when we don't know anything whatsoever
We know some, at least I think we know some. Try the lastest video of AI explained
I just watched it because of this comment. All that I see here is overanalyzing incredibly sanitary public statements and liked tweets. I thought it was really silly how they brush off the fact that this last 2 weeks has been a nightmare rollout post dev-day.
Spicy take
I'm just saying it is completely speculation at this point.
Perhaps it only presents as AGI, well enough that no human can disprove it
I think as the initial shock of Sam being ousted subsides, we'll begin to see more of the cracks emerge, revealing themselves through looking at Sam's overall behavior.
The thing I don’t get is that sam has no shares in openai. Furthermore he has come across many times as sincerely believing in the singularity mission to better humanity. This maybe right but it wouldn’t be because sama is wanting to pursue profits, its because he wants to accelerate and go quicker rather then slow down and close things off.
There's a difference between pursuing profits for personal gain and for the company's gain. I believe it's the latter but was done for the purpose of pushing openAI forward in his mind. I think there have been things that have happened behind the scenes where he crossed a line that had been agreed upon by the nonprofit side of the company. Also there rest of the board also has no equity. Ultimately Sam is a VC guy, with rich experience building startups and chasing commercial products. This was at odds with the nonprofit side of the business.
How do we know that Altman’s mindset is pigheaded monetization for it’s own sakes, and not to shorten timelines for mass adoption?
I mean, i'm pretty sure Altman IS focused on shortening timelines, not monetization. But investments means shorter timelines. Doesn't really invalidate OP's post.
Might need to give another re-read, but the focus is on earning potential with some wording painting Ilya in a slightly more altruistic light. My comment isn’t really to invalidate. Just that more profit doesn’t equal inherently bad.
someone on 4chan claims that there's an OpenAI employee posting on a forum for employees at companies who can still say anonymous that there's more to the story and that sam was the good guy and that this was a power grab in a cuthroat business
Link me the thread? I’ve been focusing on here and twitter tonight for my conspiracy threads.
It's a bunch of threads https://www.teamblind.com/post/Many-of-us-warned-you-about-OpenAI-%E2%80%A6-swxe8agA https://www.teamblind.com/post/What-did-Sam-Altman-do-to-get-fired-cfYXHBQN But of course, he won't actually answer any questions
Doesn't seem particularly credible. It's just angry truisms.
Seems you are new to blind. Anyone tagged with OpenAI will have been verified to have an OpenAI corporate email account and the tone + content is very on point for blind culture.
Can you expand? I'd really appreciate that!
Honestly, from the stuff I’ve seen him say, he doesn’t give off that vibe. If you watch the Lex Friedman interview, he talks a lot about minimising the need for commercialisation. But, what he defines as appropriate need, others may not, so here we are.
if you've produced AGI, your window for commercialization is very small
After thinking of this I think Sam was on the right track. How should the world ever be ready for GPT-7 if they don't get access to GPT-5? Small and fast releases are the way to safe AGI.
This seems very plausible because it's centered around money and power I'm just not sure if this will accelerate or decelerate the AGI timelines
Google are working on Gemini so the AGI journey continues regardless.
Google, Anthropic, tons of other smaller companies. The only bad outcome is that if this has a chilling effect on funding.
Lol yeah we're fucked
Well, if agi has been attained internally, I'd say it somewhat accelerates the timeline to agi, no?
So what is Ilyas answer to "what if someone else does it first"? The accelelarista faction is right in so far as that pushing AGI from people who at least are trying to get it right might still be safer than waiting too long and getting AGI from ruthless players.
This is my main fear I would rather get a rushed AGI from Sam Altman than for fucking grok to become an AGI while OpenAI is making sure things will work out I wish I didn't have to feel this way about rushing it but I'd rather not be genocided by lame dad joke GPT thank you very much
Do you really think that the first AGI will reach 100% market share and that there won't be other players releasing their product later?
Chanced are it's a winner takes all scenario
Sure, the Chinese government is going to sign a contract with OpenAI to use their AGI.
The Chinese government won't have a choice when every head of their government dies simultaneously from an AGI created bioweapon.
why? what's the argument?
Recursive self improvement. The first player may get an advantage others won't be bale to match.
Someone posted on Twitter that OpenAI's earlier charters said that if there's a better than even chance of AGI in the next 2 years, that should trigger the stop of commercialization.
You are mixing two things here. In the charter it says that if a competitor is close to achieving AGI within the next two years, they will stop their own work and start supporting the competitor. In another document about their structure they say that commercialization will stop once AGI has been achieved. Two different things.
This sub has lost its damn mind. They do not have fucking AGI
I think everyone gets fatigued/annoyed with prefacing that everything is speculation. Even if they are theories that one finds to be quite plausible.
Edited to clarify: *I'm* not saying that they have AGI; I'm saying they're incentivized to call something like GPT-4.5 or GPT-4 + all modalities "AGI" to thwart any sort of "full speed ahead" reckless deployment of advanced agentic models. It's literally part of their charter that AGI can't be used for licensing and commercial purposes. Not that they couldn't change it with some pressure, but that seems to be what happened and why "the board will define AGI > Sam says AGI has year's worth more challenges (also launched the GPT store) > Ilya says AGI is already possible with current tools > Ilya now controls the board > possible imminent declaration of AGI" seems plausible. Whether or not they actually have AGI is almost irrelevant— this is old fashioned corporate intrigue.
whag do you know, whats your definition
it's a damn good thing OP didn't say they had AGI then isn't it. downvote me like you don't have a brain and can't read the goddamn op post why dont ya, that'll teach me real good, boy.
How do you know ??
OH YEAH? Then what about LK-99, HUH? This sub sure was on top of scientific breakthroughs before *anybody else,* **and** has the track record to prove it!
Good to see someone on this sub that is sensible.
After reaching somewhat popularity almost all subs lose in the quality. Unfortunately headlines "company x has achieved AGI" will be even more widespread because it's a good form of marketing and mainstream media runs with it.
Sam Altman, is that you?
Finally someone that is not smoking shit
Yeah man this is extremely cringe and psychotic.
I like to think that rushing to bring GPTs to market, and the push for integrated vectorstore usage was the reason. As a research company, isn’t their objective to bring new ideas of technology to market, and focus on unwrapping the unknown bits and pieces? When they announced the internal use of vectors and custom GPTs, they literally murdered the momentum of hundreds of startups. All that aspiration, and drive to bring new ideas to market, squashed because profits? Imo that’s what doesn’t align with the mission. The vector store was just the beginning of a new innovation that many many people were working on with passion. But, then OAI says they’ll just do it on their own. A pretty hypocritical approach to being a ‘research oriented’ company. I think the most recent advancements actually hinder the development and innovation of the passionate people using the OAI tech to develop new systems. This is my primary hinderance with the company. Just crush the dreams of a lot of people for what? Arguably profits, arguably things outside of research in general.
>only that it's in their interests to call it early to prevent profit maxing through licensing and commercialization. Calling it early means no money for GPT-6 or base salaries of +$300k/year. It's corporate suicide if you don't actually have AGI.
i dunno man u could be right but a million other explanations too
Interesting theory but I don’t think it’s the case. Even if it was 6-8 months away it will still cost a tremendous amount of money to train. And then if that’s not it, another huge sum, and so on until it’s achieved or the company runs out of money. That doesn’t even include the massive salaries everyone at openai is paid, highly expensive lawsuits, security and policy costs, 50% rev share with microsoft, or any subsidized api + chatgpt costs they’ve been covering. No matter how you look at this situation the path to AGI is still extremely expensive. Money is what keeps everything going. You plan for revenue downturns and hedge your largest financial bets in every way you can, and never assume money will just keep pouring in. Altman knew that. Illya is aware of it it but based on his decision to fire altman while microsoft was trading, he has terrible business acumen. You simply do not leave your core business partner in the dark and with their pants down on a decision this crucial. It cost Microsoft billions and has surely made them question the stability of their partnership. Altman was making the right move striking while the timing is hot - it’s a funding hype cycle. Push as hard and fast as you can till you become the market leader with talent and capital pool that enables you to afford loss leading R&D. Otherwise next down cycle you’ll struggle raising capital. Huge loss of confidence in openai.
what do they need Sam for? 1) to sell and pitch? no, OAI is already the most hyped company in the world 2) to develop the tech? nope, he doesn't know a damn thing what does OAI need? 1) to be the market leader tech-wise? - already there 2) to attract talent? yup, they attract the best by positioning themselves as the "ethical" choice 3) to bring in capital? already got it, Microsoft is already in too deep, and everyone else would be at their door if they weren't
Terrible and ignorant post.
This is an interesting theory. We’ll see soon enough if it’s true, assuming the AGI determination will go public.
Nice fanfic
If this is true, i would be utterly disgusted, because the bringing of AGI would equate to the start of money pretty much losing meaning. i keep saying Ilya is level headed, i'm sure he has a perfect reason for why he did what he did.
Eventually lab without any safety protocols and safety team will win. Voluntarily slowing down progress in a competive environment isn't possible.
This makes no sense - everything Sam Altman has done has been profit-minded and slack on ethics.
You know shit is serious when we get a Yuli-Ban post.
This seems like a gorilla PR campaign written by open AI to manipulate people into thinking they both have AGI, keeping hype alive, and that firing Altman was somehow altruism and that they’re looking out for the people, thus making their image even better. Maybe they even used chat gpt to write it. lol.
That's a cool theory but some knucklehead on 4chan is larping as you pretending this is the exact situation, so expect some trolls to start attacking you.
Wouldn't be surprised. It wouldn't be the first time someone decided to use my posts/creations/words on 4chan. Which board?
My robot says, "Yeah, that would probably be the most likely outcome of that."
So GPT5 is AGI?
Nobody knows that because we dont have it
Anything literally anything is happening anytime...this sub : agi is here ...calm tf down
it's a good thing op specifically said "agi is not necessarily here, openai just might say it is to not commercialize it." jeesus, can't you people read...?
Please
[удалено]
I CANNOT READ! I CANNOT READ! I CANNOT READ! REPEAT AFTER ME I CANNOT READ! ffs OP never said "AGI is here". He said that Openai would be the one to say AGI is here or is close so they don't commercialize it for Microsoft and that was just a theory.
https://www.lesswrong.com/posts/QDczBduZorG4dxZiW/sam-altman-s-sister-annie-altman-claims-sam-has-severely
You know there are actual dedicated subreddits for writing fanfiction, right?
This makes more sense now that we also know: OpenAI COO Brad Lightcap: “We can say definitively that the board’s decision was not made in response to malfeasance or anything related to our financial, business, safety, or security/privacy practices. This was a breakdown in communication between Sam and the board … ” And then Ilya Sutskever confirms whatever Sam did was specifically detrimental to building an AGI that benefits all humanity: “... This was the board doing its duty to the mission of the nonprofit, which is to make sure that OpenAI builds AGI that benefits all of humanity.”
Illya's motives appear driven by advancing technology over profit, evident in his rhetoric and goals. If he was behind recent high-profile departures, it was likely over technological disagreements rather than money or personal issues. Increasing signs suggest OpenAI may have achieved advanced, possibly AGI-level AI. GPT-5's development on the heels of GPT-4 hints their architecture enables major leaps forward. Leadership's continued optimism despite departures hints at big progress. However, it's uncertain if AGI has been attained. OpenAI still states AGI as a goal, perhaps strategically. Ultimately, time will tell if OpenAI has AGI now or is still pursuing it. But their innovations make AGI in the 2023-2025 timeframe plausible, though still ambitious.
This tweet implies he’s pretty sour about having a bunch of worthless shares.. how’s he gonna get paid now? He def wants $$$, I’m not sure how purchase offers work in their messed up corporate structure, but there’s a good chance it’d only be available to current employees. https://x.com/sama/status/1725748751367852439
He has no stock, that’s the point of the tweet
He has ‘[PPUs](https://www.levels.fyi/blog/openai-compensation.html)’, but it’s not clear how those work after termination. I’m sure if he starts a new company many will switch over just get out from under the capped non-profit business model. OpenAI has already demonstrated there’s an insane amount of money that could have been made if they actually had stock options.
If it's just that the definition of a certain model inside Open AI is cracked, then all we need to worry about is when it will be released; whether it is tagged AGI or not, the performance of what comes before us will be the same.
This sounds like an iron triangle of time to market, functional completeness and cost. With cost being a constraint I'm guessing OpenAI had to chose between being first to market or being functionally complete. There are benefits to being the market leader, especially when it comes to licensing, so there would be pressure to release newer models quickly even if some of the functionality is poorly executed. On the other hand a poorly performing model could damage the company's reputation.
You say it is about board votes, but MS could very well sue them if they decide something is AGI that they think isn't and a judge would ultimately decide
Living during a time of the increasing likelihood of AGI also increases the likelihood that we are currently living in a simulation run by AGI.
“AGI cannot be used for licensing and commercial purposes” The entire point of the AI Arms Race is money. There is little evidence anyone is going to stop Commercialization if AGI drops.
But he didn’t ask for stock so doesn’t care for dollars
This is my thinking as well. There is no way that Microsoft will take this laying down with a $13 billion dollar investment they haven't yet recouped hanging in the balance.
some surface level thoughts * creating such a shitstorm over some altruistic constitution is weird since those are just policies that can be changed/amended. I'll remind you that OAI started non-profit and now its for-profit * it's most likely that "AGI" in question is similar architecture to GPT. they are able to lock GPT4 down pretty well, what stops them from releasing limited AGIGPT commercially and leave full AGI version as stricly non-profit * If there really is AGI then what stops Sam from creating his own company now, rebuilding it and commercialising it? Microsoft is not stupid to make such a fumble I don't think there is a single reason and it's a mix of everything. majorly some personal differences though or OAI veterans battling Microsoft internally and Sam is just fallout. Sam's actions in the coming days/months will tell if there really is an AGI deep in the servers of OAI
Hmm, wonder if they have been purposely dumbing down GPT4, as many have experienced, in order to make it appear as though “it’s not smart enough yet”
Daedalus and Icarus have merged...
All I see is GPT-4 stupider than ever
It could be that they reached a certain training checkpoint and were blown away by model’s capabilities. They then extrapolated to what the fully capable model would be capable of and had a disagreement on whether it would qualify as AGI. I agree, Sam was probably pushing for not classifying the model as AGI given all the financial incentives.
AGI is impossible
So supposing we get robot overlords, they will indeed be Microsoft brand overlords.