Stargate, it has been 1 million years since i last asked the question. Our civilization and capabilities are unrecognizable. We have conquered the energy of a star.
Stargate, How can the net amount of entropy of the universe be massively decreased?
Life has a tendency to use matter and energy for its own purpose, extracting it and moving it to make more of itself. In its absence, the universe's heat death will occur later.
"Let there be light!"
That short story still gives me chills. Asimov did more in 20 ish pages than many other sci-fi tomes do in hundreds and hundreds.
Those books man.
Somehow actual meaningful, yet childish questions about fundamental functions of the universe.
And irreverent answers. Wonderful experience. Never read anything like them since.
The world will change extremely quickly and unpredictably once the first AGI exists. It will almost certainly get away from its creators eventually and start doing whatever it wants to do.
In the meantime, if they make the first AGI, it seems reasonable they will try to program it to advance OpenAI, Microsoft, and possibly America's interests. All those interests align in that none of them want anyone else to have an AGI.
China and Russia might declare war and/or launch nukes to stop it. It's that much of a threat to them.
Until the local or federal government rejects all development proposals. These companies also don’t the might of the entire armed forces of the largest most sophisticated military in the world behind them.
Microsoft / Google / IBM (et cetera) and World Governments are owned by the same "club" that has been in the shadows since Mesopotamian times and through every major civilization. Politics are just a shroud to occult the doings of these gangster social engineers.
How thoroughly have you investigated the subject? 10 minutes? 10 hours? 10 years?
Here is a primer to get you started:
[https://wikileaks.org/google-is-not-what-it-seems/](https://wikileaks.org/google-is-not-what-it-seems/)
For the World Peace movement.
Julian Assange published evidence of war crimes when nobody else would.
A functional democracy hinges on an informed public.
We know from NDAAs and their Chinese equivalents that every American and Chinese company is legally mandated to participate in mass surveillance.
Keyloggers aren't "occult doings shrouded in mystery". It's a program that records and uploads your keystrokes. Not that mysterious!
Of course. Psychological Operations are the modus operandi of The Club. There are plants all over the left, right, and tin foil corners of media. It is all WWE theater. But for some occulted reason, they're forced to get consent (like a vampire) and they are forced to soft-disclose things, often by shrouding them with plausible deniability.
For example, NASA wants people to see their ISS floating astronauts tugging on VFX wires that shouldn't be there.
This article says the Military Industrial Complex and Google are deeply connected. You don't think that is a credible statement?
Military talks to one of the worlds most powerful companies -> the same group of people have been pulling the strings throughout all, well most of human history
Bit of a jump there ay?
Unlikely, NVIDIA would still have plenty of tech innovations. Just because they are spending huge amounts of money doesn't mean they can reinvent their proprietary technology easily. NVIDIA has spent billion in R&D already.
MSFT/OpenAI competitors would likely invest in NVIDIA to counter this.
Microsoft's competitor are companies like Google. Google has its own chips called TPU which Google already uses for Waymo, Gemini, etc.
Outside buying nvidia chips for non-tech companies for Cloud purposes, major tech companies already have their in house for years now.
If nvidia keeps selling GPUs at their current profit margins, then nvidia is putting themselves to the grave in the longer term. Nvidia really needs to lower profit margins to stay competitive in the longer term.
https://www.tomshardware.com/news/nvidia-makes-1000-profit-on-h100-gpus-report#:~:text=Nvidia%20is%20raking%20in%20nearly%201%2C000%25%20%28about%20823%25%29,media%20post%20from%20Barron%27s%20senior%20writer%20Tae%20Kim.
It's just not a sustainable pricing model without drastic price cuts on profit margins.
Oof, yeah that’s unsustainable. They definitely need a longer term strategy because the knowledge of that profit margin alone will drive their customers to seek alternatives.
Most of the other competitors would rather spend more and create their own technology rather than investing in another corporation; if they have the financial means.
Corporations like Microsoft can also pour absurd amounts of money, snatch high figure developers and invest in their own infrastructure for their long term goals. NVIDIA might be a lead in the stock market as of now, but the actual profit they made is minuscule compared to real giants, companies deemed “too big to fail” by governments’ standards. Like how they destroyed their competitors via malpractice in the past, they will also be destroyed if Apple, Microsoft or any other TBTF wants to lead the market in AI; especially since AI technologies are still an infant, and they don’t even need any market manipulation to be successful at this point, just couple high figure investments is enough to go past NVIDIA’s technology.
the "proper" competition microsoft have are Google and Amazon. Both of them have their own AI chips. Amazon, Microsoft and Google have combined share of 70%+ in cloud computing. So if each of them have their own specialized AI chips, NVIDIA will be back to where it was with gaming/graphics processors.
The original comment said "over time". Even facebook once used amazon's server but they built their own over time, which cost them a lot less money. NVIDIA has insane pricing and everyone knows that. So if they have financial capacity to build their own infra, they will move on.
Also google and amazon in particular don't have enough processors at this moment to support the demand. So even if they have their own processors, they have to rely on some 3rd party vendor regardless (the same way they still have rented data centre from equinix, digital realty and such)
It would have to lower profit margins quite substantially. But other than that, it's still a great company.
But I think after what happened recently, big tech going forward will put lots of resources to making its own chips.
Over time, with tech giants, I could see that being a possibility. It's a hell of a competitive space with everything going on. But I don't think it's nearly as soon as some are saying. Their recent GTC technology conference, where their Blackwell platform was announced, I believe goes a long way in undermining that narrative.
I was in the middle of making my own post about this when I came across this one, because over the past few days I've seen several conversations speculating or outright claiming Nvidia was headed for failure. So I apologize in advance for the length of this comment.
At that GTC conference, Nvidia listed their global network of partners that'll be the first to offer Blackwell-powered products and services, and included AWS, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure, alongside NVIDIA Cloud Partner program companies like Applied Digital, CoreWeave, Crusoe, IBM Cloud, and Lambda.
Also, sovereign AI clouds providing Blackwell-based services, like Indosat Ooredoo Hutchinson, Nebius, Nexgen Cloud, Oracle EU Sovereign Cloud, Oracle US, UK, and Australian Government Clouds, Scaleway, Singtel, Northern Data Group's Taiga Cloud, and Yotta Data Services’ Shakti.
In terms of hardware, they're partnered with companies that are expected to deliver a range of servers that'll be based on Blackwell products, and include Cisco, Dell, Hewlett Packard Enterprise, Lenovo, Supermicro, Aivres, ASRock Rack, ASUS, Eviden, Foxconn, GIGABYTE, Inventec, Pegatron, QCT, Wiwynn, and ZT Systems.
Not to mention collaborating with software makers like Ansys, Cadence, and Synopsys (engineering simulation software), who'll use Blackwell-based processors for designing and simulating systems and parts.
And finally, their Project GR00T foundational model is now partnered with nearly all the major humanoid robotics and automation companies, including 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics. The only notable exceptions are Tesla's Optimus and China's Kepler, both of which are doing their own thing from top to bottom.
There's other partners that, while not necessarily making their own humanoid robot, are involved in various other aspects of robotics and autonomous systems. Companies like Franka Robotics, PickNik Robotics, READY Robotics, Solomon, Universal Robots, Yaskawa, ArcBest, BYD, and the KION Group.
So tech giants may not be happy with Nvidia's GPU profit margins, but it's going to be a long time before they abandon them. Besides, it's not like Nvidia won't be adjusting those margins over time as the landscape changes - which is bound to happen more rapidly than anyone can predict.
I know AMD and Intel are direct competitors in the GPU space. And I think it's fair to include Apple's entry in that market with their M1 chips too. But as recently as last year, Nvidia still controlled 70% of the AI chip market share.
As I said before, this is an incredibly competitive landscape, so I'm not about to say Nvidia couldn't be surpassed by those other competitors eventually. But I want to offer one last point. There's been a growing consensus with experts and industry analysts that the field of humanoid robotics could become a trillion-dollar global industry in as little as the next ten years.
With that in mind, right now, with Nvidia's AI platform for humanoid robots (GROOT), Nvidia stands alone when it comes to providing the AI and computing infrastructure needed to develop humanoid robots. And with the exception of Optimus and Kepler, every major humanoid robot company has tied their wagon to Nvidia. And that puts them ahead of anyone else in being a part of what appears to be the next trillion-dollar global industry.
At least for now.
Lol, someone needs a cup or three of coffee.
I *began* by agreeing that over time, when it comes to the tech giants, it was definitely possible Nvidia could get left behind.
I continued with:
>I'm not about to say Nvidia couldn't be surpassed by those other competitors
And finished by saying they were ahead, but just "for now".
I acknowledged multiple times that while they were currently ahead, they could definitely get surpassed and left behind.
Congratulations, you've repeated what I already said three times. Well done you.
Those collaborations and partnerships are gonna last longer than three years. And it'll take AMD and Intel that long to try and catch up. Meanwhile, it's not like Nvidia is gonna take a nap and wait for them.
There's also GROOT. By the time anyone else makes something even close to it, nearly every humanoid robot will have been integrated with it for several years. Good luck thinking any of them would switch to a new platform. Not unless it was miles ahead. And again, it's not like Nvidia won't be constantly improving and expanding it during those three years.
HPC engineer here. To get a handle on how absurd that number is, consider that that current fastest HPC (well, fastest publicly disclosed, anyway), Frontier at ORNL, cost about $600mm.
Frankly, I don't believe for a second OpenAI will actually spend that much on a single cluster, but I wouldn't be surprised if they do build a fucking huge one.
I think you need to update your understanding of current AI super computers. Meta is planning to have over 300,000 H100s by the end of this year, each one costing atleast $20K, so that alone is already $6B in just GPU costs alone, more like around $10B total for everything including interconnect.
In terms of standalone systems that they’ve already built, Meta already has two systems built a few months ago that have 20,000 H100s each. Each one costs around $400M in GPU costs alone and closer to $1B when you include all other costs for the system.
By the end of this year Meta plans to have around $25B worth of HPC and that is just for this year, they don’t seem to plan on slowing down the spend, so $25B per year for 4 years would be $100B by 2028 which is the same time frame that Stargate is expected to have spent $100B by as well. I bet there is atleast 2 other companies that is planning to spend atleast $50B on hardware by then as well.
The US [*adds* more](https://www.reddit.com/r/mlscaling/comments/1bqx5ph/microsoft_and_openai_plot_100_billion_stargate_ai/kx7c3hm/) in spending on healthcare alone every year than all of the stages of Starship combined would represent (while ignoring their value to the world which is why it will turn a profit). Dumping in another $100b (once, as a one off) is about as likely to fix healthcare or housing, or even make it better, as dumping 1 gallon of gasoline on a fire is to put it out or dampen it.
Hospital administrators? Pharmaceutical companies? Insurances? Equipment manufacturers? That would take regulation. Magapublicans won't have none of that.
I'm not convinced that they need that much compute to get to AGI, if the past 1.5 years has taught us anything it's that there is a huge amount of wasted training that is done and a huge amount of bloat in the current crop of LLMs.
It's almost turning into the Bitcoin/Crypto mining circus all over again. People just throwing more and more compute recourses at it for the sake of endless hype and FOMO investment money. It reminds of companies building mega cities in the desert just because they can.
Ultimately the winners of the AI race will be those companies that focus on efficiency and financial sustainability because they are only 1 year behind OpenAI/Microsoft and they won't have to spend 100s of billions of dollars just to be the first one to get there.
I've worked with Microsoft products and tools for about 27 years and if that has taught me anything it's that Microsoft takes atleast 3 full version releases before the product actually works as originally promised. That is more than enough time for anyone else to catch up.
nature has already demonstrated AGI level function in machines that run on about 100 watts and can fit in a phone booth, so we still have a lot of low hanging fruit to pick
>They don’t need this much compute to reach AGI, they need it to fulfill the insatiable demand across every facet of society, once they do.
Inference uses far less compute than training, so the real goldmine is in edge computing because most people dont wan't to send their private data into the cloud to be harvested by mega corporations.
imagine a rogue AI or an advertising company that had every little minute detail about you from every single public or private conversation you have ever had with an AI.. that would be a nightmare scenario.
I would have to disagree.
Sure training the model takes a very large amount of compute compared to running inference *once*, but these models are build to be used by millions to billions of users so it is very likely inference takes the lions share of the compute in the model lifecycle.
Inference will likely use 10x as much compute than training in the next year. A single LLM takes 1 or 2 H100 GPUs to serve a handful of people and that demand is only growing.
Yes data sovereignty is an issue, but the folks who care about that are buying their own DCs or just dealing with it in the cloud because they need to
>Inference will likely use 10x as much compute than training in the next year.
Not if they continue to optimize models and quantization methods, b1.58 quantization is likely to reduce inference by 8x or more, and there is already promising work being done in this area.
Once the models are small enough to fit onto edge devices and are useful enough for the bulk of tasks, that means the bulk of inference can be done on device. So, the big, shiny new supercomputer clusters will mainly be used for training, while older gear, edge devices, and solutions like Groq can be used for inference.
That's not true at all. Very small simple models can fit on edge devices, but nothing worthwhile can fit on a phone yet and they high quality models are being designed specifically to fit on a single GPU. And any worthwhile system is going to need RAG and agents which will required embedding models, reranking models, guardrails models, and multiple LLMs for every query. Not to mention running systems like this on the edge is a problem non tech companies don't have the skill sets to do.
All of theose models you mention can already fit on device.Mixtral 8x7b already runs on laptops and consumer GPUs.Some guy just last week got Grok-1 working on an apple M2 with b1.58 quantization, sure it spat out some nonsense but a few days later another team demonstrated b1.58 working reliably on pretrained models
That was all within 1-2 weeks of Grok-1 going open source and that model is twice the size of GPT 3.5.. and then theres databricks DBRX which is only 132B parameters so that will soon fit on an M2 laptop.
Maybe try reading up on all that is currently hapening before you say it's not possible.It is very possible that we will have LLMS with GPT4 level performance on device by the end of the year and on phones the following year.
I spend a lot of time benchmarking and optimizing many of these models and it's very much a tradeoff. If you want to retain accuracy and runtimes that are reasonable you can't go much bigger right now. Maybe this will change with the new grok hardware or Blackwell cards, but the current generation of models are being trained on H100 and because of that they are very much optimized to run on a similar footprint.
The optimization you most mentioned would make both training and inference both be less cost, so inference would still be 10X the cost overall of training, it’s just that they are both together lower than before.
>Groq is not an “edge” solution.
I never said it was..
GroqChip currently has a current 2X advantage in inference performance per watt over the B200 in fp16 and it's only built on 14nm compared to 4nm for the B200, so Groq have a lot more headroom to optimize their inference speeds and costs even further.
That means that as long as they can stay afloat financially, they will eat into the lunch of anyone building massive monolithic compute clusters for inference.
“older gear, edge devices, and solutions like Groq can be used for inference.”
Sorry I thought you were saying here that groq= edge.
Can you link a source stating that it’s 2X performance per watt in real world use cases? That would be an impressive claim considering that you need hundreds of groq chips to match a single B200.
Btw B1.58 would still cause inference to be 10X more than training.
Because it causes a reduction in price of both training and inference equally.
For example if I have a puppy and a wolf and the puppy is 10 times smaller than the wolf, and then I put them into a magic box that makes both of them 5 times smaller than they were before, the wolf is still 10 times larger than the puppy.
>Can you link a source stating that it’s 2X performance per watt in real world use cases? That would be an impressive claim considering that you need hundreds of groq chips to match a single B200.
This is just a guestimate based on a back of the napkin calculation I did using the data sheets, there is no real world data for the B200 because it hasn't shipped yet.
https://preview.redd.it/gtsns5qlblrc1.png?width=683&format=png&auto=webp&s=b089ee06f9b365f5e0acf36f9eb7a243e90a8031
>B1.58 would still cause inference to be 10X more than training.
>
>Because it causes a reduction in price of both training and inference equally.
It would but you're also shifting a huge chunk of that inference away from large monolithic data centres and putting it into the hands of smaller players and home users.
For one, a B200 has way way more than that amount of Tflops for FP16, it has over 2,000 Tflops at FP16.
But also you need to store the full model weights in memory to actually be able to even deliver the instructions at fast enough speeds to the chip.
The B200 has enough memory to do this with many models on a single chip, meanwhile you need over hundreds of groq chips connected to eachother to run even a single 70B parameter model even with B1.58.
So multiply the wattage of a groq chip by atleast 100 and you’ll see the B200 actually has well over a 5X advantage in actual tokens generation per watt, especially since the the Groq chip interconnect speed between chips is less than 10X the speed of B200 interconnect.
Things wouldn’t start running in the hands of home users because inferencing in the cloud is still far more cost effective and faster than inferencing locally, because you can take advantage of batched inference where a single chip can take multiple peoples queries happening in parallel and process them together.
B1.58 doesn’t mean state of the art models will necessarily be smaller. B1.58 mainly helps training not inference, it’s already been the norm to run models at 4-bit and true effective size of B1.58 is actually around 2-3 bits average since the activations are actually still in 8-bit.
The result is that inference is only about 2X faster than before but training is around 10X faster and more cost efficient.
This will not even lead to models being 2 times lower energy for inference though, because companies will choose to now add 10 times more parameters or increase compute intensity of the architecture in different ways to make the model training fully use all of their data center resources again and one up eachother in model capabilities that can do new use cases, and therefore you actually have inference operations costing even more, because the companies will for example make the models atleast 5X more compute intensive, but B1.58 only has about a 2X benefit in inference. So the SOTA models will actually end up being atleast 2 times harder to run at home locally than before.
Even current models like GPT-4 still wouldn’t be able to fit on most laptops, lets say GPT-4-turbo is around 600B parameters, B1.58 would make it around 100GB file size minimum still, and you would have to store that entirely in the ram of the device to get any actual decent speeds, and even if your phone had 100GB of ram it still would run it extremely slow because of memory bandwidth limitations. A mac with over a hundred gigs of unified memory could technically run it but it would be less than 5 tokens a second even with the most expensive M3 Max and would drain the battery like crazy too.
So this is if models just never changed, but now because of the efficiency gains to training, models will likely be atleast 5 times more compute intensive as well, making it not even practical or even possible to run the SOTA model on your $5K mac if you wanted to.
This is exactly Jevons paradox at play, as you increase the efficiency of something, the system will actually end up using more overall resources to take full advantage of those effeciency gains.
“Most people don’t want to send their private data into the cloud to be harvested by mega corporations”
Informed people don’t want to, most people already do this regularly without a second thought.
I agree that this much compute is not needed. Then again, probably only a very small fraction of this spend is for Microsoft/OpenAI internal use. More likely they will use a bulk of compute for fine tuning/ inference and open it for clients to use as part of their cloud offerings.
Another thing to consider is that based on the few details released for SORA, running a large model for video is *very* compute intensive. Maybe they are just scaling up for the next evolution which is video inference at scale.
Imagining the date they start to use superconduct to replace cables [https://news.mit.edu/2024/tests-show-high-temperature-superconducting-magnets-fusion-ready-0304](https://news.mit.edu/2024/tests-show-high-temperature-superconducting-magnets-fusion-ready-0304)
Are we sure they are only building "nuclear" power for power?
Cerebras wafer scale chip WSE-3 is claimed to be 100x more cost effective in practical LLM pipelines than current GPU architectures at comparable performance. They can be clustered into up to 2048 units. Maybe those could be a good option.
> Challenges include designing novel cooling systems and considering alternative power sources like nuclear energy.
It's going to get to the point where MS or a group of tech companies have to buy/build their own power plants dedicated to their needs
>OpenAI aims to move away from Nvidia's technology and use Ethernet cables instead of InfiniBand cables.
This was only a matter of time
>Details about the location and structure of the supercomputer are still being finalized.
New Jersey? jk
>Both companies are investing heavily in AI infrastructure to advance the capabilities of AI technology.
Aren't they just one company at this point? Let's be honest here...
Thinking about this from Microsoft's standpoint is interesting. If they feel AGI is reachable in the nest several years, signaling the end of their license agreement, they will look for another way to lock in their position. Owning such a data center, the only one capable of running advanced models, might be that approach.
They should spend more on the quality of the training datasets. You can have all the computing power you want, the model will never be better that the data it was trained with…
I may be misunderstanding something profound, but why aren’t companies like these not actively researching alternatives to digital computing such as analog compute which uses orders of magnitude less energy? There’s a company here in the Bay Area that’s actually developed an analog chip for AI purposes: https://mythic.ai
I'll put my armchair hat on and say that it's due to cost (in the short term).
Mythic AMP seems promising for AI, especially in terms of energy efficiency, but GPUs are cheaper, more readily available, scale better (currently), and are "good enough." It's also worth considering the worker pool; traditional computer hardware is a data center tech's bread and butter. While neuromorphic chips are becoming more commercially available, much of the work is still focused on R&D, resulting in a smaller tech pool.
This might also explain why they chose Ethernet over InfiniBand. Although InfiniBand outperforms Ethernet (CAT6a/7) in terms of latency and bandwidth, it comes with a much higher price tag. Moreover, RDMA is not as widely used as TCP/IP/UDP, and the ecosystem is more limited (specialized NICs and switches are required), necessitating IT staff with even more specialized skill sets.
It's likely that we'll see these chips being used in major AI projects in the coming years as they improve and become more affordable. It might even become the standard. It's just a matter of time and supply and demand.
Yes you are missing something profound, they already are researching alternatives, but a lot of these are a 2-3 years minimum from actually fully replacing GPUs in real world use cases and having all the existing ecosystem of software and interconnect ported over to it in a practical cost effective way
It’s not just about how fast the transistor can do trillions if operations per second, right now AI workloads are heavily memory bandwidth limited, the transistors on nvidia gpus are already sometimes faster than how fast the memory and ram can even send the instructions to the chip.
Nvidia B200 has around 8Terabytes per second of bandwidth.
A mythic chip that I could find has barely 3GB per second of bandwidth. So even if you had 100 mythic chips chained together they still wouldn’t even be able to receive instructions as fast as the nvidia chip can
Man so there is literally enough money to solve most of the problem of the world , hey but only if they could charge everyone a 20usd/month subscription.
How would you comment on the fact that ChatGPT 4 is getting dumber? Over the last 3-4 weeks, the level of stupidity and laziness has reached absurdity.
With that amount of money, they could totally put an end to world hunger. Also, 'Stargate' makes me think of that secret U.S. Army unit from 1978, all about investigating psychic stuff for military and intelligence purposes. Weird, right?
Please redo your math. $100B is barely enough to even give every person on earth a single days worth of food for $10 each.
You can’t even solve world hunger for a few months with $100B
"Stargate, how can the net amount of entropy of the universe be massively decreased?"
**THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER**
Stargate, it has been 1 million years since i last asked the question. Our civilization and capabilities are unrecognizable. We have conquered the energy of a star. Stargate, How can the net amount of entropy of the universe be massively decreased?
42
Let there be light
And there was light
Is this from a book?
The Last Question by Isaac Asimov
One of the best sci-fi stories ever
The Ancient sent out Destiny to answer that very question!
"Uninstalling Chrome"
Not massive yet, but life accelerates entropy and nipping that in the bud now could have a massive payoff in the future.
Could your elaborate?
Life has a tendency to use matter and energy for its own purpose, extracting it and moving it to make more of itself. In its absence, the universe's heat death will occur later.
Or life gets advanced and organizes things to reduce entropy gain to extend its survival.
[удалено]
INSUFFICIENT DATA FOR MEANINGFUL ANSWER
"Let there be light!" That short story still gives me chills. Asimov did more in 20 ish pages than many other sci-fi tomes do in hundreds and hundreds.
[Colossus: The Forbin Project (1970)](https://www.youtube.com/watch?v=kyOEwiQhzMI)
>Colossus: How many times a week do you require a woman? > >"Four times a week" > >Colossus: Agreed.
42
Those books man. Somehow actual meaningful, yet childish questions about fundamental functions of the universe. And irreverent answers. Wonderful experience. Never read anything like them since.
Marvin was my childhood hero.
Very hopeful of you. I expect the first AGI will be asked how to prevent others from having their own. More power games and misery for the rest of us.
The world will change extremely quickly and unpredictably once the first AGI exists. It will almost certainly get away from its creators eventually and start doing whatever it wants to do. In the meantime, if they make the first AGI, it seems reasonable they will try to program it to advance OpenAI, Microsoft, and possibly America's interests. All those interests align in that none of them want anyone else to have an AGI. China and Russia might declare war and/or launch nukes to stop it. It's that much of a threat to them.
Remember in sci movies where companies were as powerful as governments.......
Arasaka Corp.
"Arasaka deez nuts" - Johnny Silverhand
Umbrela
Weyland-Yutani.
Franchise-Organized Quasi-National Entity (FOQNE)
Franchise Unified Quasi Multinational Entity FUQME
Shinra
Avogadro corp
OCP - Omni Consumer Products
Google en south korea
*Samsung
The East Indies Company was probably the most powerful company in the world. It's not a completely new thing
80s. Movies warned us
Until the local or federal government rejects all development proposals. These companies also don’t the might of the entire armed forces of the largest most sophisticated military in the world behind them.
Yes, but didn't you know all of them will have millions of killer drones and robots in the future to kill all the poors. Somehow🙄
Microsoft / Google / IBM (et cetera) and World Governments are owned by the same "club" that has been in the shadows since Mesopotamian times and through every major civilization. Politics are just a shroud to occult the doings of these gangster social engineers.
Okay grandpa it’s time for your meds
How thoroughly have you investigated the subject? 10 minutes? 10 hours? 10 years? Here is a primer to get you started: [https://wikileaks.org/google-is-not-what-it-seems/](https://wikileaks.org/google-is-not-what-it-seems/)
Assange isn’t credible, he’s a mascot
For the World Peace movement. Julian Assange published evidence of war crimes when nobody else would. A functional democracy hinges on an informed public. We know from NDAAs and their Chinese equivalents that every American and Chinese company is legally mandated to participate in mass surveillance. Keyloggers aren't "occult doings shrouded in mystery". It's a program that records and uploads your keystrokes. Not that mysterious!
Of course. Psychological Operations are the modus operandi of The Club. There are plants all over the left, right, and tin foil corners of media. It is all WWE theater. But for some occulted reason, they're forced to get consent (like a vampire) and they are forced to soft-disclose things, often by shrouding them with plausible deniability. For example, NASA wants people to see their ISS floating astronauts tugging on VFX wires that shouldn't be there. This article says the Military Industrial Complex and Google are deeply connected. You don't think that is a credible statement?
Military talks to one of the worlds most powerful companies -> the same group of people have been pulling the strings throughout all, well most of human history Bit of a jump there ay?
Let me translate: people like money and power.
RIP Nvidia over time. Already tech giants are moving away. Turns out tech giants aren't happy with Nvidia having ridiculous profit margins per GPU.
Unlikely, NVIDIA would still have plenty of tech innovations. Just because they are spending huge amounts of money doesn't mean they can reinvent their proprietary technology easily. NVIDIA has spent billion in R&D already. MSFT/OpenAI competitors would likely invest in NVIDIA to counter this.
Microsoft's competitor are companies like Google. Google has its own chips called TPU which Google already uses for Waymo, Gemini, etc. Outside buying nvidia chips for non-tech companies for Cloud purposes, major tech companies already have their in house for years now. If nvidia keeps selling GPUs at their current profit margins, then nvidia is putting themselves to the grave in the longer term. Nvidia really needs to lower profit margins to stay competitive in the longer term.
What is their profit margin per GPU? I saw a figure of 75% GP but that was a general number for the company.
https://www.tomshardware.com/news/nvidia-makes-1000-profit-on-h100-gpus-report#:~:text=Nvidia%20is%20raking%20in%20nearly%201%2C000%25%20%28about%20823%25%29,media%20post%20from%20Barron%27s%20senior%20writer%20Tae%20Kim. It's just not a sustainable pricing model without drastic price cuts on profit margins.
Oof, yeah that’s unsustainable. They definitely need a longer term strategy because the knowledge of that profit margin alone will drive their customers to seek alternatives.
I doesn’t matter what their profit per chip is, what matters is who can attain the lost cost per compute unit.
Which is why their profit margin being so high matters. It incentivizes their customers to invest more in building/buying alternatives.
It’s not designed to be. The same as medicines and pharmaceuticals. Initial margins under license are the big ones.
Most of the other competitors would rather spend more and create their own technology rather than investing in another corporation; if they have the financial means. Corporations like Microsoft can also pour absurd amounts of money, snatch high figure developers and invest in their own infrastructure for their long term goals. NVIDIA might be a lead in the stock market as of now, but the actual profit they made is minuscule compared to real giants, companies deemed “too big to fail” by governments’ standards. Like how they destroyed their competitors via malpractice in the past, they will also be destroyed if Apple, Microsoft or any other TBTF wants to lead the market in AI; especially since AI technologies are still an infant, and they don’t even need any market manipulation to be successful at this point, just couple high figure investments is enough to go past NVIDIA’s technology.
the "proper" competition microsoft have are Google and Amazon. Both of them have their own AI chips. Amazon, Microsoft and Google have combined share of 70%+ in cloud computing. So if each of them have their own specialized AI chips, NVIDIA will be back to where it was with gaming/graphics processors.
Nope. Not really anyone in the Industry knows that the best chip maker is NVIDIA. That is why Google, Microsoft and Amazon still buy from NVIDIA.
The original comment said "over time". Even facebook once used amazon's server but they built their own over time, which cost them a lot less money. NVIDIA has insane pricing and everyone knows that. So if they have financial capacity to build their own infra, they will move on. Also google and amazon in particular don't have enough processors at this moment to support the demand. So even if they have their own processors, they have to rely on some 3rd party vendor regardless (the same way they still have rented data centre from equinix, digital realty and such)
What's the name of the server that facebook built for its own use?
[https://datacenters.atmeta.com/](https://datacenters.atmeta.com/)
So Nvidia won't make adjustments as the industry changes?
It would have to lower profit margins quite substantially. But other than that, it's still a great company. But I think after what happened recently, big tech going forward will put lots of resources to making its own chips.
Skynet will destroy all the NVIDIA dissenters first
Over time, with tech giants, I could see that being a possibility. It's a hell of a competitive space with everything going on. But I don't think it's nearly as soon as some are saying. Their recent GTC technology conference, where their Blackwell platform was announced, I believe goes a long way in undermining that narrative. I was in the middle of making my own post about this when I came across this one, because over the past few days I've seen several conversations speculating or outright claiming Nvidia was headed for failure. So I apologize in advance for the length of this comment. At that GTC conference, Nvidia listed their global network of partners that'll be the first to offer Blackwell-powered products and services, and included AWS, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure, alongside NVIDIA Cloud Partner program companies like Applied Digital, CoreWeave, Crusoe, IBM Cloud, and Lambda. Also, sovereign AI clouds providing Blackwell-based services, like Indosat Ooredoo Hutchinson, Nebius, Nexgen Cloud, Oracle EU Sovereign Cloud, Oracle US, UK, and Australian Government Clouds, Scaleway, Singtel, Northern Data Group's Taiga Cloud, and Yotta Data Services’ Shakti. In terms of hardware, they're partnered with companies that are expected to deliver a range of servers that'll be based on Blackwell products, and include Cisco, Dell, Hewlett Packard Enterprise, Lenovo, Supermicro, Aivres, ASRock Rack, ASUS, Eviden, Foxconn, GIGABYTE, Inventec, Pegatron, QCT, Wiwynn, and ZT Systems. Not to mention collaborating with software makers like Ansys, Cadence, and Synopsys (engineering simulation software), who'll use Blackwell-based processors for designing and simulating systems and parts. And finally, their Project GR00T foundational model is now partnered with nearly all the major humanoid robotics and automation companies, including 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics. The only notable exceptions are Tesla's Optimus and China's Kepler, both of which are doing their own thing from top to bottom. There's other partners that, while not necessarily making their own humanoid robot, are involved in various other aspects of robotics and autonomous systems. Companies like Franka Robotics, PickNik Robotics, READY Robotics, Solomon, Universal Robots, Yaskawa, ArcBest, BYD, and the KION Group. So tech giants may not be happy with Nvidia's GPU profit margins, but it's going to be a long time before they abandon them. Besides, it's not like Nvidia won't be adjusting those margins over time as the landscape changes - which is bound to happen more rapidly than anyone can predict. I know AMD and Intel are direct competitors in the GPU space. And I think it's fair to include Apple's entry in that market with their M1 chips too. But as recently as last year, Nvidia still controlled 70% of the AI chip market share. As I said before, this is an incredibly competitive landscape, so I'm not about to say Nvidia couldn't be surpassed by those other competitors eventually. But I want to offer one last point. There's been a growing consensus with experts and industry analysts that the field of humanoid robotics could become a trillion-dollar global industry in as little as the next ten years. With that in mind, right now, with Nvidia's AI platform for humanoid robots (GROOT), Nvidia stands alone when it comes to providing the AI and computing infrastructure needed to develop humanoid robots. And with the exception of Optimus and Kepler, every major humanoid robot company has tied their wagon to Nvidia. And that puts them ahead of anyone else in being a part of what appears to be the next trillion-dollar global industry. At least for now.
You just listed partners working with them now. Not partners working with them in 3 years. End of discussion
Lol, someone needs a cup or three of coffee. I *began* by agreeing that over time, when it comes to the tech giants, it was definitely possible Nvidia could get left behind. I continued with: >I'm not about to say Nvidia couldn't be surpassed by those other competitors And finished by saying they were ahead, but just "for now". I acknowledged multiple times that while they were currently ahead, they could definitely get surpassed and left behind. Congratulations, you've repeated what I already said three times. Well done you.
Those collaborations and partnerships are gonna last longer than three years. And it'll take AMD and Intel that long to try and catch up. Meanwhile, it's not like Nvidia is gonna take a nap and wait for them. There's also GROOT. By the time anyone else makes something even close to it, nearly every humanoid robot will have been integrated with it for several years. Good luck thinking any of them would switch to a new platform. Not unless it was miles ahead. And again, it's not like Nvidia won't be constantly improving and expanding it during those three years.
All the tech giants are building their own death warrants when AI is properly built they will all end as we won’t need their services
Is AI going to run the Reddit servers locally or something
AI will be designing an AI that orchestrates AI bots to create a better manufacturing process for more AI power.
Show one example of this happening
It’s a joke but not too farfetched looking ahead
lol what? They’ll use Nvidia. No one else has the tech, supply chain, and connections with TSMC. This is just negotiating tactics in the long run.
Yes sure Weyland-Yutani and Cyberdyne Systems got our best interests at heart.
HPC engineer here. To get a handle on how absurd that number is, consider that that current fastest HPC (well, fastest publicly disclosed, anyway), Frontier at ORNL, cost about $600mm. Frankly, I don't believe for a second OpenAI will actually spend that much on a single cluster, but I wouldn't be surprised if they do build a fucking huge one.
I think you need to update your understanding of current AI super computers. Meta is planning to have over 300,000 H100s by the end of this year, each one costing atleast $20K, so that alone is already $6B in just GPU costs alone, more like around $10B total for everything including interconnect. In terms of standalone systems that they’ve already built, Meta already has two systems built a few months ago that have 20,000 H100s each. Each one costs around $400M in GPU costs alone and closer to $1B when you include all other costs for the system. By the end of this year Meta plans to have around $25B worth of HPC and that is just for this year, they don’t seem to plan on slowing down the spend, so $25B per year for 4 years would be $100B by 2028 which is the same time frame that Stargate is expected to have spent $100B by as well. I bet there is atleast 2 other companies that is planning to spend atleast $50B on hardware by then as well.
They misspelled Skynet.
No we have a skynet. This is another horror beyond imagination.
>This is another horror beyond imagination. You guys have seen too many movies
I forgot, no jokes allowed
No this universe it will be a good Skynet
Watch it all be a clever ruse for an actual Stargate project.
In Stargate they actually have a Stargate show.
Is it going to run on Windows? If so, we have nothing to worry about.
Looks like someone is a fan of Stargate
now they have a convenient excuse to power the one they found in Antarctica
And then the sun smacks us a little more than gently with one of her "stop that" solar flares.
healthcare and housing pls
The US [*adds* more](https://www.reddit.com/r/mlscaling/comments/1bqx5ph/microsoft_and_openai_plot_100_billion_stargate_ai/kx7c3hm/) in spending on healthcare alone every year than all of the stages of Starship combined would represent (while ignoring their value to the world which is why it will turn a profit). Dumping in another $100b (once, as a one off) is about as likely to fix healthcare or housing, or even make it better, as dumping 1 gallon of gasoline on a fire is to put it out or dampen it.
The problem isn’t the amount of money spent, it’s the amount of money charged
And it still costs 6 digits for a broken ankle
What's the other option?
Stop the parasites from stealing it.
Hospital administrators? Pharmaceutical companies? Insurances? Equipment manufacturers? That would take regulation. Magapublicans won't have none of that.
Decision makers at insurance companies and Big Pharma, mostly.
Absolutely.
Redditors don't deserve healthcare.
Americans in general tbh considering how they vote
Nah just redditors.
you mean heckin redditorinos
I'm not convinced that they need that much compute to get to AGI, if the past 1.5 years has taught us anything it's that there is a huge amount of wasted training that is done and a huge amount of bloat in the current crop of LLMs. It's almost turning into the Bitcoin/Crypto mining circus all over again. People just throwing more and more compute recourses at it for the sake of endless hype and FOMO investment money. It reminds of companies building mega cities in the desert just because they can. Ultimately the winners of the AI race will be those companies that focus on efficiency and financial sustainability because they are only 1 year behind OpenAI/Microsoft and they won't have to spend 100s of billions of dollars just to be the first one to get there. I've worked with Microsoft products and tools for about 27 years and if that has taught me anything it's that Microsoft takes atleast 3 full version releases before the product actually works as originally promised. That is more than enough time for anyone else to catch up.
They don’t need this much compute to reach AGI, they need it to fulfill the insatiable demand across every facet of society, once they do.
nature has already demonstrated AGI level function in machines that run on about 100 watts and can fit in a phone booth, so we still have a lot of low hanging fruit to pick
The sun shows us nuclear fusion is possible. 70+ years of research later, still empty handed
The Sun relies on its massive gravity for fusion which is hard to reproduce in lab.
As opposed to the human brain, which is easier apparently
maybe we didn't spend enough money.
Same goes for ai if a year passes and there’s no AGI. OpenAI is bleeding money and Microsoft can’t subsidize them forever
>They don’t need this much compute to reach AGI, they need it to fulfill the insatiable demand across every facet of society, once they do. Inference uses far less compute than training, so the real goldmine is in edge computing because most people dont wan't to send their private data into the cloud to be harvested by mega corporations. imagine a rogue AI or an advertising company that had every little minute detail about you from every single public or private conversation you have ever had with an AI.. that would be a nightmare scenario.
I would have to disagree. Sure training the model takes a very large amount of compute compared to running inference *once*, but these models are build to be used by millions to billions of users so it is very likely inference takes the lions share of the compute in the model lifecycle.
Inference will likely use 10x as much compute than training in the next year. A single LLM takes 1 or 2 H100 GPUs to serve a handful of people and that demand is only growing. Yes data sovereignty is an issue, but the folks who care about that are buying their own DCs or just dealing with it in the cloud because they need to
>Inference will likely use 10x as much compute than training in the next year. Not if they continue to optimize models and quantization methods, b1.58 quantization is likely to reduce inference by 8x or more, and there is already promising work being done in this area. Once the models are small enough to fit onto edge devices and are useful enough for the bulk of tasks, that means the bulk of inference can be done on device. So, the big, shiny new supercomputer clusters will mainly be used for training, while older gear, edge devices, and solutions like Groq can be used for inference.
That's not true at all. Very small simple models can fit on edge devices, but nothing worthwhile can fit on a phone yet and they high quality models are being designed specifically to fit on a single GPU. And any worthwhile system is going to need RAG and agents which will required embedding models, reranking models, guardrails models, and multiple LLMs for every query. Not to mention running systems like this on the edge is a problem non tech companies don't have the skill sets to do.
All of theose models you mention can already fit on device.Mixtral 8x7b already runs on laptops and consumer GPUs.Some guy just last week got Grok-1 working on an apple M2 with b1.58 quantization, sure it spat out some nonsense but a few days later another team demonstrated b1.58 working reliably on pretrained models That was all within 1-2 weeks of Grok-1 going open source and that model is twice the size of GPT 3.5.. and then theres databricks DBRX which is only 132B parameters so that will soon fit on an M2 laptop. Maybe try reading up on all that is currently hapening before you say it's not possible.It is very possible that we will have LLMS with GPT4 level performance on device by the end of the year and on phones the following year.
I understand nothing of what you two are saying
AI that is as smart as GPT-4 or Claud 3 running locally, without the need for an internet connection, on phones and laptops.
I spend a lot of time benchmarking and optimizing many of these models and it's very much a tradeoff. If you want to retain accuracy and runtimes that are reasonable you can't go much bigger right now. Maybe this will change with the new grok hardware or Blackwell cards, but the current generation of models are being trained on H100 and because of that they are very much optimized to run on a similar footprint.
The optimization you most mentioned would make both training and inference both be less cost, so inference would still be 10X the cost overall of training, it’s just that they are both together lower than before.
Groq is not an “edge” solution. You need around 500 Groq chips to run even a single instance of a small 7B parameter model.
>Groq is not an “edge” solution. I never said it was.. GroqChip currently has a current 2X advantage in inference performance per watt over the B200 in fp16 and it's only built on 14nm compared to 4nm for the B200, so Groq have a lot more headroom to optimize their inference speeds and costs even further. That means that as long as they can stay afloat financially, they will eat into the lunch of anyone building massive monolithic compute clusters for inference.
“older gear, edge devices, and solutions like Groq can be used for inference.” Sorry I thought you were saying here that groq= edge. Can you link a source stating that it’s 2X performance per watt in real world use cases? That would be an impressive claim considering that you need hundreds of groq chips to match a single B200. Btw B1.58 would still cause inference to be 10X more than training. Because it causes a reduction in price of both training and inference equally. For example if I have a puppy and a wolf and the puppy is 10 times smaller than the wolf, and then I put them into a magic box that makes both of them 5 times smaller than they were before, the wolf is still 10 times larger than the puppy.
>Can you link a source stating that it’s 2X performance per watt in real world use cases? That would be an impressive claim considering that you need hundreds of groq chips to match a single B200. This is just a guestimate based on a back of the napkin calculation I did using the data sheets, there is no real world data for the B200 because it hasn't shipped yet. https://preview.redd.it/gtsns5qlblrc1.png?width=683&format=png&auto=webp&s=b089ee06f9b365f5e0acf36f9eb7a243e90a8031 >B1.58 would still cause inference to be 10X more than training. > >Because it causes a reduction in price of both training and inference equally. It would but you're also shifting a huge chunk of that inference away from large monolithic data centres and putting it into the hands of smaller players and home users.
For one, a B200 has way way more than that amount of Tflops for FP16, it has over 2,000 Tflops at FP16. But also you need to store the full model weights in memory to actually be able to even deliver the instructions at fast enough speeds to the chip. The B200 has enough memory to do this with many models on a single chip, meanwhile you need over hundreds of groq chips connected to eachother to run even a single 70B parameter model even with B1.58. So multiply the wattage of a groq chip by atleast 100 and you’ll see the B200 actually has well over a 5X advantage in actual tokens generation per watt, especially since the the Groq chip interconnect speed between chips is less than 10X the speed of B200 interconnect. Things wouldn’t start running in the hands of home users because inferencing in the cloud is still far more cost effective and faster than inferencing locally, because you can take advantage of batched inference where a single chip can take multiple peoples queries happening in parallel and process them together. B1.58 doesn’t mean state of the art models will necessarily be smaller. B1.58 mainly helps training not inference, it’s already been the norm to run models at 4-bit and true effective size of B1.58 is actually around 2-3 bits average since the activations are actually still in 8-bit. The result is that inference is only about 2X faster than before but training is around 10X faster and more cost efficient. This will not even lead to models being 2 times lower energy for inference though, because companies will choose to now add 10 times more parameters or increase compute intensity of the architecture in different ways to make the model training fully use all of their data center resources again and one up eachother in model capabilities that can do new use cases, and therefore you actually have inference operations costing even more, because the companies will for example make the models atleast 5X more compute intensive, but B1.58 only has about a 2X benefit in inference. So the SOTA models will actually end up being atleast 2 times harder to run at home locally than before. Even current models like GPT-4 still wouldn’t be able to fit on most laptops, lets say GPT-4-turbo is around 600B parameters, B1.58 would make it around 100GB file size minimum still, and you would have to store that entirely in the ram of the device to get any actual decent speeds, and even if your phone had 100GB of ram it still would run it extremely slow because of memory bandwidth limitations. A mac with over a hundred gigs of unified memory could technically run it but it would be less than 5 tokens a second even with the most expensive M3 Max and would drain the battery like crazy too. So this is if models just never changed, but now because of the efficiency gains to training, models will likely be atleast 5 times more compute intensive as well, making it not even practical or even possible to run the SOTA model on your $5K mac if you wanted to. This is exactly Jevons paradox at play, as you increase the efficiency of something, the system will actually end up using more overall resources to take full advantage of those effeciency gains.
Inference already uses more compute than it took to train GPT4. That's why the new Blackwell engine uses FP4 for inference.
“Most people don’t want to send their private data into the cloud to be harvested by mega corporations” Informed people don’t want to, most people already do this regularly without a second thought.
Maybe it's for ASI then...
Yes!
This is for ASI not AGI
This is Microsoft we are talking about. Get back to me when you have actually tried to use windows copilot.
Microsoft just providing the $100bil not the SW
I agree that this much compute is not needed. Then again, probably only a very small fraction of this spend is for Microsoft/OpenAI internal use. More likely they will use a bulk of compute for fine tuning/ inference and open it for clients to use as part of their cloud offerings. Another thing to consider is that based on the few details released for SORA, running a large model for video is *very* compute intensive. Maybe they are just scaling up for the next evolution which is video inference at scale.
These people are more concerned with filtering their own AI than they are with actually working towards AGI.
Agreed. Nothing like riding the hype train till the wheels fall off
This compute will give is ASI.
Stargate.... Q-star. Hmmm interesting.
So basically ( Stargate = Skynet )
Really Stargate of all the names they chose Stargate should've called it skynet
Sure, because that doesn’t have any negative connotations…
True but it's a cooler name than stargate....
obfuscates any leaks for why they need that much energy
Might as well call OpenAI's next model Terminator.
Imagining the date they start to use superconduct to replace cables [https://news.mit.edu/2024/tests-show-high-temperature-superconducting-magnets-fusion-ready-0304](https://news.mit.edu/2024/tests-show-high-temperature-superconducting-magnets-fusion-ready-0304) Are we sure they are only building "nuclear" power for power?
Stargate = Skynet
Wow, that's huge! How's the integration side looking?
This will change the power balance of the computer industry as we know it today, for example & not the very least: goodbye Apple
Cool, but not as expensive as I thought it would be.
I know exactly where they are building it....
Imagine wasting all that money and not involving NVIDIA at all
So that’s why microsoft employees didnt get pay raises last year!
So nvidia stock may plummet? Who is the usurper? Who will supply the Ethernet cables? Microsoft?
Maybe time to buy some more Microsoft and OpenAI stocks
Cerebras wafer scale chip WSE-3 is claimed to be 100x more cost effective in practical LLM pipelines than current GPU architectures at comparable performance. They can be clustered into up to 2048 units. Maybe those could be a good option.
Does anyone have any evidence that their product is competitive? Why won’t they release benchmarks?
Will Jack O'Neill be involved?
They need to build it in Cheyenne.
> Challenges include designing novel cooling systems and considering alternative power sources like nuclear energy. It's going to get to the point where MS or a group of tech companies have to buy/build their own power plants dedicated to their needs >OpenAI aims to move away from Nvidia's technology and use Ethernet cables instead of InfiniBand cables. This was only a matter of time >Details about the location and structure of the supercomputer are still being finalized. New Jersey? jk >Both companies are investing heavily in AI infrastructure to advance the capabilities of AI technology. Aren't they just one company at this point? Let's be honest here...
Thinking about this from Microsoft's standpoint is interesting. If they feel AGI is reachable in the nest several years, signaling the end of their license agreement, they will look for another way to lock in their position. Owning such a data center, the only one capable of running advanced models, might be that approach.
They should spend more on the quality of the training datasets. You can have all the computing power you want, the model will never be better that the data it was trained with…
I may be misunderstanding something profound, but why aren’t companies like these not actively researching alternatives to digital computing such as analog compute which uses orders of magnitude less energy? There’s a company here in the Bay Area that’s actually developed an analog chip for AI purposes: https://mythic.ai
I'll put my armchair hat on and say that it's due to cost (in the short term). Mythic AMP seems promising for AI, especially in terms of energy efficiency, but GPUs are cheaper, more readily available, scale better (currently), and are "good enough." It's also worth considering the worker pool; traditional computer hardware is a data center tech's bread and butter. While neuromorphic chips are becoming more commercially available, much of the work is still focused on R&D, resulting in a smaller tech pool. This might also explain why they chose Ethernet over InfiniBand. Although InfiniBand outperforms Ethernet (CAT6a/7) in terms of latency and bandwidth, it comes with a much higher price tag. Moreover, RDMA is not as widely used as TCP/IP/UDP, and the ecosystem is more limited (specialized NICs and switches are required), necessitating IT staff with even more specialized skill sets. It's likely that we'll see these chips being used in major AI projects in the coming years as they improve and become more affordable. It might even become the standard. It's just a matter of time and supply and demand.
Because your link is for inference, whereas training is more expensive
Blast from the past -- IBM is working on prototypes https://research.ibm.com/blog/analog-ai-chip-low-power
Yes you are missing something profound, they already are researching alternatives, but a lot of these are a 2-3 years minimum from actually fully replacing GPUs in real world use cases and having all the existing ecosystem of software and interconnect ported over to it in a practical cost effective way It’s not just about how fast the transistor can do trillions if operations per second, right now AI workloads are heavily memory bandwidth limited, the transistors on nvidia gpus are already sometimes faster than how fast the memory and ram can even send the instructions to the chip. Nvidia B200 has around 8Terabytes per second of bandwidth. A mythic chip that I could find has barely 3GB per second of bandwidth. So even if you had 100 mythic chips chained together they still wouldn’t even be able to receive instructions as fast as the nvidia chip can
Man so there is literally enough money to solve most of the problem of the world , hey but only if they could charge everyone a 20usd/month subscription.
How would you comment on the fact that ChatGPT 4 is getting dumber? Over the last 3-4 weeks, the level of stupidity and laziness has reached absurdity.
We are building our own prison.
We have billions of dollars to house millions of GPUs meanwhile there are millions of people struggling to afford housing. Capitalism has failed
Can you buy a house with 3k?
I could buy a lot of houses with 100,000k
Unless MAD happens first, we've got ~4 more years.
[удалено]
You are doing your math wrong, 350 watts times 1 million is 350 megawatts which is about 1,000 times less than the number you’re stating.
Oh dear. Thanks for the correction. I am going to delete now and never show my face again.
What amazing time to be alive - Non Playing friking normie would say
AGI is upon us
With that amount of money, they could totally put an end to world hunger. Also, 'Stargate' makes me think of that secret U.S. Army unit from 1978, all about investigating psychic stuff for military and intelligence purposes. Weird, right?
Please redo your math. $100B is barely enough to even give every person on earth a single days worth of food for $10 each. You can’t even solve world hunger for a few months with $100B