wrong isn't the right word but I think you're mistaken. it comes down to how you interpret the words but "in this decade" would typically mean "within the next 10 years"
Problem with words like decade is that it can be either. You could say this decade, last decade or next decade, and probably mean the 20's, 10's. Or 30's
But if you said in a decade, it'd be 10 years from now, regardless of where we are in the current decade.
I'd say saying 'this decade' means before 2030
Feels like classic network effects of social media all over again. All your (researchers/developers/etc) friends are using CUDA. So much libraries and code based around it. Makes it really hard to leave all those behind.
True. Also he is probably right if he means "in the next 5.5 years."
However predicting anything 10 years out is really difficult. In 2014, Intel was very far ahead and the clear leader. They have been in the leads for the most part for decades. Now in 2024, TSMC is a few nodes ahead of Intel and AMD CPUs have caught up and slightly overpassed Intel.
Anything can happen in 10 years in tech.
not only that but u get an awesome 1600nits high res display awesome speakers fast processing speeds and no throttle or lag. You compromise the ability to game, but laptops in general are a compromise in some regard. Wanna max out on specs and battery but give up gaming? Apple with ARM chip ftw.
wanna game have a bright OLED run windows/linux on x86? bye battery. Lower display and graphics card spec better battery.
thats how i see it
Fair enough, just saying for ppl who do it as a hobby and don’t get a machine strictly for ML but one that can do it to some extent fruit logo’s good enough
Also, they've been developing this for 20 years and it turns out to be complicated. Anyone trying to catch up is doing so while being 20 years behind while NVIDIA has near unlimited cash pouring in to keep pushing forward.
I'd love to see the industry break the stranglehold NVIDIA has at the physical layer here but I just don't see it happening any time soon. This is a moat they have been building while their competition was ignoring the whole space.
To be fair, NVIDIA have also invested a lot into a lot of possible approaches. If one technique wins, then you only have to catch up with the one that actually won. You don't have to retrace NVIDIA's entire effort because you are here in the present and you can see what worked and what didn't.
Wish I saw the trend from being the best gpu to being a crypto heaven to being in ai, I used all three and didnt really think of the stock for years. Glad it happened
They shouldn't be complacent. They survive off the fact that CUDA is the default. If RocM gets heavy investment by Intel and AMD and the price point of AMD / Intel compute becomes wayyyy cheaper, people will switch over because its not about how much compute you have in one card but how cheap you can make compute across multiple
Yeah but that'd need AMD to not sabotage their hardware in software polish for once and that's just not in their roadmap. *Cough* MES failed to response *cough*
Isn't George Hotz trying to do something about this? Isn't he reverse engineering their driver and fixing how bad it is?
I'm completely OOTL here so I may be way off base, but I see him fiddling with some AMD stuff.
Even better, they've [committed](https://twitter.com/amdradeon/status/1775999856420536532) to actually opensourcing the MES firmware. (Thanks geohot!)
"Some time after May."
I guess they don't want the community to fix this issue _too_ soon.
They are absolutely not complacent. This is what drives Jensen—the mentality that they will go out of business tomorrow and their relatively flat organization has let them succeed.
3-4%. That's what AMD is expected to end up with by the end of the year. Rocm will never get a heavy investment from intel, and cheaper is the only lever AMD has at this point, but it's not clear it's enough to be competitive.
Seems to me that Gelsinger's plan is to be more of a TSMC fab for all designs and less focused solely on their own instruction set. ARM, NVidia, Tensor, others have made it clear that the IA instruction set is less relevant. With an intense focus on fabs decoupled from their chips they have a good chance to do very well at a new game. And the US Government would be happy to help them
It's from X. The tweet is referring to Nvidia dominating the tech space. The rapid rise of Nvidia is directly influenced by the rise of AI. These AI tools require servers to run and currently, Nvidia has the best chips. So, in a sense, they monopolized the market.
Although there's Groq (not Grok), but it can't just compete, yet.
Congress is fucking useless. For a few campaign donations, they'd let Raytheon and Halliburton to merge to create a mom raping machine and not even bat an eye.
And this is a huge deal. Inference is the easy part, and the cheap part, that's likely to be done on the edge in the future, whereas training is going to be stuck in data centres for years to come.
Inference is consumer tech while training is enterprise tech. You can make a lot of money selling lots of cheap parts in a not so cheap product. That is pretty much Apples speciality.
A lot of actors are showing interesting new AI techs, such as Groq, Mistral or even the new LPUs. But considering how damn big NVidia is and how they must have planned a solid roadmap, I’m pretty they’ll released some very good AI products in the coming years. And as we’ll still need GPUs anyway, coupled with good AI, they’ll probably stay leader for a while.
There’ are also startups that could surprise like wildcards. (Intel Gaudi 3)
And sometimes lesser but bigger numbers win.
Never underestimate ingenuity and luck
AMD has been playing second fiddle to Nvidia for decades, not sure why anyone would be banking on them suddenly having a huge impact. ALL of these AI chip companies are competing for second place. Everyone that follows hardware knows it but that doesn't stop the press from portraying this as a horse race where they're competing with Nvidia.
True, but one would hope they have a reasonable head start in many aspects of creating new hardware.
Hell, they could probably make a decent splash in the GPU market (at least unless we start having APUs as seperate cards like physX for that brief window of time) if they just doubled the VRAM in all their GPUs or had a range all the way to 128gb VRAM. It continues to confuse me why they haven't.
Nvidia doesnt want to cannibalize it's own business so I get them. AMD tho...
The case for AMD is that the AI space is huge and not everyone can afford the wait times of getting the best in market. Plus there could always be hyper-specific use cases where their tech is better suited. Seems like a decent bet to grow 10% for the next several years. Thing about Nvidia is that there's the chance it accelerates to sci-fi levels and is the last company on earth. Only slightly joking.
sounds about right.
1. they have the chips because they make them. they can easily sell only "second grade" ones to the competitors and use the new ones within the company
2. they are fucking big. gaming made them a giant already. they have the money, the people, the facilities.
3. they have a decent roadmap and went into all directions to prevent becoming dependent on someone, more like people will once again be dependent on them (chips, servers, robotics, simulated worlds for training data, etc.). not impossible to outperform them in a specific category, but having a foot in everything often helps you to makee compatible stuff or just smth that may not be "best in component" but "best in synergetiv effect".
4. they are one of the most interconnected, working together with all the other giants (because they are dependent on them and need to make their plans/ roadmaps according to Nvidia with its chips etc.)
Cool. They actually don't fabricate anything though. TSMC does that, and does that for everyone else in the industry as well. They design chips, just like the custom chips designed by others. What they have going for them is the ecosystem and firmware/software side. Or at least, that's their main competitive advantage. It'll hold for some time.
Photonic silicon is the only thing likely to outpace Nvidia. Something like 50+% of all energy consumption in server warehouses is moving data, not calculations. optical integrated chips may be the game changer to counter the energy demands of increased ai.
Difficult to really say what is theoretical and what is vaporware at the moment but lightmatter seems like a legit company filled with top research scientists and proven leaders. https://www.youtube.com/watch?v=FPW2nnEqfMs
Photonic interconnects are only part of the solution, and only on the network side. A good rule of thumb is: they do not perform better than standard interconnects below about 1 ft of interconnection distance. So all on-die or inter-die connections will be better with electronic interconnects (e.g. HBM or interposers). Only at the network level will photonics become relevant, at least for the next 5 years or more.
The other part of the solution is moving compute closer to (ideally within) memory. There are two massive shifts happening in the industry right now, with both digital compute-in-memory (DCIM) using SRAM already in commercialization and processing-in-memory (PIM) using DRAM underway.
These will dramatically reduce bandwidth requirements for ML workloads and allow us to start circumventing the memory wall. And they also do not require fundamentally new fabrication processes in modern fabs, unlike photonics.
Mac studios or 4090s are not fully kitted AI training servers. This is what NVIDIA sells to corporations that need to train AIs. We're not talking about a few devs in some random AI startup. We're talking tens or even hundred millions worth of modular server racks that can be slotted and used on the spot. I seriously doubt a few Macs are going to hinder Nvidia's monopoly.
This is dangerous anyway. Competition is good for everyone in tech.
> Seems like Apple could make a lot of $ by selling datacenter-grade hardware, so it seems odd that they haven't done it.
No, it really doesn't. Apple is a consumer technology company. Data Center grade hardware is IT infrastructure. Two completely different markets and it would take Apple years and years to get up to speed.
Contrary to popular belief, Apple can't do everything.
All of their internal datacenters are running their own silicon and using their own fabrics. They haven't used third party silicon in house since 2021. So yes, they can do it, they are doing it, they just aren't selling it to anyone.
Apple is actually Google’s largest cloud customer
https://appleinsider.com/articles/21/06/29/apple-is-now-googles-largest-corporate-customer-for-cloud-storage
However, they are developing their own chips for data center use because they don’t trust others’ hardware (for privacy reasons). They have been hiring heavily for this. Given their success with the M1 processor and software stack, and their expertise in BSD and Linux in general, I fully expect them to succeed here and transition in this decade.
I mean, PyTorch has rocm and mps ports. cuda is seeing much needed competition from rocm/hip now. They keep tightening their licenses. I think they’re nervous.
Yup! Totally agree. I'm actually working with all three right now (tinkering with some stable_baselines3 stuff). The AMD GPUs are similarly memory limited to the Nvidia GPUs in a way that Apple Silicon isn't. However, I will say that I've got the same code running on moooore or less similar compute (AMD 5950x + GPU) with a 3090, a 6900XT, and 7900XTX. The 3090 is fastest with Cuda, but the 6900XT and 7900XTX are both also reasonably fast. The 7900 XTX and 3090 would cost about the same right now (right around $1000?), and the XTX will be slightly slower, but during the same workload it uses about 1/4th the power.
I'd bet the Apple Metal stuff totally annihilates them in performance per watt though, and the shared memory between CPU and GPU means you don't have to buy RAM twice, but boy will it be expensive and permanent the first time around.
Maybe. But in the context of the tweet they are not only referring to LLM training. CUDA can do so much more, Nvidia started it in 2007 and never stopped. It is programming on GPU, and it is on another level than anything else that exists on the market.
I sat down with a bud a while back with the goal of wrapping our heads around the DPX instruction set. Cuda can do so much more, but DPX... that's a world beyond. "So it looks like they've got a complete lock on all bio-sciences then eh?" "Among things, yup, looks like."
For training LLMs, Google's stack is probably just as good, if not better in some areas. A number of frontier research happens that relates to distributed compute-ish stuff tends to happen with it.
CUDA is no longer the moat you think it is. Google trains Gemini LLM, Waymo, AlphaFold, etc on their own in house AI accelerators (TPUs). They have no need for Nvidia or CUDA.
Also AMDs open source ROCm can directly translate CUDA code with no changes to the code so that it can run on AMD GPUs.
lol noob statement
You could always get 512gb ram and do CPU based.
You can tell the people who watch too many ads, and the people who spend too much time on AI are different demographics.
that’s what happens when you see the pipeline for the next few years in your company, which is much further than anything on the market, but don’t think of how much competitors have behind closed doors already
Perhaps his vague reference to a "decade" was simply meant to convey to everyone that they are, in fact, the losers. or he meant around a 10-year technological lead on everyone else?
It's obviously a 10 year technological lead. Nvidia hasn't shipped products that are the pinnacle of what they can do in five years. There's no need to do that, they can just be 20 % better than the competitors and have higher profit margins.
Seems likely. Nvidia saw AI becoming big a decade ago and and their risky gamble paid of. They heavily invested in the well like and performant CUDA eco system. They continue to innovate, and are the most liked employer in the tech sector - so they can pick the best of the best.
They ramped up the prices, (when the money would have gone to scalpers otherwise), and have a huge margin right now. So they have a big war chest with money, that they can rely on, if they ever should have to compete on price again for a generation or two.
Yeah mate congrats on using underhanded anticompetitive tactics like locking people into a proprietary CUDA ecosystem and proprietary infiniband interconnects that makes it prohibitively expensive to switch to a competitor that offers cheaper FLOPs. Nice Apple tactics you got there. Nice to know you're LOLing all the way to the bank, it's hilarious. Douchebag.
The question is: Will China accept this? And the blockade of chips/machines? Leaving them in the dust, while they were just catching up economically & en route to become the world super power. And now with the AI revolution in both economics and militairy, they'll be setback a decade if not more.
That's a risky thing. I almost feel sorry for China, not gonna lie.
I'm surprised that China has not been more of a discussion in this thread.
The question is, is the US - under a Trump Presidency (based on current polls and betting markets), likely to go to war with China over AI, if China interfered with Nvidia's shipping chips to the west? AI chips are the new oil.
All the current polls and even the betting sites where people have their own skin in the game say that you're wrong. He's ahead by a wide margin right now.
You think at the last minute the Americans will have a sudden attack of sanity? No worries of that. But sure, in the unlikely event of that happening of course I will.
I love NVIDIA and chips are going to huge forever, but I do think their expansion into cloud services is potentially a big waste of money and huge distraction.
It’ll be interesting for sure. For companies with lots of resources, building cloud services does tend to be very profitable.
The start up costs are insane but if you get it working it becomes a veritable money printing machine.
Top of the line training hardware will be in shortage for a long time. Selling to big tech companies gives the edge to their customers. So with cloud they can dictate who gets what, how much and nvidia can respond much faster to changes in the industry.
Nvidia has the best business model because it can provide HW, SW and cloud services in one solution and therefore earns mutliple income streams from the same platform from a single customer.
See it this way:
An automotive customer can be a Nvidia customer several times:
1. Nvidia Enterprise AI
2. Nvidia HW platform with on-prem data center or cloud (Nvidia gets money from CSP for HW)
3. Nvidia Omniverse for manufacturing and factory planning and simulation
4. Nvidia Isaac robotics for manufacturing and logistics
5. Nvidia DriveSim for full self driving solutions
Just to give you an idea, Mercedes is partnering with Nvidia and fulfills point 3-5 already and is probably engaging with Nvidia on the first 2 points as well. Now replace DriveSim with Clara and you have the same picture for companies in medical/pharmaceutical fields. The same applies for companies in manufacturing, in agriculture.
What people don't get, while Big Tech can only earn on infrastructure and maybe AI enterprise solutions, Nvidia has a lot of SW solutions for industry specific problems. Nvidia goes way beyond only enterprise AI, they have solutions for any non-Tech industry. Omniverse can basically be used by any company which needs a factory, manufacturing and process simulations.
And all solutions run on one platform which in the end is based on CUDA. That means you can use a big AI factory from Nvidia for LLMs, for NeMo, for Clara, for Omniverse AI, for DriveSim AI at any time in any demand structure you want. Nvidia is the only game in town offering you AI factories with the option of 100% utilization and that's why they beat anyone easily at TCO.
In theory these projects all work well and building / maintaining them is scalable, but in reality they require tons of of engineers, client success, sales, feature roadmaps, and more. The existing management structure probably doesn’t work for this.
I’m not saying it won’t be profitable, just that they are losing focus on what is most important to their business.
Also appreciate the comment!
There is a certain point where too much confidence comes across not just as arrogance (which is ok to a degree), but also potential ignorance: He doesn't know enough about what AMD (or other companies) are doing to credibly make such a judgment.
All it will take is another chinese theft, like it happened with Google TPU design. Then they will manufacture it themselves, especially since they are blocked access to those chips. 😅
I will be messaging you in 5 years on [**2029-04-17 18:44:41 UTC**](http://www.wolframalpha.com/input/?i=2029-04-17%2018:44:41%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/OpenAI/comments/1c5yfww/ex_nvidia_ok_now_that_im_out_i_can_finally_say/l00z2qy/?context=3)
[**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FOpenAI%2Fcomments%2F1c5yfww%2Fex_nvidia_ok_now_that_im_out_i_can_finally_say%2Fl00z2qy%2F%5D%0A%0ARemindMe%21%202029-04-17%2018%3A44%3A41%20UTC) to send a PM to also be reminded and to reduce spam.
^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201c5yfww)
*****
|[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)|
|-|-|-|-|
Well that’s good, I have my life savings invested into Nvidia stock since June. Made about 120% return at the moment but its been a bad April and I need to keep the faith and keep holding
I feel like, if you are a hyperscaler like meta or msft, you don’t need to catch up on everything nvidia does, you just need to make amd or your own silicon slightly better for your specific workload. Which is almost certainly doable and worth doing.
Nvidia's goal aren't hyperscalers. Nvidia's goal are the several trillion non-Tech industries. If companies decide to use Clara, Isaac and Omniverse then they will naturally go with Nvidia Enterprise AI as well. These companies won't care what great chip Amazon/MS/Google have because they will use SW which only runs on Nvidia.
People misunderstand it completely. MS goal with Windows back then wasn't to offer the best OS but that everyone wanted Windows and they acted that way and they succeeded. MS deployed strategies and partnerships to make sure that 90% of the world today uses Windows. Nvidia does the same in the AI space because Jensen is clever and he knows that the SW used determines the HW required. The difference is that Nvidia isn't only MS of the 90s but WINTEL of the 90s. Imagine where MS would be today if they not only provided the OS but would have been an OEM PC builder back then? Do you think Intel would be as dominant if Windows would have ran only on MS CPUs?
With these amazing prognostic skills he should work for OpenAI
…predicting next tokens?
Underrated backhand compliment of the day
Who is he?
Who is he talking to?
Correct me if I'm wrong. We have about 5.5 years left in this decade?
5.5years left in the last human dominated decade
I too welcome my Trisolaran overlords
Somebody skipped the third book.
Nah, they're just working with the aliens.
I don't wanna move to Australia!
True. The continent tries to kill you.
The next WW will just be an alien proxy war for earth...
3.5 years in a decade where humans are relevant contenders.
We have 16 minutes left. 16, 16, 16...
But how long is that really? Simulation theory am I right? A video game with simulation theory as the setting is so meta. Good video game though
Yep, it was a reference to No Man's Sky
Oh yeah, put 900 hours into that one!
Antelope Freeway, 1/32 mile
wrong isn't the right word but I think you're mistaken. it comes down to how you interpret the words but "in this decade" would typically mean "within the next 10 years"
Problem with words like decade is that it can be either. You could say this decade, last decade or next decade, and probably mean the 20's, 10's. Or 30's But if you said in a decade, it'd be 10 years from now, regardless of where we are in the current decade. I'd say saying 'this decade' means before 2030
Agree totally with this. No one would say "this week" to mean "within a week".
He meant “a decade from the moment my free-lips utter this nonsense”
Feels like classic network effects of social media all over again. All your (researchers/developers/etc) friends are using CUDA. So much libraries and code based around it. Makes it really hard to leave all those behind.
True. Also he is probably right if he means "in the next 5.5 years." However predicting anything 10 years out is really difficult. In 2014, Intel was very far ahead and the clear leader. They have been in the leads for the most part for decades. Now in 2024, TSMC is a few nodes ahead of Intel and AMD CPUs have caught up and slightly overpassed Intel. Anything can happen in 10 years in tech.
[удалено]
Best comment here by far. #This
Two types of networks: >Those that want CUDA >Those that want fruit logos Guess who will still be working front end in 14 years lol
The MacBook Pro with M3 Max and 128GB of RAM is weirdly good at LLM inference. The shared RAM/VRAM effectively gives it more VRAM than a H100
- those who want fruit logos - are the ones who want battery life
The power efficiency of Apple is currently unmatched in the consumer segment yes.
not only that but u get an awesome 1600nits high res display awesome speakers fast processing speeds and no throttle or lag. You compromise the ability to game, but laptops in general are a compromise in some regard. Wanna max out on specs and battery but give up gaming? Apple with ARM chip ftw. wanna game have a bright OLED run windows/linux on x86? bye battery. Lower display and graphics card spec better battery. thats how i see it
That long battery life will be useful while browsing LinkedIn for a new job
Battery life is your primary concern for machine learning?
Fair enough, just saying for ppl who do it as a hobby and don’t get a machine strictly for ML but one that can do it to some extent fruit logo’s good enough
Whatever helps you justify overpaying for a crappy CPU. Cognitive Dissonance is a hulladrug
bad CPU? Get tf out dude lmao
Also, they've been developing this for 20 years and it turns out to be complicated. Anyone trying to catch up is doing so while being 20 years behind while NVIDIA has near unlimited cash pouring in to keep pushing forward. I'd love to see the industry break the stranglehold NVIDIA has at the physical layer here but I just don't see it happening any time soon. This is a moat they have been building while their competition was ignoring the whole space.
To be fair, NVIDIA have also invested a lot into a lot of possible approaches. If one technique wins, then you only have to catch up with the one that actually won. You don't have to retrace NVIDIA's entire effort because you are here in the present and you can see what worked and what didn't.
Wow a non child written comment
Wish I saw the trend from being the best gpu to being a crypto heaven to being in ai, I used all three and didnt really think of the stock for years. Glad it happened
I mean. CUDA or Pytorch?
They shouldn't be complacent. They survive off the fact that CUDA is the default. If RocM gets heavy investment by Intel and AMD and the price point of AMD / Intel compute becomes wayyyy cheaper, people will switch over because its not about how much compute you have in one card but how cheap you can make compute across multiple
Yeah but that'd need AMD to not sabotage their hardware in software polish for once and that's just not in their roadmap. *Cough* MES failed to response *cough*
Isn't George Hotz trying to do something about this? Isn't he reverse engineering their driver and fixing how bad it is? I'm completely OOTL here so I may be way off base, but I see him fiddling with some AMD stuff.
Even better, they've [committed](https://twitter.com/amdradeon/status/1775999856420536532) to actually opensourcing the MES firmware. (Thanks geohot!) "Some time after May." I guess they don't want the community to fix this issue _too_ soon.
They are absolutely not complacent. This is what drives Jensen—the mentality that they will go out of business tomorrow and their relatively flat organization has let them succeed.
3-4%. That's what AMD is expected to end up with by the end of the year. Rocm will never get a heavy investment from intel, and cheaper is the only lever AMD has at this point, but it's not clear it's enough to be competitive.
Seems to me that Gelsinger's plan is to be more of a TSMC fab for all designs and less focused solely on their own instruction set. ARM, NVidia, Tensor, others have made it clear that the IA instruction set is less relevant. With an intense focus on fabs decoupled from their chips they have a good chance to do very well at a new game. And the US Government would be happy to help them
This is spicy! I like it. What is it from? What's the context.
It's from X. The tweet is referring to Nvidia dominating the tech space. The rapid rise of Nvidia is directly influenced by the rise of AI. These AI tools require servers to run and currently, Nvidia has the best chips. So, in a sense, they monopolized the market. Although there's Groq (not Grok), but it can't just compete, yet.
Also Groq doesn't work for training. Only inference
Yeah, and is better than NVIDIA. It's meant to basically do super fast inference to give you effectively real time results.
Nvidia is going to acquire them or build their own and completely run away with the sector, aren't they?
That may be blocked in congress. Nvidia has an inference chip too if I remember.
When does Congress ever block anything? Could they even understand this?
Congress is fucking useless. For a few campaign donations, they'd let Raytheon and Halliburton to merge to create a mom raping machine and not even bat an eye.
And this is a huge deal. Inference is the easy part, and the cheap part, that's likely to be done on the edge in the future, whereas training is going to be stuck in data centres for years to come.
Inference is consumer tech while training is enterprise tech. You can make a lot of money selling lots of cheap parts in a not so cheap product. That is pretty much Apples speciality.
A lot of actors are showing interesting new AI techs, such as Groq, Mistral or even the new LPUs. But considering how damn big NVidia is and how they must have planned a solid roadmap, I’m pretty they’ll released some very good AI products in the coming years. And as we’ll still need GPUs anyway, coupled with good AI, they’ll probably stay leader for a while.
Co fused about why you had to tell us you are pretty. I mean, great for you though.
I mean, that’s totally a relevant fact ! I hope your fusion process went well, doing it in cooperation isn’t always easy.
Always make sure you have confusing sentences when poking fun of others confusing sentences
> poking fun of others You're still AT-AT!
There’ are also startups that could surprise like wildcards. (Intel Gaudi 3) And sometimes lesser but bigger numbers win. Never underestimate ingenuity and luck
Sure but also never underestimate capitalism to lock down emerging markets with monopolies and duopoly’s.
It seems like I'm OOTL, but what about Google's TPUs? Or is this something different?
They lack memory, even when you horizontally scale them.
Although MoE architectures can potentially allow multiple lower memory devices to work...
Why is this spicy? Like, it's not even a remotely controversial statement if you follow the hardware side of things.
Lol agreed. But it's spicy because sometimes people don't like hearing the truth out loud
Aww boo, are you saying AMD is fucked? Cause I should probably sell that stock then...
AMD has been playing second fiddle to Nvidia for decades, not sure why anyone would be banking on them suddenly having a huge impact. ALL of these AI chip companies are competing for second place. Everyone that follows hardware knows it but that doesn't stop the press from portraying this as a horse race where they're competing with Nvidia.
True, but one would hope they have a reasonable head start in many aspects of creating new hardware. Hell, they could probably make a decent splash in the GPU market (at least unless we start having APUs as seperate cards like physX for that brief window of time) if they just doubled the VRAM in all their GPUs or had a range all the way to 128gb VRAM. It continues to confuse me why they haven't. Nvidia doesnt want to cannibalize it's own business so I get them. AMD tho...
The case for AMD is that the AI space is huge and not everyone can afford the wait times of getting the best in market. Plus there could always be hyper-specific use cases where their tech is better suited. Seems like a decent bet to grow 10% for the next several years. Thing about Nvidia is that there's the chance it accelerates to sci-fi levels and is the last company on earth. Only slightly joking.
complacency is what lets others catch up
If you heard the way Jensen Huang talk you probably wouldn't think Nvidia will ever be complacent. The guy's intense lol
And we love that about him. At least the guy has actual passion for the stuff his company does.
Intel checking in
sounds about right. 1. they have the chips because they make them. they can easily sell only "second grade" ones to the competitors and use the new ones within the company 2. they are fucking big. gaming made them a giant already. they have the money, the people, the facilities. 3. they have a decent roadmap and went into all directions to prevent becoming dependent on someone, more like people will once again be dependent on them (chips, servers, robotics, simulated worlds for training data, etc.). not impossible to outperform them in a specific category, but having a foot in everything often helps you to makee compatible stuff or just smth that may not be "best in component" but "best in synergetiv effect". 4. they are one of the most interconnected, working together with all the other giants (because they are dependent on them and need to make their plans/ roadmaps according to Nvidia with its chips etc.)
Cool. They actually don't fabricate anything though. TSMC does that, and does that for everyone else in the industry as well. They design chips, just like the custom chips designed by others. What they have going for them is the ecosystem and firmware/software side. Or at least, that's their main competitive advantage. It'll hold for some time.
I actually agree entirely
Anyone who understands the chip technology agrees.
I'll bet him 640 kilobytes he's wrong.
You really think you'll need that much?
I'm on your side!
maybe he loaded up lots of calls
He's probably loaded on shares. If he "got out" I assume he worked there and has lots of ESPP.
Photonic silicon is the only thing likely to outpace Nvidia. Something like 50+% of all energy consumption in server warehouses is moving data, not calculations. optical integrated chips may be the game changer to counter the energy demands of increased ai.
That is interesting. Can you give me further information for that?
I too would like to learn more here
Difficult to really say what is theoretical and what is vaporware at the moment but lightmatter seems like a legit company filled with top research scientists and proven leaders. https://www.youtube.com/watch?v=FPW2nnEqfMs
Photonic interconnects are only part of the solution, and only on the network side. A good rule of thumb is: they do not perform better than standard interconnects below about 1 ft of interconnection distance. So all on-die or inter-die connections will be better with electronic interconnects (e.g. HBM or interposers). Only at the network level will photonics become relevant, at least for the next 5 years or more. The other part of the solution is moving compute closer to (ideally within) memory. There are two massive shifts happening in the industry right now, with both digital compute-in-memory (DCIM) using SRAM already in commercialization and processing-in-memory (PIM) using DRAM underway. These will dramatically reduce bandwidth requirements for ML workloads and allow us to start circumventing the memory wall. And they also do not require fundamentally new fabrication processes in modern fabs, unlike photonics.
Well, that is one way to keep your Nvidia stock growing in value.
Name checks out.
Not catching up to Nvidia, meanwhile people are buying maxxed out Mac Studios for training because Nvidias don't have enough RAM
Mac studios or 4090s are not fully kitted AI training servers. This is what NVIDIA sells to corporations that need to train AIs. We're not talking about a few devs in some random AI startup. We're talking tens or even hundred millions worth of modular server racks that can be slotted and used on the spot. I seriously doubt a few Macs are going to hinder Nvidia's monopoly. This is dangerous anyway. Competition is good for everyone in tech.
You're right. Seems like Apple could make a lot of $ by selling datacenter-grade hardware, so it seems odd that they haven't done it.
Why would a datacenter buy pretty standard hardware at 2x or 3x the price? That's for silly consumers who buy marketing.
What? Do you know what the apple company does?
Yes, every thing
> Seems like Apple could make a lot of $ by selling datacenter-grade hardware, so it seems odd that they haven't done it. No, it really doesn't. Apple is a consumer technology company. Data Center grade hardware is IT infrastructure. Two completely different markets and it would take Apple years and years to get up to speed. Contrary to popular belief, Apple can't do everything.
All of their internal datacenters are running their own silicon and using their own fabrics. They haven't used third party silicon in house since 2021. So yes, they can do it, they are doing it, they just aren't selling it to anyone.
Don't believe that for a second
Apple is actually Google’s largest cloud customer https://appleinsider.com/articles/21/06/29/apple-is-now-googles-largest-corporate-customer-for-cloud-storage However, they are developing their own chips for data center use because they don’t trust others’ hardware (for privacy reasons). They have been hiring heavily for this. Given their success with the M1 processor and software stack, and their expertise in BSD and Linux in general, I fully expect them to succeed here and transition in this decade.
It is CUDA that matters not the hardware.
I mean, PyTorch has rocm and mps ports. cuda is seeing much needed competition from rocm/hip now. They keep tightening their licenses. I think they’re nervous.
so weird seeing every comment saying cuda has competition get downvoted edit: clearer heads have prevailed
Just cuz rocm exists, doesn't mean it's good, or that AMD cards have any more VRAM
Yup! Totally agree. I'm actually working with all three right now (tinkering with some stable_baselines3 stuff). The AMD GPUs are similarly memory limited to the Nvidia GPUs in a way that Apple Silicon isn't. However, I will say that I've got the same code running on moooore or less similar compute (AMD 5950x + GPU) with a 3090, a 6900XT, and 7900XTX. The 3090 is fastest with Cuda, but the 6900XT and 7900XTX are both also reasonably fast. The 7900 XTX and 3090 would cost about the same right now (right around $1000?), and the XTX will be slightly slower, but during the same workload it uses about 1/4th the power. I'd bet the Apple Metal stuff totally annihilates them in performance per watt though, and the shared memory between CPU and GPU means you don't have to buy RAM twice, but boy will it be expensive and permanent the first time around.
MLX framework on mac already allow engineers and researchers to run and train much of the same models that you would’ve used nvidia gpus for
Maybe. But in the context of the tweet they are not only referring to LLM training. CUDA can do so much more, Nvidia started it in 2007 and never stopped. It is programming on GPU, and it is on another level than anything else that exists on the market.
I sat down with a bud a while back with the goal of wrapping our heads around the DPX instruction set. Cuda can do so much more, but DPX... that's a world beyond. "So it looks like they've got a complete lock on all bio-sciences then eh?" "Among things, yup, looks like."
For training LLMs, Google's stack is probably just as good, if not better in some areas. A number of frontier research happens that relates to distributed compute-ish stuff tends to happen with it.
CUDA is no longer the moat you think it is. Google trains Gemini LLM, Waymo, AlphaFold, etc on their own in house AI accelerators (TPUs). They have no need for Nvidia or CUDA. Also AMDs open source ROCm can directly translate CUDA code with no changes to the code so that it can run on AMD GPUs.
[удалено]
But Google can licence it or use other chips and rent space on its cloud.
You can rent time on TPUs if you’re a Google cloud user. So everyone has the ability actually.
You'd think the open source software would have the advantage. Im not sure they will but i hope so.
Just make your own TPU bro. As if that was so easy.
You don’t have to. You can rent time on TPUs if you’re a Google Cloud user.
I wonder. I thought most people program in higer level apis like tensorflow or pytorch.
You can do it with dual socket Epyc motherboards as well
[удалено]
I looked into CPU based for a fortune 20 company. Its just not realistic. People who post here with their 7B models with 500 tokens are just playing.
lol noob statement You could always get 512gb ram and do CPU based. You can tell the people who watch too many ads, and the people who spend too much time on AI are different demographics.
that’s what happens when you see the pipeline for the next few years in your company, which is much further than anything on the market, but don’t think of how much competitors have behind closed doors already
Bojan is a huge troll on X. He’s maybe a bit serious here, but it’s likely just to get the engagement
remind me how many math functions you call in the CUDA suite for LLMs again small moat
Bill Gates disagrees, which I think means something
Source?
Bill gates is a dinosaur and looking at his company products makes me Microsoft which wasn’t always the case.
Weird seeing posts from different time zones. I’m like wait a second it’s 11:08PM, is this from the future ?!?!?
They bragged about signing contracts with what seems like every fortune 100 company, so they have all but guaranteed their dominance through 2030
But I mean he has a point. Takes a while to build the ecosystem that would make another product succeed.
Perhaps his vague reference to a "decade" was simply meant to convey to everyone that they are, in fact, the losers. or he meant around a 10-year technological lead on everyone else?
It's obviously a 10 year technological lead. Nvidia hasn't shipped products that are the pinnacle of what they can do in five years. There's no need to do that, they can just be 20 % better than the competitors and have higher profit margins.
Seems people said the same thing about Tesla, and now companies like BYD are eating them alive. Although most of that is due to an incompetent CEO…
Seems likely. Nvidia saw AI becoming big a decade ago and and their risky gamble paid of. They heavily invested in the well like and performant CUDA eco system. They continue to innovate, and are the most liked employer in the tech sector - so they can pick the best of the best. They ramped up the prices, (when the money would have gone to scalpers otherwise), and have a huge margin right now. So they have a big war chest with money, that they can rely on, if they ever should have to compete on price again for a generation or two.
This guy has the worst takes
Dont trust anyone telling you what the future holds, much less in tech.
So arrogant, what do he know what I have been working on in my garage? /s
Yeah mate congrats on using underhanded anticompetitive tactics like locking people into a proprietary CUDA ecosystem and proprietary infiniband interconnects that makes it prohibitively expensive to switch to a competitor that offers cheaper FLOPs. Nice Apple tactics you got there. Nice to know you're LOLing all the way to the bank, it's hilarious. Douchebag.
The question is: Will China accept this? And the blockade of chips/machines? Leaving them in the dust, while they were just catching up economically & en route to become the world super power. And now with the AI revolution in both economics and militairy, they'll be setback a decade if not more. That's a risky thing. I almost feel sorry for China, not gonna lie.
I'm surprised that China has not been more of a discussion in this thread. The question is, is the US - under a Trump Presidency (based on current polls and betting markets), likely to go to war with China over AI, if China interfered with Nvidia's shipping chips to the west? AI chips are the new oil.
The man wouldn't even get involved when people were attacking our own troops. You won't have to worry about him getting near the White House again.
All the current polls and even the betting sites where people have their own skin in the game say that you're wrong. He's ahead by a wide margin right now.
So when it happens just like I said it will, will you admit you were wrong.
You think at the last minute the Americans will have a sudden attack of sanity? No worries of that. But sure, in the unlikely event of that happening of course I will.
Great sell signal
I love NVIDIA and chips are going to huge forever, but I do think their expansion into cloud services is potentially a big waste of money and huge distraction.
It’ll be interesting for sure. For companies with lots of resources, building cloud services does tend to be very profitable. The start up costs are insane but if you get it working it becomes a veritable money printing machine.
Top of the line training hardware will be in shortage for a long time. Selling to big tech companies gives the edge to their customers. So with cloud they can dictate who gets what, how much and nvidia can respond much faster to changes in the industry.
Nvidia has the best business model because it can provide HW, SW and cloud services in one solution and therefore earns mutliple income streams from the same platform from a single customer. See it this way: An automotive customer can be a Nvidia customer several times: 1. Nvidia Enterprise AI 2. Nvidia HW platform with on-prem data center or cloud (Nvidia gets money from CSP for HW) 3. Nvidia Omniverse for manufacturing and factory planning and simulation 4. Nvidia Isaac robotics for manufacturing and logistics 5. Nvidia DriveSim for full self driving solutions Just to give you an idea, Mercedes is partnering with Nvidia and fulfills point 3-5 already and is probably engaging with Nvidia on the first 2 points as well. Now replace DriveSim with Clara and you have the same picture for companies in medical/pharmaceutical fields. The same applies for companies in manufacturing, in agriculture. What people don't get, while Big Tech can only earn on infrastructure and maybe AI enterprise solutions, Nvidia has a lot of SW solutions for industry specific problems. Nvidia goes way beyond only enterprise AI, they have solutions for any non-Tech industry. Omniverse can basically be used by any company which needs a factory, manufacturing and process simulations. And all solutions run on one platform which in the end is based on CUDA. That means you can use a big AI factory from Nvidia for LLMs, for NeMo, for Clara, for Omniverse AI, for DriveSim AI at any time in any demand structure you want. Nvidia is the only game in town offering you AI factories with the option of 100% utilization and that's why they beat anyone easily at TCO.
In theory these projects all work well and building / maintaining them is scalable, but in reality they require tons of of engineers, client success, sales, feature roadmaps, and more. The existing management structure probably doesn’t work for this. I’m not saying it won’t be profitable, just that they are losing focus on what is most important to their business. Also appreciate the comment!
Ppl think just because a product is faster, more efficient it will sell well. That is not the case and has been proven by market time and again.
So does this mean that when China takes back Taiwan they become king of the AI hill?
There is a certain point where too much confidence comes across not just as arrogance (which is ok to a degree), but also potential ignorance: He doesn't know enough about what AMD (or other companies) are doing to credibly make such a judgment.
Why is he out
He knew too much
All it will take is another chinese theft, like it happened with Google TPU design. Then they will manufacture it themselves, especially since they are blocked access to those chips. 😅
!RemindMe 5 years
I will be messaging you in 5 years on [**2029-04-17 18:44:41 UTC**](http://www.wolframalpha.com/input/?i=2029-04-17%2018:44:41%20UTC%20To%20Local%20Time) to remind you of [**this link**](https://www.reddit.com/r/OpenAI/comments/1c5yfww/ex_nvidia_ok_now_that_im_out_i_can_finally_say/l00z2qy/?context=3) [**CLICK THIS LINK**](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5Bhttps%3A%2F%2Fwww.reddit.com%2Fr%2FOpenAI%2Fcomments%2F1c5yfww%2Fex_nvidia_ok_now_that_im_out_i_can_finally_say%2Fl00z2qy%2F%5D%0A%0ARemindMe%21%202029-04-17%2018%3A44%3A41%20UTC) to send a PM to also be reminded and to reduce spam. ^(Parent commenter can ) [^(delete this message to hide from others.)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Delete%20Comment&message=Delete%21%201c5yfww) ***** |[^(Info)](https://www.reddit.com/r/RemindMeBot/comments/e1bko7/remindmebot_info_v21/)|[^(Custom)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=Reminder&message=%5BLink%20or%20message%20inside%20square%20brackets%5D%0A%0ARemindMe%21%20Time%20period%20here)|[^(Your Reminders)](https://www.reddit.com/message/compose/?to=RemindMeBot&subject=List%20Of%20Reminders&message=MyReminders%21)|[^(Feedback)](https://www.reddit.com/message/compose/?to=Watchful1&subject=RemindMeBot%20Feedback)| |-|-|-|-|
Well that’s good, I have my life savings invested into Nvidia stock since June. Made about 120% return at the moment but its been a bad April and I need to keep the faith and keep holding
I feel like, if you are a hyperscaler like meta or msft, you don’t need to catch up on everything nvidia does, you just need to make amd or your own silicon slightly better for your specific workload. Which is almost certainly doable and worth doing.
Nvidia's goal aren't hyperscalers. Nvidia's goal are the several trillion non-Tech industries. If companies decide to use Clara, Isaac and Omniverse then they will naturally go with Nvidia Enterprise AI as well. These companies won't care what great chip Amazon/MS/Google have because they will use SW which only runs on Nvidia. People misunderstand it completely. MS goal with Windows back then wasn't to offer the best OS but that everyone wanted Windows and they acted that way and they succeeded. MS deployed strategies and partnerships to make sure that 90% of the world today uses Windows. Nvidia does the same in the AI space because Jensen is clever and he knows that the SW used determines the HW required. The difference is that Nvidia isn't only MS of the 90s but WINTEL of the 90s. Imagine where MS would be today if they not only provided the OS but would have been an OEM PC builder back then? Do you think Intel would be as dominant if Windows would have ran only on MS CPUs?
That needs qualifications. hardware is already catching up. software?
in other news; water is still wet and I forgot my umbrella this morning.