This is the related benchmark blog from Redpanda [disclosure: I work for Redpanda and I helped write this. Credit to Travis Downs & others at Redpanda for the heavy lifting on the testing and analysis.]
Vera does what NVIDIA calls Spatial Multithreading, "physically partitioning each core’s resources rather than time slicing them, allowing the system to optimize for performance or density at runtime." A kind of static hyperthreading; you get two threads per core.
It's somewhat different from how x86 chips do simultaneous multithreading (SMT),
Seems like curious terminology from NV. In estabilished use, SMT means executing instructions from several cpu threads concurrently in the OOO CPU's execution units so they are not starved from work, whereas timeslicing conventionally means context switching between threads/processes, alternating temporally.
In operating systems timeslicing means giving a quantum of execution time to each process, and context switching between processes. Not normally a term used in computer architecture but possibly the characterisation would fit a barrer processor rather than SMT.
I own one of these systems. My interpretation is the Ampere systems are targeted at lower cost scale out. The Ampere Altra CPUs are limited to DDR4. The raw single core performance doesn’t match Intel or AMD offerings. You get a lot of cores for a lower hardware cost and at lower energy usage.
The Nvidia CPUs are designed for a very specific use case. They are designed for high performance with less concern about cost control.
The newer AmpereOne CPUs use DDR5 with the AmpereOne M supporting even higher memory bandwidth. Even then, I doubt the AmpereOne CPUs will match the performance of the Nvidia Rubin CPUs. But the Ampere processors are available for general use. I am guessing that Nvidia is only going to sell the complete rack system and only to high-volume customers.
But doesn't the Apple M series NPU support FP8, and as it's a monolithic die (except for the GPU in the M5 Pro and Max) it could be argued it has hardware FP8 support, no?
By that logic, on the M4 (which still has the GPU on the same die as the CPU), CPU cores have hardware accelerated raytracing, which is obviously nonsense.
Apple's hardware does not support FP8 (neither the ANE NPU, or the new "neural accelerator" tensor cores), though the most recent variant supports INT8.
If M5 has 9-18 cores and takes ~20w, then that's ~1-2w per CPU core. If these are 200-300W, and have ~100-200 CPU cores, then guess what? That's also ~1-2w per CPU core.
Xeons, Epycs, whatever this is - they are all also typically optimized for power efficiency. That's how they can fit so many CPU cores in 200-300W.
So does this cut out Intel/x86 from all the massive new datacenter buildouts entirely? They've already lost Apple as a customer and are not competitive in the consumer space. I don't see how they can realistically grow at all with x86.
Even Apple hardware looks inexpensive compared to Nvidia's huge premium. And never mind the order backlog.
x86 and Apple already sell CPUs with integrated memory and high bandwidth interconnects. And I bet eventually Intel's beancounter board will wake up and allow engineering to make one, too.
Worse, because since they no longer care about workstation market, there are pluggable cards, and no update to the chese grater, which the Studio is not comparable to.
They also dropped the ball on the data center, having left OS X Server behind.
Those markets are now served by Windows or Linux based configurations.
"And as the initial crop of Apple Intelligence features hasn’t been used as much as Apple expected"
Nah, as so-called "analysts" expected. The no-effort crybabies deriding Apple for being "behind on AI" have turned out to be, shocker of shockers, wrong. Anyone who even put a few minutes of thought into Apple's business realized that it (and its customers) didn't stand to benefit much from "AI."
It's sad that Apple hurried to pander to these clowns, only to be derided further... and to encounter the appropriate apathy from customers, who were and are doing just fine without asinine "AI" gimmicks.
Apple wouldn't have built the server capacity if they thought it wouldn't be used. It's indeed their own analysis.
In any case, that article is also looking forward to next-gen models like the sparse Gemini model Google trained for Siri. Apple Silicon simply isn't powerful enough to compete for that inference.
AFAIK they still dominate on clock rate, which I was surprised to see when doing some back of the envelope calculations regarding core counts.
I felt my 8 core i9 9900K was inadequate, so shopped around for something AMD, and IIRC the core multiplier of the chip I found was dominated by the clock rate multiplier so it’s possible that at full utilization my i9 is still towards the best I can get at the price.
Not sure if I’m the typical consumer in this case however.
Your 9900k at 5ghz does work slower than a Ryzen 9800X3D at 5ghz. A lot slower (1700 single core geekbench vs 3300, and just about any benchmark will tell the same story). Clock speed alone doesn't mean anything.
>8 Cores and 16 processing threads, based on AMD "Zen 5" architecture
which is the same thread geometry as my 9900K.
My main concerns at the time were:
1. More cores for running large workloads on k8s since I had just upgraded to 128G RAM
2. More thread level parallelism for my C++ code
Naively I thought that, ceteris paribus and assuming good L1 cache utilization, having more physical cores with a higher clock rate would be the ticket for 2.
Does the 9800X3D have a wider pipeline or is it some other microarchitectural feature that makes it faster?
You don't even need to go into the pipeline details. The 9800X3D has 8x more L2 cache, 6x more L3 cache, 2x the memory bandwidth than the now 8 years old i9 9900K. 3D V-cache is pretty cool.
I purposely picked a CPU with the same thread geometry as your 9900K to avoid calls of "apples & oranges" or whatever. If you want more threads, the 9950X is right there in the same socket. Or Core Ultra 9 285k. Either of which will run circles around a 9900K in code compilation.
I think my i9 was released right after the Spectre and Meltdown mitigations in 2019, but I seem to remember even more recent vulns in that family… so that could also be a factor.
I replied to the sibling comment: I was making simplifying assumptions for two specific use cases and naively treated physical cores and clock rate as my variables.
Ahhh, so is this a chip "more optimised" for connecting GPU's to reality ... or are they skipping the GPU step entirely? Are GPU's only for training now?
Is this an ASIC? Or FPGA? Or something even more exotic?
I’m guessing it’s some form of ASIC because I can’t imagine crafting the logic of Llama on silicon is a very quick or easy job. Not that doing it on an ASIC is a piece of cake either.
"Taalas is borrowing some ideas from the structured ASICs of the early 2000s to make its hardwired model-specific chips. Structured ASICs used gate arrays and hardened IP blocks, changing only the interconnect layers to adapt the chip to a specific workload. At the time, this was seen as a more cost-effective alternative to a full-custom ASIC that was more performant than an FPGA."
"Taalas changes only two masks to customize a chip for a specific model, but the two masks can change both model weights and dataflow through the chip. On the HC1, the model and its weights are stored on the chip using a mask-ROM-based recall fabric paired with a (programmable) SRAM, which can be used to hold fine-tuned weights and/or the KV cache. Future generations of chips may split the SRAM onto a separate chip, meaning they could be denser than the HC1."
The problem is not that gaming GPUs are in demand, it’s that selling silicon to AI center buildouts is so absurdly profitable right now they just aren’t making many gaming GPUs.
If you can only get so many mm^2 of dies from TSMC, might as well make 50x selling to AI providers.
I'm assuming this is for tool call and orchestration. I didn't know we needed higher exploitable parallelism from the hardware, we had software bottlenecks (you're not running 10,000 agents concurrently or downstream tool calls)
Can someone explain what is Vera CPU doing that a traditional CPU doesn't?
But at what stage are we asking for that RAM? if it's the inference stage then doesn't that belong to the GPU<>Memory which has nothing to do with the CPU?
I did see they have the unified CPU/GPU memory which may reduce the cost of host/kernel transactions especially now that we're probably lifting more and more memory with longer context tasks.
Given the price of these systems the ridiculously expensive network cards isn't such a huge huge deal, but I can't help but wonder at the absurdly amazing bandwidth hanging off Vera, the amazing brags about "7x more bandwidth than pcie gen 6" (amazing), but then having to go to pcie to network to chat with anyone else. It might be 800Gbe but it's still so many hops, pcie is weighty.
I keep expecting we see fabric gains, see something where the host chip has a better way to talk to other host chips.
It's hard to deny the advantages of central switching as something easy & effective to build, but reciprocally the amazing high radix systems Google has been building have just been amazing. Microsoft Mia 200 did a gobsmacking amount of Ethernet on chip 2.8Tbps, but it's still feels so little, like such a bare start. For reference pcie6 x16 is a bit shy of 1Tbps, vaguely ~45 ish lanes of that.
It will be interesting to see what other bandwidth massive workloads evolve over time. Or if this throughout era all really ends up serving AI alone. Hoping CXL or someone else slims down the overhead and latency of attachment, soon-ish.
> It might be 800Gbe but it's still so many hops, pcie is weighty.
Once you need to reach beyond L2/L3 it is often the case that perfectly viable experiments cannot be executed in reasonable timeframes anymore. The current machine learning paradigm isn't that latency sensitive, but there are other paradigms that can't be parallelized in the same way and are very sensitive to latency.
Most of the big AI/HPC clusters these systems are aimed at aren’t running regular PCIe Ethernet between nodes, they’re usually wired up with InfiniBand fabrics (HDR/NDR now, XDR soon)
Are we rapidly careening towards a world where _only_ AI “computing” is possible?
Wanted to do general purpose stuff? Too bad, we watched the price of everything up, and then started producing only chips designed to run “ai” workloads.
Oh you wanted a local machine? Too bad, we priced you out, but you can rent time with an ai!
Feels like another ratchet on the “war on general purpose computing” but from a rather different direction.
From the "fridge purpose-built for storing only yellow tomatoes" and "car only built for people whose last name contains the letter W" series.
When can this insanity end? It is a completely normal garden-variety ARM SoC, it'll run Linux, same as every other ARM SoC does. It is as related to "Agentic $whatever" as your toaster is related to it
> It is as related to "Agentic $whatever" as your toaster is related to it
These things have hardware FP8 support, and a 1.8TB/s full mesh interconnect between CPUs and GPUs. We can argue about the "agentic" bit, but those are features that don't really matter for any workload other than AI.
Don't think they would. Games aren't nearly as hungry for memory bandwidth as LLMs are. Also, I expect that the VRAM/GPU/CPU balance would be completely out of whack. Something would be twiddling its thumbs waiting for the rest of the hardware.
mem bw between cores matters for .... literally all workloads that are not single-core (read: all). And FP8 matters not at all cause inference on cpu is too slow to be of any use whatsoever in the days of proper accelerators
What the heck is agentic inference and how is it supposed to be different from LLM inference? That's a rhetorical question. Screw marketing and screw hype.
Agentic AI CPU? No.
It’s a CPU designed for an AI cluster. Their last CPU Grace was the same thing and no one called it agentic.
Vera now just has more performance/more bandwidth. It’s cool, I’d like to have one of these clusters, but this is not new.
It’s marketed as agentic AI because that’s fashionable in 2026.
They significantly lowered latency compared to EPYC/Xeon, which is critical for streaming agents (e.g. text/audio/video agents).
What latency? How much is it compared to LLM inference speed?
See the Redpanda comment/link here.
This is the related benchmark blog from Redpanda [disclosure: I work for Redpanda and I helped write this. Credit to Travis Downs & others at Redpanda for the heavy lifting on the testing and analysis.]
https://www.redpanda.com/blog/nvidia-vera-cpu-performance-be...
It is a 88-core ARM v9 chip, for somewhat more detailed spec.
Vera does what NVIDIA calls Spatial Multithreading, "physically partitioning each core’s resources rather than time slicing them, allowing the system to optimize for performance or density at runtime." A kind of static hyperthreading; you get two threads per core.
It's somewhat different from how x86 chips do simultaneous multithreading (SMT),
Seems like curious terminology from NV. In estabilished use, SMT means executing instructions from several cpu threads concurrently in the OOO CPU's execution units so they are not starved from work, whereas timeslicing conventionally means context switching between threads/processes, alternating temporally.
In operating systems timeslicing means giving a quantum of execution time to each process, and context switching between processes. Not normally a term used in computer architecture but possibly the characterisation would fit a barrer processor rather than SMT.
Hmm, the 128-core Ampere Altra CPU is already available, and in a case from System76. I wonder what else differentiates it.
If they're going to build CPUs I wish they had used Risc-V instead. They are using it somewhat already.
You can see here[1] what the specs are for the CPU (listed as "NVIDIA Vera Rubin Superchip").
The CPU is integrated with two Rubin GPUs but each of the CPU cores has dedicated FP8 acceleration as well.
1. https://www.nvidia.com/en-us/data-center/vera-rubin-nvl72/
I own one of these systems. My interpretation is the Ampere systems are targeted at lower cost scale out. The Ampere Altra CPUs are limited to DDR4. The raw single core performance doesn’t match Intel or AMD offerings. You get a lot of cores for a lower hardware cost and at lower energy usage.
The Nvidia CPUs are designed for a very specific use case. They are designed for high performance with less concern about cost control.
The newer AmpereOne CPUs use DDR5 with the AmpereOne M supporting even higher memory bandwidth. Even then, I doubt the AmpereOne CPUs will match the performance of the Nvidia Rubin CPUs. But the Ampere processors are available for general use. I am guessing that Nvidia is only going to sell the complete rack system and only to high-volume customers.
Say what you want about NVIDIA (to me they are just doing what every company would do in their place), but they create engineering marvels.
Anyone know how this compares to Apple’s M5 chips? Or is that comparison <takes off sunglasses> apples to oranges.
Features like hardware FP8 support definitely make it apples-to-oranges.
But doesn't the Apple M series NPU support FP8, and as it's a monolithic die (except for the GPU in the M5 Pro and Max) it could be argued it has hardware FP8 support, no?
By that logic, on the M4 (which still has the GPU on the same die as the CPU), CPU cores have hardware accelerated raytracing, which is obviously nonsense.
I thought the M5 had FP16 support, and not FP8.
Apple's hardware does not support FP8 (neither the ANE NPU, or the new "neural accelerator" tensor cores), though the most recent variant supports INT8.
It doesn't matter, because you will never find M5 chips on cloud offerings, or server racks.
It is kind of rediculous that the only server option with Apple hardware has been to stack up mac minis.
They got rid of the server and workstation market, focusing on consumers only.
Grace GB10, Vera's predecessor, had a single core performance comparable to M3 so I guess we can expect at least M4 level performance now.
Isn't the GB10 a Mediatek chip and not directly related to the Grace datacenter CPU?
More fair to say it's completely unrelated to the Grace data center CPU.
The DGX Spark (and the white box variants of it) run on the Grace Blackwell GB10 "superchip".
M5 are 9-18 cores and optimized for power-efficiency, those are more like Xeons, with 200-300W TDP, I'd bet.
If M5 has 9-18 cores and takes ~20w, then that's ~1-2w per CPU core. If these are 200-300W, and have ~100-200 CPU cores, then guess what? That's also ~1-2w per CPU core.
Xeons, Epycs, whatever this is - they are all also typically optimized for power efficiency. That's how they can fit so many CPU cores in 200-300W.
So does this cut out Intel/x86 from all the massive new datacenter buildouts entirely? They've already lost Apple as a customer and are not competitive in the consumer space. I don't see how they can realistically grow at all with x86.
Even Apple hardware looks inexpensive compared to Nvidia's huge premium. And never mind the order backlog.
x86 and Apple already sell CPUs with integrated memory and high bandwidth interconnects. And I bet eventually Intel's beancounter board will wake up and allow engineering to make one, too.
But competition is good for the market.
Apple went from a high-end PC to a low-end AI provider due to blocking Nvidia on their platform.
Worse, because since they no longer care about workstation market, there are pluggable cards, and no update to the chese grater, which the Studio is not comparable to.
They also dropped the ball on the data center, having left OS X Server behind.
Those markets are now served by Windows or Linux based configurations.
Even with those advantages, Apple can't even sell datacenter hardware to themselves: https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep...
"And as the initial crop of Apple Intelligence features hasn’t been used as much as Apple expected"
Nah, as so-called "analysts" expected. The no-effort crybabies deriding Apple for being "behind on AI" have turned out to be, shocker of shockers, wrong. Anyone who even put a few minutes of thought into Apple's business realized that it (and its customers) didn't stand to benefit much from "AI."
It's sad that Apple hurried to pander to these clowns, only to be derided further... and to encounter the appropriate apathy from customers, who were and are doing just fine without asinine "AI" gimmicks.
Apple wouldn't have built the server capacity if they thought it wouldn't be used. It's indeed their own analysis.
In any case, that article is also looking forward to next-gen models like the sparse Gemini model Google trained for Siri. Apple Silicon simply isn't powerful enough to compete for that inference.
>are not competitive in the consumer space
AFAIK they still dominate on clock rate, which I was surprised to see when doing some back of the envelope calculations regarding core counts.
I felt my 8 core i9 9900K was inadequate, so shopped around for something AMD, and IIRC the core multiplier of the chip I found was dominated by the clock rate multiplier so it’s possible that at full utilization my i9 is still towards the best I can get at the price.
Not sure if I’m the typical consumer in this case however.
Your 9900k at 5ghz does work slower than a Ryzen 9800X3D at 5ghz. A lot slower (1700 single core geekbench vs 3300, and just about any benchmark will tell the same story). Clock speed alone doesn't mean anything.
From the newegg listing:
>8 Cores and 16 processing threads, based on AMD "Zen 5" architecture
which is the same thread geometry as my 9900K.
My main concerns at the time were:
1. More cores for running large workloads on k8s since I had just upgraded to 128G RAM
2. More thread level parallelism for my C++ code
Naively I thought that, ceteris paribus and assuming good L1 cache utilization, having more physical cores with a higher clock rate would be the ticket for 2.
Does the 9800X3D have a wider pipeline or is it some other microarchitectural feature that makes it faster?
Comparing CPUs by clock speed doesn’t work. New CPUs are do more work per clock cycle.
A 9800X3D is twice as fast as your 9900K in benchmarks like GeekBench, despite having similar clock speed and the same core count.
If you could downclock the AMD part to 2.5GHz as an experiment it would still beat your 5GHz 9900K.
You don't even need to go into the pipeline details. The 9800X3D has 8x more L2 cache, 6x more L3 cache, 2x the memory bandwidth than the now 8 years old i9 9900K. 3D V-cache is pretty cool.
I purposely picked a CPU with the same thread geometry as your 9900K to avoid calls of "apples & oranges" or whatever. If you want more threads, the 9950X is right there in the same socket. Or Core Ultra 9 285k. Either of which will run circles around a 9900K in code compilation.
You can research microarchitecture differences if you want, it's a fascinating world, or you can just skip to looking at benchmarks/reviews. Little hard to compare against quite that large of a generation gap, but eg https://gamersnexus.net/cpus/rip-intel-amd-ryzen-7-9800x3d-c... or https://www.phoronix.com/review/amd-ryzen-7-9800x3d-linux/2
The 9800X3D has wider everything. Decoder, execution ports, vectors, cache, memory bandwidth...
I think my i9 was released right after the Spectre and Meltdown mitigations in 2019, but I seem to remember even more recent vulns in that family… so that could also be a factor.
A 9700X is twice the performance of a 9900K and M5 Max is almost 3X the performance. The megahertz myth is a myth.
I replied to the sibling comment: I was making simplifying assumptions for two specific use cases and naively treated physical cores and clock rate as my variables.
But why? That's like trying to determine which car is faster by looking at only at the rpm.
Yes, but core count and clock speed of a nearly 10 year old CPU are meaningless when comparing to current processors.
Ahhh, so is this a chip "more optimised" for connecting GPU's to reality ... or are they skipping the GPU step entirely? Are GPU's only for training now?
have you seen this: https://chatjimmy.ai/
It's quite impressive what purpose build inference can/will do once everyone stops trying to become kind of the best model.
Wow impressive. What's the story with this?
It's a tech demonstrator for a company that turns models into custom silicon for fast inference. In this case llama3.1-8b https://taalas.com/products/
Is this an ASIC? Or FPGA? Or something even more exotic?
I’m guessing it’s some form of ASIC because I can’t imagine crafting the logic of Llama on silicon is a very quick or easy job. Not that doing it on an ASIC is a piece of cake either.
An ASIC is custom silicon, no?
Anyways, I found this article discussing it a bit more: https://www.eetimes.com/taalas-specializes-to-extremes-for-e...
"Taalas is borrowing some ideas from the structured ASICs of the early 2000s to make its hardwired model-specific chips. Structured ASICs used gate arrays and hardened IP blocks, changing only the interconnect layers to adapt the chip to a specific workload. At the time, this was seen as a more cost-effective alternative to a full-custom ASIC that was more performant than an FPGA."
"Taalas changes only two masks to customize a chip for a specific model, but the two masks can change both model weights and dataflow through the chip. On the HC1, the model and its weights are stored on the chip using a mask-ROM-based recall fabric paired with a (programmable) SRAM, which can be used to hold fine-tuned weights and/or the KV cache. Future generations of chips may split the SRAM onto a separate chip, meaning they could be denser than the HC1."
Taalas hardware implementation of Llama 3.1 8B They claim 16k tok/s vs Cerbras at 2k. https://taalas.com/products/
"democratize access to AI and accelerating innovation."
So they make inference cheaper and the models get even worse. Or Jensen Huang has AI psychosis. Or both.
Here is a new business idea for Nvidia: Give me $3000 in a circular deal which I will then spend on a graphics card.
Me too plz. To quote (more or less) Harvey Pekar: “I’m trying to sell out, but nobody’s buying!”
This is yet not the grok acquisition, so there is another update coming with that claiming more improvements?
https://developer.nvidia.com/blog/inside-nvidia-groq-3-lpx-t...
Does this mean their gaming GPUs are becoming less in demand, and therefore cheaper/more available again?
Absolutely not, unfortunately.
The problem is not that gaming GPUs are in demand, it’s that selling silicon to AI center buildouts is so absurdly profitable right now they just aren’t making many gaming GPUs.
If you can only get so many mm^2 of dies from TSMC, might as well make 50x selling to AI providers.
Check the GTC 2026 agenda, there are hardly any graphics programming talks.
At least there are a few cool ones about programming CUDA directly in Python.
Yep, it's a sad time for the gaming world.
It means it will be profitable to mine crypto again
No.
I'm assuming this is for tool call and orchestration. I didn't know we needed higher exploitable parallelism from the hardware, we had software bottlenecks (you're not running 10,000 agents concurrently or downstream tool calls)
Can someone explain what is Vera CPU doing that a traditional CPU doesn't?
> you're not running 10,000 agents concurrently or downstream tool calls
Cursor seem to be doing exactly that though
Lots and lots of CPUs pooled. Faster more efficient power RAM accessible to both GPU and CPU. IIUC.
But at what stage are we asking for that RAM? if it's the inference stage then doesn't that belong to the GPU<>Memory which has nothing to do with the CPU?
I did see they have the unified CPU/GPU memory which may reduce the cost of host/kernel transactions especially now that we're probably lifting more and more memory with longer context tasks.
Given the price of these systems the ridiculously expensive network cards isn't such a huge huge deal, but I can't help but wonder at the absurdly amazing bandwidth hanging off Vera, the amazing brags about "7x more bandwidth than pcie gen 6" (amazing), but then having to go to pcie to network to chat with anyone else. It might be 800Gbe but it's still so many hops, pcie is weighty.
I keep expecting we see fabric gains, see something where the host chip has a better way to talk to other host chips.
It's hard to deny the advantages of central switching as something easy & effective to build, but reciprocally the amazing high radix systems Google has been building have just been amazing. Microsoft Mia 200 did a gobsmacking amount of Ethernet on chip 2.8Tbps, but it's still feels so little, like such a bare start. For reference pcie6 x16 is a bit shy of 1Tbps, vaguely ~45 ish lanes of that.
It will be interesting to see what other bandwidth massive workloads evolve over time. Or if this throughout era all really ends up serving AI alone. Hoping CXL or someone else slims down the overhead and latency of attachment, soon-ish.
Maia 200: https://www.techpowerup.com/345639/microsoft-introduces-its-...
> It might be 800Gbe but it's still so many hops, pcie is weighty.
Once you need to reach beyond L2/L3 it is often the case that perfectly viable experiments cannot be executed in reasonable timeframes anymore. The current machine learning paradigm isn't that latency sensitive, but there are other paradigms that can't be parallelized in the same way and are very sensitive to latency.
Most of the big AI/HPC clusters these systems are aimed at aren’t running regular PCIe Ethernet between nodes, they’re usually wired up with InfiniBand fabrics (HDR/NDR now, XDR soon)
Infiniband cards are connected to the rest of the machine with the PCIe as well
Am I crazy, or is Jensen's statement a copy-paste from ChatGPT?
(Could be both)
If AI is so great why should he not use it?
Should work on building the AI Jensen. Maybe it's already the AI Jensen
Are we rapidly careening towards a world where _only_ AI “computing” is possible?
Wanted to do general purpose stuff? Too bad, we watched the price of everything up, and then started producing only chips designed to run “ai” workloads.
Oh you wanted a local machine? Too bad, we priced you out, but you can rent time with an ai!
Feels like another ratchet on the “war on general purpose computing” but from a rather different direction.
> Purpose-Built for Agentic AI
From the "fridge purpose-built for storing only yellow tomatoes" and "car only built for people whose last name contains the letter W" series.
When can this insanity end? It is a completely normal garden-variety ARM SoC, it'll run Linux, same as every other ARM SoC does. It is as related to "Agentic $whatever" as your toaster is related to it
> It is as related to "Agentic $whatever" as your toaster is related to it
These things have hardware FP8 support, and a 1.8TB/s full mesh interconnect between CPUs and GPUs. We can argue about the "agentic" bit, but those are features that don't really matter for any workload other than AI.
The huge interconnect would also useful be for HPC tasks. The FP8 not so much, HPC still loves FP64.
Would cloud gaming platforms benefit from the interconnect?
Don't think they would. Games aren't nearly as hungry for memory bandwidth as LLMs are. Also, I expect that the VRAM/GPU/CPU balance would be completely out of whack. Something would be twiddling its thumbs waiting for the rest of the hardware.
mem bw between cores matters for .... literally all workloads that are not single-core (read: all). And FP8 matters not at all cause inference on cpu is too slow to be of any use whatsoever in the days of proper accelerators
The power and importance of marketing is deeply underappreciated by us technical types.
And yet more than a little Gavin Belson "Box III" vibes here. Fortunately, no signature edition.
I don’t underappreciate it, but I do despise it.
> It is a completely normal garden-variety ARM SoC
To mis-quote the politician quip:
How can you tell a marketer is lying?
Answer: His/her mouth is moving.
The World's First Central Sloppressing Unit
what a bizarre title
They should've called it Vega: https://doom.fandom.com/wiki/VEGA
Perhaps, but consider the existence of the AMD Vega GPU line https://en.wikipedia.org/wiki/Radeon_RX_Vega_series
A GPU purpose-built for Slop.
What the heck is agentic inference and how is it supposed to be different from LLM inference? That's a rhetorical question. Screw marketing and screw hype.
Who wants general computing anyways?
China will beat this....
Seems like a triumph of hype over reality.
China can do breathless hype just as well as Nvidia.