part 4: The processor can’t do this alone, its needs a supporting cast
When I read Data Center Frontier's account of AMD's Helios system, the important point was the package: accelerators, CPUs, memory, networking and software designed around the rack. The discussion had moved beyond selecting a GPU. That is where I want to pick up this series.
Just like a great quarterback needs a great team so does a data center need a supporting cast of compute. The accelerator (GPU) gets the attention, but I want to understand what it needs to perform useful work. Permission to compute is not a chip specification. It is an operating arrangement.
Start with the job to be done
Training and inference tasks require different things of the system. NVIDIA distinguishes teaching a model through training from applying the trained model to new inputs during inference (these are the basic definitions btw). I would settle that workload discussion before starting a processor comparison.
There is plenty of overlap in the equipment offered. AMD positions Instinct GPUs for training, inference and fine tuning, supported by its ROCm programming tools, libraries and runtimes. That makes software compatibility part of the buying decision from the start.
The data center operator also has to choose a system configuration. NVIDIA's enterprise architecture guidance distinguishes air cooled RTX PRO designs from HGX and NVL72 options, according to the workload and available power and cooling. A facility's limits belong in the computing specification.
So how do you determine which hardware “setup” to run? My first questions would be practical. What model and software must run for optimal results? What response time or training schedule is required? Which existing applications and data sources must the system support? I would want those answers before debating a generation of silicon.
Memory has several jobs
The word memory can hide different requirements. Micron distinguishes memory as: high bandwidth memory (HBM) close to accelerators, DRAM for active system work and SSD storage for datasets, models and outputs. So you need each of these as part of your design discussion, so keep them separate from each other.
HBM addresses the movement of data close to the accelerator. SK hynix describes stacked DRAM dies providing capacity and throughput for that role, while Samsung and Micron also supply HBM products. So HBM is a manufacturing category in its own right.
Packaging connects the pieces. TSMC's CoWoS technologies use interposers and other interconnect structures to integrate logic chips and HBM within an advanced package. I would examine the package as part of the processor platform, rather than treat it as a box added afterwards.
Capacity, bandwidth and software support belong in the same analysis. I would be wary of a processor evaluation that presents a headline arithmetic figure but leaves the intended model, memory requirements and operating software unspecified, so make sure you have the full picture and a holistic view aimed towards the objective.
The server is part of the product
System manufacturers turn components into equipment the operator can deploy. Dell describes servers and rack systems integrating NVIDIA platforms, AMD CPUs, storage and cooling technology. That creates another set of engineering and support decisions.
I would ask who owns the tested configuration. If the software needs a particular driver or the cooling requires a particular server arrangement, who signs off on this combination? I want a deployment and support plan, not a collection of individually impressive components.
The software layer can also determine how the system is shared. NVIDIA Run:ai pools and schedules GPU resources across workloads, using quotas and priorities to manage access. We will return to that operational layer in my next post.
For this discussion, I would simply include it in the design budget. The value of additional processors depends on what the operator can actually run on them (meaning the type of workloads). I would ask about the queue as well as the equipment order.
Where the Israeli alternatives fit
NeuReality approaches inference as a pipeline, combining its NR1 AI CPU, a network engine and an AI Hypervisor that schedules work on accelerators. The interesting comparison is at the system level.
The architecture includes engines for processing before and after the neural network calculation, and NeuReality's modules can connect to external GPUs and other accelerator types. I would test it with the customer's complete request path, from arrival to returned result before any full scale deployment.
That suggests a specific diligence question. Does the installation produce more useful completed work with the intended models and traffic? I would want to see that result under a representative mix of requests, not only under a carefully chosen demonstration. Meaning that controlled demos are easy, but everyone has a plan until they get punched in the face or in this case, an unexpected workload spike.
NextSilicon offers another approach through Maverick, describing runtime optimization of application hot spots and support for languages including C++, FORTRAN and OpenMP. Its published specifications include HBM, which places it in a different part of the discussion from orchestration software.
The software boundary matters here too. NextSilicon lists CUDA, HIP/ROCm and major AI framework integrations as upcoming on its product page. I would separate the applications supported today from the integration roadmap when discussing a customer deployment.
These are different propositions within the Israeli compute ecosystem for AI focused data centers. I would assess NeuReality around the inference pipeline and NextSilicon around the applications its accelerator supports. A precise workload comparison is more useful than asking which company becomes “the next NVIDIA.”
Final Thoughts
The supporting cast of compute deserves a place in every processor discussion. My preference is to evaluate a tested workload on a complete system, including memory, software and the operational constraints around it. That is how I would compare an established global platform with a more specialized Israeli contribution.