AI Data Centers For Dummies: Part 1

The GPU gets the headline. The building does the work.

This is the first in a series about the technologies behind AI data centers. Reading the International Energy Agency’s Key Questions on Energy and AI, I kept returning to one uncomfortable point: buying the computing equipment and getting a facility ready to use it are different challenges. A developer can move faster than the electricity infrastructure needed to serve the site, and putting gas turbines beside the building is not necessarily a quick escape. So I want to start somewhere other than the chip.

I think the more useful frame is permission to compute. Not a regulatory permit, although those matter too. I mean the practical permission that comes from having every supporting system ready when the processor needs it. Buying the most expensive component is not the same as building a working business.

That is how I want to examine AI data centers, including the Israeli companies supplying particular pieces. Some compete directly with global manufacturers. Others improve a specific part of a larger system. The interesting question is where each company solves a problem well enough to earn its place.

A data center is more than a building

NVIDIA distinguishes between training, which teaches a model by adjusting its internal parameters using data, and inference, which applies the trained model to new inputs to produce predictions or answers. They use related technology but do different jobs.

The distinction matters when designing the facility. NVIDIA's enterprise architecture guidance treats compute, networking, storage, software and security as a connected system, with configurations determined by the customer's workloads and available power and cooling. There is no single shopping list that makes every building an AI factory.

Electricity before intelligence

As Eaton illustrates, power arrives through a chain of equipment: transformers and switchgear for supply and distribution, uninterruptible power supply (UPS) systems to bridge interruptions and distribution equipment carrying electricity towards the racks. A processor is at the end of that chain, not outside it.

Eaton, Schneider Electric and Vertiv supply data center power infrastructure, while GE Vernova supplies generation configurations for primary, standby and transitional power needs through its gas turbines. They serve different points in the chain.

The IEA identifies another complication for AI: rapid changes in computing activity can produce rapid changes in electricity demand, making storage and controls relevant to how a facility interacts with its power source. The problem is not just securing enough electricity over a year. It is also about managing the spikes, peaks and valleys of compute and electrical demand.

Israel's Tower Semiconductor belongs in this discussion at the processor power delivery layer: its Gen3 power management technology targets the devices used to deliver power to AI processors. Its role is inside the power delivery chain, close to the processor.

Heating & cooling: more than standard HVAC

Cooling is an entire system, not one appliance beside a server. Vertiv's infrastructure portfolio includes air systems, chillers, coolant distribution units, manifolds and other equipment that moves heat from computing hardware towards the facility's heat rejection systems.

Much like the cooling system in your car, liquid cooling brings fluid close to hot components, but it does not mean every AI installation needs the same design: NVIDIA describes both air cooled enterprise configurations and denser systems suited to customers with different power and cooling capabilities.

ZutaCore, with its R&D in Israel, offers HyperCool, a system that removes heat directly from chips using a dielectric fluid that changes phase through boiling and condensation. Its role is at the chip cooling layer of that larger system.

There is another distinction worth keeping. Processors need more than processing power

As NVIDIA explains, GPUs, or graphics processing units, suit AI because they perform many calculations in parallel. But selecting the processor is only the start.

Now we can talk about the chips. NVIDIA supplies accelerated computing platforms, AMD supplies Instinct GPUs with its ROCm software and server manufacturers such as Dell integrate processors, networking and cooling into deployable systems. The operator has to make the hardware and software work together.

Memory matters because the accelerator needs data close to the processing units and available quickly. Micron, SK hynix and Samsung supply high bandwidth memory, while TSMC's CoWoS packaging connects logic chips and that memory within an advanced package. A fast processor cannot wish away a slow data path.

NeuReality, with offices in Caesarea, Tel Aviv and Krakow, addresses a different part of this problem: serving inference workloads. Its NR1 architecture works with external accelerators, including GPUs, and handles functions around the inference pipeline.

Here, the Israeli contribution sits around the accelerator: coordinating the work needed to serve inference requests. For an operator, the question is whether the complete system delivers more useful work from the hardware already installed.

Communication is more than networking

A cluster needs communication between its computing systems and access to the data those systems use; NVIDIA's design guidance explicitly includes both networking and storage paths, including retrieval and checkpoint traffic. I would not leave these paths as an afterthought in a capacity discussion.

There are different networks inside the network. NVIDIA uses NVLink for closely coupled GPU communication and offers InfiniBand and Ethernet platforms for connecting computing systems more widely. The point is to let processors exchange work without communication becoming the constraint.

Arista and Broadcom supply different layers of AI networking, from switches and management software to networking silicon and adapters. Corning supplies optical connectivity components, including fiber interfaces used with optical technologies close to networking chips.

Xsight Labs, headquartered in Kiryat Gat, develops switch silicon and data processing units for AI and large data center networks. Its role is at the semiconductor layer of the networking system.

Storage has its own supplier map. Dell offers PowerScale systems for AI data workflows, while WEKA and VAST supply data infrastructure platforms with different architectures and functions from their global operations in the US and Israel.

Someone has to run it. And protect it.

The software above the hardware determines who can use resources and when. NVIDIA Run:ai pools GPU capacity, schedules workloads and applies quotas and priorities across shared infrastructure. The Israeli developed platform is now offered under NVIDIA’s name. For the data center operator, the job is to allocate GPU capacity across competing workloads.

Orchestrating GPU workloads is a different job from monitoring the building. Schneider Electric's EcoStruxure IT addresses infrastructure visibility, capacity planning and asset management across equipment from multiple vendors. GPU scheduling and cooling alarms may meet in the operator's workflow, but they are not the same software market.

Security is another necessary layer. Palo Alto Networks supplies AI workload security, while Israel's Check Point offers network, host and application protections for AI infrastructure, including integration with NVIDIA BlueField data processing units. These are distinct security functions, so I would start with what needs protecting and where each product operates.

Physical access controls, fire protection and environmental monitoring remain basic requirements in IBM’s data center overview. AI does not give the operator permission to skip the ordinary work. Johnson Controls supplies facility security and fire protection systems for data centers.

Final Thoughts

In the Data Center, the GPU deserves attention, I just do not think it deserves all of it. For the rest of this series, I want to follow permission to compute through the systems that make useful work possible, then examine where Israeli suppliers compete directly and where they improve somebody else’s platform. For founders, the question is where the product fits and who must approve its deployment. For investors, it is whether solving that problem produces repeat business and cash.