Semiconductors & Advanced Manufacturing

The biggest story in chips right now isn't a new chip — it's everything a chip needs to keep running. AI accelerators are generating heat and consuming power at levels that are fundamentally breaking the infrastructure data centers were built around. This week's coverage from Data Center Dynamics makes clear the bottlenecks have shifted: the question is no longer whether companies can design or manufacture powerful AI chips, but whether the physical world can keep up with them.

The Thermal Wall: AI Chips Are Melting the Old Playbook

Modern AI accelerators — the GPUs and custom chips that power large language models — run at power densities that would have seemed absurd a decade ago. A rack of traditional servers might draw 5–10 kilowatts of power and produce proportional heat. A dense AI training rack today can exceed 50–100 kilowatts, generating heat that standard air-cooling simply cannot remove fast enough.

Vertiv's Maurizio Frizziero (Vertiv is one of the dominant suppliers of power and cooling infrastructure to data centers globally) told Data Center Dynamics this week that rising chip temperatures are forcing a wholesale rethink of cooling architecture. The debate has moved past whether to adopt liquid cooling and onto which liquid cooling architecture. The current front line: in-rack CDUs (Coolant Distribution Units — think plumbing that circulates chilled liquid directly inside each server rack, right next to the hot components) versus floor-mounted CDUs (larger, centralized units that serve multiple racks from a distance). Neither is universally better; the right answer depends on the specific chip mix, rack density, and whether the facility was built from scratch for AI or retrofitted.

This matters beyond the data center industry because it's a cost multiplier. Liquid cooling infrastructure is expensive to install and operate, and it raises the total cost of running AI workloads — costs that eventually flow through to AI service pricing and to the economics of companies building their own AI infrastructure.

Power Is Becoming the Binding Constraint

Building compute capacity for AI at scale is hitting a power wall. Three stories this week illustrate how serious this is becoming:

  • Nuclear exploration: Riot Platforms (best known as a Bitcoin miner that pivoted into AI infrastructure) signed a memorandum of understanding with Terrestrial Energy to explore co-locating data centers next to SMRs — Small Modular Reactors, a new generation of compact nuclear plants that proponents argue can be sited more flexibly than traditional large-scale nuclear. The appeal is obvious: a dedicated, always-on power source that doesn't depend on a stressed grid. No sites have been committed; this is still exploratory.
  • Coal country opens its doors: Wyoming governor Mark Gordon is actively courting hyperscalers (the term for the largest cloud providers — Amazon Web Services, Microsoft Azure, Google Cloud, and a handful of others at comparable scale). Wyoming's pitch is cheap land and access to substantial existing power infrastructure, even if that power comes primarily from coal. Data center developers are not being particularly choosy about carbon sources when the alternative is waiting years for grid interconnection elsewhere.
  • Power kills a flagship project: Microsoft and G42 (a major UAE-based AI investment firm) have a data center in Kenya that has been stalled since its 2024 announcement — not by regulatory issues or geopolitics, but simply because there isn't enough power capacity available. This is a recurring story across emerging markets: the demand for AI infrastructure exists, but grid buildout lags by years.

The throughline is that chip performance has outpaced energy infrastructure. The semiconductor industry can design a faster accelerator in a few years; building new power generation takes a decade.

AMD Takes Another Swing at Nvidia's Crown

AMD this week debuted the MI350 PCIe card, its latest AI accelerator targeting enterprise workloads. The PCIe format (Peripheral Component Interconnect Express — the standard slot that connects add-in cards to a server's motherboard) is significant: it's easier to retrofit into existing server infrastructure than AMD's higher-end rack-scale systems, making it a more accessible entry point for enterprises that want AI acceleration without buying entirely new hardware.

AMD has been the credible challenger to Nvidia's dominance in AI accelerators — Nvidia holds an estimated 70–80%+ of the AI chip market — but "credible challenger" has not yet translated into meaningful market share erosion. The MI350 is aimed at the enterprise segment where Nvidia's grip is somewhat less total than in hyperscale AI training. Whether it gains traction will depend on software support: Nvidia's CUDA ecosystem (the programming framework that developers use to write code that runs on Nvidia GPUs) has a decade-plus head start that hardware specs alone can't overcome.

The Global AI Infrastructure Buildout Keeps Spreading

Data center construction activity this week spans São Paulo, Bangkok, Wyoming, Pakistan, and France (where TotalEnergies, Nvidia, and Dell are building a supercomputer in Pau for seismic imaging — geophysics being one of the less-discussed but computationally intensive applications of AI). Separately, QuantumDiamonds, a startup specializing in testing quantum chips using diamond-based sensors, installed its second QDm.1 system in Taiwan in as many months — a small but telling data point about the emerging quantum computing supply chain taking root in the same geography as conventional advanced chip manufacturing.

The Neocloud buildout is also worth watching. Neoclouds — a new category of cloud provider that specializes specifically in renting GPU capacity for AI workloads, distinct from generalist hyperscalers like AWS — are, per Data Center Dynamics' analysis this week, reshaping how AI compute is provisioned. They move faster than hyperscalers on GPU deployment, compete aggressively on price, and are drawing workloads away from the established players. The competitive pressure this creates for AWS and Azure is real, and it's accelerating the overall buildout of AI-optimized infrastructure.

The Trend to Watch

The semiconductor industry is deep into a phase where chip design has outrun everything else. Fabs (semiconductor manufacturing plants) can produce more powerful processors, but data centers can't always cool them, grids can't always power them, and in some markets, the basic infrastructure to host them doesn't exist yet. The next 2–3 years will see enormous capital flowing into solving these adjacent problems — liquid cooling, SMRs, grid interconnection, subsea cables. The companies positioned at the intersection of chip performance and physical infrastructure are the ones to follow.

TL;DR - AI chips now generate so much heat that liquid cooling is becoming mandatory infrastructure, not optional — the industry is debating which liquid cooling architecture, not whether to adopt it - Power is emerging as the single biggest bottleneck to AI infrastructure growth: companies are exploring nuclear reactors, building in coal country, and watching projects stall entirely for lack of grid capacity - AMD launched a new AI accelerator (MI350) targeting enterprises, but Nvidia's software ecosystem advantage remains the harder obstacle to overcome than any hardware gap - The AI data center buildout is now a genuinely global phenomenon — from Pakistan to Thailand to Wyoming — with neoclouds (GPU-specialist cloud providers) accelerating competition against the traditional hyperscalers
Compiled from 1 source · 20 items
  • Data Center Dynamics (20)