We had $12K to spend on GPUs today. There Were None to Buy.

Today’s Reality

This is what trying to provision GPUs looks like today.

You open a provider dashboard expecting to spin up a serious training cluster. The budget is approved. The workload is defined. You are ready to pay. And everything you actually want is red.

Every B200 configuration is unavailable.
8x nodes, gone.
4x nodes, gone.
2x nodes, gone.
Even single-GPU B200 machines are gone.

There is some H100 capacity, but only in fragments. A few 1x nodes. Occasionally, a 4x or 8x if you are lucky and fast enough.

Older GPUs are scattered around the edges. A100s, A10s. Technically usable, operationally constraining.

This is not a planning exercise. This is not a hypothetical future state. This is what happens when you try to provision a cluster today.

Here is the math that makes this painful.

We planned for 8 next-generation GPUs with roughly 1.4TB of aggregate memory.

What we could actually secure was 8 H100 GPUs at 80GB each. Total available capacity: 640GB of VRAM. Less than half of what was architected.

This is not a performance hit. It is a feasibility constraint.

It forces real engineering tradeoffs. Batch sizes have to be redesigned. Model sharding strategies have to change. Training runs get longer. Some experiments are no longer viable at all.

And the most important detail is this:

A Quick Primer for Non-Technical Readers

GPUs, or Graphics Processing Units, are the hardware that powers modern AI.

Unlike traditional CPUs that execute a small number of sequential tasks, GPUs are designed to perform many calculations in parallel. That parallelism is exactly what large AI models require for training and inference.

They also include large amounts of high-bandwidth memory, commonly referred to as VRAM, which is necessary to hold and process the billions of parameters inside modern models.

NVIDIA is currently the dominant manufacturer in enterprise AI hardware. Most business leaders recognize the name, even if they have never had to think about what the hardware actually does.

Their current flagship enterprise AI chips include:

• H100, the workhorse of enterprise AI
• B200, the newest and most powerful generation

When teams talk about “8x B200” or “8x H100,” they are referring to a single server with eight of these GPUs working together as a unified compute system. These are enterprise-grade AI systems that quickly move into 6-figure territory once fully deployed.

Why We Run Our Own Infrastructure

Scarcity alone does not explain the decision to run controlled environments. We do not operate our own infrastructure because it is cheaper. It is not. We do it because of governance, privacy, and control.

When workloads run inside our environment, data can be encrypted and secured across the entire pipeline, including at the GPU layer. We control how data enters, how it is processed, and where it resides. That level of control is difficult to achieve with API-only providers or fully shared public services.

For enterprise clients, this is not theoretical. Data sovereignty requirements are explicit. Compliance requirements are contractual. In some cases, shared infrastructure simply does not meet regulatory obligations.

The tradeoff is straightforward. We do not own every piece of hardware. But when the system is running, we control the environment end-to-end. That control is the point. GPU scarcity is the exposure that comes with it.

The Economics of Enterprise GPU Infrastructure

It helps to ground this in something tangible and in financial terms.

We recently pulled a vendor quote for DGX H100 systems configured with 8x 80GB GPUs. 2 systems came in at just over $60,000 in base hardware before additional components, services, and tax were included. That is real. That is documentable. That is budgetable.

So the natural reaction is simple: if hardware requires capital commitment, use the cloud.

Cloud rental exists precisely for this reason. Published market analyses place H100 rental pricing in the single-digit dollar range per GPU-hour, depending on the provider, contract structure, and availability. On paper, that model is elegant. You avoid capital expense. You scale when needed. You shut it down when you are done.

But cloud pricing is not the constraint. Availability is.

When we checked real-time listings, capacity fluctuated dramatically. Certain GPU classes were unavailable. Cluster configurations disappeared. Nodes that existed one hour were gone the next.

The issue was not whether we were willing to pay. It was whether the infrastructure existed at that moment.

And that is where the conversation shifts.

The GPU Hunger Games

This is where the model breaks down.

GPUs are increasingly unavailable when you need them, even when the budget is approved, and pricing is flexible. The cluster you planned for may not exist. The specific GPU class required for your workload may not be available. In some cases, there is no cluster at all, only individual servers that must be stitched together to approximate the original architecture.

Teams end up constantly adjusting workloads on the fly. Operationally, this is punishing. Infrastructure planning becomes probabilistic rather than deterministic. Engineering teams are forced to adapt, often daily. Training runs stretch beyond expected timelines. Costs rise quietly through longer runtimes rather than higher hourly rates.

Some workloads simply cannot run at reduced capacity. They are not linearly scalable. They either fit the available infrastructure or they do not.

In practice, teams end up settling along a spectrum:

• Securing exactly what they need, though this is rare
• Accepting fewer nodes and adjusting parallelism
• Switching GPU classes and redesigning around memory and compute limits
• Stitching together individual servers instead of a unified cluster
• Waiting because nothing suitable is available

This is why it feels like a game, not because anyone is acting maliciously, but because demand and supply are structurally misaligned.

Why It Gets Worse

This situation is not stabilizing. It is accelerating.

On the demand side, enterprise AI adoption continues to expand. Many organizations are shifting beyond API experimentation toward more controlled, self-hosted environments for governance and privacy reasons. As that shift occurs, infrastructure requirements increase.

Workloads are also growing. Models are larger. Context windows are longer. Data volumes continue to expand. Inference demand is rising as AI features move into production systems. Training is no longer the only driver. Running models at scale is becoming a persistent operational load.

On the supply side, advanced GPU manufacturing remains complex and capacity-bound. Foundry output is finite. High-performance memory components are constrained. Datacenter power and cooling infrastructure cannot expand infinitely or instantly. New hardware generations enter the market gradually, while existing capacity is often heavily allocated.

The result is a structural imbalance. Demand is growing faster than deployable capacity, and the gap is widening rather than closing. This is not just an operational inconvenience. It is a structural infrastructure risk — one explored more deeply in The GPU Problem: AI Infrastructure Risk.

What This Means for Business Leaders

The key takeaway is simple:

Rigid infrastructure plans will fail in environments defined by supply volatility. Flexibility is not optional. The cloud is not infinite, and “on demand” does not mean always available.

Every serious AI initiative requires a degraded-mode plan. What happens if you receive 50% of the capacity you designed for? What breaks first? What can be delayed? What cannot?

Relationships matter. Reserved capacity matters. Diversification across providers matters. These are not engineering details. They are operational risk decisions.

Leadership teams should be asking clear questions. What is our GPU procurement strategy? What fallback plans exist if preferred hardware is unavailable? Are workloads designed to tolerate variable infrastructure? What is the cost of delay when GPUs cannot be provisioned?

These are not theoretical considerations. They lead directly to a broader executive conversation about infrastructure ownership, risk tolerance, and long-term control, a dynamic GPU question every CEO will be forced to answer.

This is the current state of the market.

Teams that understand this plan differently. Those that do not are surprised at the worst possible time.

The competition for compute is not theoretical. It is structural.