The short version: prices for AI training and inference compute are being bid UP in the near term, with the squeeze falling hardest on young AI startups, while big tech incumbents and the neoclouds selling capacity are cashing in. And of course Nvidia as well, whose chips from 2022 and later are seeing rental price bumps at the cloud and neocloud companies.
“The AI chip shortage is causing headaches for tech giants such as OpenAI. Among those hardest hit, though, are the startups trying to build their own AI models, so-called neolabs that need to use their limited funding to secure access to enormous clusters of graphics processing units for the compute-intensive task of model training.”
The ground level detail is striking.
Evan Morikawa of robotics AI startup Generalist canvassed some 17 AI cloud providers for compute this year. Last winter, providers were offering reasonable prices on contracts as short as one year. By May and June, prices had jumped, contract terms had stretched to 3 to 5 years, and clusters of the latest chips carried 12 to 18 month waits. “It’s like VC currency right now to know the current price of GPUs,” he told The Information. And: “Now 1-year contracts are almost impossible to get.”
“If your company is 1 year old, signing a 5 year contract makes no sense. The value of that 5 year contract is probably 3 times what you have raised, so you’re already underwater on paper.”
It gets more granular. A mid-size AI safety research organization talked to over 20 providers between March and July in search of just 60 chips. Three separate times, capacity that providers were advertising was sold to someone else within two days of the meeting. The org finally signed a deal worth several million dollars with Prime Intellect, reselling compute from data centers in India. An employee bought a cake to celebrate. “It’s a new landscape. We’re all adjusting.”
And at the bigger end of the neolab spectrum, Nvidia-backed Reflection AI is renting chips from SpaceXAI for $150 million a month, plus a multiyear Nebius deal totaling over $1 billion. That is a hefty chunk of the $2.5 billion the startup raised in its last round. Compute is the main use of new funds for these companies.
The number that matters most: the hourly rental price of an Nvidia H100 on a 1-year contract rose 50% from $1.73 in mid-December to $2.60 in mid-June, per Semianalysis. The H100 is a 2022 chip, TWO generations behind Nvidia’s latest. For on-demand H100s, Semianalysis often cannot quote a price at all in recent months. The column just reads ‘sold out’.
Even at the largest cloud vendors like Amazon AWS.
That datapoint cuts directly against the bear case argued by market shorts like Michael Burry, that AI GPUs are short-lived assets whose useful lives are overstated on the accounting books. The third-party market is voting the other way right now: four-year-old silicon is repricing UP, not down, as global demand for training and inference collide. To be fair to the bears, that is a shortage market talking, not a permanent re-rating. But the current trend lines are what they are. For the chip and cloud/neocloud vendors like CoreWeave in particular.
CoreWeave, which I profiled early as Nvidia’s rocket fuel propulsion of CoreWeave (RTZ #657), reported Tuesday evening: topped estimates, raised guidance, and another volatile week for the stock, this time to the upside, up 19% Wednesday. CEO Michael Intrator on the call:
“Our near-term capacity remains effectively sold out… [that] is translating into signed commitments on increasingly favorable terms from a broadening set of customers.”
Pricing and margins for its newest Blackwell and Vera Rubin chips are
“setting new highs, while pricing for prior-generation [chips] is at or above where it was years ago.” CoreWeave raised prices 25% in July, per CFO Nitin Agrawal: “We are more confident than ever in the long-term ROI of our product.”
Nebius, one of the ‘Neoclouds’ I flagged back in RTZ #558, grew revenue 454% to $582 million last quarter, and held its first ever capacity AUCTION, clearing Blackwell prices “15% above the highest price we ever charged before” per CEO Arkady Volozh. The stock soared 34%.
Even SpaceXAI, which pioneered the short-term premium rentals to Anthropic and Google I covered in part 1, jumped 9.7% Wednesday to $146.15, up 35% from last week. BNP Paribas notes Nebius’s short-term pricing is roughly equivalent to the economics of SpaceXAI’s recent deals. All of it under the wing of Kingmaker Nvidia, now ‘Uber AI Investor & Banker’ (AI-RTZ #1171).
“The problem with the large hyperscalers like AWS is availability and pricing. They don’t have anything, and if they do, the pricing is insane.”
Some 75% of Nebius’s business is startups that maxed out capacity at the big clouds. Consultants describe clients paying to run extra servers just in case: ‘a lot of defensive buying’.
The other side of the trade deserves stating. Duckbill’s chief cloud economist Corey Quinn asks the right question: “At some point, the supply shortage is going to resolve itself, at which point you can get the GPU that you need from any provider you care to name, why would you pick a neo-cloud?” And Peers notes the premiums are specifically for short-term, time-bounded needs. Multi-year deals are not paying them, and nobody pays a premium for 2029 capacity while the industry spends hundreds of billions expanding supply.
My take: this is the ground level confirmation of the view I have laid out in parts 1 and 2, and for months.
The collision between global training and inference demand is pushing AI Compute pricing UP in the short term, even for older chips in third-party markets, with the pain concentrated on the youngest AI startups and the pricing power concentrated in the neoclouds and their Kingmaker. Consumers are paying their own version of this bill via ‘RAMageddon’ (Google just added $100 to every Pixel 11). A few years out, as those hundreds of billions of data center buildout land as live capacity, I expect the pressure to temper.
That is the subject of the coming part 4 soon, back up at the macro level of the AI Compute economy: the supply and the demand together.
Here’s a preview of what we’ll cover this week: Macro: FED Rate Cuts Markets: Cycle Highs, Meltup, Meltdown, Mayhem Company Earnings: Consumer Staples...
The Bigger Picture, September 22, 2024 I’ve said before that we’re just in the early rush to build sufficient AI infrastructure in this AI Tech Wave . We...
Stocktoberfest is less than ONE month away. Come to Coronado, San Diego October 20-22 An intimate event for people that love to invest and trade and...
