116 Tech CapEx Spend Driver = Compute Spend to Train & Run Models To understand the evolution of AI computing economics, it’s constructive to look at where costs are concentrated – And where they’re headed. The bulk of spending in AI large language model (LLM) development is still dominated by compute – specifically, the compute needed to train and run models. Training costs remain extraordinarily high and are rising fast, often exceeding $100 million per model today. As Dario Amodei, CEO of Anthropic, noted in mid-2024, Right now, [AI model training costs] $100 million. There are models in training today that are more like a billion… I think that the training of…$10 billion models, yeah, could start sometime in 2025. Around these core compute costs sit additional high-cost layers: research, data acquisition and hosting, and a mix of salaries, general overhead, and go-to-market operations. Even as the cost to train models climbs, a growing share of total AI spend is shifting toward inference – the cost of running models at scale in real-time. Inference happens constantly, across billions of prompts, queries, and decisions, whereas model training is episodic. As Amazon CEO Andy Jassy noted in his April 2025 letter to shareholders, While model training still accounts for a large amount of the total AI spend, inference… will represent the overwhelming majority of future AI cost because customers train their models periodically but produce inferences constantly. NVIDIA Co-Founder & CEO Jensen Huang noted the same in NVIDIA’s FQ1:26 earnings call, saying Inference is exploding. Reasoning AI agents require orders of magnitude more compute. At scale, inference becomes a persistent cost center – one that grows in parallel with usage, despite declines in unit inference costs. The broader dynamic is clear: lower per-unit costs are fueling higher overall spend. As inference becomes cheaper, AI gets used more. And as AI gets used more, total infrastructure and compute demand rises – dragging costs up again. The result is a flywheel of growth that puts pressure on cloud providers, chipmakers, and enterprise IT budgets alike. The economics of AI are evolving quickly – but for now, they remain driven by heavy capital intensity, large-scale infrastructure, and a race to serve exponentially expanding usage.