Your laptop’s memory has a job even when it isn’t working: being ready when you are. Sam Altman, CEO of OpenAI, the company behind ChatGPT, seems concerned about its work ethic.
Altman made the case during an Oct. 29 appearance on Cheeky Pint, the podcast hosted by Stripe co-founder John Collison. They were discussing why high prices hadn’t prompted memory manufacturers to expand supply faster. Altman challenged the idea that running artificial intelligence (AI) models on personal computers would ease the need for data centers. With memory scarce, he argued, centralized systems could put it to more consistent use.
Altman strikes me as earnest. I think he means it when he says that, from a “global allocation perspective,” it’s “quite terrible to have everybody have a bunch of the world’s limited supply of DRAM sitting in their own computer… being used 5% of the time.” DRAM—dynamic random-access memory—is the working memory computers use to run programs.
His premise is straightforward: Memory is scarce, so it should go wherever it gets the heaviest, most consistent use. By that logic, memory sitting idle in a laptop 95% of the time would serve us better in a data center, working almost continuously for thousands of users.
That view is understandable coming from someone whose firm needs vast data centers to run AI models. But it mistakes how often we use something for how much we value it. Assets we expect to use for years can justify their cost simply by being available when we need them. Privacy, control, and peace of mind also count, even if they don’t show up in a utilization rate.
More subtle versions of this mistake can be just as well-intentioned. In a recent interview with Ben Thompson, Palantir Chief Technology Officer Shyam Sankar aptly compared the current AI revolution to the spread of electricity in the late 19th and early 20th centuries. His central point is correct: Realizing the gains from this new technology requires redesigning how we work.
But the electricity analogy has limits. Altman’s complaint about idle memory shows where it begins to break down.
Your Guest Room Is Not a Policy Failure
Electricity is perishable. Unless we store it, energy generated but not consumed goes to waste. Grid operators must continually match supply and demand, and persistent excess capacity can suggest that a grid is overbuilt—though some spare capacity is essential for reliability.
Sankar uses the analogy to make a sound point. The bottleneck isn’t the supply of the most advanced AI, but getting it into widespread, productive use. Factories reaped electricity’s benefits when they redesigned production around electric motors. Businesses adopting AI face a similar challenge.
But the analogy can smuggle in a second, mistaken lesson: An asset’s value depends on the share of time it spends in use. That’s true of almost no durable good we own. It also misses why we invest time and resources in less tangible benefits, such as security and peace of mind.
A homeowner hasn’t wasted the guest room because someone sleeps there only a dozen nights a year. A fire department hasn’t bought too many engines just because they spend most of their time parked. And Altman’s own laptop hardly represents a tragic misallocation of processing power whenever it goes to sleep. An insurance policy that never pays a claim is, all things considered, a pretty good outcome.
In each case, we pay for something to be there when we need it. How often we expect to need it already figures into the decision, whether we do the math explicitly or simply judge it worthwhile. Low use alone tells us little about whether the investment makes sense.
The Spare Tire Isn’t Slacking
Economists have a name for this: investment under uncertainty. When future demand is unpredictable and you can’t fully recover what you spend on capacity, having an asset ready when conditions warrant can justify the investment—even if it sits idle much of the time.
The real-options literature develops a related idea: The flexibility to use an asset when it’s especially valuable has value today. An investment that looks like a losing proposition based on average use can still make sense once you account for what access is worth at the moment of need. Idle capacity isn’t necessarily mere “slack.” It can be the price of keeping an option available, and those options have value even when we never exercise them.
The same logic applies to memory in household and business computers. A consumer who buys enough memory to run a capable AI model on her own machine is buying an option: AI that works offline, carries no provider charge for each use, keeps her data on her device, and responds when she needs it without requiring access through a provider’s application programming interface (API)—the connection software uses to request a service. Using that capacity just 5% of the time doesn’t make it a bad purchase. Spare tires, fire extinguishers, and, on most days, second bathrooms would also fare poorly under that test.
Whether the option is worth buying depends on the extra hardware’s cost over time compared with paying for remote access. That’s an empirical question, and the answer will vary by buyer. Calling it a problem of “global allocation” doesn’t settle it. Millions of people spending their own money and weighing their own tradeoffs are doing the allocating—the very mechanism we otherwise praise.
The Cloud Keeps a Guest Room Too
The same logic applies to the models themselves. As Sankar correctly notes, a snapshot of benchmark scores—results on standardized tests—isn’t the comparison that matters. A firm needs to know what it costs to get useful, correct decisions over time in its actual operations, accounting for privacy, availability, and other benefits.
Developers train models to perform well on benchmarks. Even setting that aside, those tests measure performance under laboratory conditions. They don’t capture how a model performs over time on hardware a firm controls, tailored to its “operational exhaust”—the data its everyday activities generate. A model that trails the latest systems by six months on a benchmark can still outperform them where it counts if it’s available on the firm’s schedule and closely integrated into its operations. A modest car you own can likewise beat a much better one you rent by the minute.
How firms assign tasks to models supports this point. They increasingly choose cheaper models suited to particular tasks instead of defaulting to the most capable system. Models with openly available weights—the numerical settings learned during training—can lower access costs and make local adaptation easier. The latest capabilities become one input among several. Firms also pay for availability and fit.
Ironically, data centers themselves run on much the same logic, just at a larger scale. Like electric utilities, data centers that run AI models must plan for demand that arrives in bursts. An efficient facility needs capacity above its average workload.
Suppose average demand is 100 units but unpredictable peaks reach 150. Building capacity for 130 might make sense: It leaves room for surges without paying to meet every possible peak. The right capacity depends on how often peaks occur and what turning away customers would cost. “Match the average” isn’t much of a business plan.
The operator buys capacity that will sometimes sit idle because having it available for future demand is valuable. Holding that reserve can be the profit-maximizing strategy. Judged solely by average utilization, a well-run data center can look wasteful. Its idle capacity serves the same purpose as the reserve a small business or individual buys to run AI locally.
That doesn’t make every investment in local AI sensible. Firms and individuals will waste plenty of money figuring out what works. That trial and error is part of how technology spreads—and one reason the process takes time.
Nor must running AI become an either/or choice. The likely outcome is a mix: small models on phones and other devices, medium-sized jobs on local computers, and heavier work in remote data centers, or “the cloud.” That varied, evolving arrangement is precisely what central allocation struggles to get right.
The Case for Leaving Well Enough Idle
Altman’s sincerity isn’t the issue. But when prominent voices mistake low utilization for waste, a common economic error can become a policy problem. That rhetoric invites utility-style regulation that could do far more harm than good to a technology still taking shape. It also encourages the idea that large firms running AI models need to coordinate their business decisions to manage the industry effectively—a terrible idea.
Patience would serve us better. People and firms can rationally hold capacity against uncertain demand. Visible idleness may show that they’ve bought useful flexibility, not that the market has failed. As they work out how AI fits into their lives and businesses, running it locally remains a reasonable investment.
Before anyone repossesses the world’s underused DRAM, the question is whether having it available is worth the price. A memory chip can earn its keep without working around the clock.