Abstract #
In 2024, I made a prediction—or, to be more precise, a bet—about the future of artificial intelligence: as AI became mainstream, inference would become inexpensive enough to run locally, and large-scale AI data centers would eventually lose some of their centrality. Specialized processors, increasingly capable SIMD/vector units, improved software optimization, and larger amounts of affordable memory would allow ordinary computers to handle a growing range of AI tasks.
By October 2026, the picture is more complicated. Local inference has become practical for many workloads, including classification, information extraction, and document processing. At the same time, surging demand for AI infrastructure has placed substantial pressure on the global memory supply. Manufacturers have increasingly prioritized higher-margin server and AI-related memory products, while supply agreements and purchasing power have dictated which customers are exposed to rising prices.
This article revisits the original prediction, examines the technical and economic developments that have complicated it, and evaluates whether the outcome should be classified as a win, a loss, or a draw.
The central conclusion is that the prediction was substantially correct for useful, bounded inference workloads, but remains unproven for frontier-scale inference. The final assessment depends on how the original prediction is interpreted—and whether the relevant benchmark is practical utility, hardware accessibility, or the ability to run the largest models locally.
The Original Hypothesis #
The prediction made in 2024 was based on a familiar pattern in computing history.
Capabilities that initially required expensive, specialized infrastructure have often become widely available through hardware integration—superseding implementations on general-purpose hardware—and economies of scale. Graphics acceleration, video encoding, and cryptographic operations are classic examples of workloads that migrated from specialized systems into ordinary consumer devices. My hypothesis was that AI inference would follow a similar trajectory, driven by three reinforcing developments:
- Hardware specialization: CPUs, GPUs, and dedicated neural processing units (NPUs) would increasingly incorporate silicon optimized for matrix operations, vector processing, and lower-precision arithmetic.
- Software optimization: Quantization, model distillation, improved inference runtimes, and more efficient architectures would reduce the computational and memory footprints of capable models.
- Increasing local capacity: Consumer computers would ship with stronger accelerators, greater memory capacity, and sufficient memory bandwidth to run increasingly sophisticated models.
Under this hypothesis, AI data centers would remain essential during the transition, but would eventually become less necessary for a substantial share of inference workloads.
An important ambiguity was present from the beginning, however: what did it mean for AI to become a commodity? It could mean that useful AI inference became inexpensive and widely accessible. Alternatively, it could mean that the largest and most capable models could run locally on ordinary computers. These are fundamentally different propositions. The first concerns the economics of practical applications; the second concerns the physical and economic scalability of consumer hardware. This distinction is central to evaluating the prediction.
Setting the Bar for Local Inference #
Local Inference as a Practical Counterexample to Centralized Dependence #
My current local AI setup can be summarized with a concrete example: a reasonably priced Apple Mac mini M4 with 16 GB of unified memory, running Ollama and a quantized Mistral model. The system handles several practical tasks:
- Classifying and flagging forum entries
- Processing and summarizing speech-to-text QSOs from my local amateur radio repeater
- Summarizing, normalizing, and cleaning up documents for my document management system (DMS)
- Executing routine information-processing tasks that do not require frontier-level reasoning
The machine spends much of its time idle, and all workloads execute with reasonable latency. For heavier tasks, my 2020s ThinkPad P1 runs a stronger model, also locally.
This setup is not a controlled benchmark and does not prove that all comparable workloads can be processed economically on similar hardware. It nevertheless demonstrates an important engineering point: a relatively inexpensive, general-purpose computer can perform useful inference without continuous dependence on a remote AI service. Furthermore, today’s “general-purpose” consumer hardware already integrates high-speed vector units and competitive memory bandwidth.
Several characteristics make this viable:
First, the workloads are bounded. Classification, extraction, and summarization require far less general reasoning capability than open-ended research or complex planning. Because the context window remains controlled, the computational effort required to generate a response stays manageable, assuming a suitable software harness.
Second, the workloads are not strictly latency-sensitive. A background document-processing queue can tolerate moderate delays without requiring the over-provisioned infrastructure needed to handle peak demand instantaneously—unlike a real-time human-to-machine conversational interface, which demands minimal latency.
Third, smaller general-purpose models have become dramatically more efficient. Aggressive quantization, Mixture-of-Experts (MoE) architectures, and KV/token caching have substantially lowered the hardware threshold required for consumer devices to run useful inference.
Finally, local execution offers advantages beyond direct cost savings, including data privacy, operational independence, and immunity to external service outages. The relevant metric is therefore not simply tokens per second. It is the total cost of completing a useful task at an acceptable quality level—accounting for hardware, energy, maintenance, and operational overhead. For this class of applications, the original hypothesis has already received strong practical validation.
Local AI-Assisted Software Development: Capability Is Not Reliability #
A second example illustrates both the progress and the limitations of local inference: AI-assisted software development.
Local models already provide meaningful assistance with code analysis, small feature implementations, refactoring, documentation, and drafting initial architecture plans. For these workflows, the economic benefits of local inference are clear: a developer can run continuous iterations without incurring per-token API fees, keep proprietary source code entirely local, and integrate inference directly into their development environment.
In my experience, smaller local models can often formulate a solid implementation plan, yet falter when translating that plan into working code once a task crosses a certain complexity threshold. The bottleneck is not solely model size. Complex tasks require tighter supervision and an agentic harness that decomposes work into manageable steps—much like established practices in agile software development.
A local model may have a near-zero marginal inference cost, yet still demand heavy human oversight. If a developer must repeatedly fix broken tool calls, repair faulty logic, and verify every intermediate output, the efficiency gains of an AI-assisted workflow over manual coding shrink considerably.
That said, the complexity and viability threshold is shifting upward rapidly. Quite recently, I re-implemented my “developer tools” single-page application (SPA) using a “dark-factory” autonomous approach with flint on my older ThinkPad. In the time it took to drink a coffee, that 2020s machine built an entire web application that would have taken me a full day to write manually. Similarly, modernizing the visual styling of a three-year-old, abandoned Hugo theme took roughly ten minutes. By contrast, during my first agentic AI experiments 18 months ago, implementing a basic sorting algorithm was the upper limit of what any model could reliably achieve on my local machine.
This progression reinforces the broader thesis of this article: the democratization of AI does not depend solely on whether a model can run on commodity hardware. It depends on whether the resulting system can execute tasks reliably enough to be genuinely useful.
In short, local AI has crossed the threshold of practical utility for many workflows, even if it has not yet crossed the threshold of dependable autonomy.
The Global Memory Market #
As AI infrastructure expanded, hardware supply constraints became far more consequential. Large-scale AI deployments require not just accelerators, but also conventional dynamic random-access memory (DRAM), high-bandwidth memory (HBM), high-speed storage, and supporting server infrastructure.
HBM is critical for modern AI accelerators, while server DRAM underpins the host systems those accelerators rely on. While HBM and conventional DRAM are not interchangeable, their production competes for the same wafer capacity, capital expenditure, and advanced packaging resources. When demand for AI infrastructure surges, memory manufacturers have strong financial incentives to allocate fabrication capacity toward higher-margin enterprise products.
This dynamic restricts the supply of conventional memory even when consumer demand has not spiked proportionally. On September 30, 2026, market intelligence firm TrendForce projected that conventional DRAM contract prices would rise by 10–15% quarter-on-quarter in Q4 2026, attributing the tight supply largely to manufacturers prioritizing advanced-node capacity for high-performance server products—a constraint expected to persist into 2027.
Contracts and Purchasing Power #
The memory crunch also highlights how supply constraints affect market participants unevenly. In a competitive market with abundant supply, buyers can negotiate pricing and delivery schedules across multiple vendors. When supply tightens, bargaining power shifts sharply. Hyperscalers and major enterprises lock in capacity through multiyear supply agreements, leaving smaller buyers, unallocated customers, and anyone purchasing above their contracted volume exposed to spot-market volatility.
This is an important caveat to the assumption that the market sets a single, uniform price for memory. In practice, pricing and availability depend heavily on contractual commitments, order timing, guaranteed volumes, and buyer leverage. Reports from German data-center operators (known to the author) describe suppliers demanding substantially higher prices despite long-standing commercial relationships—a clear sign of a market defined by unequal contractual protection.
For an infrastructure provider locked into fixed-price customer contracts, this creates a painful squeeze between downstream revenue commitments and rising upstream procurement costs. Unable to pass those increases on immediately, operators see memory-market volatility ripple across the entire IT supply chain—driving up hosting rates, delaying hardware rollouts, and undermining the economics of affordable local hardware.
Impact on the Bet #
Surging memory demand—driven overwhelmingly by hyperscale data centers—has tilted the economics back toward centralized infrastructure. By keeping RAM prices elevated and constraining affordable high-memory consumer configurations, the memory crunch has slowed the proliferation of capable consumer-level AI hardware, significantly complicating my original two-year timeline.
Reassessing the Bet #
The original 2024 prediction can be evaluated against three distinct interpretations:
-
Interpretation A: Useful AI inference becomes a commodity. Under this reading, ordinary hardware becomes capable of running practical AI workloads at low ongoing cost without relying on cloud APIs. The Mac mini example offers clear evidence in favor of this outcome: an affordable desktop can handle multiple real-world inference tasks locally while remaining idle most of the day. While this does not cover every AI use case, the prediction has largely held true for bounded, practical applications. Assessment: Largely correct.
-
Interpretation B: High-end local inference becomes broadly accessible. Under this reading, mainstream consumer workstations can run larger, more capable models at practical speeds and reasonable cost. Advances in quantization, NPUs, and inference engines support this trajectory. However, memory capacity, bandwidth, and hardware costs remain major bottlenecks. The RAM shortage has driven up the cost of high-memory systems at the exact moment local AI workloads demand more RAM. Assessment: Partially realized, depending heavily on model size and workload.
-
Interpretation C: Frontier-scale AI becomes a commodity workload on ordinary computers. Under this reading, the largest and most capable models run locally on consumer machines, rendering centralized data centers largely redundant. A 600-billion-parameter model illustrates the scale of this gap: even at 4-bit precision, the model weights alone require roughly 300 GB of memory before accounting for context (KV cache) and runtime overhead—far beyond the reach of a typical 16 GB or 32 GB desktop. Moreover, centralized data centers offer structural advantages beyond raw memory capacity: they pool hardware utilization, amortize fixed costs across millions of users, and absorb bursty peak demand that would be uneconomical to provision locally. Assessment: Not established.
Verdict: A Draw, Subject to the Original Terms #
My 2024 prediction was directionally right about the falling computational cost and rising accessibility of useful local AI inference. Where it fell short was underestimating how rapidly aggregate demand for AI infrastructure would grow—creating fierce competition for memory and semiconductor manufacturing capacity.
The memory shortage does not mean local AI has failed; rather, it proves that algorithmic efficiency, component pricing, and infrastructure economics are independent variables. A local model can be remarkably cost-effective for bounded daily tasks while a frontier model remains completely impractical on the exact same machine. Both realities coexist without contradiction. Consequently, the bet cannot be judged without defining what AI becoming a commodity actually entailed by 2026.
If the benchmark was inexpensive local inference for practical, everyday tasks, the prediction succeeded. If the benchmark was running frontier-scale models on ordinary consumer hardware, it has not materialized. Because the original claim was broad enough to imply both, the fairest retrospective verdict is a draw.
Local AI has progressed along the expected technical path: consumer chips are faster, inference software is leaner, and genuinely useful models now run on modest hardware. Yet the global memory market has served as a sharp reminder that software breakthroughs do not happen in a vacuum—they remain bound by fabrication capacity, supply-chain power, and industrial economics.
The bet is therefore best settled as a draw: local inference has become a practical commodity for everyday workloads, while the broader vision of running frontier-scale AI on ordinary hardware remains out of reach.
Outlook #
The original bet scales perfectly for a draw. Doing a hub service on the winner’s road bike while they enjoy a cold drink translates seamlessly into servicing both of our bikes together—and sharing a cold drink afterward.
