At four in the morning, Peter Diamandis and Emad Mostaque were exchanging messages about two charts. When Diamandis brought them to the Moonshots discussion, the first showed a steep fall in the memory needed by DeepSeek’s newest open-weight model. The second plotted quality against cost.
“Every day is a freaking new model now, right?” Mostaque said. This one interested him because of what its designers had changed inside it: rather than simply demanding more expensive memory, they had found ways to retain less information and reuse more of what the model had already computed.
DeepSeek’s September 10 release announcement presents V4.1-Flash as a lower-cost successor with native visual understanding—the ability to process images as well as text. The panel’s argument was that its memory efficiency could also change the calculation for companies deciding whether to build their own AI systems.
What the 890-byte figure measures
As a language model generates an answer, it uses information computed from earlier parts of the conversation. Saving that information avoids repeating work for every new token, the small pieces of text a model processes. The saved intermediate values are called the key–value cache, or KV cache.
Longer conversations and more simultaneous users make this cache a substantial expense. It often occupies high-bandwidth memory, or HBM: fast memory placed close to the processor so information can reach the arithmetic units quickly.
DeepSeek’s official model card reports a global KV cache of 890 bytes per token, approximately a quarter of V4-Flash’s. Several changes work together:
- Reuse between reading and writing. The model separates prompt processing from answer generation through an encoder–decoder design. The decoder reuses representations produced by the encoder instead of creating a separate set of attention representations from scratch.
- Compression and selective attention. Attention is how a model draws on relevant parts of its context. DeepSeek’s compressed sparse attention shares cached information and selected token positions between layers, reducing duplication.
- Four-bit storage. Cached values use a compact numerical format, reducing the space each value occupies.
- Recomputation instead of retention. Missing short-range cache states can be reconstructed from recent tokens rather than kept on a solid-state drive, or SSD. This trades some repeated calculation for less storage.
Alongside those changes, a separate component called Engram provides sparsely accessed, token-based memory: the model draws on selected entries rather than using the whole store at every step. Its 196 billion parameters are separate from the reported conversation-cache figure.
The distinction matters for deployment. Parameters are the model’s learned numerical values, not its temporary conversation notes. The card lists 552 billion backbone parameters, with eight billion active per token during prompt processing and sixteen billion during generation, plus Engram. Activating only part of a model reduces computation; it does not make the remaining parameters disappear from storage.
The release announcement says the cache requires one-quarter as much HBM and one-eighth as much SSD capacity as its predecessor. Those are cache savings, not equivalent reductions in total model memory or the cost of an entire server.
Engineering inside a small box
Mostaque described DeepSeek as “routing around the HBM memory that is so ridiculously expensive.” Another panelist contrasted two approaches: placing ever-faster memory closer to processors, and changing the model so it needs less of that expensive hardware.
In the panel’s interpretation, restrictions on Chinese access to advanced hardware create pressure to pursue the second route. The speakers expected Western labs to adopt useful architectural improvements too. Their entrepreneurial lesson was to “think in a really, really small box”—treat a scarce resource as a design constraint rather than assume more of it will arrive.
The discussion quickly expanded into forecasts about data-center spending. One speaker estimated that HBM accounted for 40% of the American capital-expenditure buildout and treated the cache improvement as a major threat to that spending. That was the panel’s estimate and prediction; DeepSeek’s reported fourfold cache reduction does not establish a fourfold reduction in overall HBM demand.
Cheaper service is not automatic workstation deployment
DeepSeek’s commercial changes are concrete. Its announcement introduces peak and off-peak API pricing, with off-peak rates half the peak level. For work that can wait, that offers a scheduling-based saving alongside the architectural changes.
The company also scheduled requests addressed to V4-Pro to route to V4.1-Flash from September 14 until V4.1-Pro launches, billed at Flash rates. Customers using that model name therefore face a planned change in the system answering their requests, not just a different bill.
The panel’s quality-and-cost comparisons were more sweeping. Discussing the Open Design Benchmark, which he described as testing website design, Mostaque claimed advantages over named rival models and described Flash as twenty times faster and cheaper. He also called it a roughly 500-gigabyte model that could fit on a Mac Studio.
Those comparisons and the workstation claim came from the discussion, rather than an established deployment test in DeepSeek’s release materials. A model’s storage footprint alone does not settle whether a workstation can run it usefully: runtime memory, bandwidth and compatible serving software also matter. DeepSeek’s announcement invites work on additional open-source deployment options, while Mostaque himself warned that the unusual architecture could be difficult for inference providers to support.
The enterprise opening
The panel made its enterprise argument through Moderna. A speaker urged the company to build an internal AI capability using open weights—the downloadable numerical values that allow a model to be run and adapted—alongside its proprietary biological data.
The proposal was not that a website-design score establishes expertise in biology. It was that a company need not fund a frontier model from scratch to start building useful internal systems. An efficient general model could provide a starting point, while the company supplies its own data and domain-specific development. The speakers did not report that Moderna had begun such a deployment.
Training economics prompted a separate dispute. One panelist calculated a $10 million training cost; another asked how much might have come from distillation, in which a model learns from another model’s outputs or reasoning traces. The $10 million figure was the panelist’s estimate, not a training bill established by the cited release materials.
Mostaque pointed instead to DeepSeek’s description of simulated training environments. That is a different claim from learning by copying rival models’ reasoning. He argued that once a model is good enough for its intended uses, a provider can concentrate on serving it cheaply rather than continually chasing a stronger model through distillation.
Efficiency can coexist with scarcity
Earlier in the discussion, Mostaque argued that the same generation of AI chips now delivers much more capability than when it first appeared. Yet he also described demand spreading into visual and physical AI, keeping hardware difficult to obtain. Lower resource use per answer and rising total demand can happen together: cheaper answers make more applications economical.
GPU rental prices are one way to follow that market, but their scope matters. Ornn’s methodology measures prices paid in sampled active on-demand rentals, excluding reserved capacity and bespoke long-term contracts. The methodology alone does not establish a shortage or a price trend.
For an enterprise choosing a model, DeepSeek’s release presents a more immediate decision than the panel’s predictions about future chip spending. A company can weigh a cheaper hosted service—including half-price off-peak use—against the hardware and engineering needed to run open weights itself. The 890-byte cache improves that calculation without doing away with it.