A few months ago, according to the Moonshots panel, Jalapeño was just an idea inside OpenAI. On the show's 29 August episode, it was a production chip with published performance numbers — and the panel spent the segment working out what that means for the company that currently sells almost everyone their hardware.
Jalapeño is a custom inference accelerator OpenAI developed with Broadcom. OpenAI announced it on 24 June 2026 as the first chip in a planned multigeneration platform, describing engineering samples running machine-learning workloads — including its GPT-5.3-Codex-Spark model — at the frequency and power targeted for production. Broadcom supplies the silicon implementation and networking, including its Tomahawk technology. OpenAI said the design took about nine months, credited partly to help from its own models, and said it planned gigawatt-scale deployment with data-center partners.
Training and inference, the two words the panel kept using
Earlier in the same episode, Dave Blundin offered listeners a way to make sense of the enormous numbers flying around the AI industry. Almost everything, he said, can be understood through two things: AI training and AI inference.
Training is the expensive process of building a model by learning from data. Inference is what happens afterwards, every time a model answers a question, writes code or generates a video. Blundin used Elon Musk's satellite plans as an illustration: what is going into orbit is "purely inference," with revenue coming from connecting AI-generated content to people. Training, he said, "is a very different beast."
That split is what makes a chip like Jalapeño interesting. It is built for the second job.
The numbers read out on the show
The panel relayed OpenAI's figures: 1.5 to 1.9 times more AI work per watt and up to 3.6 times lower end-to-end latency compared with NVIDIA's GB200 and GB300 systems, with Jalapeño drawing 700 watts against the GB300's 1,400 watts. Latency here means the delay between a request and a completed answer.
Alex picked out a different statistic as the most startling one: throughput per second per user — how fast tokens come back to a single person using the model. He said OpenAI claimed almost a 54-fold increase hosting GPT-OSS, its open-weight model, versus what the company "obliquely" referred to as the existing best, which he presumed meant an NVIDIA architecture. He also said the only plausible way OpenAI reached a tape-out — the point at which a chip design is finalized for manufacturing — that quickly was in conjunction with Broadcom.
These are OpenAI's own published comparisons, read out by the panel rather than independently measured on the show. One panelist mentioned having seen an interview in which a reporter asked NVIDIA's Jensen Huang how he felt about his largest customer now competing with him.
"I think this is what we're going to see everybody doing," one panelist said. "Everybody's moving up and down the stack. They want independence."
Why Blundin thinks NVIDIA is fine anyway
Blundin agreed that inference is moving off NVIDIA — he called Jalapeño "the first 2x of 100x performance gains" still to come — but said that does not mean NVIDIA has a problem. Training, in his account, is staying on NVIDIA, and training capacity is "infinitely sold out" just as inference is.
His larger point was about size. He put inference at around 90% of compute, which in his framing means an enormous new industry is opening up for inference-only hardware rather than a fixed market being divided. He also drew a lesson from the timeline itself: an idea at OpenAI a few months ago is now a production chip, which he took as a measure of how short innovation cycles are becoming with AI assistance.
"OpenAI Compute": the plot twist Alex floated
Alex's thought was the one that stopped the table. If Jalapeño really balances prefill — processing the user's prompt — against the memory-bandwidth-limited work of generating each next token, he wondered whether OpenAI might want the optionality, in a year or two, of becoming a hyperscaler itself: a company that rents out large-scale computing.
He imagined the announcement: in addition to its own models, OpenAI offers its own cloud, hosted on OpenAI chips, as a response to the competitive pressure from Chinese and other open-weight models. He called it OpenAI Compute. His model for it was Musk, who he said is making money "hand over fist" hosting everyone else's models, including Anthropic's. "I could imagine a world where, say, an OpenAI compute cloud hosts an Anthropic model and OpenAI and Anthropic win at the same time." He described it as a hypothetical and "a heck of a plot twist," not something OpenAI has signalled.
Is CUDA's moat dead?
That prompted a question from the table: NVIDIA's CUDA moat has been said to be dying — how big a deal is that?
Blundin explained CUDA for listeners who had not met it. AI researchers work in Python and PyTorch because ideas can be tried quickly there. To run those ideas at scale, something has to translate that code into instructions the hardware executes fast, and CUDA is the layer that turns PyTorch into NVIDIA's micro kernels. NVIDIA paid for that translation in human effort, years before AI could write code. It became so pervasive, he said, that when AI took off only NVIDIA chips could realistically be used, because porting the underlying matrix-multiplication routines to AMD or another vendor would have been a nightmare. That early support of researchers, in his account, is specifically why NVIDIA is the most valuable company in the world today.
What has changed, he argued, is inference. Inference algorithms are easy enough to port that a developer can "vibe up" a version for AMD or Intel and stop caring about CUDA. Advanced researchers doing new training still use CUDA overwhelmingly. "CUDA as the inference time mode is already dead," he said, using the show's shorthand for moat; as a training-time moat he expects a limited lifespan.
The show has been here before from another direction. On an earlier episode, investor Ramez Naam predicted NVIDIA's software moat would break "this year and next year" — 2026–2027 — because AI could rewrite the small pieces of optimized code called kernels for a rival's chip, while disclosing that a company in his venture portfolio was working on the problem. That was a forecast plus one panelist's encouraging experience with AI-generated kernels, not a measured demonstration that production workloads could move vendors without losing speed. Jalapeño puts a named platform and a business-model question behind the same argument.
The next moat is the wiring
Blundin's closing point was that NVIDIA saw this coming. It bought Mellanox to build high-speed interconnect — the links that let chips exchange data while working on a shared problem. If you want to run 100,000 or a million GPUs as one coherent cluster, he said, you still have to go with NVIDIA. "Your moat is only good until the next moat," he said of Jensen Huang. "He's moving the moat very intelligently, and he has a huge amount of capital to keep moving with it."
That leaves a narrower version of the interconnect question, one the panel has raised before: on the earlier episode, the table pushed back on the idea that inference escapes the need for fast links, offering the example of roughly 10 to 20 GPUs inside a single rack serving one large model and needing to talk to each other quickly — local coordination, even if an entire supercluster need not stay in step the way a training run does. Jalapeño's own networking comes from Broadcom. The scale at which fast links matter is what is being contested, and OpenAI's answer arrives with the gigawatt-scale deployments it says it is planning with data-center partners.