Aaron Levie has a rule for anyone running the kind of software company he runs — a platform where a customer's data and daily workflow already live. There are two things you have to do, he said, and anyone attempting one without the other is going to lose.
"You have to build an agent that is insanely great at your product," the Box chief executive said on Sequoia's Training Data podcast. "That agent has to be — you have to provably be, like, 10 or 20 points better than an off-the-shelf agent using your system. Not because it's hobbled the other side. It's just like you are so eval-maxed and you're, like, so tuned to your particular workflow."
The second: "Obviously you have to go headless. You literally have to make sure that your APIs are exposed to Claude and ChatGPT and, you know, all the different platforms."
An agent here means software that plans a task and uses tools on its own — searching, reading files, calling APIs — rather than answering a single question. Box stores unstructured corporate content in the cloud: contracts, research files, marketing assets, loan documents. Levie said the company sits on hundreds of billions of files, and the recurring problem is that companies "rarely know what's actually inside of it." Unless someone opens a document and reads it, the information stays invisible.
From a shelved experiment to a company-wide pivot
Box had been "flirting with AI kind of products" since around 2015, Levie said. Image classification and optical character recognition looked useful for an enterprise — label the image data, pull the text out of a scan — but the economics did not work: it was "insanely expensive," and each workflow needed its own hyper-trained model. "So we kind of shelved it."
What restarted the effort was ChatGPT, which he called the "head-exploding moment" — less for the underlying model than for the form factor that showed everyone these could be interactive systems you simply ask. Box then did what he described as the textbook version of a pivot: a carved-out team, the best people on it, daily reviews. In May 2023 the company announced its first Box AI features: asking questions of documents, in preview, and generating or rewriting content in Box Notes, using OpenAI models and enforcing the file permissions customers already had. Access was initially through a selective design partner program. What Levie described in this interview is the stack built out since then, and an agent that can now be pointed at that content at scale.
What the harness actually does
When the conversation turned to harnesses — the machinery around a model that decides which tools to call and in what order — Levie's argument was about accumulated knowledge of one system. Box knows how people search Box, and how they decide, when ten files come back, which one to open. That heuristic is what the harness tries to encode.
It has access to the file system and the search engine, plus tools for pulling the plain text of a document, pulling chunks out of it, or computing embeddings on the fly — numerical representations that let a system match meaning rather than exact words. Box's published agent framework describes the same shape: plan, find the relevant files, reason over their contents, act on the result, with retrieval governed by the customer's content permissions.
Levie used his own account as the test case. He has something on the order of tens of millions of files, twenty years of accumulation. Asking a question of all of it, the agent "does multiple searches in one, it re-ranks it, it then very quickly sort of pulls out the most relevant information, then it'll, in some cases, read the full document." Box compares that against handing the same API to Claude or OpenAI directly, and sees "meaningfully better results on accuracy and latency," he said, "because again, we kind of know exactly how to tune it for our workflows."
Measuring half a point
The claim rests on evaluations. Box runs hundreds of tests on every new model, Levie said, and maintains two main ones. The first is the Complex Work Eval, a document-centric set of domain tasks — life sciences, financial services, public sector, technology — where a model is given a handful of documents and a set of problems, and run inside Box's own agent framework. Box's published methodology covers tasks across 12 industries, scored with weighted binary criteria, often dozens per task, checking expected facts, figures and conclusions, and aggregated over repeated trials. In its reported comparison of Anthropic's Sonnet 5 against Sonnet 4.6, Box found broadly comparable quality overall, with gains of four percentage points each in energy and retail, two in professional services and one in technology. The second eval is held back: Box's own instance, and how Box employees use their data.
The point of running both, Levie said, is resolution. "We see when things move by half a point in terms of model improvement." Box then sets default models against cost and accuracy thresholds, while letting customers pick any model from what he called a model garden.
The applied research team — Box Labs, he said, though he was not sure they had capitalized the L yet — spends its time on the gap between a working demo and an automatable process. One group is working on a kind of automated hill-climbing at the harness level: given a customer's data, search for the agent configuration that answers their questions best. His example was a bank receiving 100-page loan documents. Getting 70% accuracy from an off-the-shelf model or 97% from a tuned system is, he said, "a world of difference in can you actually go and automate that process."
The work nobody asks for
Asked for the use cases that land every time, Levie gave the plain one: you have a million contracts, find out what is inside them. Pull the structured data out of a million research documents, put it in a database, query it. Reading every contract was never worth a human's time, so it never happened.
The cases he finds more interesting run in the background. He described a customer's idea — an agent continuously watching whether documents meet the company's governance policies, flagging what should go to an archive or a legal hold. Every company has a head of compliance and a head of governance, he said, and they "can only be, like, overseeing the whole sort of enterprise. They've never been able to be everywhere at once." The agent would sit beside the employee: "Nah, that's going to break your governance policy," instead of the user having to predict the rule.
The version he called the dream state is client onboarding at a bank: every automatable step automated, the process jumping out to a person for extra review or verification, and the whole thing collapsing from "that sort of one- or two-week back and forth" to about an hour. He was describing the target, not a shipped result.
That changes the interface. A chat box is not going away — "it's just like your product has a search box. Obviously it does," he said — and it works for one-off asks. But for agents reviewing contracts, reading logs or triaging security incidents while you are elsewhere, chatting is at best a catch-up. "I want a dashboard. I want a workflow. I want a queue. I want a task list." A horizontal assistant that tried to grow all of those surfaces would stop being simple and delightful, in his view, which is why he expects vertical products that know the process to build them. His bet: "in five years from now, I would bet like 90% of all tokens in the enterprise are things that a user never kicked off, and they just see a result."
Why letting rivals in pays
The headless half is where Levie thinks other software executives got it wrong, by underestimating how much of the demand is new — "complete white-space opportunities," as he put it. Box has taken this route: its hosted MCP server — MCP being the standard connector between assistants and outside tools and data — is generally available, offering external agents multi-file search, metadata extraction and the ability to use Box documents as context for new content saved back into Box, under existing access policies. Box named Claude, Microsoft Copilot Studio and Azure API Center at launch, with Salesforce Agentforce and GitHub Copilot described as coming.
His evidence for the upside is his own behavior. "I use Salesforce more today, probably by an order of magnitude, than I ever have because I MCP into it via Claude or ChatGPT," he said, constantly querying the CRM data. He has told LinkedIn's product managers the same thing: "I'd probably pay 10x more for LinkedIn if I just could MCP into it," to keep track of which executive is doing what and when to reach out.
A host asked whether that makes systems of record into tollbooth businesses. "I don't love that term because, like, no one's had a good experience at a tollbooth," Levie said. What he thinks it does require is a volume-oriented business model on the headless side — the depth of purpose, as he put it, is still organizing the workflow, managing and securing the data and providing guardrails. "If you're solving real problems for customers, it'll just, like, make money."