Mercury 2: The First Reasoning Diffusion LLM, and Why Its Speed Matters

There is a category of AI application where latency is not an inconvenience. It is a product failure. When someone is on the phone and the system pauses for two seconds before responding, the caller does not think the AI is considering the question carefully. The caller thinks the call has dropped, or the system is broken, or something is simply wrong and they should hang up and try somewhere else. That reaction happens at the level of the nervous system, not at the level of rational evaluation. No amount of subsequent accuracy repairs the experience once that pause has registered as wrong.

This distinction between synchronous and asynchronous AI interactions defines what speed actually means in practice. In an asynchronous context, you submit a request and check back later. A few seconds of processing time is irrelevant because you moved on. In a synchronous context, you are in the experience continuously, waiting for each response as it arrives. Latency in that context is not measured in seconds. It is measured in the quality of the interaction, and it degrades fast.

Mercury 2 is the first reasoning model built on a diffusion architecture, generating tokens in parallel rather than one at a time. At approximately 1,000 tokens per second, it runs roughly five times faster than speed-tuned models in head-to-head tests. The architecture change is not an optimization of the existing approach. It is a different approach entirely, and it opens a set of use cases that were previously out of reach regardless of how much the sequential model was optimized.

Why sequential token generation was always going to be the bottleneck for real-time applications

Standard large language models generate text the same way a typewriter works: one token at a time, each one dependent on all the tokens that came before it. The architectural reason is that attention mechanisms in transformer models weight every previous token when predicting the next one. That attention pass happens for each token, and because tokens cannot be predicted independently they must be generated sequentially. The model cannot skip ahead and come back, because each prediction depends on what came before.

For most use cases this constraint is invisible. When you submit a writing task and come back later, whether the model took 28 or 30 seconds to finish is not meaningful. The output is what matters. But for real-time applications, the sequential constraint creates a ceiling that optimization cannot break through. You can run the model on faster hardware. You can reduce the output length. You can cache common prefixes. All of those help at the margin. None of them change the fundamental architecture that requires token N to be complete before token N+1 can begin.

This ceiling is what made real-time voice and agent applications feel second-rate. A customer service voice agent on a standard model pauses in ways that feel unnatural in human conversation. An agentic workflow that chains many steps, each requiring a model call, accumulates latency at every link in the chain. A real-time coding assistant feels sluggish compared to a human typing in the same editor. The ceiling was not a hardware problem or an optimization problem. It was an architecture problem, and architecture problems require architecture solutions.

What diffusion adds to reasoning models and why it took until now to combine them

Diffusion models generate by starting with structured noise across the full output length and refining that noise in parallel passes until it converges on a coherent result. Rather than generating token 1, then token 2, then token 3, the diffusion approach refines many tokens simultaneously across multiple passes. The throughput of tokens per second is therefore much higher than any sequential approach at comparable quality levels, because the compute per token is distributed across the full output rather than applied sequentially to each position.

Diffusion language models existed before Mercury 2. What did not exist was a reasoning diffusion model. Reasoning models require the ability to deliberate, to consider multiple angles, to catch their own errors and correct them. Early diffusion language models could generate text quickly but did not demonstrate the kind of structured reasoning that makes a model trustworthy for business-critical questions. The skeptical view was that the diffusion architecture was inherently less suited to step-by-step deliberation, because the parallel generation process does not obviously lend itself to sequential logical chains.

Mercury 2 challenges that view with demonstrated results. It generates at approximately 1,000 tokens per second, roughly five times faster than Claude Haiku in direct comparison on the same prompts, and also outpacing another speed-focused model in the same category. It wrote nearly 600 lines of working code for a chess game in a single pass and handled follow-up prompts by rewriting the full implementation rather than patching it. That combination of speed and functional output quality is what makes it notable: not the speed alone, and not the reasoning alone, but the first time both appeared together in the same model at this tier. The playground also provides an adjustable reasoning effort slider, so you can set the depth of deliberation for a given use case and observe the trade-off between reasoning depth and output speed for your specific questions.

The applications where 1,000 tokens per second changes the experience rather than just the benchmark number

Speed improvements matter most in the applications where users are continuously present in the interaction and any pause breaks the experience. Voice assistants sit at the top of that list. A voice agent that answers a phone call needs to respond in natural conversation time, which means the gap between the customer finishing a sentence and the agent beginning its reply should be nearly imperceptible. At 1,000 tokens per second, a 200-token response arrives in roughly 0.2 seconds before the text-to-speech layer converts it to audio. That is within the range of a natural human response time. At a standard sequential model generating 40 to 60 tokens per second, the same response takes 3 to 5 seconds of model generation time before the audio even begins, and that pause is the product failure described at the start.

Customer service agents face the same requirement. Real-time search tools that respond as a user types benefit because each query can receive a model response that arrives before the user has finished revising the search. Multi-step agents gain because latency compounds across every link in the chain, and a five-step agent at 1,000 tokens per second finishes in a fraction of the time of the same chain at 60 tokens per second.

The API pricing makes this model accessible for production use without a large budget commitment. At $0.25 per million input tokens and $0.75 per million output tokens, the cost of running a production voice agent handling 50 conversations per day stays well under a meaningful monthly budget line. The combination of speed and price positions Mercury 2 as the first model in this tier where building a real-time application is not primarily a budget decision. It is an architecture decision, and the architecture case is strong.

The limits that still apply and the tasks where Mercury 2 is the wrong model to reach for

Speed as the primary optimization criterion means Mercury 2 is not the right model for every task. Tasks that require extended deliberation, complex multi-step reasoning chains where each step builds carefully on the last, and long-form analysis that benefits from deep context integration are better served by larger reasoning models that trade speed for depth. A detailed legal brief, a complex technical architecture review, or a financial model with many interdependent assumptions is not the Mercury 2 use case.

The reasoning effort slider introduces a useful trade-off within the model itself. At lower effort settings it generates faster with shallower reasoning. At higher effort settings it reasons more carefully but the speed advantage narrows relative to the ceiling. Calibrating the right effort level for each use case requires testing with actual business questions, not synthetic benchmarks. A booking agent and a code generator will land in different places on the effort dial, and discovering where each should sit is a runtime experiment.

The third limit is the full latency stack. The model generates tokens quickly, but a voice agent also needs to transcribe the user's speech before the model sees it, and convert the model's text response back to audio before the user hears it. Those two additional steps add time that exists outside the model. The relevant question for a voice application is not whether the model is fast enough in isolation. It is whether the end-to-end stack, model plus transcription plus speech synthesis, is fast enough. Testing the full loop before committing to production is not optional.

What an auto repair shop can capture with a real-time voice and chat agent

A busy auto repair shop misses calls whenever the team is under a hood, and in high-intent service categories the majority of callers who reach voicemail contact a competitor rather than wait for a callback. Here is what the numbers look like for a shop handling 50 inbound inquiries per day.

Before the agent: 40 percent of calls during peak hours go to voicemail. Of the 20 inquiries that hit voicemail, approximately 12 call a competitor rather than waiting, based on typical conversion patterns in time-sensitive service categories. Of those 12, roughly 40 percent would have booked an appointment with a live response. That is approximately 5 lost bookings per day from unanswered calls alone. With an average repair ticket of $285, the monthly lost revenue from voicemail reaches 5 bookings times $285 times 22 working days, which equals $31,350 per month.

With a Mercury 2 agent answering every call and website chat inquiry in real time: the agent consults the shop's calendar and replies with available slots and a rough price range before the caller has had time to consider hanging up. The response feels immediate because at 1,000 tokens per second it is immediate. After hours, the same agent captures inquiries that previously went to a dead voicemail and either books them into the next available slot or sends a confirmation text with the scheduled time.

Token cost for 50 inquiries per day averaging 500 tokens of conversation per inquiry: 25,000 output tokens per day. At $0.75 per million output tokens: under $0.02 per day. Monthly token cost: under $0.45. Even with the full infrastructure cost of the voice stack, the monthly operating cost of the agent stays well under $25. Recovering three of those five lost daily bookings, which is a conservative estimate given that the agent answers every call, adds $3 bookings times $285 times 22 working days, which equals $18,810 per month in additional booked revenue against a monthly cost of under $25.

The ratio of return to cost is not a marginal improvement. It is a different order of magnitude. The reason the economics work this clearly is that the speed of the model closes the gap between what callers experience and what they expect from a live service response. A slow model closes 0 of those 5 daily gaps because callers hang up before the response arrives. A fast model that reasons correctly closes most of them. Mercury 2 is the first model in the speed tier that delivers both properties at the same time. There is a secondary application worth naming that compounds on the voice and customer service use case. Mercury 2's speed makes it a strong candidate for the agent chains that run inside a business system rather than facing a customer directly. An agent that routes incoming inquiries to the right department, enriches a lead record with publicly available information, drafts a follow-up message for review, and logs the interaction to the CRM in one chained workflow runs noticeably faster at 1,000 tokens per second than at 60. The latency reduction is not always felt by a human in these backend chains, but the throughput gain means the system handles more volume before hitting practical limits. A repair shop handling 50 inquiries per day has room to grow that volume without re-architecting the agent stack. That headroom is a form of operational leverage that compounds as the business grows.