Rohit Ghumare · August 18, 2026 · 10 min read

DFlash 2: The Drafter Stopped Guessing Alone

Speculative decoding has one rule. A small drafter guesses a block of tokens, and the big model verifies the whole block in a single forward pass, so every guess that survives is a token the big model did not have to decode on its own. DFlash, published in January, made the drafter itself one-pass: every position in the block predicted at once, in parallel. That parallelism is also its weakness. A position that cannot see what its neighbor chose will sometimes choose the same word, and a block with a stutter in it dies at verification. Today Inco AI shipped DFlash 2, which keeps the draft parallel and adds two small parts: a selector that traces one coherent path through candidates the drafter had already produced, and a two-tap convolution that stops the draft from decaying toward the end of the block. Together they add about 1% to the draft-verify cycle and return around 21% more accepted tokens per pass, with the output provably unchanged.

One pass to draft, one pass to verify

Decoding is a memory problem before it is a compute problem. Every generated token reads the full set of weights once, so a 27B model in 16-bit numbers moves about 54 GB through the memory bus per token, and the arithmetic units sit mostly idle while that happens. I wrote the arithmetic up in Tokens per Second Is a Memory Bandwidth Number. Speculative decoding is the standard way to buy back that idle compute: a drafter proposes several tokens, the target model scores all of them in one forward pass, and the pass costs about the same as decoding one token because the weights are read once either way. The accepted prefix is kept, the first miss is replaced by the target's own choice, and the rest of the block is thrown away. Greedy output is identical to running the target alone, and with sampling the distribution is preserved by rejection sampling. Nothing about the answer changes, only the number of passes it takes to write it.

The catch has always been the drafter. For years it was a smaller autoregressive model, which means it wrote its guesses one token at a time and the block took as many small passes as it had positions. DFlash treated drafting as a block-diffusion problem: mask the whole block, predict every position in one pass over a few layers. By Inco AI's count the method now runs in SGLang, vLLM, TensorRT-LLM, and llama.cpp, NVIDIA measured up to 15x throughput with it on Blackwell, Google reported 3x on TPUs, and Meta shipped a DFlash drafter alongside Muse Glimmer. The drafters have been downloaded more than 3.5 million times on Hugging Face, again by Inco's own tally. So the interesting question in DFlash 2 is not whether parallel drafting works. It is what a one-pass drafter gets wrong, and how little it costs to fix.

The stutter

Predict every position independently and each pick is plausible on its own. Nothing makes the picks fit together. Position two does not know that position one already used the verb, so if the verb is also the most likely token in position two's marginal, position two says it again. The block reads "verifies verifies", the target rejects the second one, and every correct token drafted after it is discarded along with it. One repeated word can cost four accepted tokens.

The fix in DFlash 2 starts from an observation about the candidates the drafter already produces. Inco measured a five-layer Qwen3-4B drafter on GSM8K and looked not just at the top pick but at the top sixteen. At the first draft position the top pick is right 85.4% of the time. The right token is somewhere in the top sixteen 99.5% of the time. The pattern holds down the block: by position seven the top pick is right 72.9% of the time and the top sixteen still contain the answer 87.8% of the time. An oracle that always chose the right candidate from the sixteen would lift the mean acceptance length from 4.27 tokens per pass to 6.79. That gap is pure selection headroom. The drafter already knows; it just picks alone.

Selection beats correction

Two earlier methods, Domino and DSpark, buy coherence the expensive way: sequential heads that rewrite each position's full-vocabulary distribution after seeing the position before it. That is autoregression sneaking back into the drafter. DFlash 2 keeps the top sixteen candidates per position and scores every adjacent pair with a single expression: the drafter's own logit for the candidate, plus a bilinear term that asks how well the candidate follows its predecessor. Each token gets a compact 256-dimensional embedding on each side of the pair, and a gate computed from the hidden state decides which parts of the match count. Inco calls it a low-rank bilinear attention over adjacent candidates, which is exactly what it is.

The scoring stays parallel. Every adjacent pair at every position is scored in one shot, with no extra backbone pass and no extra language-model head. The only sequential work is a walk over scores that already exist: start from the last verified token, follow the best successor at each step. Under sampling the walk draws from the same scores and rejection sampling restores the target distribution, so losslessness survives. On the five-layer Qwen3-4B drafter, the selector alone adds 2.0 million parameters and 0.6% to the draft-verify cycle and lifts acceptance from 4.27 to 4.61 tokens at temperature zero, 3.78 to 4.25 at temperature one. DSpark's sequential correction adds 77.8 million parameters and 9.6% latency for 4.49 and 4.08. Roughly forty times fewer parameters, sixteen times less overhead, better result. The oracle is still at 6.79, so pairwise scoring is the simplest selector that works, not the last word.

The decay at the end of the block

Look again at that recall table and a second problem shows up. Both rows decline toward the end of the block, and the oracle declines with them: even perfect selection falls from 99.5% at the first position to 87.8% at the last, because the candidates themselves are running out. Inco calls this suffix decay, and it belongs to the backbone, not the selector. The obvious suspect is capacity. Three-, five-, and fifteen-layer drafters are nearly identical at the first position and fan apart down the block, so depth does help where the decay lives. But depth is indiscriminate: ten extra layers add capacity everywhere, including early positions that had nothing left to gain, and they add 15.2% to cycle latency, which erases much of what made a one-pass drafter attractive.

The attention maps say where the work actually is. A drafter's attention has two jobs, reading the context before the block and modeling dependencies inside it, and it spends less and less on the second: the block's share of attention falls from about 30% in the first layer to 8% in the fifth, concentrated in a shrinking handful of heads. So DFlash 2 splits the jobs. Following Canon Layers and dynamic short convolutions, it inserts a two-tap depthwise convolution before and after every attention and feed-forward sublayer: each position mixes its own representation with its predecessor's, with weights that combine a learned base kernel and a small correction from the current hidden state, one correction shared per sixteen channels. The first position reads the last verified token. Information crosses the block one neighbor at a time while every position still computes in parallel, and the module is stateless and block-local, so attention, the head, and verification are untouched.

The numbers behind the figure: the convolution adds 16.5 million parameters, about 3% of the drafter, and 0.7% to cycle latency, and it carries the five-layer model most of the way to the fifteen-layer one on late positions. Average within-block attention in layers four and five drops from 9.4% to 0.5% once the convolution is in, which is what you would expect if the local work moved into the local operator and attention went back to reading context. A kernel that reaches one position back recovers most of what ten extra layers buy. Suffix decay is mostly local.

What the two parts add up to

Measured together on Qwen3.5-4B with thinking on and the model's default sampling, the mean acceptance length across GSM8K, MATH-500, HumanEval, MBPP, and MT-Bench goes from 4.54 for the model's built-in MTP head to 4.92 for DFlash, 5.49 for DSpark, and 5.97 for DFlash 2. That is 1.05 tokens over DFlash, a 21% gain, and 0.48 over DSpark, for 1.3% added cycle latency. On MATH-500 the conditional acceptance rate for DFlash 2 sits near 86% all the way to the fifteenth position; every baseline ends the block six to nine points lower. The selector fixes the picks; the convolution keeps the candidates alive to the end.

The release ships two production drafters. For Qwen3.8-27B, a 2B-parameter drafter at block size 8 reaches a mean acceptance of 4.80 tokens against 4.28 for the model's own seven-token MTP path and 3.62 for a community DSpark drafter. On one H200 with SGLang, that turns 68.9 tokens per second of autoregressive decoding into 236.1 on GSM8K at concurrency 1, a 3.43x speedup, and between 2.67x and 3.43x across the five tasks. For Meta's Muse Glimmer at block size 16, the mean acceptance is 5.70 against 4.44 for the DFlash drafter Meta ships with the model and 4.48 for a DSpark one, which Inco reports as 3.1x to 4.6x over autoregressive. All of these are the vendor's numbers on the vendor's hardware. The shape of the table matters more than any single cell, and the shape is the same on every row: a full token per pass over the previous best.

Where it helps and where it stops

Read the concurrency-32 column before you plan capacity around the batch-1 number. At 32 concurrent requests on the same H200, autoregressive decoding already produces 1,330 tokens per second on GSM8K because the batch fills the arithmetic units on its own. DFlash 2 still adds 45% there, but MTP and DSpark are worth almost nothing, and on MT-Bench the DFlash 2 speedup is 1.01x. Speculative decoding sells idle compute back to you; when a big batch has already bought it, there is less left to sell. The place this drafter changes the bill is exactly where agents live: one request, one long trace, many thousands of tokens of tool calls and reasoning, latency-bound rather than throughput-bound. Inco's line is that an agent writes in an afternoon what a chatbot writes in a month, and that afternoon is spent in decode passes; by their arithmetic DFlash 2 takes about a third of the compute per token at that end of the curve.

Two more things worth holding in mind. Acceptance is task-shaped: GSM8K and MATH-500 accept more than five tokens per pass because arithmetic and proof steps are predictable, HumanEval and MBPP a little over four, and open-ended MT-Bench 4.10. Your prompts will land somewhere on that range, and the only way to know where is to measure your own acceptance length. And the drafter is tied to its target: the Qwen3.8-27B drafter drafts for Qwen3.8-27B, at the model's own sampling parameters, and a different fine-tune or a different quantization of the target shifts the distribution the drafter was trained to predict. Inco's post ends with an offer to train drafters for custom fine-tunes, which tells you how the company expects to make money from an open method.

Running it

Both mainstream GPU engines carry the method now. The vLLM implementation, pull request 52816, merged on August 21; the llama.cpp implementation, pull request 27342, merged on August 27, after a reviewer on the vLLM side had first asked where the method came from, because the pull request went up before the checkpoints and the post were public. Inco's own instructions, written the day of release, still point at the pull-request branches, so treat these as the current shape and check that your build is on a release that includes the merge.

# SGLang
python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
  --speculative-num-draft-tokens 8

# vLLM (mainline since the August 21 merge)
vllm serve Qwen/Qwen3.8-27B \
  --speculative-config '{"method": "dflash", "model": "incoai/Qwen3.8-27B-DFlash2", "num_speculative_tokens": 7}'

# llama.cpp (mainline since the August 27 merge)
./build/bin/llama-server \
  -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M \
  -hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \
  --spec-type draft-dflash --spec-draft-n-max 7

Note the two conventions for the same thing: SGLang counts the block including the verification token, so 8, while vLLM and llama.cpp count draft tokens, so 7. Ollama has a pull request and a Modelfile recipe with a DRAFT line, and oMLX ships a prebuilt build for Apple Silicon that Inco's demo video runs on an M5 Max. The GGUF drafter is quantized alongside the target, and the community has started measuring: one r/LocalLLaMA benchmark of the pull-request build on Qwen3.8-27B reported 2.26x on 100 real coding prompts, which is a useful sanity check against the vendor's 3.11x on HumanEval, since real coding prompts are longer, messier, and less predictable than a benchmark suite. For how the verify step fits into a scheduler, the block pool, and continuous batching, the vLLM internals guide walks the same code paths, and The Inference Engine Underneath covers which engines expose which drafter types.

What to check on your own serving

Three numbers decide whether this is worth a deploy. First, your acceptance length on your prompts at your temperature, because it is the whole gain and it is task-shaped. Second, your typical concurrency per GPU, because the gain shrinks as the batch grows and can round to nothing at high load. Third, the block size, which is a knob: the paper's drafters were measured at 8 for Qwen and 16 for Muse Glimmer, and a block longer than your acceptance length is wasted draft compute while a shorter one leaves tokens on the table. The pleasant part is what you do not have to check. Output is unchanged by construction, the drafter is a 2B-parameter sidecar rather than a second serving stack, and the two additions that make it DFlash 2 cost 1.3% of a cycle. Connect the candidates, keep drafting parallel. The drafter already had the right tokens; it needed a way to agree with itself.