Cheap First, Escalate on Doubt: The Arithmetic of a Model Cascade

A model cascade sends every item to a cheap model first. A confidence signal then decides which items a bigger model also sees. The threshold on that signal is the one control you own, and it trades cost against error. In the first week of November, two teams published their cascades with the numbers attached. A router added a third rule in the same week: do not switch models when the switch costs you the cache.

Hightouch graded 1,000 synthetic identities with Jev first and Claude Opus 5.5 on the doubtful ones, and printed every operating point on November 4. Kiro described the cascade inside its Auto mode on November 6, with a background checker as the signal. OpenRouter released its rebuilt Auto Router the same day. Three days earlier, Google's developer blog had put a Jev classifier in front of Gemini models. This essay follows two earlier pieces on Jev. Laya and Jev covers what a decision model returns and how to measure its calibration. Can Jev See shows that a threshold belongs to one model version. This one works the arithmetic of the gate, shows what calibration does to the cost curve, and ends with a sweep for your own labels.


Machine · A sorting line with a confidence gate

Hightouch's identity grader, drawn as a line. The share sent, the errors, the prices, and the seconds per identity are Hightouch's published numbers for each band. The six-bucket confidence histogram is derived from those bands, so it is as exact as the printed shares. On the harder identities only one band per scorer was published, so the gate locks to it, and the share comes from the printed cost. The three red bars are errors per 1,000 for the cheap model alone, the cascade, and Opus alone. The three stacks beside them are the same three bills. The card movement is illustrative.

One gate, two bills

Call the cheap model's price per item ccheap, the big model's price cbig, and the share of items the gate escalates p. Every item pays the first price. Only the escalated share pays the second. The expected cost per item is ccheap + p × cbig. Latency has the same shape, because an escalated item waits for both answers in turn.

Hightouch's table lets you check that formula against production numbers. Jev cost them $0.05 per 1,000 identities and answered in 0.15 s. Claude Opus 5.5 cost $3.86 per 1,000 and took 2.2 s. Their 30 to 90 confidence band sent 22% of identities to Opus. The formula gives 0.05 + 0.22 × 3.86 = $0.90 per 1,000, and they printed $0.88. The gap is the share rounded to a whole percent. Time works the same way: 0.15 + 0.22 × 2.2 = 0.63 s, which is the number they printed. Every row in their table reproduces this way, and the machine above does the arithmetic for each one.

With a 77x price gap, cost rarely decides whether a cascade pays. The cascade stays cheaper than the big model until the gate escalates 98.7% of items, which is 1 − 0.05 / 3.86. Error is the hard part. The cascade's error rate has two parts: the cheap model's mistakes on kept items, and the big model's mistakes on escalated items. The kept items are the ones the cheap model scored with high confidence. So the first part shrinks only if that confidence tracks correctness. The escalated items are the hard ones, so the second part sits above the big model's average error.

On Hightouch's easier set, Jev alone made 44 errors in 1,000. The 30 to 90 cascade made 12. Opus alone made 10. Sending 220 identities removed 32 of 44 errors for $0.83 more per 1,000. Removing the last two errors took the 10 to 100 band, which sent 74% and cost $2.92. The returns fall off fast, and they fall off differently on each side of the band. Moving the upper edge from 80 to 90 at a fixed lower edge of 50 removed 4 errors for $0.40. Moving the lower edge from 50 to 30 at a fixed upper edge of 90 removed 9 errors for $0.17. The low side removed about five times more errors per dollar.

That inequality says which side to widen next. A 2026 paper, Is Escalation Worth It?, derives the stopping rule. At the optimum, the ratio of expected accuracy gain to expected downstream cost is equal across every deferral boundary. Hightouch did not run that optimizer. Their rows show it unmet near the 50 to 90 band. There the high edge removed about 10 errors per dollar and the low edge about 53. So the low edge is the one to widen first.

Fig. 1 · Published frontier board

Every operating point Hightouch printed, placed by cost per 1,000 identities and errors per 1,000. Walk the band from nothing sent to everything sent. The verdict recomputes the printed cost from the share, prices each error removed since the previous band, and on the harder set derives the hidden share from the printed cost.

30 to 90
selected scorer, bands in orderthe other scorerselected bandOpus alone

Shares, accuracies, costs, and seconds are Hightouch's, from one run on 1,000 pseudopeople identities. Errors per 1,000 are 1,000 minus the printed accuracy times 10, which matches their false positive plus false negative counts. The harder-set share is (cost minus cheap cost) over the Opus cost, because Hightouch printed the cost and not the share.

What calibration does to the curve

Hightouch ran two cheap scorers through the same gate. GPT-6 Luna alone was the better grader: 97.3% accuracy against Jev's 95.6% at a 50% cutoff. In the cascade the order flipped. Jev followed by Opus on the 30 to 90 band reached 98.8% at $0.88. Luna followed by Opus on the 10 to 90 band, the one Hightouch carried forward, reached 98.5% at $0.33. Luna sent only 6% of identities. The authors wrote that most of Jev's mistakes came with low confidence, and most of Luna's came with high confidence. A band catches only the mistakes that arrive with middling confidence.

The published shares show the shape of each scorer's confidence. For Jev, 26% of identities scored below 10 and 34% above 90, so 40% sat in between where a band can reach them. For Luna, 46% scored below 10 and 48% above 90, with 6% in between. Luna left almost nothing in the middle to escalate, and its errors sat at the ends.

Then Hightouch built a harder set of twins, juniors and seniors, and typos in birth dates. The band did not change. Opus alone fell to 95.1% at $5.39 per 1,000. Jev then Opus held 95.3% at $2.94. Luna then Opus fell to 91.6% at $1.15. The printed costs give the shares the post does not print. For Jev, (2.94 − 0.06) / 5.39 is 53%, so the band filled from 22% to 53% of identities. For Luna it rose from 6% to 19%. A calibrated scorer raises its own escalation rate when the data gets harder, so the bill rises and the error stays near the big model's. The price of that property is a bill that moves with no change to the code. The headline 4x holds on the easier set, where 3.86 / 0.88 is 4.4. On the harder set the saving is 5.39 / 2.94, or 1.8x, and the cascade's own cost rose 3.3x.

You can see the same effect without a vendor's data. The script below builds 2,000 labeled items with a fixed random generator. Each item carries the probability a calibrated cheap scorer would report. Its label is drawn from that probability, so the scorer is calibrated by construction. An overconfident scorer reports the same probabilities pushed toward 0 and 1. It makes the same decisions, so its accuracy alone is identical. The big model is wrong on 0.5% to 2.5% of items, more often near the middle. The costs are Hightouch's unit prices.

def lcg(seed=7):
    while True:
        seed = seed * 16807 % 2147483647
        yield seed / 2147483647

def sharpen(q, k):
    a = b = 1.0
    for _ in range(k):
        a, b = a * q, b * (1 - q)
    return a / (a + b)

def make(n=2000, hard=False):
    u, items = lcg(), []
    for _ in range(n):
        p = sharpen(next(u), 2 if hard else 4)
        label = next(u) < p
        big_ok = next(u) >= 0.005 + 0.02 * (1 - abs(2 * p - 1))
        items.append((p, label, big_ok))
    return items

def cascade(items, lo, hi, overconfident=False, c_cheap=0.05, c_big=3.86):
    sent = wrong = 0
    for p, label, big_ok in items:
        r = sharpen(p, 3) if overconfident else p
        if lo <= r <= hi:
            sent, wrong = sent + 1, wrong + (not big_ok)
        else:
            wrong += (r >= 0.5) != label
    n = len(items)
    return sent / n, wrong / n, c_cheap + sent / n * c_big

items = make()
for lo, hi in [(0.5, 0.5), (0.4, 0.6), (0.3, 0.9), (0.1, 0.9), (0.0, 1.0)]:
    for over in (False, True):
        share, err, cost = cascade(items, lo, hi, over)
        kind = "overconfident" if over else "calibrated   "
        print(f"{lo:.1f}-{hi:.1f} {kind} sent {share:7.2%} error {err:5.2%} ${cost:.2f}")
0.5-0.5 calibrated    sent   0.00% error 8.35% $0.05
0.5-0.5 overconfident sent   0.00% error 8.35% $0.05
0.4-0.6 calibrated    sent   5.15% error 6.25% $0.25
0.4-0.6 overconfident sent   1.80% error 7.80% $0.12
0.3-0.9 calibrated    sent  19.25% error 3.40% $0.79
0.3-0.9 overconfident sent   6.65% error 5.70% $0.31
0.1-0.9 calibrated    sent  28.15% error 1.85% $1.14
0.1-0.9 overconfident sent   8.85% error 4.90% $0.39
0.0-1.0 calibrated    sent 100.00% error 1.25% $3.91
0.0-1.0 overconfident sent 100.00% error 1.25% $3.91

Python 3.14, standard library only. The figure below runs the same generator in the browser and prints the same numbers.

Both scorers start at 8.35% error. At the 0.3 to 0.9 band the calibrated one sends 19.25% and reaches 3.40%. The overconfident one sends 6.65% and stays at 5.70%, because its mistakes now report 0.93 instead of 0.70 and sit outside the band. Pass hard=True to make and the calibrated scorer's band fills to 36.75% on its own, while its error at that band is 5.20%. The overconfident scorer sends 12.90% and leaves 10.65% error.

The stretch in this script keeps the order of the items, so no information is lost. A band tuned on the overconfident scorer's own numbers gets the calibrated result back. The band from about 0.073 to 0.9986 holds the same 385 items as 0.3 to 0.9 on the calibrated scale. So this kind of overconfidence hurts in two cases. One is a band carried from one scorer to another. The other is a useful edge that sits too close to 0 or 1 to set with care. Luna is the harder case. Its mistakes sat in the 90 to 100 bucket beside many of its right answers, and an edge at 90 cannot split them.

That is also why the two curves in the figure below share their points. Each curve traces symmetric bands, and a symmetric band on the stretched scale holds the same items as a narrower one on the true scale. The overconfident curve stops early and jumps to everything sent. The asymmetric bands from the sliders are where the two scorers differ.

Fig. 2 · Threshold sweep lab

The 2,000 items from the script above, generated in the browser. Drag the two band edges, pick the scorer and the set, and set the big model's price. The curves trace every symmetric band from nothing sent to everything sent. The dot is your band. The reliability strip bins the chosen scorer's reported probability against the share of items whose label is true.

0.30
0.90
$3.86
calibrated scoreroverconfident scoreryour band on the chosen scorer

Generated data, not a vendor's. The cheap price is fixed at $0.05 per 1,000 and the times at 0.15 s and 2.2 s per item, which are Hightouch's. Error is the share of 2,000 items answered wrong by whichever model answered. The p99 wait is the big model's time plus the cheap model's whenever more than 1% of items escalate.

Picking the threshold on your own data

The order of the steps matters. Start with a few hundred labeled items held out from any tuning, as the Laya essay describes for a single model. Score every item with the cheap model and store the confidence. Then run the big model on every item once and store its answer too. From here every band's share, error, and cost come from stored numbers, with no new calls. A threshold sweep is then a loop over a stored table.

Draw the reliability diagram before you sweep. Bin the stored confidences, and in each bin compare the share of correct answers with the mean confidence. If the cheap model's mistakes sit in the 0.9 to 1.0 bin, no band will find them. The sweep then removes few errors until the band takes in almost every item. The escalation paper gives the reason in one identity. The accuracy gain of score-based escalation over random escalation splits into two terms. One is how well the score separates the cheap model's right answers from its wrong ones. The other is how well it does the same for the later model. A score that also tracks difficulty shared by both models sends items the big model also gets wrong.

Set the target before you look at the curve. Either an error ceiling, measured against the big model's own answers, or a budget per 1,000. CascadeFilter, published on November 5, states the threshold problem this way. Minimize token-aware cost subject to an error constraint relative to the oracle. Then pick the least-cost configuration in a grid you fixed in advance. The fixed grid matters. Every extra configuration you test on the same labeled set raises the chance that the best one looks good by luck. That paper allocates a multiple-testing budget for that reason.

Record the operating point as a share sent, with the band that produced it, and store both with the model version. Can Jev See showed the same thresholds sending 6 photos to review on one Jev version and 20 on the next. In production, the share sent is also your first alarm. Hightouch's band moved from 22% to 53% when the identities got harder, long before anyone could have labeled the new data.

Keep the cascade to two stages. The escalation paper tested five benchmarks and nine confidence scores. Two-model cascades beat the best single model at the same budget by 2.1 to 8.2 points. Allowing more than two models added at most 0.43 points, and in 118 of 135 comparisons it added nothing.

A signal that is not a probability

Kiro's cascade uses a different signal, because its cheap model is the whole coding agent. Every Auto session starts on an efficient model. An independent model reads the session's trajectory in the background, so the session never waits. In Kiro's words, "the working model never grades its own progress." When the check sees "repeated exploration of the same code" or work being redone, Auto hands the session to a more capable model. The handoff carries the work so far. Handoffs only go up.

Kiro measured it on DeepSWE v1.1, three runs each. The initial model alone solved 63.96% ± 2.70 of tasks. The capable model alone solved 70.57% ± 2.08. The cascade solved 69.37% ± 1.56. Across the runs, 80.48% of tasks finished on the initial model and 19.52% were handed off. In the terms above, the cascade recovered 5.41 of the 6.61-point gap, or 82%. It did that with one task in five on the big model. Kiro notes that the handoff tended to come late, during verification and integration, when the initial model worked to show that every requirement held. The numbers come from an internal harness that Kiro says is not identical to the public leaderboard, so compare them only with each other. Kiro's models page prices Auto at 1.0x credits and Opus 5.5 at 2.0x, which is how the cascade shows up on a bill.

The cost formula changes shape here. An escalated session pays the initial model's tokens, then the capable model's tokens from the handoff onward, not a second full pass. The checker is a third term, and it runs on every session. Kiro does not publish the three terms. Two more signals are common. Agreement between samples asks the cheap model twice at a nonzero temperature and escalates when the answers differ. A separate scorer is the oldest form. FrugalGPT in 2023 trained one on the cheap model's own output to decide when to call the next model. On one dataset it matched GPT-4 at 2% of the cost. Before that, pattern recognition called the same idea the reject option. Geifman and El-Yaniv framed it for neural networks as a trade between risk and coverage. In a cascade, the rejected items go to a second model.

A router that counts the cache

A router decides before the cheap model answers. A cascade decides after. The cascade sees the answer and its confidence, which is a richer signal, and it pays for the cheap call on every item. The router sees only the request and pays nothing extra, so it needs a classifier that reads difficulty from the prompt. OpenRouter's rebuilt Auto Router does that with a fast decision model. In one call it rates eleven aspects of a request. Among them are the kind of task, whether the task just changed, how hard it is, and whether speed matters. It also rates whether a bigger model or more thinking would help. A policy then weighs that rating against live prices, cache status, and benchmark results, and picks a model and a reasoning effort.

The rule that matters for an agent loop is about switching. OpenRouter writes that the router prefers a change in reasoning effort when that is enough. It "switches only when the gain is worth the cache it gives up." In its beta it "stayed on the same model for nearly nine of every ten agent turns." Both figures are OpenRouter's. The company also reports that it verified the router on production workloads through typesafe/jev-router and its own beta channel. It claims Auto Code scores in line with Claude Opus 5.5 on its five benchmarks at 74% less per task. Treat that as a vendor benchmark. OpenRouter reports the routing step itself at 0.65 s at the 90th percentile.

The response tells you what happened. Set the slug as the model, opt in to metadata with one header, and the pipeline names the stage that picked the model:

curl https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-OpenRouter-Metadata: enabled" \
  -d '{
    "model": "openrouter/auto",
    "session_id": "ticket-4471",
    "plugins": [{"id": "auto-router", "cost_tier": "medium"}],
    "messages": [{"role": "user", "content": "Add a retry with backoff to fetchUser in api/users.ts"}]
  }'

From the Auto Router and router metadata docs. The model field names the model that answered. The jev-router stage in openrouter_metadata.pipeline lists the candidates with their effort and estimated cost, and the reason for the pick.

Use openrouter/auto-code for a coding harness. Keep the session_id stable across turns, because the router recognizes the conversation from it. cost_tier selects a routing policy, not a reasoning effort. If you want a hard allowlist, use allowed_models, and expect a 404 that names the field when the list leaves nothing eligible.

Google's post is a small-scale version of the same router, and it gives the cache problem in numbers. The team asked Jev one atomic yes-or-no question per task: does the answer need more than one sequential calculation or deduction. A keyword heuristic written on 16 development tasks scored 93.8% on them and 70.6% on 34 unseen tasks. Jev classified all 34. Gemini 3.5 Flash-Lite alone missed 6 of 25 hard tasks. Jev flagged all 6 and escalated them to Gemini 3.8 Flash on low thinking, and the pair solved all 50. That is one run on 50 short prompts, and the post says so. The part that carries over is about context. The post notes that context caching cuts the price of resending a repeated block by 75% to 90%. A cache belongs to one model. An agent that switches models from turn to turn keeps two caches or loses the discount. Their split: route short, self-contained tasks, and keep long, context-heavy turns on one model.

Where it breaks

The cheap model is confident when it is wrong. This is the Luna case. The Laya essay found the same thing in Laya itself: on 30 tickets, 5 of its 9 errors came at 0.98 or above. No band finds those. The fix comes before the sweep: draw the reliability diagram, and if the errors sit in the top bin, change the signal. A separate verifier, agreement between two samples, or a second model's cheap opinion all score the answer from outside. Widening the band instead just buys the big model for items that did not need it.

The data shifts. With a calibrated scorer the band fills and the bill rises, as Jev's 22% became 53% and $0.88 became $2.94. With an overconfident scorer the band fills less and the error climbs further. Luna's share went from 6% to 19%, and its errors went from 15 to 84 per 1,000. Set an alarm on the share sent, in both directions. Pair it with a small weekly labeled sample. Add a budget cap that sends overflow to a review queue instead of keeping the cheap answer with no record.

A switch mid-session empties the cache. An agent turn carries the repository map and the tool definitions on every call. A cache of that prefix belongs to one model. A handoff re-reads it at full price, so the escalated turn costs more than cbig alone. Put the rebuilt prefix into cbig when you price a cascade on long contexts. Prefer an effort change over a model change when the signal is weak, as the router does. Switch at task boundaries, or carry forward only the crux of the work so far, as Kiro's handoff does.

The mean hides the tail. At the 30 to 90 band the mean wait was 0.63 s, and no identity waited 0.63 s. Seventy-eight percent waited 0.15 s and the rest waited 2.35 s, because an escalated item pays both latencies in turn. For a batch grader that is fine. For an interactive path, set a latency budget and route before generation when the big call cannot fit. Or run the check beside the work as Kiro does, so the session never waits on it. On short prompts, Google's test found the shorter wait mattered more than the cost.

The big model is the reference, not a floor. On escalated items the cascade inherits the big model's mistakes. On kept items it beats the big model only where the cheap model is right and the big one is wrong. That happened on Hightouch's harder set, where the cascade made 47 errors against Opus's 49, inside the noise of one run on 1,000 items. On the easier set no band got under Opus's 10. Measure a cascade against the big model's own answers on the same items, and expect to land near them.

Build the sweep this week

  1. Label a few hundred held-out items for one decision, and write down the error you can afford on it.
  2. Score every item with the cheap model and store the confidence. Run the big model on every item once and store its answer.
  3. Draw the reliability diagram. If the errors sit in the top bin, change the signal before you go on.
  4. Sweep a fixed grid of bands over the stored table. Pick the least-cost band that meets your error ceiling, and check the two edges remove errors at a similar price.
  5. Record the band, the share sent, and the model version together. Re-derive all three when either model changes.
  6. Set an alarm on the share sent in production, and price the rebuilt cache into the big model's cost wherever the context is long.

Hightouch's table is the shape to aim for. Each row gives a band, a share, an error count, a price, and a time. Each row also reproduces from the two unit prices and the share.

rg
Rohit Ghumare

CNCF Ambassador and Google Developer Expert. I build agent infrastructure and write about the fundamentals underneath the AI stack. I have no relationship with Hightouch, TypeSafe AI, Kiro, OpenRouter, or Google. Every vendor number here comes from the posts linked inline, read on November 9, 2026, and the text says who measured each one. The 2,000-item set in the sweep is generated on this page by the code shown, with no model calls.

Related: Build a Personal RAG Harness · The Inference Bill · More posts · X