Rohit Ghumare · September 25, 2026 · 19 min read

Laya and Jev: System 1 Models Answer With a Type, Not Text

A new class of model takes your text and a question with a fixed set of answers. It returns a probability for every answer and generates no text. Laya does this in one pass through a bidirectional encoder. Jev also scores typed questions in parallel, but TypeSafe has not published its architecture. Because nothing is generated, a routing or yes/no call takes tens of milliseconds and costs a tiny fraction of an LLM call. The result is a type your code can branch on. The probability is worth only what your own labeled data says it is worth. On the open checkpoint I ran, it was often confidently wrong.

The hosted, closed version is Jev from TypeSafe AI, which TypeSafe launched on September 15, 2026. TechCrunch covered it on September 18. The open one is Laya, with Apache-2.0 weights and code by Nandakishor M of Convai Innovations. Laya went public on GitHub and PyPI on September 18, and it ships a server that speaks Jev's wire format. On September 25 I installed Laya 0.3.20 and ran its smallest checkpoint on a laptop CPU. Each Laya number below says whether I measured it or Laya's README reports it. Each Jev number says whether it comes from TypeSafe or a third party.

I follow one support ticket through both systems: the request shape, how Laya turns it into probabilities, and the cost in milliseconds. Then come the places where the probabilities lie, how to pick a confidence threshold, and whether to move real ticket volume off an LLM.

A decision is a probability vector

When you classify with an LLM, it writes the label as text and you parse the text. The model assigns probabilities to next tokens, not directly to your options. You can constrain decoding to your labels, or normalize token probabilities over them. Both methods sit on top of text generation. A System 1 model is trained on the decision itself. You send a state (the ticket, a JSON record, a transcript) and a set of named questions. Each question has a type and a closed set of answers. For each question, the model returns a distribution over exactly the answers you wrote. TypeSafe's primitives page defines three types:

• Choice picks one option from a set and returns the choice, a probability per option, and a confidence.
• Score rates the state against ordered, described levels and returns a score, a probability per level, and a confidence.
• Noul returns one number, the probability that a statement holds, for use in an if. It carries no confidence field.

Here is the ticket this page uses, as a request to Jev's endpoint in the shape its quick start documents:

curl https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "jev-1.13.0",
       "state": "I was charged twice for my September invoice. Please refund the duplicate charge today.",
       "questions": {
         "department": {"type": "choice",
                        "instructions": "Which team should handle this message?",
                        "criteria": {"billing": "payments, invoices, refunds, charges",
                                     "technical": "bugs, errors, outages, login problems",
                                     "sales": "pricing questions, upgrades, new plans"}},
         "refund_requested": {"type": "noul",
                              "instructions": "Is the customer asking for a refund?"}}}'

I pin the model to jev-1.13.0 instead of the jev-latest alias. TypeSafe's models page warns that an alias moves when a release ships. It says to pin the versioned ID once you have tuned thresholds against it. I had no TypeSafe key, so I did not send this request. Every response on this page comes from Laya, which accepts the same body.

What "System 1" means here, and what it does not

The name is Kahneman's fast, automatic mode from Thinking, Fast and Slow, which TypeSafe credits. In machine learning research the phrase already had a narrower meaning. Meta FAIR's Distilling System 2 into System 1 (arXiv 2407.06023, July 2024) trains an LLM to answer without writing a chain of thought. The answer should match what the chain would have reached. UNC's System-1.x (arXiv 2407.14414, July 2024) frames planning as a controllable mix of fast direct plan generation and slow explicit search. In both, System 1 means answering without visible intermediate reasoning, and the model still generates text.

Jev and Laya narrow it again: no generation at all, and an output space the caller fixes in advance. The name does not say the model reasons like human intuition, and it does not mean a small or cheap LLM. The probabilities are also not calibrated for your data by default. Calibration is a training target. You measure whether it holds on your labels, and the rest of this page shows how.

Jev: one set of weights behind an API

TechCrunch names the founder as Diogo Almeida, a former OpenAI researcher who worked on ChatGPT. It reports that he is tight-lipped about the architecture, and observers in that piece suspect an open-weight LLM underneath. TypeSafe documents the contract instead. Its machine learning primer says Jev is trained with RLCD, "Reinforcement Learning for Calibrated Decisions."

The models page says the same weights serve every account. Jev is not fine-tuned or adapted with customer data, so you shape it through the state, instructions, and criteria. The page prices Jev 1.13 at $0.042 per million input tokens, with output tokens free. A request can hold 64k tokens, and the state plus the longest question can use 32k. Rate limits are 250,000 tokens per second and 1,200 requests per minute, which the page says are adjusting under demand. It calls English Jev's strongest language.

TypeSafe launched Jev on September 15, 2026, according to its launch post. The post page also shows a September 25 timestamp, which is its last edit. A funding figure appears only in secondary coverage, so I left it out.

TypeSafe's own performance claims are vendor claims: an end-to-end response time of 70 to 500 ms, and 40x to 200x faster than LLMs. The customer numbers in TechCrunch are the customers' own. Vercel saw 5 to 18 times faster results than an OpenAI model. Bryo AI found Jev 10 to 20 times cheaper than Gemini for email classification.

One detail changes how you port code between the two systems. Jev's confidence page derives confidence from the probability distribution. Its three-option demo uses (3 × p_max − 1)/2. The general form of that demo formula is (n·p_max − 1)/(n − 1): 0 for a uniform distribution, 1 for a certain one. Laya's confidence is 1 minus normalized entropy. For the urgency answer below (probabilities 0.0118, 0.4629, 0.5253), the demo formula gives (3 × 0.5253 − 1) / 2 = 0.288, while Laya reported 0.3199. Laya also exposes the probability of the chosen answer as answer_confidence, which is 0.5253. One distribution gives three numbers, and a threshold tuned on one does not carry to the others.

Laya reads the answer off mask tokens

Laya's architecture is all in the repository. build_sequence (laya/common.py:94-146) renders one question into one token sequence. The sequence holds the question type and instructions, then a <mask> marker per option with the option's key and description, then the state. The model reads answers only from those marker positions. The encoder is bidirectional, a ModernBERT or mmBERT. So before scoring, each marker's hidden state has attended to the whole state and to every other option.

DecisionModel.forward (laya/common.py:181-217) adds a learned embedding for the question type. It can refine the result with a two-layer transformer head. Then torch.gather pulls the hidden state at each marker position, 768 numbers each on mmBERT-base. One shared scorer (LayerNorm, Linear, GELU, Linear to 1) turns each into a single logit.

Decoding (laya/agent.py:759-814) depends on the type. A Choice is the argmax of a softmax over the logits. A Score is the expected level Σ i·p_i. A Noul renders two markers, false and true, and reports p(true). The figure replays that path on the real token sequences and real logits from my run.

Two things in that figure matter for practice. First, each question is its own sequence, and Laya encodes the state again inside each one. The three questions for this ticket render as 66, 55, and 55 tokens. That sum, 66 + 55 + 55 = 176, is exactly the input_tokens Laya reported for the call. Jev's models page says it ingests the state once and evaluates every question against it. On Laya, cost grows with questions times state length.

Second, the negated example shows an encoder error. The reader arithmetic is correct, and the encoder put more weight on "cancel" than on "do not."

Laya divides the logits by a temperature before the softmax. It fits one temperature per question type and option-count bucket (2, 3 to 5, 6 to 10, 11 or more). A clamp keeps each temperature between 0.5 and 5.0. Per a comment in laya/common.py, the clamp exists because the shipped choice:11+ temperature was 0.1006. That value would publish a 0.24 top probability as 0.99. On the four questions in the figure, the reported probabilities equal a plain softmax of the raw logits.

Laya ships three checkpoints (README.md:113-119). The option text and the state share one window, split at head_max_len. The room left for your state is the context minus that option budget:

A router (laya/lang.py, laya/router.py) picks the checkpoint per request from script and stopword detection. It uses no learned model and, according to the README, decides in under 0.5 ms. An explicit model field overrides it. Skipping the router is costly: the README reports the English checkpoint at 0.000 accuracy on Khmer, with 0.952 mean confidence.

Running the smallest checkpoint on a laptop

The multilingual checkpoint is the smallest at 322M parameters. The install is one package, and it pulls in torch and transformers (I got torch 2.14.0 and transformers 5.17.0):

python3.12 -m venv .venv
.venv/bin/pip install "laya==0.3.20"

Load the checkpoint and ask all three question types at once. The dict has the same shape Jev takes. The state here is a JSON object, which both systems accept:

import json
import laya

agent = laya.load("convaiinnovations/laya", subfolder="multilingual", device="cpu")

QUESTIONS = {
    "department": {
        "type": "choice",
        "instructions": "Which team should handle this message?",
        "criteria": {
            "billing": "payments, invoices, refunds, charges",
            "technical": "bugs, errors, outages, login problems",
            "sales": "pricing questions, upgrades, new plans",
        },
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this message?",
        "criteria": ["not urgent", "somewhat urgent", "very urgent"],
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Is the customer asking for a refund?",
    },
}

ticket = "I was charged twice for my September invoice. Please refund the duplicate charge today."
result = agent.predict({"body": ticket}, QUESTIONS)
print(json.dumps(result["answers"], indent=1))
print(result["usage"])

I removed each answer's action block from the output below. It read {"act_probability": 1.0} on every answer, and the failure table explains why to ignore it.

"department": {"type": "choice", "choice": "billing",
  "probabilities": {"billing": 1.0, "technical": 0.0, "sales": 0.0},
  "confidence": 1.0, "answer_confidence": 1.0}
"urgency": {"type": "score", "score": 1.5135,
  "legend": {"0": "not urgent", "1": "somewhat urgent", "2": "very urgent"},
  "probabilities": {"0": 0.0118, "1": 0.4629, "2": 0.5253},
  "confidence": 0.3199, "answer_confidence": 0.5253}
"refund_requested": {"type": "noul", "noul": 0.9516,
  "confidence": 0.9516, "answer_confidence": 0.9516}
{'input_tokens': 176, 'output_tokens': 0}

The department and refund answers are right and sure. The urgency score of 1.51 sits between "somewhat" and "very," and its distribution is nearly split. A near split is a fair answer to a vague rubric. Keep that 0.5253 in mind, because it decides the routing example later.

I measured latency with time.perf_counter around predict(), after three warm-up calls, with 30 timed calls each. The machine was an Apple M1 Max with 64 GB, CPU only, 8 torch threads:

The first load, including the checkpoint download, took 190 seconds. Three questions cost less than three single calls because predict() batches the three sequences. For comparison, Laya's README reports 32.8 ms for one question on this checkpoint on a T4 GPU. It reports 7.2 ms per question when ten are batched. Laya measured those numbers, and I did not.

The Jev-compatible server

laya-serve exposes the router on POST /v1/systemone (README.md:435-469). I ran it on CPU and pinned it to the multilingual checkpoint, so it would not download the English one:

.venv/bin/pip install "laya[serve]==0.3.20"
LAYA_DEVICE=cpu LAYA_PRELOAD=1 LAYA_MODELS=multilingual LAYA_PORT=8791 .venv/bin/laya-serve

curl -s localhost:8791/v1/systemone -H 'content-type: application/json' -d '{
  "model": "multilingual",
  "state": {"body": "I was charged twice for my September invoice. Please refund the duplicate charge today."},
  "questions": {
    "department": {"type": "choice", "instructions": "Which team should handle this message?",
                   "criteria": {"billing": "payments, invoices, refunds, charges",
                                "technical": "bugs, errors, outages, login problems",
                                "sales": "pricing questions, upgrades, new plans"}},
    "refund_requested": {"type": "noul", "instructions": "Is the customer asking for a refund?"}}}'

{"model":"laya-rl-agent","answers":{
  "department":{"type":"choice","choice":"billing","probabilities":{"billing":1.0,"technical":0.0,"sales":0.0},
                "confidence":1.0,"answer_confidence":1.0,"action":{"act_probability":1.0}},
  "refund_requested":{"type":"noul","noul":0.9516,"confidence":0.9516,"answer_confidence":0.9516,
                      "action":{"act_probability":1.0}}},
 "usage":{"input_tokens":121,"output_tokens":0},
 "routing":{"model":"multilingual","repo":"convaiinnovations/laya/multilingual",
            "reason":"explicit model='multilingual'","detection":null,"workflow":null}}

A plain string state, the form in Jev's quick start, also works. If you set LAYA_API_KEY, the server requires Authorization: Bearer. A Jev client then needs only its base URL changed.

The README lists what does not port. Options share the head_max_len budget, while Jev caps options at 255. Labels start getting trimmed around 20 options. Above 126 short options on laya, or 254 on the others, the server rejects the request with 422. Every Score level needs a description. Laya also computes confidence differently, as shown above.

What the training optimizes

Laya's README says it is trained with reinforcement learning against strictly proper scoring rules. It uses the same acronym TypeSafe uses, RLCD. Neither side documents whether the two methods are related. The shipped reward is proper_reward (laya/common.py:278-304):

def proper_reward(q, target, qtype, mask, w_sph=0.5, w_rps=1.0, log_floor=-9.21):
    q = q * mask
    logq = torch.log(q.clamp_min(1e-12)).clamp_min(log_floor)
    log_score = (target * logq).sum(-1)
    sph = (target * q).sum(-1) / q.norm(dim=-1).clamp_min(1e-9)
    r = log_score + w_sph * sph
    is_score = (qtype == QTYPES["score"]).float()
    if is_score.any():
        k = mask.sum(-1).clamp(min=2).float()
        cdf_q = torch.cumsum(q, -1); cdf_t = torch.cumsum(target, -1)
        rps = (((cdf_q - cdf_t) ** 2) * mask).sum(-1) / (k - 1)
        r = r - w_rps * rps * is_score
    return r

A scoring rule is strictly proper when the only way to maximize the expected reward is to report the true distribution. The log score and the spherical score both have that property. At the level of the objective, the model gains nothing by rounding 0.7 up to 0.99. Finite training, approximation error, and distribution shift can still make the trained model overconfident, as my calibration check shows. Score questions also subtract the ranked probability score, which compares cumulative distributions. If the true answer is "very urgent," mass on "not urgent" costs more than mass on "somewhat urgent" does. Order matters for scores and not for choices, and the reward encodes that.

The optimizer is in the one training notebook in the repo, notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb. It produces the laya-typed-decisions checkpoint from a base checkpoint. The method is closer to evolution strategies than to the PPO used for chat models. For each example, it adds Gaussian noise to the logits four times (GROUP_SIZE = 4, noise scale annealed from 0.4 to 0.1). It scores each noisy distribution with proper_reward and subtracts the group mean to get an advantage. A Gaussian log-likelihood surrogate then pushes the logits toward the better samples, plus a soft cross-entropy term against teacher distributions.

Training uses AdamW at 2.5e-5 for the encoder and 1e-4 for the head, for four epochs. The README puts it at roughly four to five hours on two T4s for about 30,000 questions. A final step fits the temperatures on 400 held-out items. The repo has two gaps: the dataset is missing, and so is the code that trained the two base checkpoints. So nobody can confirm from the repo that the base checkpoints were trained with RL.

The numbers, and who measured them

Laya's README compares itself to Jev 1.13.0. Laya's numbers are its own measurements. Every Jev number in that table is marked "third-party published, never measured here," because the author had no TypeSafe API access. The only live paired run is an archived Chinese benchmark under research/benchmarks/feishu_zh/.

Read the base rows first: on the same typed-decisions set, the two base checkpoints score 0.362 and 0.352. The baselines are 0.461 for the majority class and 0.318 for random, and the teacher ceiling is 0.735. The README says all of the typed-decisions capability comes from fine-tuning. It also says Jev assigned zero probability to the true label on 16% of DAIR Emotion items. That number comes from Laya's side, about the third-party outputs. Banking77 shows the option budget at work: 72 to 77 intents do not fit 256 tokens of option text without trimming.

Where it breaks

Laya's README is unusually frank about its failures, and two of them reproduced in my runs on the first try. The negation run used a two-option intent Choice (cancel_account, keep_account) on four states:

"Please cancel my account today."                -> cancel_account  p=1.0
"Do NOT cancel my account, I still need it."     -> keep_account    p=0.9858
"I do not want to cancel my account."            -> cancel_account  p=0.837   wrong
"Please keep my account open, do not close it."  -> keep_account    p=1.0

Picking a threshold you can defend

Calibration means that among answers reported at 0.8, about 80% are right. Expected calibration error (ECE) measures the gap. You bin answers by reported probability and take the difference between accuracy and mean confidence in each bin. The ECE is the average of those differences, weighted by bin size. It is cheap to compute from any labeled set. I ran this check on 30 tickets I wrote and labeled by hand, 8 to 11 per department, with a few deliberately ambiguous:

import numpy as np

LABELED = [
    ("I was charged twice this month, please refund one of them.", "billing"),
    ("The dashboard shows a 500 error when I open reports.", "technical"),
    ("What does the enterprise plan cost for 200 seats?", "sales"),
    # 27 more (text, gold) pairs
]

rows = []
for text, gold in LABELED:
    a = agent.predict({"body": text}, {"dept": QUESTIONS["department"]})["answers"]["dept"]
    rows.append((a["answer_confidence"], a["choice"] == gold))
conf = np.array([c for c, _ in rows])
hit = np.array([h for _, h in rows], dtype=float)


def ece(conf, hit, bins=10):
    edges = np.linspace(0, 1, bins + 1)
    total = 0.0
    for lo, hi in zip(edges[:-1], edges[1:]):
        m = (conf > lo) & (conf <= hi)
        if m.any():
            total += m.mean() * abs(hit[m].mean() - conf[m].mean())
    return total


print("accuracy", hit.mean().round(3), "ece", round(ece(conf, hit), 3))
for t in (0.5, 0.7, 0.8, 0.9, 0.95):
    keep = conf >= t
    print(f"threshold {t:.2f}: auto {keep.mean():.0%}, accuracy on those {hit[keep].mean():.3f}")

accuracy 0.7 ece 0.215
threshold 0.50: auto 90%, accuracy on those 0.741
threshold 0.70: auto 80%, accuracy on those 0.750
threshold 0.80: auto 80%, accuracy on those 0.750
threshold 0.90: auto 80%, accuracy on those 0.750
threshold 0.95: auto 73%, accuracy on those 0.773

Thirty items are far too few for a real estimate, so look at the shape of the result. Raising the threshold from 0.5 to 0.95 escalated five more tickets and moved accuracy on the rest by three points. Most errors came with high confidence.

"What does the enterprise plan cost for 200 seats?" went to billing at 0.993. "Can we get a discount if we pay yearly?" went to billing at 1.0. The model read price words as billing words, with full conviction. A threshold only helps a model whose confidence tracks its errors. So the order is: fine-tune, measure ECE, then choose the threshold.

The procedure I use needs a few hundred labeled examples held out from any fine-tuning. For each question, sweep thresholds. Take the lowest one where accuracy on the answers kept meets your target for that action. The share of answers below it is your escalation rate, which the cost board below needs as input.

TypeSafe's confidence page makes the same point from the other direction. It suggests 0.5 as a floor for genuinely uncertain answers, and higher thresholds for actions that are harder to undo. Set one threshold per action, not one per model.

Routing 10,000 tickets a day

The pattern below is mine, not from either project's docs. Ask every typed question up front. Act on the answers only when each one clears its own floor. If any answer falls under its floor, send the ticket to an LLM.

FLOOR = {"department": 0.90, "urgency": 0.60, "refund_requested": 0.90}


def triage(text):
    answers = agent.predict({"body": text}, QUESTIONS)["answers"]
    unsure = [q for q, a in answers.items() if a["answer_confidence"] < FLOOR[q]]
    if unsure:
        return {"route": "llm", "unsure": unsure}
    return {
        "route": answers["department"]["choice"],
        "priority": round(answers["urgency"]["score"]),
        "refund": answers["refund_requested"]["noul"] >= 0.5,
    }

I was charged twice for my September invoice. Please refund the duplicate charge today. -> {'route': 'llm', 'unsure': ['urgency']}
The API returns timeouts since this morning. -> {'route': 'llm', 'unsure': ['urgency']}
Hi, quick question about your plans. -> {'route': 'llm', 'unsure': ['department', 'urgency']}

Every ticket escalated, which is what the gate should do here. On this checkpoint the urgency answer never reached 0.6. That is the position bias from the failure table, showing up on real inputs. The department answer on the vague third ticket also fell under 0.9. Before this goes near production, urgency moves to the English checkpoint or gets fine-tuned. The floors should also come from the sweep above, not from me.

For a worked example, take a support queue with 10,000 tickets a day. Suppose 8,000 only need triage (team, priority, refund flag). The other 2,000 need a drafted reply, which stays with the LLM. After fine-tuning, the sweep gives a 10% escalation rate at your target accuracy. Assume the LLM call uses 600 input and 80 output tokens and takes 1.5 seconds per ticket.

Claude Haiku 4.5 has a list price of $1 and $5 per million input and output tokens. At that price, sending everything to the LLM costs 10,000 × $0.001 = $10.00 a day. Routing sends 2,000 drafts plus 800 escalations to the LLM, for $2.80 a day. It also runs 8,000 three-question Laya calls. At my measured 81.2 ms each, that is 650 seconds, about 11 minutes, of one laptop CPU stream. Mean time to a decision falls from 1,500 ms to 485 ms.

With Jev in place of Laya, the triage calls add 8,000 × 392 tokens × $0.042 per million = $0.13 a day. The 392 input tokens come from TypeSafe's own three-question sample. Check three things weekly: a sample of auto-routed tickets against human labels, the escalation rate against the sweep, and every negated request.

Who built it first

The public record gives dates, and it does not settle the question. The typesafe-ai GitHub organization's first public repository, an adapter backed by LLM APIs, is dated August 8, 2026. TypeSafe launched Jev on September 15, and TechCrunch covered it on September 18. Laya's GitHub repository dates from September 18 at 04:46 UTC, and laya 0.1.0 reached PyPI the same day.

On September 19 Nandakishor M posted "I built non-autoregressive decision models with RL a year ago" to Hacker News. The post had 1,353 points and 316 comments when I checked on September 25, and a longer version is on dev.to. The earlier work cited in the post is two solo-authored preprints. They are SalesRLAgent (March 30, 2025) and Confidence-Aware Routing (September 23, 2025). Neither abstract describes a marker-token decision head. On r/LocalLLaMA, one thread asked the opposite question: did TypeSafe's work derive from Laya's author?

In public, the two converged on the same interface and the same acronym within days of each other. Laya's package dates from Jev's launch week, and no earlier. Outsiders cannot settle the question while Jev's architecture stays unpublished. For a builder, the difference is practical. Laya lets you read and retrain the weights. Jev gives you one set of weights and a support contract.

When a System 1 model is the right call

The mechanism gives a rule with four conditions. When all four hold, route a decision to a System 1 model. If any one fails, keep it on an LLM:

1. The answer is a closed set you can describe. Every option needs a key and a one-line description the encoder can read. On Laya, that means about 20 options before trimming. On Jev, it means up to 255.
2. The state fits the window next to the options. The limit is 320 tokens on laya and 768 on the multilingual and fine-tuned checkpoints at default. Jev allows 32k for the state plus the longest question. Longer inputs need summarizing first, and that is an LLM call again.
3. You have labels to calibrate against. You need a few hundred held-out examples per question, an ECE number, and a swept threshold. Without them you are trusting a probability nobody checked. My 30 tickets show how that goes.
4. A wrong answer is cheap or caught. Routing a ticket to the wrong queue is recoverable. Cancelling an account on a negated sentence is not. Put destructive actions behind an independent verifier or a human. A second question to the same model shares its errors.

Then pick between the two. If you need to fine-tune on your own labels, keep data on your machines, or run under 100 ms on commodity hardware, pick Laya. The fine-tuning notebook is the real entry cost there. If you need many options, long states, or English accuracy without training anything, pick Jev at $0.042 per million input tokens. Re-derive your thresholds on the pinned version.