---
title: "When not to use Jev: rules, embeddings, trained classifiers, small and frontier LLMs"
type: synthesis
source_tier: mixed
tags: [decision-guide, alternatives, cascade, rules, embeddings, classifiers]
created: 2026-09-23
updated: 2026-09-23
confidence: medium
sources:
  - raw/docs/model-jaggedness__jev-1.13.md
  - raw/docs/concepts__how-to-build-with-system-one.md
  - raw/docs/concepts__system-one.md
  - raw/docs/introduction__machine-learning-primer.md
  - raw/docs/models.md
  - raw/docs/concepts__use-case-map.md
  - raw/docs/patterns__confidence-routing.md
  - raw/docs/cookbooks__sde_cascade.md
  - raw/docs/cookbooks__pre_parsed_value_extraction_cookbook.md
  - raw/docs/cookbooks__rerank_typesafe.md
  - raw/docs/cookbooks__classifying_rag_passages.md
  - raw/docs/cookbooks__entity_alignment.md
  - raw/docs/cookbooks__autoresearch_feature_discovery.md
  - raw/docs/cookbooks__classification_using_confidence.md
  - raw/docs/cookbooks__consistency_choice_cookbook.md
  - raw/github/skills/skills/typesafe-ai/SKILL.md
  - raw/site/blog-introducing-system-one.txt
  - raw/community/latent-space-jev-diogo-almeida.md
  - raw/community/diogo-almeida-coding-agent-notes.md
  - raw/community/archerhume-jevs-architecture-unmasked.md
  - raw/nate/jev-shaped-problems.md
  - raw/x/trycua-2101437979180904640.md
  - raw/x/0xidanlevin-2100937437325205568.md
  - raw/x/jpschroeder-2100952337904767258.md
  - raw/x/LangChain-2101454284927959080.md
  - raw/x-repos/kyotofin__tax-doc-classifier.md
  - wiki/syntheses/jev-vs-llm-structured-outputs.md
  - wiki/ideas/field-reports.md
  - wiki/ideas/measurements.md
jev_version: "jev-1.13.0"
summary: "Per alternative (rules, embeddings, trained classifiers, small and frontier LLMs, a human): where it beats Jev, where Jev wins, how to combine them, plus a situation-by-situation verdict table."
---

# When not to use Jev: rules, embeddings, trained classifiers, small and frontier LLMs

> **TL;DR** Jev for closed-set language judgements at volume that code consumes; code for anything computable exactly; an LLM for anything generated or reasoned over several steps; a person for uncertain or irreversible cases. Default design: **rules floor → Jev → escalate** on confidence. The sources barely discuss embeddings or trained classifiers as *rivals*; those claims are labelled (inferred) or community.

## The question

"For this decision, is Jev the right tool, or does something else win — and when should they be combined?" Jev vs an LLM in JSON mode is covered in [[syntheses/jev-vs-llm-structured-outputs]]; this page covers the wider field.

Labels: **docs** (wins every conflict), **first-party** (TypeSafe staff outside the docs), **community** (`verified` / `contradicts docs` / `unverified`). Every number is one attributed run.

## What Jev says it is bad at

From [[concepts/jaggedness-jev-1-13]] and [[concepts/system-one]] (docs):

| Out of scope for Jev | Docs' remedy | So the work goes to |
|---|---|---|
| Arithmetic and counting: "Jev is not a calculator" | "Keep the arithmetic in code". If a regex or parser can find the unit, "the count belongs in code" | Code |
| Date ordering, durations, windows | "Extract components; compare in code" ([[cookbooks/date-extraction]]) | Jev extracts, code computes |
| Generation: "not trained to generate text"; chaining choices "will not work well and will be very slow" | "Use a generative model" | LLM |
| Multi-hop indirection, System Two tasks | Reduce hops. The model "does not generate code or choose its own next action" | Code or a reasoning LLM |
| Large state with irrelevant detail | "Filter first" | Code or retrieval before Jev |
| Numeric representations (hex, RGB, binary) | Convert in code, pass a number or named bucket | Code |
| Images, audio, video | Text only; pre-process ([[reference/models-and-pricing]]) | Another model upstream |

The design rule behind the table, from [[concepts/how-to-build]]: "Keep control flow, deterministic rules, and side effects in code." Step one there is "Use code when you can… It is reliable and cheap."

## Alternative by alternative

### Deterministic code, rules, regex

- **Beats Jev:** anything exact, such as arithmetic, dates, counting, identities, schema checks and side effects (docs, above). Jev does not guarantee identities: `P(refund)` + `P(not refund)` summed to 1.19 in the docs' own example ([[concepts/jaggedness-jev-1-13]] §8).
- **Jev beats it:** language judgement that keywords miss. In [Nate B. Jones](https://unlock-ai.natebjones.com/guides/jev-shaped-problems)'s tests, keyword rules got 4/12 and Jev got 11/12 on tricky support messages. A "v means major release" rule got 1/8 and Jev got 8/8 on changelog titles (community, `unverified`: small hand-picked sets. The direction matches the docs).
- **Combine:** code settles deterministic states before the call ("Handle deterministic states without calling a model", docs' `triage_ticket`). A recall-tuned regex over-finds, Jev picks one, so the value is a verbatim match ([[cookbooks/pre-parsed-value-extraction]]); [[cookbooks/citation-check]] string-matches first. Code owns thresholds and weights ([[patterns/composite-scoring]]).

### Embeddings, nearest neighbour, similarity search

- **Beats Jev:** the first pass over a whole corpus. The rerank cookbook calls BM25 or dense embeddings "fast search" that can "run on the whole pile"; judging every document means "millions of comparisons per query" (docs). No source gives embedding latency or price.
- **Jev beats it:** judging each shortlisted candidate. On 40 CLERC legal queries, BM25 shortlists were reranked with one `Noul` per pair (1,200 calls, $0.0645). Top-1 accuracy rose from 5% to 18% and top-10 from 38% to 62% ([[cookbooks/rerank]], docs, TypeSafe's run). Jev also labels relevance, contradiction and injected instructions per passage ([[cookbooks/classifying-rag-passages]]).
- **Combine:** retrieve with embeddings or BM25, then have Jev judge the top-k. The RAG cookbook keeps the top 12 by cosine similarity. [[cookbooks/entity-alignment]] judges 450 *pre-generated* candidate pairs; Jev does not generate them. The use-case map's "Replace or supplement embeddings" is TypeSafe's claim; its cookbooks only show "supplement".
- **Silent:** no source compares embeddings with Jev for dedup or clustering at scale. Using embeddings to block and Jev to confirm near-duplicates is (inferred).

### Trained classifiers (classic ML, task-trained small models)

- **Docs position:** Jev "is not fine-tuned or LoRA-adapted with customer data". You adapt it through `state`, `instructions`, `criteria` ([[reference/models-and-pricing]]). When you have labels, the documented route is Jev probabilities as **features** for a classical model: CatBoost in [[cookbooks/autoresearch-feature-discovery]], and "feed into a classical ML model" in [[concepts/how-to-build]]. Without labels, the docs suggest an ensemble of reasoning models to make them.
- **Beats Jev (community):** a task-trained specialist scored 99.7% decision accuracy against 83.6% for hosted Jev on one form task, at 7–9 ms local against 260–280 ms hosted ([@trycua](https://x.com/trycua/status/2101437979180904640); `unverified`, narrow). The authors say the two measure "different boundaries". A 0.6B replica beat Jev on ViZDoom Basic ([[ideas/field-reports]]; `unverified`).
- **Jev beats it:** no training set or training loop, and new categories need only edited `criteria` (docs). @nedwize's tax-form classifier shipped without training: 0 wrong on 1,067 pages, compared with a Claude Sonnet classifier rather than a trained model ([[ideas/measurements]]; community, `unverified`).
- **Combine:** Jev answers become features and a small labelled set fits the weights. @razeden0 fitted weights over five Nouls on 1,000 labelled examples ([[ideas/field-reports]]; community, numbers `unverified`).
- **Silent:** no source measures a classic classifier (logistic regression, gradient boosting, fine-tuned BERT) against Jev on stable categories with many labels. **Graduate-to-classifier** is (inferred): launch on Jev, log answers and corrections, and once a labelled set proves a local model matches *your* decisions, move the stable high-volume categories to it; Jev keeps the long tail. [[ideas/measurements]] advises the same for self-hosted replicas (community).

### Small fast chat LLMs with JSON mode

- **Beats Jev:** generation, including tool arguments, search queries and open extraction. In WindTunnel, Jev picked tools and Mercury 2.5 wrote the arguments ([@0xidanlevin](https://x.com/0xidanlevin/status/2100937437325205568); `verified` direction, jaggedness §9). Cached input can be cheaper per token: @jpschroeder puts Jev at about 7x a DeepSeek V4.1 Flash cache read ($0.006) ([[ideas/field-reports]]; price `verified`). In TypeSafe's consistency run, Haiku 4.5 at temperature 0 was *more* repeatable than Jev (mean probability std dev `0.0012` against `0.0098`) ([[cookbooks/consistency-choice]], docs).
- **Jev beats it:** in that same run, Jev took 114 ms and $0.000046 per call. The cheapest small-LLM setting (gpt-5.4-mini single-pick, t=0) took 826 ms and $0.000936. Jev also returns calibrated `probabilities` and cannot return a value outside your schema. The JSON-mode comparison is on [[syntheses/jev-vs-llm-structured-outputs]].
- **Combine:** the small LLM extracts or generates and Jev verifies. In the [[cookbooks/sde-cascade]] cascade, `gpt-5.4-mini` extracts, a battery of Jev `Noul`s scores P(wrong) for each field, and anything over `0.7` escalates. Or the LLM proposes candidates and Jev selects: "Select instead of generate" ([[reference/agent-skill]]).

### Frontier and reasoning LLMs

- **Beats Jev:** multi-hop reasoning, choosing the next action, writing prose or code, explanations for a reviewer, unfamiliar processes. TypeSafe itself lists chatbots, copilots and coding agents as LLM territory ([[concepts/system-one]]). Jev is "not a drop-in replacement" for a coding agent's LLM ([[concepts/jev-with-coding-agents]]).
- **Jev beats it:** narrow calls at volume. In TypeSafe's consistency run, gpt-5.5-reasoning took 12,978 ms and $0.041255 per call against Jev's 114 ms and $0.000046 (docs, TypeSafe's own run; TypeSafe says its published evals are "generally run from our laptops on the West Coast"). On judging, @LangChain reports quality-score variance 92–913x lower than GPT-5.6 Luna, Terra and Claude Sonnet 4.6 (community, `verified` in direction).
- **Combine:** use the frontier model as the **escalation rung** of a cascade. Alone, gpt-5.5-reasoning reached about 0.81 quality at about $0.10 per extraction. The cascade's frontier sits "up-and-left of every single model" ([[cookbooks/sde-cascade]], internal TypeSafe results). They also generate labels.
- **first-party caveat:** in an agent loop, routing to a smaller model and back "can cost more because the context needs to be re-processed by the larger model" (Diogo Almeida, [[ideas/field-reports]]). Cascade independent items, not mid-session turns (inferred).

### A human

- **Beats Jev:** high-stakes or irreversible actions, and cases where the signal isn't in the text. Diogo Almeida gives the example of sarcasm combined with a VIP customer: "this is when we escalate to a human" (first-party). Jev judges the text; it cannot forecast outcomes ([[ideas/field-reports]], community).
- **Jev beats it:** volume and speed. TypeSafe's target is automation that is "99% machine-to-machine interactions and 1% human interaction" ([[concepts/machine-learning-primer]]).
- **Combine:** use a confidence floor. In the docs' banking example, anything below `0.6` goes to a person, and a transfer needs more than `0.85` or else the user is asked to confirm ([[patterns/confidence-routing]]). In [[cookbooks/classification-using-confidence]], a `0.9` cutoff split 60 filings in half: 90% correct above it, 40% below. Human labels are also how you tune those thresholds ([[guides/testing-and-evaluation]]).

## Decision table

| Situation | Recommended tool | Why | Combine with |
|---|---|---|---|
| Closed-set language judgement at volume | **Jev** | Narrow, typed, calibrated, about 100 ms ("Most queries complete in about 100 ms", docs) | Rules floor before it, confidence gate after it |
| Needs generation (text, code, arguments, summaries) | **LLM** (small if routine, frontier if hard) | Jaggedness §9 | Jev as guardrail or verifier ([[cookbooks/llm-guardrails]], [[cookbooks/sde-cascade]]) |
| Needs arithmetic, counting or date math | **Code** | Jaggedness §2–3 | Jev extracts the components as a Choice ([[cookbooks/date-extraction]]) |
| Stable categories with lots of labels | **Trained classifier** (inferred; no source compares) | Task-trained specialists won narrow tasks (community, `unverified`) | Jev probabilities as features, Jev for the long tail |
| Similarity search or dedup over a large corpus | **Embeddings or BM25** for recall | The docs' "fast search" step | Jev judges the shortlist or candidate pairs ([[cookbooks/rerank]]) |
| One-off, low-volume decision | **Frontier LLM or a person** (inferred) | Writing and testing criteria doesn't pay off at low volume (inferred) | Nothing needed |
| Must be bit-reproducible or audited | **Code**. Jev only with its answers stored | Docs promise "extremely consistent", not bit-exact. There is no seed or temperature parameter. Archer Hume saw small differences between identical requests (community, `unverified`) | Pin `jev-1.13.0`, log the response `model` field and the probabilities ([[reference/models-and-pricing]]) |
| Latency budget in single-digit ms | **Code, or a local trained model** | Hosted round trip was 260–280 ms against 7–9 ms local (@trycua, community). Docs say "about 100 ms" | Jev off the hot path: async, cached, or to label training data (inferred) |
| High-stakes, irreversible action | **A person or a confirmation step** | Confidence is "not overall workflow correctness or permission to act" ([[reference/agent-skill]]) | Per-action thresholds ([[patterns/confidence-routing]]) |

## The default composite: rules floor, Jev, escalate

1. **Rules floor.** Code settles everything exact and deterministic, filters the state, and builds the candidate list, using a regex, retrieval or an LLM proposer ([[concepts/how-to-build]]).
2. **Jev.** Many atomic questions in one request, each over a closed option set with an explicit `other` / `none` ([[patterns/fan-out]], [[concepts/choice]]).
3. **Compose in code.** Use weights or a classical model over the probabilities ([[patterns/composite-scoring]]).
4. **Escalate on uncertainty.** Send cases to a frontier model or a person: "if it's in the middle, then you do the next bigger model" (Diogo Almeida, first-party; [[cookbooks/sde-cascade]] is the documented form).
5. **Graduate** stable, high-volume slices to a trained model once your labels support it (inferred).

## Related

- [[syntheses/jev-vs-llm-structured-outputs]]: the Jev vs JSON-mode comparison this page does not repeat
- [[concepts/jaggedness-jev-1-13]]: the official failure-mode list
- [[concepts/how-to-build]]: keep code in control
- [[patterns/confidence-routing]]: floors and per-action thresholds
- [[cookbooks/sde-cascade]]: the documented cascade
- [[ideas/field-reports]]: community wins and failures
- [[ideas/measurements]]: community numbers
- [[entities/latent-space-interview]]: first-party remarks on cascades, regex and fine-tuning

## Sources

- raw/docs/model-jaggedness__jev-1.13.md (https://docs.typesafe.ai/model-jaggedness/jev-1.13): failure modes 1–9
- raw/docs/concepts__how-to-build-with-system-one.md (https://docs.typesafe.ai/concepts/how-to-build-with-system-one): "Use code when you can", classical-ML composition, routing on uncertainty, about 100 ms
- raw/docs/concepts__system-one.md, raw/site/blog-introducing-system-one.txt: scope and TypeSafe's LLM comparison
- raw/docs/introduction__machine-learning-primer.md: 99%/1% machine-to-human framing
- raw/docs/models.md: no fine-tuning, text only, aliases and pinning
- raw/docs/cookbooks__sde_cascade.md, cookbooks__pre_parsed_value_extraction_cookbook.md, cookbooks__rerank_typesafe.md, cookbooks__classifying_rag_passages.md, cookbooks__entity_alignment.md, cookbooks__autoresearch_feature_discovery.md, cookbooks__classification_using_confidence.md, cookbooks__consistency_choice_cookbook.md (https://docs.typesafe.ai/cookbooks)
- raw/docs/patterns__confidence-routing.md, raw/docs/concepts__use-case-map.md, raw/github/skills/skills/typesafe-ai/SKILL.md
- First-party: raw/community/latent-space-jev-diogo-almeida.md (Latent Space, 2026-09-21: regex, fine-tuning, cascade, human escalation); raw/community/diogo-almeida-coding-agent-notes.md (routing cost)
- Community: raw/nate/jev-shaped-problems.md (Nate B. Jones, https://unlock-ai.natebjones.com/guides/jev-shaped-problems); raw/x/trycua-2101437979180904640.md; raw/x/0xidanlevin-2100937437325205568.md; raw/x/jpschroeder-2100952337904767258.md; raw/x/LangChain-2101454284927959080.md; raw/x-repos/kyotofin__tax-doc-classifier.md; raw/community/archerhume-jevs-architecture-unmasked.md