Agents: read the raw Markdown of this page, or start at llms.txt.
~/wiki/ideas
Advisor checklist: what to check before recommending Jev for a decision
TL;DR Run these ten checks on every decision that survives the fit test in Consult guide: could Jev help this project?, before it goes on the shortlist. Each item names the evidence behind it; open that page only if the user asks why. Moved from the end of Failure reports: where Jev broke, lost, or was the wrong tool on 2026-09-25 (every item and link kept) so a consult reads ~1.4k tokens here instead of ~9k there. All community tier; TypeSafe's docs win.
When to run it
- Every decision, after step 3 of Consult guide: could Jev help this project? (the fit test) and before the shortlist.
- A consequential gate (blocks, pays, deletes or sends) also gets the mandatory failure check in the consult's "Rules for the advisor": read Failure reports: confidence misread and calibration and make validating the cut on the user's own labels part of the recommendation.
- Evidence rows cited below sit on Failure reports: where Jev broke, lost, or was the wrong tool unless another page is named; section names are given so you can jump straight to the row.
The checklist
| # | Check | Evidence |
|---|---|---|
| 1 | Does the answer need generation, counting, date maths, geometry or lookahead? Put that in code and let Jev pick from the candidates (§2–§4, §9). | Jev 1.13 jaggedness: known failure modes §2 math, §3 dates, §4 indirection, §9 generation; failure-reports sections "Generation or sampling misuse" and "Arithmetic, numbers, spatial state and time series" |
| 2 | Is the needed evidence actually in state? Test against a stub that always answers the same; measure any trim before using it (a short prefix trim collapsed agreement in jev-ultralightspeed). |
jev-ultralightspeed row (collapseindex), failure-reports section "Wrong role, or options that don't fit" |
| 3 | Do the options cover reality, with "other/none", one judgment each, and few enough to tell apart (one Choice over a dozen close accounts lost badly: agentjournal)? Shuffle the order in evals; check no option is never chosen. | agentjournal's numbers: bookkeeping row on Head-to-head: Jev against other models and methods ("vs classic methods, engines and people"); pointer row in failure-reports "Wrong role, or options that don't fit"; many-class caveat on P26 (Patterns: judging, search, documents, real-time and markets) |
| 4 | Faster than reported round trips (260–280 ms hosted form-fill; 976.8 ms median in a browser, Mario)? Pause per decision or skip Jev. | @trycua's form-fill row on Field reports: what held up, what broke, what it means when advising ("Where it disappointed or broke"); Mario 1-1 row (@umezawakanta13), failure-reports "Speed: too slow for games, or fast but less accurate"; latency vs question count: Request mechanics: billing, limits, latency, calibration and stability |
| 5 | Price it: calls/hour × input tokens (state plus every question's wording) × $0.042/Mtok, with grouped questions. |
Cost ledger: published cost per Jev decision; list price: Models, aliases, pricing, rate limits, context |
| 6 | Gate on confidence only after calibrating on the project's own labels, per question; Noul for gates. | Failure reports: confidence misread and calibration |
| 7 | Adding Jev to an agent? A/B the whole agent: in most builder A/Bs success did not rise and time often did. | Head-to-head: Jev inside agents, routers and tool gates |
| 8 | Baseline rules, keywords, a regex, grep, n-grams, BERT, a solver or a small LLM on the same items; use Jev where it wins or ties cheaper. | failure-reports "A simpler tool won"; Head-to-head: Jev against other models and methods, Head-to-head benchmarks: security (phishing, spam, injection, vulnerable code); When not to use Jev: rules, embeddings, trained classifiers, small and frontier LLMs |
| 9 | Never the sole security gate (self-describing text was scored as its own label; a floor that caught all 5 successful injections escalated 58% of clean traffic); no trading recommendations. | jevgrep (allebee) and jev-engineering (eugeniughelbur) rows, failure-reports "Wrong role, or options that don't fit"; Head-to-head benchmarks: security (phishing, spam, injection, vulnerable code); P03 on Patterns: agent internals, routing, gates, context and memory; trading: P25 (poor) on Patterns: judging, search, documents, real-time and markets |
| 10 | Naming a community tool, route or package? Read its row flag and the Warnings block first. Never offer a Warnings-listed service as the way in, even while direct signup is closed; call anything else "community, unofficial". | Warnings: not-Jev services, key safety, look-alikes and install names (receipts: Warning receipts: what we checked behind each warning); vendor routes: Platforms and gateways: Zapier, LangChain, Spring AI, Cloudflare, Netlify, Vercel, OpenRouter, Fly.io, Pydantic AI and other hosted routes to Jev; access: console.typesafe.ai (console + playground) |
Related
- Consult guide: could Jev help this project? — the procedure this checklist belongs to
- Failure reports: where Jev broke, lost, or was the wrong tool — the failure rows behind items 1-4, 8 and 9; Failure reports: confidence misread and calibration — item 6
- Jev 1.13 jaggedness: known failure modes — the official failure modes; Testing and evaluating a Jev workflow — the 50-case experiment
Sources
Files in frontmatter sources: (captured by 2026-09-25); original URLs are on the evidence rows linked above. The price is TypeSafe's (Models, aliases, pricing, rate limits, context).