---
title: "Head-to-head: Jev vs OpenAI's Decisions API"
type: community
source_tier: community
tags: [head-to-head, openai, decisions-api, community]
created: 2026-10-12
updated: 2026-10-25
confidence: medium
sources: "42 files, listed under this page in https://jevwiki.ai/index.json (raw/ paths map to their URLs in https://jevwiki.ai/raw/MANIFEST.json); links are inline in the body"
summary: "Builder comparisons of Jev vs OpenAI's Decisions API (GPT-6 Luna; public beta 2026-10-06): accuracy, latency, cost per run; split from head-to-head 2026-10-12, rows unchanged."
---

# Head-to-head: Jev vs OpenAI's Decisions API

> **TL;DR** OpenAI's Decisions API (GPT-6 Luna behind a decision endpoint, public beta since 2026-10-06) is the one rival on these pages that reads images, which Jev cannot. In these builder runs, all small or single-team and `unverified` unless marked, Jev was usually cheaper and often faster or more decisive; Decisions was ahead on images, date arithmetic and lenient grading. A threshold tuned on one model does not carry over to the other. One builder's run per row; test on your own labels. Split from [[ideas/head-to-head]] on 2026-10-12, rows unchanged.

OpenAI's Decisions API (GPT-6 Luna behind a decision endpoint; public beta 2026-10-06; facts and the first forum comparisons, community and `unverified`, on [[syntheses/when-not-to-use-jev]]) is the one rival here that read images when these runs were made; Jev could not until 2026-10-14 and since then reads still images as an experimental, header-gated option ([[concepts/images]], [[reference/models-and-pricing]]). First-day runs, all small or single-team; vendor facts for other decision models: [[ideas/rival-decision-models]]. Multi-model runs that include OpenAI's Decisions API (Liqa Labs slide visuals, CalibratedAgents answer verification by input size, @eandualem dictation): [[ideas/head-to-head-rival-models]] (Liqa Labs on [[ideas/head-to-head-small-models]]).

| Task (builder) | Compared | Numbers as published | Won | Caveat |
|---|---|---|---|---|
| Relevance of a search query, and of a resume, to a job description on 1-10, for HiringCafe (2.5M users, author) ([@h_nilforoshan](https://x.com/h_nilforoshan/status/2107643820150308932) thread, [results](https://x.com/h_nilforoshan/status/2107644661448024403), [labels](https://x.com/h_nilforoshan/status/2107697251510624263), 2026-10-07) | Jev vs OpenAI Decisions vs Gemini 3.1 Flash Lite | Spearman with labels: query × job 0.74 / 0.71 / 0.63; resume × job 0.71 / 0.67 / 0.61. OpenAI about 2× Jev's cost; headline "5-10% worse" (the posted correlations are 4-6% lower, our arithmetic) | Jev on both tasks and cost | `unverified`: 500 samples hand-labelled by one teammate (whether per task not stated); cost method not given. HiringCafe runs Jev in production and calls Jev tokens its biggest expense ([reply](https://x.com/h_nilforoshan/status/2107644837407404340)) |
| Chat-support router and policy classifier, earlier failure cases (answer in chat history, new policy lookups, ambiguous follow-ups) ([@sacino](https://x.com/sacino/status/2107721893893779876), 2026-10-07; attached chart not transcribed) | Jev vs OpenAI Decisions vs OpenAI structured outputs (model not named) | Routing: Jev 12/12, Decisions 11/12; Jev cheapest and fastest. Policy selection: all required policies found Decisions 4/6, structured outputs 4/6, Jev 3/6; picks without irrelevant extras structured outputs 4/6, both decision models 1/6. Decisions faster than Jev on classification; about half structured outputs' latency at ~28% of its estimated uncached cost | Jev routing; **LOST** classification | n = 12 and 6, hard cases chosen on purpose; prompts adapted per provider (Decisions sent more input tokens), not like for like (author). `unverified` |
| Evidence rerank: score 30 candidates, keep 6; 127 questions (101 answerable from the docs), averaged over 3 embedding models ([@Vayllon301](https://x.com/Vayllon301/status/2107672153579495755), 2026-10-07) | Jev vs OpenAI Decisions; embedding order as baseline | Hit@6 97.4 vs 97.0% (baseline 87.8); Jev ~65% lower rerank cost, slightly better p95 latency | Cost; accuracy tie | `unverified`: post only, no absolute cost or latency; the author calls them neck and neck and values Decisions' image input |
| Booking-agent labels, 5 fixture prompts ([@ssbrouhard](https://x.com/ssbrouhard/status/2107588213540098474), 2026-10-06) | Jev vs OpenAI Decisions (and Luna via the Responses API) | ~171 vs ~134 ms; Decisions ~12× faster than Luna via Responses; same labels as Jev; ~$22 vs ~$56 per million calls (ledger: [[ideas/cost-ledger]]) | Cost; **LOST** latency narrowly | n = 5; `unverified` |
| 500 synthetic cases, task not stated ([@argbeknwn](https://x.com/argbeknwn/status/2107734411915440265), 2026-10-07) | Jev 1.13 vs OpenAI Decisions | Accuracy 97.0 vs 98.0%; 321 vs 329 ms; Decisions about one cent more for the whole run | **LOST** accuracy narrowly (5 items); latency, cost about tied | Separate runs, one synthetic test (author); `unverified` |
| Moderation pilot (48 synthetic cases × 2 rounds) and translation QA (23 synthetic examples) for seedit, a peer-to-peer Reddit alternative ([comparison at eaed0a6](https://github.com/bitsocialnet/seedit/blob/eaed0a6d7676b70089873e6707b5e7959031b4f2/scripts/visual-qa/comparison-2026-10-06.md), [@tomcasaburi](https://x.com/tomcasaburi/status/2107617363294105886), 2026-10-06) | Jev 1.13 vs OpenAI Decisions, paired sequential calls from one Mac, concurrency 1, no retries | Moderation labels 48/48 vs 45/48 each round (Decisions: one review case allowed, two date-arithmetic cases sent to review; without the 4 date cases 44/44 vs 43/44); median 276 vs 271 ms, p95 343 vs 550 ms; estimated input cost $120.81 vs $257.95 per million checks (53% less; cost row on [[ideas/cost-ledger]]). 28 stress cases: both matched. Translation: both flagged all 12 issues, no false passes; at the shared 0.95 pass line Jev left 1 good example unverified, Decisions 5. Decisions answered 4 easy questions on one screenshot right in ~979 ms (no Jev arm: text only) | Jev on text; kept Jev, adds Decisions for screenshot checks | Synthetic, private policy; translation labels model-written; prompts and the 0.95 line came from the Jev workflow, which may favour Jev (author). Decisions priced at the author's assumed $0.10 per M input tokens. The two date-arithmetic misses were Decisions', on a task type Jev's own docs flag as weak ([[concepts/jaggedness-jev-1-13]]) |
| Routing, the author's own benchmark (set and size not stated) ([@LessonHist](https://x.com/LessonHist/status/2107767932008960055), 2026-10-07) | Jev vs OpenAI Decisions | Routing accuracy 91.7 vs 74.7%; Decisions "slightly slower" | Jev; kept Jev | `unverified`: a reply, no method or n |
| Game-state facts given in `state`; missing or conflicting information; Titanic survival prediction; over 8,000 calls in all ([@ragebuilds](https://x.com/ragebuilds/status/2107961027711357248), 2026-10-07) | Jev vs GPT-6 Luna on the Decisions API | Explicit facts 100 vs 80.8%; missing or conflicting 71.3 vs 66.9%; Titanic, plain prompt 75.65 vs 69.92%, code-controlled harness 77.84 vs **78.44%**; Jev also lower latency and cost (no numbers) | Jev; **LOST** Titanic in the harness narrowly | `unverified`: post only, per-task n not stated. Author: swapping is easy, checking for regressions is the cost; Luna's edge is vision |
| 43 questions, the author's decision-API benchmark rerun ([@geoffHulten](https://x.com/geoffHulten/status/2107895733060026741), 2026-10-07; chart transcribed) | Jev vs OpenAI Decisions; GPT-6 Luna and Luna Flex (chat), GPT-6 Sol, Claude Haiku 4.5, local DeepSeek V4 Flash from the author's September baseline | Decisions 1.63× Jev's cost; median 0.196 vs 0.221 s; both 0 wrong (Luna 2, Luna Flex 3, Haiku 1, Sol and DeepSeek 0). Chart: under 0.2¢ for all 43 on either decision API vs ~5.9¢ Sol, ~6.6¢ Haiku | Cost; **LOST** latency narrowly | Decisions run 2026-10-06, the other points in September; n = 43; chart positions approximate; `unverified` |
| 50 questions (34 everyday, 16 known weak spots) × 5 reps, 500 calls, written and run by a Claude Opus 5.5 agent in the author's itr8.ai workspace ([@cantrell](https://x.com/cantrell/status/2107928782959202490), 2026-10-07; table and screenshot transcribed; the itr8.ai page gave no extractable text) | Jev 1.13 vs OpenAI Decisions | 49/50 vs 50/50 (everyday 100% both; weak spots 15/16 vs 16/16); same answer every rep for both; median 89 vs 101 ms, p90 128 vs 208, p99 209 vs 400; Jev faster on 41 of 50. Jev's miss, all 5 reps: was an invoice issued March 3 on 30-day terms and paid April 10 paid on time (Jev 0.56, Luna 0.00) | Latency; **LOST** accuracy by one item | The miss is date arithmetic, a documented Jev failure mode (`verified`: [[concepts/jaggedness-jev-1-13]] mode 3, compare dates in code). The agent's caveat: too easy to rank the two. Author: Decisions "the clear winner" on images (no Jev arm: text only); hard to justify Jev if you already use OpenAI's APIs |
| Grading a 50-question reading test on a Tagore novel as correct / partial / incorrect: 104 answer files from 51 answering models × 50 questions, English and Japanese, 5,200 requests ([@7shi](https://x.com/7shi/status/2107762089427386545), 2026-10-07; [results](https://github.com/7shi/bou-thakuranir_haat/blob/384df61bc17e18e9c1c1a3b63653e6856cecb864/qa-eval/results/OPENAI.md)) | GPT-6 Luna on the Decisions API vs Jev (82 files), Bespoke Nimble (local), Qwen 3.6 (Ollama) as graders | Under Jev no model passed 97 (en) / 96 (ja); OpenAI gave the best 98-100. Verdict agreement with Jev 90.1% en / 87.2% ja, all at the correct / partial boundary; Pearson r of model scores 0.960. Per file of 50: 11.1 vs 12.3 s, $0.0017 vs $0.0027 (1 : 1.11 time, 1 : 1.60 cost; post: +10% time, +60% cost). Jev's earlier full grading run: ~15 min, 3.3M tokens, $0.14 ([quoted post](https://x.com/7shi/status/2103499518415417739)) | Decisions separates the top; Jev cheaper | No human labels: graders compared with each other. Author's reading: Jev holds to the gold answer's details and its near-even correct / partial splits round down to partial, so 96 is its practical ceiling. Jev's run covers the first 82 files only |
| 355 decisions per model: 50 business documents in 6 types (misleading titles), 24 support messages under written house rules, AskOne moderation (100 audience questions, 43 display names; English and Polish) ([@PawelHuryn](https://x.com/PawelHuryn/status/2107883346202075488), 2026-10-07; card transcribed) | Jev 1.13 vs OpenAI Decisions vs Cloudflare Clef, Clef-flash | Documents with definitions 50 / 48 / 47 / 48 of 50, names only 47 / 49 / 49 / 47; house rules 24/24 for all; errors above 0.80 confidence 0 / 1 / 0 / 0. Decided without a person (top-two gap ≥ 0.80 questions, ≥ 0.90 names): questions 81 / 71 / 77 / 74 of 100, names **31** / 8 / 22 / 4 of 43; none let a bad name through. Median 350 / **212** / 571 / 376 ms; $0.0247 / $0.0353 / $0.1000 / $0.0375 per 1,000 | Jev on autonomy and cost; kept Jev; **LOST** latency to Decisions | Moderation prompts and thresholds tuned on Jev, author-written sets, one run (author). Jev and Clef via OpenRouter, OpenAI direct, from Poland. "Jev reports the top-two gap as `confidence`": `verified` for two options (`abs(2p − 1)`), not beyond (Jev rescales the top probability, [[concepts/confidence]]). Clef reports the yes/no gap squared: read raw, it decides 3 names, not 22 (author). `unverified` |
| Chatbot intent routing (refund vs support), the same 30 messages as the author's September run ([[ideas/head-to-head-small-models]]), one warm-up call then 30 timed from Germany ([ChatFrom blog](https://chatfrom.io/blog/openai-decisions-api-chatbot-intent-routing), [@JustRouzbeh](https://x.com/JustRouzbeh/status/2108573738765578383), 2026-10-09; charts transcribed; **vendor**: ChatFrom sells chatbot workflows) | Jev vs OpenAI Decisions (choice and yes/no wording) vs GPT-4.1 mini, GPT-6 Luna as chat (minimal and default reasoning), GPT-5 nano, Gemini 3.1 Flash-Lite | Correct: Jev 30, Decisions choice 29, yes/no 28, GPT-4.1 mini and Luna chat 30, Flash-Lite 27, GPT-5 nano 26 (30 in September). Median / slowest 5%: Jev 0.24 / 0.27 s, Decisions 0.18 / 0.90 s (yes/no 0.18 / 0.73), chat 0.75-1.88 s. Per 100,000: Jev $1.41, Decisions $1.40 (yes/no $1.66), GPT-4.1 mini $2.52. At a 0.8 gate Jev asked only on the 2 vague messages; Decisions would ask 4 times (including "hello"), and its one error (0.74) is caught at 0.8, not at 0.7 | Accuracy, tail latency, confidence; cost tie; **LOST** median latency; kept Jev | n = 30, one run per setting; Decisions in beta. Billed ~334 tokens at $0.042/M vs ~140 at $0.10/M: the same bill (Jev `verified` at list price, [[reference/models-and-pricing]]). Yes/no wording was worse: a 0.51 coin toss, and it refused "hello" (author: treat a refusal as unsure). Jev's vague-message confidence moved between runs (0.21 → 0.30, 0.54 → 0.60). GPT-5 nano was called by an unpinned alias ([reply](https://x.com/JustRouzbeh/status/2108637029433061574)) |
| 1,000 Bitext customer-support messages balanced over 27 intents, 7 questions in one request (intent Choice of 27, topic Choice of 11, politeness Score, 4 yes/no), concurrency 16, 2026-10-10 ([blog](https://devardagur.com/blog/jev-vs-decisions), [RESULTS.md](https://github.com/devardagur/jev-vs-decisions/blob/6448d3a1e212219a21edda5a7ea2895bc8b5628e/RESULTS.md), [@devardagur](https://x.com/devardagur/status/2108855034238861676)) | `jev-1.13.0` vs OpenAI Decisions (`gpt-6-luna`) | Mean of the 7 per-question scores (balanced accuracy for yes/no) 92.6 vs 92.3%; intent 94.9 vs 93.4, typos 84.6 vs 90.9, offensive 96.4 vs 98.8, negation 93.2 vs 85.3; intent on keyword-only messages 96.4 vs 87.3% (n = 55). Confidence ≥ 0.9: 855 messages at 98.2% vs 748 at 99.1%; below 0.5: 17 at 82.4% vs 44 at 56.8%. $0.054 vs $0.151 per 1,000 messages (1,295 vs 1,508 mean input tokens; Jev reconciles at list price, our arithmetic); p50 301 vs 217 ms, p95 414 vs 446, p99 591 vs 1,255 | Cost; accuracy tie; **LOST** median latency | One run on live models (author); expected answers derived from the dataset's tags, typos and politeness approximate (author). `unverified` |
| Classifying the parts of an agent conversation into a known set of components, in a context-observability tool ([@withloveaal](https://x.com/withloveaal/status/2109653753754480915), nilenso, 2026-10-12, "my nth follow-up"; video not transcribed; [context-viewer](https://github.com/nilenso/context-viewer)) | luna-6-decisions vs Jev | Luna **1.6× faster, 2.2× cheaper**, 94.4% agreement with Jev | **LOST** speed and cost (as posted) | `unverified`: no n, latencies or costs; agreement, not accuracy. The linked repo's README names neither model; our 2026-10-14 look at its branches found only an optional Jev classifier (one Choice per part, batches of 20) on unmerged branches dated before Decisions went public, and no Luna-decisions run or these numbers |
| Follow-up questions in a data app's fast path (source, metric, filters and dates as bounded choices, a 0.50 confidence gate, anything unresolved handed to the full agent team): Isotopes' private regression suite, 88 supported scenarios × 3 runs plus 19 safety and edge checks per run, runs of 2026-10-14 ([Isotopes blog](https://isotopes.ai/blog/decision-models), 2026-10-14; [@prasanth_j](https://x.com/prasanth_j/status/2110783275526758555), 2026-10-15) | `jev-1.13.0` vs `jev-1.14.0` (then in preview) vs OpenAI Decisions (`gpt-6-luna`) vs Microsoft Decision 1 (through OpenRouter on Azure) | Passed 264/264 (100%) / 247/264 (93.6%) / 180/264 (68.2%) / 164/264 (62.1%); safety and edge 57/57 / 54/57 / 45/57 / 50/57. Median decision-plus-composition time 0.807 / 0.838 / 1.546 / 2.708 s (on the 38 scenarios every model passed every run: 0.727 / 0.743 / 1.418 / 2.518 s). Jev 1.14's 17 misses: 14 under the gate, 3 with no candidate (its numeric-bucket and entity-boundary scenarios fell back every run). OpenAI and Microsoft also produced wrong candidates at or above 0.50 (9 and 21). Repeated identical requests: Jev 1.13 changed its pick on 46 of 7,244 question comparisons and crossed 0.50 on 73, with no scenario outcome changed; OpenAI 0 of 7,242; Microsoft 76 of 6,465, 10 scenarios flipping | Jev 1.13 on every count; **1.14 preview below 1.13** | Builder's private suite, aggregates only (`unverified`). Repeats of the same scenarios, not independent tasks; 1.14 ran in a later batch, where a fresh 1.13 run passed 263/264. The post's production claim (2-5× lower latency after moving steps from Luna, better accuracy) comes with no production numbers. The authors plan to rerun when 1.14 leaves preview. Clef not tested (not on their zero-retention route) |
| Classifier Arena: three browser games, same game state, prompt, options (one shared reshuffle per question) and timing (one request in flight, at least 100 ms apart, 1 s timeout), one frozen prompt per game; matches 2026-10-12 to 16 from one Mac, $4.24 in API calls ([Malleable blog](https://www.malleable.ai/blog/classifier-arena), Brian Coombs, 2026-10-16; [@_briancoombs](https://x.com/_briancoombs/status/2111141267971219603); [repo](https://github.com/Malleable-ai/classifier-arena)) | Jev (version not stated) vs OpenAI Decisions (`gpt-6-luna`) | Head-to-head wins: Fighter 42 of 56 (75%), Inbox Invaders (label falling SMS messages legit, spam or smishing, public dataset) 48 of 52 (92.3%), Clue Clash (pick the defined word from 16 close nouns) 49 of 60 (81.7%); the post's 83% overall. Inbox accuracy 95.7 vs 95.0%, the faster model won 49 of 52; Clue Clash right picks at the three reveal steps 619 / 823 / 904 vs 574 / 773 / 883 of 960. p50 147-164 vs 188-208 ms; $0.024-0.049 vs $0.034-0.077 per 1,000 decisions. OpenAI refused 283 of 17,940 fighter moves (1.6%) and 13 of 3,408 Clue Clash picks, with no reason given; Jev never refused. One fighter state, one reshuffle of its 11 options: OpenAI's top pick changed on 20 of 20 calls, Jev's on 0 of 20 | Jev in all three games | `unverified`: one machine and place (part of the speed edge may be distance, the author says); one state for the order test. Fighter wins tracked the latency gap; against a scripted bot OpenAI won 7 of 12 fights, Jev 2 of 10. Jev counted the fighter prompt as 1,166 input tokens to OpenAI's 782, so the 2.4× list-price gap ($0.042 vs $0.10 per million input, Jev's `verified` on [[reference/models-and-pricing]]) overstates the per-decision gap |
| 123 cases from the author's product: request routing 17, rule judgments 72, production-shaped rules 32, image types 9 ([@DenisVodchyts](https://x.com/DenisVodchyts/status/2111511277394972736), 2026-10-17; tables transcribed) | "Default AI" (model not named) vs OpenAI Decisions vs Jev (version not stated) | Rules 59/72, 60/72, 59/72; production-shaped rules 17/32 each; image types 9/9, 8/9, 9/9 at median 1.95 / 0.46 / 0.37 s (the post's 81% time cut is against the default). Routing median 3.23 / 3.74 / 3.89 s: simple conversation (4 cases) Jev 0.44 s and $0.018 per 1,000 vs default 1.91 s and $0.166 (the post's 89% cost cut) and Decisions 0.57 s and $0.061; complex, supervisor required (10 cases) Jev slowest and dearest, 5.65 s and $0.323 vs 4.09 s and $0.296. Estimated total for all 123 cases $0.05049 / $0.05234 / $0.05138 | Jev on simple routing and image time; **LOST** complex routing; totals about tied | `unverified`: per-category n of 3-10; the default arm is not named; routing times include the rest of the author's pipeline (inferred from the seconds-long medians). Image input is experimental on Jev ([[concepts/images]]). The author: no clean winner |
| Labels in a training-data pipeline at moderate volume, both scored against an LLM benchmark from Opus ([@genmon](https://x.com/genmon/status/2111038036289208431), 2026-10-16) | Jev vs `gpt-6-luna` | Jev better on quality (no numbers) | Jev | `unverified`: post only; no task, n or metric |
| Does a search query need broad market research (a search-routing yes/no gate whose criteria end "when in doubt, false"): 238 queries, two annotators, about 15% positive; Jev called 2026-09-28 through TypeSafe's API, Luna 2026-10-08 through OpenRouter ([Samaya blog](https://samaya.ai/blog/calibrated-decision-models), 2026-10-20) | `jev-1.13.0` vs OpenAI Decisions (`openai/gpt-6-luna-decisions-20261006`) vs djev (open, not Jev: [[ideas/open-replicas-servers]]) vs Samaya's prompted LLM classifier | F1 at 0.5: Jev 76.8, Luna 71.3, djev 54.5 (76.7 with a threshold tuned on half the queries), LLM 79.1 (annotators vs each other 81.7). AUROC 0.95 / 0.91 / 0.95, ECE 0.033 / 0.062 / 0.089 (Jev / Luna / djev). Median 194 vs 269 ms; p99 15.1 s vs 759 ms (five Jev calls over 2 s). Sending Jev's 0.2-0.8 band (7.6% of queries) to the LLM: F1 79.7, level with the LLM alone, catching 5 of 8 clear mistakes; the same band on Luna sent 4.2% and caught 3 of 10, as a third of Luna's mistakes sat at P exactly 0 or 1. Summed min(P, 1 − P) predicted 17.5 Jev errors (14.5 made) and 5.3 for Luna (16.5 made). Price per M input tokens $0.042 vs $0.10 | Jev on accuracy, calibration, median latency, price; **LOST** tail latency | Independent (Samaya builds search for research). One task, about 36 positives per annotator; the authors call the 5.5-point Jev-Luna gap not significant (bootstrap 95% −2.6 to 14.2). The two models were called on different dates. Tuning did not help Jev (75.6 held out). Jev's price `verified` ([[reference/models-and-pricing]]); Luna's `unverified`. The post's two-decimal probabilities match the 0.01 grid measured on [[ideas/request-mechanics]] (not in the docs) |
| Labelling ("coding") task with very long prompts; task, dataset and n not stated, results table only ([@jon_mellon](https://x.com/jon_mellon/status/2114164167792112090), 2026-10-25; table transcribed) | Jev (version not stated) vs "Luna" at three reasoning settings (none, low, medium), reported with output tokens per call, which suggests GPT-6 Luna as a chat model rather than the Decisions API (inferred) | Accuracy (95% CI): Jev 74.3% (72.6-75.9) vs Luna none 72.9%, low 73.6%, medium **77.1%** (75.5-78.7); macro F1 0.711 vs 0.699 / 0.708 / **0.759**. Cost per 1,000 calls $0.56 vs $0.14 / $0.14 / $0.19; "full 100k" $93 vs $22 / $24 / **$31**. Luna output 15 / 28 / 118 tokens per call | **LOST** cost at every setting and accuracy to Luna medium; level with Luna none and low (CIs overlap) | `unverified`: no method, prompts or data. Author: Luna's 10× caching discount puts it ahead on very long prompts (OpenAI's prices are not in this wiki's sources). Jev's $0.56 per 1,000 calls ≈ 13,300 input tokens a call at list price (our arithmetic). The "full 100k" column is not 100× the per-1,000 cost for any row; the post does not explain it |

A review-queue threshold tuned on Jev, reused on OpenAI's Decisions API, sent about twice as many households to review at the same accuracy (@venkat_peri): [[ideas/failure-reports-confidence]].

Also against OpenAI's decision model: flagging problem steps in 30,943 steps of five real agent runs (Kalmia; Jev caught more, OpenAI faster and more precise) on [[ideas/head-to-head-agents]]; MCP tool shortlists on three public suites (jev-experiments, [[ideas/head-to-head-agents]]); Lightfield's agent-written classifiers run unchanged on both ([[ideas/head-to-head-benchmarks]]); Abnormal's email-security test of 40 models, Jev best calibrated ([[ideas/head-to-head-rival-models]]).

Weekly drone race of decision models (Jev, OpenAI Decisions, Clef-Flash, GLiDE; @Vybhav): [[ideas/eval-boards]].

Snake and Pac-Man on Opper's decision-benchmark boards, where GPT-6 Luna Decisions led Jev 1.13 on Snake and trailed it on Pac-Man: [[ideas/head-to-head-games]].

Tetris on autotris, where Luna beat Jev on play at a higher cost per move: [[ideas/head-to-head-games]].

Bluesky starter-pack tagging (@hboon, 2026-10-20): Jev vs GPT-6 Luna run as "Luna medium", most likely the chat model at a reasoning setting rather than the Decisions API (inferred); the author kept Luna: [[ideas/head-to-head-small-models]].

GPT-6 Luna as a chat model vs the same model through the Decisions API on claim checks and Banking77, with Jev, Decision-1, Laya and Clef calibration figures (@drinkzima, 2026-10-19): [[ideas/head-to-head-rival-models]].

Lead qualification on 79 homepage screenshots: Jev 1.13 and GPT-6 Luna on OpenAI Decisions tied with Gemini and Claude models; Jev cheapest (distribute.you, vendor, 2026-10-21 run): [[ideas/head-to-head]]. Screenshots of AI-search answers, Jev 1.14 vs GPT-6 Luna (API route not stated), Haiku 5.5 and Gemini (@arunagrahri, 2026-10-21): [[ideas/head-to-head-small-models]].

One scam text ("Hi Mum, new number, can you pay a bill") through five decision APIs on decidify.dev, the author's comparison site ([@tech2hard](https://x.com/tech2hard/status/2109008367159316501), 2026-10-10; card transcribed; n = 1): all five called it a scam and an impersonation; on "how risky to send money" Microsoft-Decision-1 alone said risky rather than dangerous. Jev 74 ms, OpenAI Decisions 284, Perplexity 366, Microsoft 802, Clef-flash 4.6 s; $0.021 / $0.049 / $0.0097 / $0.0079 / $0.017 per 1,000 as the card shows them. One input shows nothing about accuracy (`unverified`). Vendor facts: [[ideas/rival-decision-models]].

Radiology report checks (Hypervelocity, 2026-10-12): Jev, Liquid d1, GPT-6 Luna Decisions and Tev1 4B on six report-error questions; AUROC, calibration and cost per 1,000 reports on [[ideas/builds-apps]].

Option-order bias on RewardBench 2, OpenAI Decisions included, and HeySam sales-call routing: multi-model rows on [[ideas/head-to-head-small-models]].

qbash company matching: Jev vs OpenAI Decisions on naming the company in a call transcript, no per-model figures (vendor): [[ideas/builds-business]]. Chest X-ray findings: Jev vs GPT Luna decisions, experimental image input: [[ideas/builds-apps]].

OpenAI Decisions also appears in the memonsystems answer judge and mblukac's Games2p2k human-choice predictions ([[ideas/head-to-head-small-models]]), the DecisionRobust attack board ([[ideas/head-to-head-security]]), System1-Arena ([[ideas/head-to-head-reasoning-and-science]]) and OpenRouter Armor's vendor chart ([[ideas/tools-guardrails]]).

Trolley problem variants (Zach Landes): GPT-6 Luna never kept its own server over people; Jev and most local models sometimes did: [[ideas/failure-reports-wrong-role]].

Live Q&A comment check: Jev as a second check on Gemini 3.8 Flash and GPT-6 Luna hide calls (@WasimJalali, 2026-10-19): [[ideas/builds-content-and-media]]. Amp's plugin decision API offers Jev (default) and GPT-6 Luna per call and warns that thresholds do not carry over between them: [[ideas/apps-with-jev]].
Added 2026-10-24: chess, Jev vs the Decisions API (@itssotsot), mostly threefold-repetition draws, and the system-one arcade shoot-outs incl. GPT-6 Luna: [[ideas/head-to-head-games]]. Realtime-Bench live-clock web tasks, Jev 1.13 ahead of OpenAI Decisions on reliable success and latency: [[ideas/head-to-head-reasoning-and-science]]. Agent-trace eval judge (Hamel Husain and Antaripa Saha): OpenAI Decisions had the best tuned F1, Jev 1.13 missed the authors' 80/80 floor: [[ideas/failure-reports-confidence]].


## How to read these

- **Builder-run, small and varied**: tasks, labels, reasoning settings and concurrency differ between rows; most are tens to a few hundred cases. Read each row's caveat before quoting a number. Cross-row caveats: [[ideas/head-to-head]] ("How to read these").
- **Costs** reconcile against each list price on [[ideas/cost-ledger]]; Jev's price: [[reference/models-and-pricing]]. OpenAI's API facts: [[syntheses/when-not-to-use-jev]].
- **Builds on the Decisions API alone are not comparisons.** [plugin-recall](https://github.com/just-every/plugin-recall) (memory cards for Claude Code and Codex; [@zemaj](https://x.com/zemaj/status/2110348876767179247), 2026-10-14) is announced as using "Jev/Decisions API", but its README and code call only OpenAI's `gpt-6-luna`, with an OpenAI key (our reading at `c2fb11d`).
- **Model names.** Since the 2026-10-21 capture, Opper's evaluations guide names OpenAI's decision model `gpt-6-luna-decisions` (catalog ID `openai/gpt-6-luna-decisions`); the guide used `gpt-6-luna` before, and now calls `gpt-6-luna` OpenAI's chat model, which both of Opper's decision endpoints refuse with `400`. Samaya called it as `openai/gpt-6-luna-decisions-20261006` through OpenRouter; other rows use the name each builder gave. The guide says `gpt-6-luna-decisions` "reads images" (inline base64) on Opper's Decisions route, while other models there (`clef`, `jev-1.14.0` among its examples) take text only and return `400` for images, although Jev reads images natively behind an experimental header ([[concepts/images]]). Vendor docs ([Opper guide](https://docs.opper.ai/build/gateway/evaluations)); route details on [[ideas/platforms-and-gateways]].
- **Other vendors' decision models** (Clef, Microsoft-Decision-1, Mercury Decide and others): [[ideas/rival-decision-models]]; independent runs against Jev: [[ideas/head-to-head-rival-models]].

## Related

- [[ideas/head-to-head]]: Jev vs frontier, small and classic methods; cascades
- [[ideas/head-to-head-agents]]: Jev inside agents, including runs against OpenAI's decision model
- [[ideas/rival-decision-models]], [[ideas/cost-ledger]], [[ideas/failure-reports-confidence]]
- [[syntheses/when-not-to-use-jev]]: when OpenAI's API may beat Jev

## Sources

Links inline; raw captures listed in the frontmatter (not published).