$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people

Agents: read the raw Markdown of this page, or start at llms.txt.

~/wiki/ideas

Head-to-head: Jev vs OpenAI's Decisions API

[ community tier ][ updated 2026-10-25 ][ confidence medium ]#head-to-head · openai · decisions-api · community

TL;DR OpenAI's Decisions API (GPT-6 Luna behind a decision endpoint, public beta since 2026-10-06) is the one rival on these pages that reads images, which Jev cannot. In these builder runs, all small or single-team and unverified unless marked, Jev was usually cheaper and often faster or more decisive; Decisions was ahead on images, date arithmetic and lenient grading. A threshold tuned on one model does not carry over to the other. One builder's run per row; test on your own labels. Split from Head-to-head: Jev against other models and methods on 2026-10-12, rows unchanged.

OpenAI's Decisions API (GPT-6 Luna behind a decision endpoint; public beta 2026-10-06; facts and the first forum comparisons, community and unverified, on When not to use Jev: rules, embeddings, trained classifiers, small and frontier LLMs) is the one rival here that read images when these runs were made; Jev could not until 2026-10-14 and since then reads still images as an experimental, header-gated option (Images and assets (experimental), Models, aliases, pricing, rate limits, context). First-day runs, all small or single-team; vendor facts for other decision models: Rival decision models: Clef, Liquid d1, GLiDE, Drex and other non-Jev System One models. Multi-model runs that include OpenAI's Decisions API (Liqa Labs slide visuals, CalibratedAgents answer verification by input size, @eandualem dictation): Head-to-head: Jev vs rival decision models (Liqa Labs on Head-to-head: Jev vs small, fast and local models).

Task (builder) Compared Numbers as published Won Caveat
Relevance of a search query, and of a resume, to a job description on 1-10, for HiringCafe (2.5M users, author) (@h_nilforoshan thread, results, labels, 2026-10-07) Jev vs OpenAI Decisions vs Gemini 3.1 Flash Lite Spearman with labels: query × job 0.74 / 0.71 / 0.63; resume × job 0.71 / 0.67 / 0.61. OpenAI about 2× Jev's cost; headline "5-10% worse" (the posted correlations are 4-6% lower, our arithmetic) Jev on both tasks and cost unverified: 500 samples hand-labelled by one teammate (whether per task not stated); cost method not given. HiringCafe runs Jev in production and calls Jev tokens its biggest expense (reply)
Chat-support router and policy classifier, earlier failure cases (answer in chat history, new policy lookups, ambiguous follow-ups) (@sacino, 2026-10-07; attached chart not transcribed) Jev vs OpenAI Decisions vs OpenAI structured outputs (model not named) Routing: Jev 12/12, Decisions 11/12; Jev cheapest and fastest. Policy selection: all required policies found Decisions 4/6, structured outputs 4/6, Jev 3/6; picks without irrelevant extras structured outputs 4/6, both decision models 1/6. Decisions faster than Jev on classification; about half structured outputs' latency at ~28% of its estimated uncached cost Jev routing; LOST classification n = 12 and 6, hard cases chosen on purpose; prompts adapted per provider (Decisions sent more input tokens), not like for like (author). unverified
Evidence rerank: score 30 candidates, keep 6; 127 questions (101 answerable from the docs), averaged over 3 embedding models (@Vayllon301, 2026-10-07) Jev vs OpenAI Decisions; embedding order as baseline Hit@6 97.4 vs 97.0% (baseline 87.8); Jev ~65% lower rerank cost, slightly better p95 latency Cost; accuracy tie unverified: post only, no absolute cost or latency; the author calls them neck and neck and values Decisions' image input
Booking-agent labels, 5 fixture prompts (@ssbrouhard, 2026-10-06) Jev vs OpenAI Decisions (and Luna via the Responses API) ~171 vs ~134 ms; Decisions ~12× faster than Luna via Responses; same labels as Jev; ~$22 vs ~$56 per million calls (ledger: Cost ledger: published cost per Jev decision) Cost; LOST latency narrowly n = 5; unverified
500 synthetic cases, task not stated (@argbeknwn, 2026-10-07) Jev 1.13 vs OpenAI Decisions Accuracy 97.0 vs 98.0%; 321 vs 329 ms; Decisions about one cent more for the whole run LOST accuracy narrowly (5 items); latency, cost about tied Separate runs, one synthetic test (author); unverified
Moderation pilot (48 synthetic cases × 2 rounds) and translation QA (23 synthetic examples) for seedit, a peer-to-peer Reddit alternative (comparison at eaed0a6, @tomcasaburi, 2026-10-06) Jev 1.13 vs OpenAI Decisions, paired sequential calls from one Mac, concurrency 1, no retries Moderation labels 48/48 vs 45/48 each round (Decisions: one review case allowed, two date-arithmetic cases sent to review; without the 4 date cases 44/44 vs 43/44); median 276 vs 271 ms, p95 343 vs 550 ms; estimated input cost $120.81 vs $257.95 per million checks (53% less; cost row on Cost ledger: published cost per Jev decision). 28 stress cases: both matched. Translation: both flagged all 12 issues, no false passes; at the shared 0.95 pass line Jev left 1 good example unverified, Decisions 5. Decisions answered 4 easy questions on one screenshot right in ~979 ms (no Jev arm: text only) Jev on text; kept Jev, adds Decisions for screenshot checks Synthetic, private policy; translation labels model-written; prompts and the 0.95 line came from the Jev workflow, which may favour Jev (author). Decisions priced at the author's assumed $0.10 per M input tokens. The two date-arithmetic misses were Decisions', on a task type Jev's own docs flag as weak (Jev 1.13 jaggedness: known failure modes)
Routing, the author's own benchmark (set and size not stated) (@LessonHist, 2026-10-07) Jev vs OpenAI Decisions Routing accuracy 91.7 vs 74.7%; Decisions "slightly slower" Jev; kept Jev unverified: a reply, no method or n
Game-state facts given in state; missing or conflicting information; Titanic survival prediction; over 8,000 calls in all (@ragebuilds, 2026-10-07) Jev vs GPT-6 Luna on the Decisions API Explicit facts 100 vs 80.8%; missing or conflicting 71.3 vs 66.9%; Titanic, plain prompt 75.65 vs 69.92%, code-controlled harness 77.84 vs 78.44%; Jev also lower latency and cost (no numbers) Jev; LOST Titanic in the harness narrowly unverified: post only, per-task n not stated. Author: swapping is easy, checking for regressions is the cost; Luna's edge is vision
43 questions, the author's decision-API benchmark rerun (@geoffHulten, 2026-10-07; chart transcribed) Jev vs OpenAI Decisions; GPT-6 Luna and Luna Flex (chat), GPT-6 Sol, Claude Haiku 4.5, local DeepSeek V4 Flash from the author's September baseline Decisions 1.63× Jev's cost; median 0.196 vs 0.221 s; both 0 wrong (Luna 2, Luna Flex 3, Haiku 1, Sol and DeepSeek 0). Chart: under 0.2¢ for all 43 on either decision API vs ~5.9¢ Sol, ~6.6¢ Haiku Cost; LOST latency narrowly Decisions run 2026-10-06, the other points in September; n = 43; chart positions approximate; unverified
50 questions (34 everyday, 16 known weak spots) × 5 reps, 500 calls, written and run by a Claude Opus 5.5 agent in the author's itr8.ai workspace (@cantrell, 2026-10-07; table and screenshot transcribed; the itr8.ai page gave no extractable text) Jev 1.13 vs OpenAI Decisions 49/50 vs 50/50 (everyday 100% both; weak spots 15/16 vs 16/16); same answer every rep for both; median 89 vs 101 ms, p90 128 vs 208, p99 209 vs 400; Jev faster on 41 of 50. Jev's miss, all 5 reps: was an invoice issued March 3 on 30-day terms and paid April 10 paid on time (Jev 0.56, Luna 0.00) Latency; LOST accuracy by one item The miss is date arithmetic, a documented Jev failure mode (verified: Jev 1.13 jaggedness: known failure modes mode 3, compare dates in code). The agent's caveat: too easy to rank the two. Author: Decisions "the clear winner" on images (no Jev arm: text only); hard to justify Jev if you already use OpenAI's APIs
Grading a 50-question reading test on a Tagore novel as correct / partial / incorrect: 104 answer files from 51 answering models × 50 questions, English and Japanese, 5,200 requests (@7shi, 2026-10-07; results) GPT-6 Luna on the Decisions API vs Jev (82 files), Bespoke Nimble (local), Qwen 3.6 (Ollama) as graders Under Jev no model passed 97 (en) / 96 (ja); OpenAI gave the best 98-100. Verdict agreement with Jev 90.1% en / 87.2% ja, all at the correct / partial boundary; Pearson r of model scores 0.960. Per file of 50: 11.1 vs 12.3 s, $0.0017 vs $0.0027 (1 : 1.11 time, 1 : 1.60 cost; post: +10% time, +60% cost). Jev's earlier full grading run: ~15 min, 3.3M tokens, $0.14 (quoted post) Decisions separates the top; Jev cheaper No human labels: graders compared with each other. Author's reading: Jev holds to the gold answer's details and its near-even correct / partial splits round down to partial, so 96 is its practical ceiling. Jev's run covers the first 82 files only
355 decisions per model: 50 business documents in 6 types (misleading titles), 24 support messages under written house rules, AskOne moderation (100 audience questions, 43 display names; English and Polish) (@PawelHuryn, 2026-10-07; card transcribed) Jev 1.13 vs OpenAI Decisions vs Cloudflare Clef, Clef-flash Documents with definitions 50 / 48 / 47 / 48 of 50, names only 47 / 49 / 49 / 47; house rules 24/24 for all; errors above 0.80 confidence 0 / 1 / 0 / 0. Decided without a person (top-two gap ≥ 0.80 questions, ≥ 0.90 names): questions 81 / 71 / 77 / 74 of 100, names 31 / 8 / 22 / 4 of 43; none let a bad name through. Median 350 / 212 / 571 / 376 ms; $0.0247 / $0.0353 / $0.1000 / $0.0375 per 1,000 Jev on autonomy and cost; kept Jev; LOST latency to Decisions Moderation prompts and thresholds tuned on Jev, author-written sets, one run (author). Jev and Clef via OpenRouter, OpenAI direct, from Poland. "Jev reports the top-two gap as confidence": verified for two options (abs(2p − 1)), not beyond (Jev rescales the top probability, Confidence vs probability). Clef reports the yes/no gap squared: read raw, it decides 3 names, not 22 (author). unverified
Chatbot intent routing (refund vs support), the same 30 messages as the author's September run (Head-to-head: Jev vs small, fast and local models), one warm-up call then 30 timed from Germany (ChatFrom blog, @JustRouzbeh, 2026-10-09; charts transcribed; vendor: ChatFrom sells chatbot workflows) Jev vs OpenAI Decisions (choice and yes/no wording) vs GPT-4.1 mini, GPT-6 Luna as chat (minimal and default reasoning), GPT-5 nano, Gemini 3.1 Flash-Lite Correct: Jev 30, Decisions choice 29, yes/no 28, GPT-4.1 mini and Luna chat 30, Flash-Lite 27, GPT-5 nano 26 (30 in September). Median / slowest 5%: Jev 0.24 / 0.27 s, Decisions 0.18 / 0.90 s (yes/no 0.18 / 0.73), chat 0.75-1.88 s. Per 100,000: Jev $1.41, Decisions $1.40 (yes/no $1.66), GPT-4.1 mini $2.52. At a 0.8 gate Jev asked only on the 2 vague messages; Decisions would ask 4 times (including "hello"), and its one error (0.74) is caught at 0.8, not at 0.7 Accuracy, tail latency, confidence; cost tie; LOST median latency; kept Jev n = 30, one run per setting; Decisions in beta. Billed ~334 tokens at $0.042/M vs ~140 at $0.10/M: the same bill (Jev verified at list price, Models, aliases, pricing, rate limits, context). Yes/no wording was worse: a 0.51 coin toss, and it refused "hello" (author: treat a refusal as unsure). Jev's vague-message confidence moved between runs (0.21 → 0.30, 0.54 → 0.60). GPT-5 nano was called by an unpinned alias (reply)
1,000 Bitext customer-support messages balanced over 27 intents, 7 questions in one request (intent Choice of 27, topic Choice of 11, politeness Score, 4 yes/no), concurrency 16, 2026-10-10 (blog, RESULTS.md, @devardagur) jev-1.13.0 vs OpenAI Decisions (gpt-6-luna) Mean of the 7 per-question scores (balanced accuracy for yes/no) 92.6 vs 92.3%; intent 94.9 vs 93.4, typos 84.6 vs 90.9, offensive 96.4 vs 98.8, negation 93.2 vs 85.3; intent on keyword-only messages 96.4 vs 87.3% (n = 55). Confidence ≥ 0.9: 855 messages at 98.2% vs 748 at 99.1%; below 0.5: 17 at 82.4% vs 44 at 56.8%. $0.054 vs $0.151 per 1,000 messages (1,295 vs 1,508 mean input tokens; Jev reconciles at list price, our arithmetic); p50 301 vs 217 ms, p95 414 vs 446, p99 591 vs 1,255 Cost; accuracy tie; LOST median latency One run on live models (author); expected answers derived from the dataset's tags, typos and politeness approximate (author). unverified
Classifying the parts of an agent conversation into a known set of components, in a context-observability tool (@withloveaal, nilenso, 2026-10-12, "my nth follow-up"; video not transcribed; context-viewer) luna-6-decisions vs Jev Luna 1.6× faster, 2.2× cheaper, 94.4% agreement with Jev LOST speed and cost (as posted) unverified: no n, latencies or costs; agreement, not accuracy. The linked repo's README names neither model; our 2026-10-14 look at its branches found only an optional Jev classifier (one Choice per part, batches of 20) on unmerged branches dated before Decisions went public, and no Luna-decisions run or these numbers
Follow-up questions in a data app's fast path (source, metric, filters and dates as bounded choices, a 0.50 confidence gate, anything unresolved handed to the full agent team): Isotopes' private regression suite, 88 supported scenarios × 3 runs plus 19 safety and edge checks per run, runs of 2026-10-14 (Isotopes blog, 2026-10-14; @prasanth_j, 2026-10-15) jev-1.13.0 vs jev-1.14.0 (then in preview) vs OpenAI Decisions (gpt-6-luna) vs Microsoft Decision 1 (through OpenRouter on Azure) Passed 264/264 (100%) / 247/264 (93.6%) / 180/264 (68.2%) / 164/264 (62.1%); safety and edge 57/57 / 54/57 / 45/57 / 50/57. Median decision-plus-composition time 0.807 / 0.838 / 1.546 / 2.708 s (on the 38 scenarios every model passed every run: 0.727 / 0.743 / 1.418 / 2.518 s). Jev 1.14's 17 misses: 14 under the gate, 3 with no candidate (its numeric-bucket and entity-boundary scenarios fell back every run). OpenAI and Microsoft also produced wrong candidates at or above 0.50 (9 and 21). Repeated identical requests: Jev 1.13 changed its pick on 46 of 7,244 question comparisons and crossed 0.50 on 73, with no scenario outcome changed; OpenAI 0 of 7,242; Microsoft 76 of 6,465, 10 scenarios flipping Jev 1.13 on every count; 1.14 preview below 1.13 Builder's private suite, aggregates only (unverified). Repeats of the same scenarios, not independent tasks; 1.14 ran in a later batch, where a fresh 1.13 run passed 263/264. The post's production claim (2-5× lower latency after moving steps from Luna, better accuracy) comes with no production numbers. The authors plan to rerun when 1.14 leaves preview. Clef not tested (not on their zero-retention route)
Classifier Arena: three browser games, same game state, prompt, options (one shared reshuffle per question) and timing (one request in flight, at least 100 ms apart, 1 s timeout), one frozen prompt per game; matches 2026-10-12 to 16 from one Mac, $4.24 in API calls (Malleable blog, Brian Coombs, 2026-10-16; @_briancoombs; repo) Jev (version not stated) vs OpenAI Decisions (gpt-6-luna) Head-to-head wins: Fighter 42 of 56 (75%), Inbox Invaders (label falling SMS messages legit, spam or smishing, public dataset) 48 of 52 (92.3%), Clue Clash (pick the defined word from 16 close nouns) 49 of 60 (81.7%); the post's 83% overall. Inbox accuracy 95.7 vs 95.0%, the faster model won 49 of 52; Clue Clash right picks at the three reveal steps 619 / 823 / 904 vs 574 / 773 / 883 of 960. p50 147-164 vs 188-208 ms; $0.024-0.049 vs $0.034-0.077 per 1,000 decisions. OpenAI refused 283 of 17,940 fighter moves (1.6%) and 13 of 3,408 Clue Clash picks, with no reason given; Jev never refused. One fighter state, one reshuffle of its 11 options: OpenAI's top pick changed on 20 of 20 calls, Jev's on 0 of 20 Jev in all three games unverified: one machine and place (part of the speed edge may be distance, the author says); one state for the order test. Fighter wins tracked the latency gap; against a scripted bot OpenAI won 7 of 12 fights, Jev 2 of 10. Jev counted the fighter prompt as 1,166 input tokens to OpenAI's 782, so the 2.4× list-price gap ($0.042 vs $0.10 per million input, Jev's verified on Models, aliases, pricing, rate limits, context) overstates the per-decision gap
123 cases from the author's product: request routing 17, rule judgments 72, production-shaped rules 32, image types 9 (@DenisVodchyts, 2026-10-17; tables transcribed) "Default AI" (model not named) vs OpenAI Decisions vs Jev (version not stated) Rules 59/72, 60/72, 59/72; production-shaped rules 17/32 each; image types 9/9, 8/9, 9/9 at median 1.95 / 0.46 / 0.37 s (the post's 81% time cut is against the default). Routing median 3.23 / 3.74 / 3.89 s: simple conversation (4 cases) Jev 0.44 s and $0.018 per 1,000 vs default 1.91 s and $0.166 (the post's 89% cost cut) and Decisions 0.57 s and $0.061; complex, supervisor required (10 cases) Jev slowest and dearest, 5.65 s and $0.323 vs 4.09 s and $0.296. Estimated total for all 123 cases $0.05049 / $0.05234 / $0.05138 Jev on simple routing and image time; LOST complex routing; totals about tied unverified: per-category n of 3-10; the default arm is not named; routing times include the rest of the author's pipeline (inferred from the seconds-long medians). Image input is experimental on Jev (Images and assets (experimental)). The author: no clean winner
Labels in a training-data pipeline at moderate volume, both scored against an LLM benchmark from Opus (@genmon, 2026-10-16) Jev vs gpt-6-luna Jev better on quality (no numbers) Jev unverified: post only; no task, n or metric
Does a search query need broad market research (a search-routing yes/no gate whose criteria end "when in doubt, false"): 238 queries, two annotators, about 15% positive; Jev called 2026-09-28 through TypeSafe's API, Luna 2026-10-08 through OpenRouter (Samaya blog, 2026-10-20) jev-1.13.0 vs OpenAI Decisions (openai/gpt-6-luna-decisions-20261006) vs djev (open, not Jev: Open replicas: servers and logit readers over other models) vs Samaya's prompted LLM classifier F1 at 0.5: Jev 76.8, Luna 71.3, djev 54.5 (76.7 with a threshold tuned on half the queries), LLM 79.1 (annotators vs each other 81.7). AUROC 0.95 / 0.91 / 0.95, ECE 0.033 / 0.062 / 0.089 (Jev / Luna / djev). Median 194 vs 269 ms; p99 15.1 s vs 759 ms (five Jev calls over 2 s). Sending Jev's 0.2-0.8 band (7.6% of queries) to the LLM: F1 79.7, level with the LLM alone, catching 5 of 8 clear mistakes; the same band on Luna sent 4.2% and caught 3 of 10, as a third of Luna's mistakes sat at P exactly 0 or 1. Summed min(P, 1 − P) predicted 17.5 Jev errors (14.5 made) and 5.3 for Luna (16.5 made). Price per M input tokens $0.042 vs $0.10 Jev on accuracy, calibration, median latency, price; LOST tail latency Independent (Samaya builds search for research). One task, about 36 positives per annotator; the authors call the 5.5-point Jev-Luna gap not significant (bootstrap 95% −2.6 to 14.2). The two models were called on different dates. Tuning did not help Jev (75.6 held out). Jev's price verified (Models, aliases, pricing, rate limits, context); Luna's unverified. The post's two-decimal probabilities match the 0.01 grid measured on Request mechanics: billing, limits, latency, calibration and stability (not in the docs)
Labelling ("coding") task with very long prompts; task, dataset and n not stated, results table only (@jon_mellon, 2026-10-25; table transcribed) Jev (version not stated) vs "Luna" at three reasoning settings (none, low, medium), reported with output tokens per call, which suggests GPT-6 Luna as a chat model rather than the Decisions API (inferred) Accuracy (95% CI): Jev 74.3% (72.6-75.9) vs Luna none 72.9%, low 73.6%, medium 77.1% (75.5-78.7); macro F1 0.711 vs 0.699 / 0.708 / 0.759. Cost per 1,000 calls $0.56 vs $0.14 / $0.14 / $0.19; "full 100k" $93 vs $22 / $24 / $31. Luna output 15 / 28 / 118 tokens per call LOST cost at every setting and accuracy to Luna medium; level with Luna none and low (CIs overlap) unverified: no method, prompts or data. Author: Luna's 10× caching discount puts it ahead on very long prompts (OpenAI's prices are not in this wiki's sources). Jev's $0.56 per 1,000 calls ≈ 13,300 input tokens a call at list price (our arithmetic). The "full 100k" column is not 100× the per-1,000 cost for any row; the post does not explain it

A review-queue threshold tuned on Jev, reused on OpenAI's Decisions API, sent about twice as many households to review at the same accuracy (@venkat_peri): Failure reports: confidence misread and calibration.

Also against OpenAI's decision model: flagging problem steps in 30,943 steps of five real agent runs (Kalmia; Jev caught more, OpenAI faster and more precise) on Head-to-head: Jev inside agents, routers and tool gates; MCP tool shortlists on three public suites (jev-experiments, Head-to-head: Jev inside agents, routers and tool gates); Lightfield's agent-written classifiers run unchanged on both (Head-to-head benchmarks: Jev on public datasets and suites); Abnormal's email-security test of 40 models, Jev best calibrated (Head-to-head: Jev vs rival decision models).

Weekly drone race of decision models (Jev, OpenAI Decisions, Clef-Flash, GLiDE; @Vybhav): Eval boards: JevBench by version, Jevals.com, typed-decisions and other independent suites.

Snake and Pac-Man on Opper's decision-benchmark boards, where GPT-6 Luna Decisions led Jev 1.13 on Snake and trailed it on Pac-Man: Head-to-head: games, simulators and real-time control.

Tetris on autotris, where Luna beat Jev on play at a higher cost per move: Head-to-head: games, simulators and real-time control.

Bluesky starter-pack tagging (@hboon, 2026-10-20): Jev vs GPT-6 Luna run as "Luna medium", most likely the chat model at a reasoning setting rather than the Decisions API (inferred); the author kept Luna: Head-to-head: Jev vs small, fast and local models.

GPT-6 Luna as a chat model vs the same model through the Decisions API on claim checks and Banking77, with Jev, Decision-1, Laya and Clef calibration figures (@drinkzima, 2026-10-19): Head-to-head: Jev vs rival decision models.

Lead qualification on 79 homepage screenshots: Jev 1.13 and GPT-6 Luna on OpenAI Decisions tied with Gemini and Claude models; Jev cheapest (distribute.you, vendor, 2026-10-21 run): Head-to-head: Jev against other models and methods. Screenshots of AI-search answers, Jev 1.14 vs GPT-6 Luna (API route not stated), Haiku 5.5 and Gemini (@arunagrahri, 2026-10-21): Head-to-head: Jev vs small, fast and local models.

One scam text ("Hi Mum, new number, can you pay a bill") through five decision APIs on decidify.dev, the author's comparison site (@tech2hard, 2026-10-10; card transcribed; n = 1): all five called it a scam and an impersonation; on "how risky to send money" Microsoft-Decision-1 alone said risky rather than dangerous. Jev 74 ms, OpenAI Decisions 284, Perplexity 366, Microsoft 802, Clef-flash 4.6 s; $0.021 / $0.049 / $0.0097 / $0.0079 / $0.017 per 1,000 as the card shows them. One input shows nothing about accuracy (unverified). Vendor facts: Rival decision models: Clef, Liquid d1, GLiDE, Drex and other non-Jev System One models.

Radiology report checks (Hypervelocity, 2026-10-12): Jev, Liquid d1, GPT-6 Luna Decisions and Tev1 4B on six report-error questions; AUROC, calibration and cost per 1,000 reports on Builds: data, search and business.

Option-order bias on RewardBench 2, OpenAI Decisions included, and HeySam sales-call routing: multi-model rows on Head-to-head: Jev vs small, fast and local models.

qbash company matching: Jev vs OpenAI Decisions on naming the company in a call transcript, no per-model figures (vendor): Builds: business ops and markets. Chest X-ray findings: Jev vs GPT Luna decisions, experimental image input: Builds: data, search and business.

OpenAI Decisions also appears in the memonsystems answer judge and mblukac's Games2p2k human-choice predictions (Head-to-head: Jev vs small, fast and local models), the DecisionRobust attack board (Head-to-head benchmarks: security (phishing, spam, injection, vulnerable code)), System1-Arena (Head-to-head benchmarks: reasoning, agent, robot and science suites) and OpenRouter Armor's vendor chart (Tools: guardrails for agents (tool-call gates, permission hooks, injection screens, rule checks)).

Trolley problem variants (Zach Landes): GPT-6 Luna never kept its own server over people; Jev and most local models sometimes did: Failure reports: wrong role, or options that don't fit.

Live Q&A comment check: Jev as a second check on Gemini 3.8 Flash and GPT-6 Luna hide calls (@WasimJalali, 2026-10-19): Builds: content, media, email, calls and reading. Amp's plugin decision API offers Jev (default) and GPT-6 Luna per call and warns that thresholds do not carry over between them: Apps with a built-in Jev option. Added 2026-10-24: chess, Jev vs the Decisions API (@itssotsot), mostly threefold-repetition draws, and the system-one arcade shoot-outs incl. GPT-6 Luna: Head-to-head: games, simulators and real-time control. Realtime-Bench live-clock web tasks, Jev 1.13 ahead of OpenAI Decisions on reliable success and latency: Head-to-head benchmarks: reasoning, agent, robot and science suites. Agent-trace eval judge (Hamel Husain and Antaripa Saha): OpenAI Decisions had the best tuned F1, Jev 1.13 missed the authors' 80/80 floor: Failure reports: confidence misread and calibration.

How to read these

Sources

Links inline; raw captures listed in the frontmatter (not published).