Agents: read the raw Markdown of this page, or start at llms.txt.
~/wiki/ideas
Head-to-head: games, simulators and real-time control
TL;DR In games Jev's edge is decision speed and cost, not play strength: it beat slower chat models where a clock or real-time loop punishes latency (Tetris, Pong, Mario) but lost Tetris with no clock, lost to local replicas where raw decisions per second win (Snake, Clash Royale), never beat a chess engine and lost a blitz game to GPT-6 Astra, and did no better than random at StarCraft II until a planner constrained its actions. Code that builds the legal-move menu does part of the work. Split from Head-to-head: Jev against other models and methods on 2026-09-25; rows moved unchanged.
Numbers as each builder published them, mostly one run or a handful of episodes, in 2026-09, when jev-latest pointed at jev-1.13.0 (Models, aliases, pricing, rate limits, context); rows that name no version are assumed on it (inferred). LOST = Jev lost on the stated goal. Pattern: Patterns: judging, search, documents, real-time and markets P24 games. The builds themselves: Builds: games, simulators, robots and devices; robot and embodied suites (Meta-World, LIBERO, ALFWorld) and the LLM Chess numbers: Head-to-head benchmarks: Jev on public datasets and suites.
Chess
| Task (builder) | Compared | Numbers as published | Won | Caveat |
|---|---|---|---|---|
| Chess, LLM Chess leaderboard (Maxim Saplin) | vs the board's LLMs | Elo in the band of mid-pack reasoning models at a small fraction of their cost and time per game; numbers on Head-to-head benchmarks: Jev on public datasets and suites | Cost, latency; accuracy tie | Wide Elo interval; no Jev row on the live leaderboard yet |
| Blitz chess 5+0, one API call per move (@aimlapi) | vs Fable 5.1, GPT-6 Astra | Fable was +16 in material at move 29 (a second queen), then lost it back to Jev's king; with king and pawn each, Fable flagged at move 51 (Fable 3.3-15.0 s per move, median 5.4 s; Jev median 2.1 s). Astra mated Jev on its 18th move (Qe1#) with 2:27.7 left; Jev 1.0-1.7 s per move in that game | Clock vs Fable; LOST to Astra | One game each (video transcribed 2026-09-25). Jev's 1-2 s per move is above TypeSafe's 70–500 ms (System One Models), so harness or route overhead is likely included (inferred) |
| Chess vs Komodo Dragon (Maxim Saplin) | vs chess engine | Drew half its games at the low levels and never won; our tally of the unmerged harness branch found no engine wins at any level (unverified); numbers on Head-to-head benchmarks: Jev on public datasets and suites |
LOST | Same run as the leaderboard row above. Failure reports: where Jev broke, lost, or was the wrong tool |
| Crowd vs Jev, jevchess.com (@staskulesh, algo repo) | Jev vs anonymous human players | Share card of 2026-09-21 (image transcribed): 19-6 over 25 games (16 wins, 6 draws, 3 losses); compute shown as "18.2C" (last glyph unclear, a C or a cent sign) | Jev | Casual anonymous pool; every legal move is a Choice option plus a code-side hanging-piece check, so code does part of the work |
Arcade and real-time games
| Task (builder) | Compared | Numbers as published | Won | Caveat |
|---|---|---|---|---|
| Three-option paddle decision, 40 states × 2 (jev-use, shitianfang) | vs Haiku 4.5 and Gemini 3 Flash, free-form and schema-constrained | p50 225 ms vs 691 (Haiku strict) / 1,027 (Gemini strict); $0.018 vs $0.30 / $0.09 per 1,000; quality 24-30 of 40 for every arm | Latency (3.0-3.1× vs constrained), cost; quality tie | The author withdrew his own "86 vs 6 decisions" Pong headline as an unconfigured baseline. Jev never chose stay in 80 calls: right on every move state, 0 of 13 hold states |
| Pong, 30 game states, Jev + 13 chat-model settings (jev-pong, Ably Labs) | vs chat models via Vercel AI Gateway | 30-state run 2026-09-19: Jev 28/29 at 222 ms median (one Gateway 503 not counted); Ministral 3B 26/30 at 1.8× the latency; the other 12 configurations (11 models; GPT-6 Astra at two effort settings) 28-30/30 at 2.7-20.4×. Live lanes 2026-09-17: 4.4 vs 0.28-0.40 decisions/s, Jev 227 ms average | Latency; LOST a point | Author: a latency demo, not an intelligence demo |
| Super Mario harness, same machine (nwn, Zenn, Japanese) | vs Llama 3.1 8B, Gemma 4 26B A4B, Qwen3 235B, Gemini 2.5 Flash-Lite (first-token logit read) | Distance 1,226 / 1,138 / 1,127 / 686 / 296 at 276 / 356 / 572 / 593 / 719 ms mean | All; narrowly vs Llama and Gemma | Only logprob-capable older, non-reasoning LLMs could be compared; his Jev latency was ~3× the public demo's. His logit read on Gemma3 270M ran 77× faster than generating JSON |
| Real-time Tetris, seed 42 (trungdq88) | vs Claude Haiku 4.5 | Won all 3 modes; ~220 vs ~750 ms per move; ~20x cheaper per move | All | One run per mode |
| Same, Gemini 3.8 Flash | Real time: Jev won (Gemini missed 8 deadlines). No clock: Gemini 0.32 lines per piece vs 0.25 at ~17x the cost per move | LOST with no clock | One seed | |
| Tetris (trungdq88) | vs Laya, 4-core CPU | Jev won at 0:14: 7 lines vs 0; 232 vs 614 ms per move | All | Laya near-flat; prefers first-listed options |
| Snake, 30 s (@sl1ma4) | vs Laya, local | Laya 86.5 vs Jev 3.2 decisions/s (2,588 vs 95 decisions); score Laya 75 vs Jev 3 at 30 s (the post's 46 vs 1 is the 15 s mark); Jev P50 305.7 ms per step | LOST | Speed-bound game; Laya's window 512-1024 tokens |
| Snake race and fight arena (laya-vs-jev-arena, PromptEngineer48; not virajbhartiya's repo) | vs Laya, local | Video: Laya won the snake race 90-50 and the first fight; ~133 ms p50 local vs Jev ~962 ms from India. 8 fixed steering positions: Jev 8/8, Laya 7/8 | LOST speed; judgment Jev | Author: "Laya's edge here is speed, not judgment." |
| Chrome T-Rex (CLM, Contrastive-LM) | Jev vs a frozen embedder with a contrastive head, both under a physics-planner shield | Both survived; Jev agreed with the planner far more often, CLM was much faster. Numbers on Eval boards: JevBench by version, Jevals.com, typed-decisions and other independent suites | Judgment Jev; LOST latency | CLM's survival is mostly the shield; its verifier claims against Jev are unverified. Replica row: Open replicas and Jev-compatible servers |
Strategy games
| Task (builder) | Compared | Numbers as published | Won | Caveat |
|---|---|---|---|---|
| StarCraft II, realtime, Lv7 AI, 10 seeds per arm (JEV-Star, sc2musa; not the StarCraft 1998 row on Builds: games, simulators, robots and devices) | Jev vs uniform random, with and without a GPT-6 Astra planner (run through the Codex CLI). Macro: Protoss, 73 actions, realtime games (so decision latency is game time, inferred). Micro: SMAC-Hard with fixed stepping, one planner plan per map, Jev acting per unit | Wins / losses / time limits: Jev alone 0 / 10 / 0, as random; planner-constrained random 0 / 10 / 0; planner-constrained Jev 9 / 1 / 0; planner advisory 3 / 3 / 4. Micro, 35 maps × 3: Jev alone 3 wins, 2 draws, 100 losses; with planner 6-7 wins. Median 0.375-0.421 s over 19,605 responses | LOST alone; planner + Jev won | Author: version and timing differences block a causal reading. No top-level licence (third-party notices only) |
| Clash Royale, 1 match (@KashyapVisharad) | vs Laya on a Mac | Laya won | LOST | n = 1 |
How to read these
- Speed decides real-time games. Where the game waits for no one, the faster decider wins: Jev over chat models (Tetris, Pong, Mario), local replicas over Jev (Snake, Clash Royale). Remove the clock and a slower model can out-play it (Tetris vs Gemini). Jev Pong's author calls it a latency demo, not an intelligence demo.
- Judgment and speed separate. In fixed steering positions and next to a physics shield Jev judged as well or better than the local model that beat it on speed (laya-vs-jev-arena, CLM T-Rex).
- Code does part of the work. Legal-move menus, hanging-piece checks, shields and planners carry much of the result: trust notes say planner-assisted game scores measure the planner plus the model (laya-vs-jev); JEV-Star only won with a planner constraining its actions.
- Chess: cheap, not strong. Jev never beat an engine and lost a blitz game to GPT-6 Astra; its cheap, fast games still placed it near mid-pack reasoning models on Elo.
- Check the answer distribution, not only accuracy: the jev-use paddle question never chose
stay(Failure reports: confidence misread and calibration).
Related
- Head-to-head: Jev against other models and methods: app-level comparisons and cascades; Head-to-head benchmarks: Jev on public datasets and suites: public suites; Head-to-head: Jev inside agents, routers and tool gates: agent A/Bs; Cost ledger: published cost per Jev decision: cost per decision
- Builds: games, simulators, robots and devices: the game and device builds; Failure reports: where Jev broke, lost, or was the wrong tool: where games broke; Open replicas and Jev-compatible servers: Laya and other replicas
- Jev 1.13 jaggedness: known failure modes: documented failure modes to check before a game build
Sources
Links inline; raw captures listed in the frontmatter (private repo). The llm_chess engine tally comes from our own 2026-09-24 vetting of the harness branch, not a capture (Head-to-head benchmarks: Jev on public datasets and suites).