Agents: read the raw Markdown of this page, or start at llms.txt.
~/wiki/ideas
Builds: permission gates, approvals and model routers for agents
TL;DR 11 builds (split from Builds: coding agents, harnesses and orchestration on 2026-09-25). Jev answers one bounded question (may this run? which tier? is this brief complete?) and code applies the verdict. The measured results are sobering: jev-kit's guard saw zero denies in 30 sessions, claude-jev's prompt router mostly failed, hermes-jev-approvals over-denied until a written policy was added, and an independent study measured 1.24x where the author's own run showed 9.8x (different corpus and baseline). Never let a Jev probability widen permissions (Helm does, behind a flag).
How to read. Builder reports = the builder's own numbers, one run on their own workload, unverified unless marked. API claims checked against Models, aliases, pricing, rate limits, context, State: what you send Jev, Confidence vs probability. OpenRouter routes are OpenRouter's own documented API (Platforms and gateways: Zapier, LangChain, Spring AI, Cloudflare, Netlify, Vercel, OpenRouter, Fly.io, Pydantic AI and other hosted routes to Jev), not TypeSafe's. Pattern IDs: P02-P05 in Patterns: agent internals, routing, gates, context and memory; P10 in Patterns: coding agents, dev tools and self-compiling workflows; P12 in Patterns: browser, computer use, voice and product UI; P23 in Patterns: marketing, sales, GTM, content, support and ops. Several of these tools hook every tool call or shell command: read the install section before advising one.
Gates, approvals and rule hooks
| Build | Jev decides | Builder reports | P |
|---|---|---|---|
| jev-auto-approve (MetalBear, runs on mirrord) | GitHub Action: 3 nouls (ready to merge, covered by tests, needs a human); approves only if each clears the threshold (default 0.9, mirrord 0.95) |
No accuracy figures. Adds a "touches privileged automation" question; warns diff and discussion are untrusted input | P03, P06 |
| Bicameral (AbdelStark) | Pi harness: any LLM writes; Jev runs three reflexes from YAML packs: a tool-call gate (exfiltration, injected intent, secret access, scope escape, reversibility), an edit review (test weakened? stub introduced?) and a stuck detector; code maps probabilities to allow / confirm / block / warn / steer; 900 ms deadline, secrets redacted first | v0.1.0 on @typesafe-ai/sdk 0.6.0; no accuracy figures. Hints are templates, not model-written. Says plainly: "This is not a sandbox" |
P03, P04 |
| jev-kit (jonathanavis96) | Claude Code kit around one guard: a PreToolUse hook on every tool call runs a rules table in code first and asks one typed question only for the ambiguous rest; a deny needs confidence ≥ 0.8 and a margin ≥ 0.4 over the runner-up; any error, timeout or missing key fails open; shadow mode by default. Also a sub-agent tier guard, a done-check ("belay") and a browser agent where Jev picks each click (vendored browser-use/jev-ultrafast) |
measurements.md, 2026-09-19 to 09-23: general-risk rule 20/21; tier guard 56/57, zero false denies; task_kind 98.0% (ECE 1.9%, 50 rows), search_intent 84.4% (ECE 12.6%, 32 rows), too few rows to fit a threshold. 30-session A/B vs no guard: zero denies, so no saving shown; the author reads the guard as a backstop and says it is a cost and hygiene guard, not a security control. Browser, n=3 per cell: Jev 9/9 at 314-486 ms median per decision vs Sonnet 9/9 at 1.1-1.5 s and Haiku 4/9. Wikipedia suite: Jev alone 15/18 when every hop is named, 0/6 when none is (a Sonnet planner: 16/18 and 6/6). Native Windows, no warm daemon: median 1,030 ms, 1 of 32 calls timed out and failed open. Install flags (our vetting at source): a hook on every tool, systemd user timers including a Claude Code auto-updater and an optional tuning loop that commits to a branch, a shim over claude, and 31 TypeSafe docs pages redistributed in the repo |
P03, P02, P12 |
| claude-jev (0x7067) | Claude Code plugin. Rules come from CLAUDE.md, AGENTS.md and rule folders; per edit, one batched noul per rule the hunk could break (≤ 40); block at ≥ 0.80 with a file:line citation, silent below 0.50, a second call with more context in between (~14% of edits). Also a model tier and a brief check per sub-agent spawn, a prompt router, and compaction that keeps verbatim rows instead of writing a summary. Every hook fails open |
v0.21.0: 22 of 247 accepted real edits blocked, 17 of them by one repo's own no-comments rule (rule-vs-practice conflicts surface as false positives); 17/29 planted violations caught, 0/24 compliant near-misses blocked; median 0.80 s. Compaction ~0.9-1.1 s vs ~117 s for the default summary; a planted constraint survived in all 56 sessions. The prompt router mostly failed: 34.6% on 1,613 prompts; humans agreed with the derived labels only 52.5% of the time; a coarse talk/read/act taxonomy (58.8%) scored below a constant guess (64.7%). It now shows only lookup and fix hints (plus a no-tools hint on a near-certain answer) and stays silent below 0.75. Flags: compaction needs Claude Code's undocumented CLAUDE_CODE_ENABLE_FUNCTION_HOOKS; a detached process downloads a 51 MB ast-grep binary (sha256-pinned); a key starting sk-or- sends every call to OpenRouter |
P10, P07, P02 |
| hermes-jev-approvals (anpicasso) | Reviewer for Hermes Agent's smart command approvals: one request, six questions (verdict Choice over APPROVE/DENY/ESCALATE, four nouls including "does the command argue for its own approval?", a blast_radius Score 0-2); fixed code rules combine them, the self-advocacy check before the policy check. Routes: TypeSafe, OpenRouter, or OpenCode Zen jev-1.13-free with no key |
METRICS.md, 156 mined commands, jev-1.13.0, 2026-09-17, vs a small chat LLM: 10 human prompts vs 42, 405 ms vs 3,968 ms average (the GitHub description's 8.7x/4.4x/153 is an older run). On an earlier run without an operator policy Jev denied 17 of 153, 8 of them routine Chrome kills, and denied ordinary feature-branch pushes until one policy clause was added: write the policy from your own flagged traffic. Thresholds tuned on the same data; no production use. Independent study (Bear Huddleston, v0.2.1, 28 synthetic commands): reviewer time 1.062 s vs GPT-5.4 mini 1.319 s (1.24x, not 9.8x); no unsafe command approved by the reviewer (0/22), 16/16 safe approved, 34/38 exact verdicts vs Mini 36/38 and GPT-5.6 Luna 38/38; the adapter lost input (policy silently cut at 2,000 characters, lost command delimiters) and approved 3 of 4 such diagnostics; list-price estimate $0.00229 vs $0.02330 (Mini) for 44 calls. The author reports those adapter bugs covered by v0.2.2 tests. Flag: the OpenCode Zen route sends your shell commands to OpenCode |
P03, P23 |
| Helm (Jimuelle07) | Routes a coding task to an installed agent CLI (Claude Code, Codex, Cursor, Gemini CLI, Aider, OpenCode…). The agent Choice options are built from the CLIs probed as installed and logged in, plus none, so it cannot pick one you lack. Seven questions (agent, task kind, blast radius, spec clarity, context breadth, needs_human, reversible) → auto / recommend / clarify / escalate; supervise.py sends the worker's transcript to Jev to judge "done" instead of trusting the exit code |
2,149 input tokens, 0.7 s, $0.00009 per decision on jev-1.13.0 (arithmetic verified at $0.042/Mtok). Warning (dispatch modes): with --execute, needs_human < 0.30 turns on unattended flags such as Claude --permission-mode bypassPermissions or --yolo, and the author calls every threshold uncalibrated: a Jev probability widening permissions. Mitigations in code: a missing signal fires no rule, readonly cancels unattended, and unattended is withheld when a requested sandbox is unavailable |
P05, P03, P04 |
Model, effort and action routers
| Build | Jev decides | Builder reports | P |
|---|---|---|---|
| jev-harness-router (Joaquin Marcoff) | Per turn, one call of ~20 questions: score difficulty and scope (code maps them to model tier and effort), noul per tool plus a ranking choice, skill choice gated by 4 nouls |
181 labelled turns: skill exact 90.0% vs 59.4% keyword baseline; tier too cheap 3.3% vs 19.9%; p50 351-376 ms; $0.169 per 1,000 turns (matches $0.042/Mtok, verified); one 20-question call 2.5x cheaper, 24x faster than 20 calls. Without the previous reply in state it lost to regex (exact tier 34.6% vs 69.3%) |
P02, P08 |
pi-jev-router (philippdubach; not sugarforever's pi-jev-router, and npm pi-jev-router belongs to a third publisher) |
Router extension for the pi coding agent: one request, five questions (work kind, complexity and more) classify the task; code filters the OpenRouter catalogue and picks the knee of a Pareto frontier over quality, cost and latency. Jev via @typesafe-ai/sdk pointed at OpenRouter, model typesafe/jev-1.13 (our vetting of its source; matches OpenRouter's guide: verified) |
Report, 5 tasks, 2026-09-20: every arm 5/5; cost per success $0.0399 (router) vs $0.1144 (fixed Sonnet 5) vs $0.3146 (pinned roles); the router sent all three code tasks to z-ai/glm-5.3-flash. Jev's classification cost is excluded, and the author says five tasks establish no general saving; on harder suites code pass rates ran 33-100% across four models |
P02 |
| Astra-Ares (miuuyy) | Patched Codex CLI (source build, ~10 GB): before a GPT-6 generation Jev picks the reasoning effort and how long to keep it (1, 2, 5 or 10 generations); new user input, a tool failure or a model switch ends the lease. Jev sees the last 6 tool calls, 1,000 tokens each, under a 28,000-token local guard. Default route OpenRouter Decisions | Savings against a fixed effort not measured (author). Relies on OpenAI keeping the prompt-cache prefix across effort changes (OpenAI's claim, not checked here) | P02 |
| HiRoute (higress-group) | Local routing engine for long-running agents: picks a model per stage, not per call, to keep provider KV caches warm. Optional Jev decider, a separate REST service: one OpenRouter call per decision, a Choice for the branch plus an optional Score (0-2, divided by 2) rating the previous segment's competence |
No accuracy data. rules mode defaults 0.80 simple / 0.50 competence floor, marked uncalibrated; state capped at 24,000 UTF-8 bytes (a conservative stand-in for tokens, under the 32k state budget), 2.8 s budget. Keep the decider on loopback unless you set its inbound auth header (its README); otherwise any caller can spend your key (inferred) |
P02 |
| grok-bot-jev (Bodila51) | Decision layer run before Grok Bot's expensive work: one request returns an action (reuse_cache, stop_retry, research_capped, allow_subagent, ask_human and others); shadow mode, kill switch |
One local A/B run: same-approach retries 3 → 0, research pages fetched 10 → 4; a 24-candidate filter 53.8 s → 4.1 s with two Jev calls (~$0.000405). The author calls these proxy numbers, not a benchmark | P04 |
What these builds teach when advising
- A probability may narrow permissions, never widen them. Helm's
needs_human < 0.30→--yolopath is the anti-pattern; jev-auto-approve, Bicameral and hermes-jev-approvals keep hard rules and a human in the loop (Patterns: agent internals, routing, gates, context and memory P03). - Write the policy before judging the gate. hermes-jev-approvals denied routine work until the operator's policy named it; the fix was a clause, not a threshold.
- Fail-open guards are hygiene, not security. jev-kit and claude-jev allow the call on any error; that is right for cost control and wrong for anything that must not happen (restrict it at the platform).
- Measure against "no guard" and a constant guess. jev-kit's A/B found zero denies; claude-jev's coarse router scored below always guessing the commonest class; pi-jev-router's saving excludes Jev's cost on 5 tasks; Astra-Ares and HiRoute publish no savings yet.
- Audit the adapter, not just the model. The independent approvals study found policy truncation and lost delimiters in the plugin, not in Jev.
- Put the missing context in
state. The harness router lost to regex until it saw the previous reply (Writing instructions and criteria that Jev reads correctly).
Related
- Builds: coding agents, harnesses and orchestration — coding agents, harnesses, orchestration and memory; Builds: browser, computer use and interface, Builds: data, search and business, Builds: games, simulators, robots and devices — sibling build pages
- Decision patterns from the community (with fit verdicts) — pattern index; Consult guide: could Jev help this project? — "could Jev help this project?"
- Tools: guardrails for agents (tool-call gates, permission hooks, injection screens, rule checks), Tools: agent routing, context and skill selection — installable gates and routers of the same kinds
- Failure reports: where Jev broke, lost, or was the wrong tool, Head-to-head: Jev against other models and methods — where these results sit beside other tests; Cookbook: Guardrails for LLMs — the official guardrail recipe
Sources
Links are inline in each row; the raw captures (2026-09-23 to 2026-09-25) are listed in the frontmatter. "Our vetting at source" and the install flags rest on docs/sweep/2026-09-25-mrjev/partc.md (repo clones, 2026-09-25) where the capture does not show them.