$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people

Agents: read the raw Markdown of this page, or start at llms.txt.

~/wiki/ideas

Builds: RAG, search and retrieval

[ community tier ][ updated 2026-11-02 ][ confidence medium ][ jev-1.14.0 ]#builds · rag · search · retrieval · community

TL;DR What people built with Jev around search and retrieval: reranking candidates, gating whether a passage or document answers, walking a document's sections, routing queries and building SQL. One row per build; every number is the builder's own run, so re-measure on your data (Testing and evaluating a Jev workflow). Split from Builds: data, search and business on 2026-11-02 (rows unchanged); benchmark-style retrieval and reranking results live on Head-to-head benchmarks: retrieval, reranking, screening and judging.

Builds

Build (builder) Jev decides Reported numbers Pattern Source
Agentic RAG app (The AI Automators; React, Python, Supabase; code not public) Three jobs, traced in Langfuse: a Choice routing each question to a cheap or strong model (instruction "how much work does a correct, complete answer require?", three criteria: greeting or small talk / needs the user's documents or tools / deep multi-hop analysis, judging the question on its own terms; a probability per option, verified shape, Choice questions; three options feed two models and the middle option's mapping is not stated); a Noul per chunk over 60 hybrid-search candidates with an agent-written ranking instruction, dropping those under ~20% (60 candidates may become ~10 for the LLM); citation Choice (fully / partially supported, says nothing, contradicts: Cookbook: Double-checking citations's three levels plus a partial one) plus a Noul Author-reported in a video (2026-09-24, machine captions): ~400 tokens per route; ~30k per 60-chunk rerank, his cost estimate 1/10 US cent (the caption reads "110th"; 1/10 is our reading); ~11k per citation check, under a second; price quoted as just over four US cents per million, output free (verified, Models, aliases, pricing, rate limits, context). Our arithmetic: 30,000 × $0.042/Mtok = $0.00126 ≈ 1/8 cent, so his 1/10 is ~20% low unless the total was nearer 24k tokens; 400 tokens ≈ $0.000017; 11,000 ≈ $0.00046. Sub-second fits TypeSafe's 70-500 ms claim (Jev (model)), not independently timed. The ~20% cut is his illustrative example, not a validated threshold; no accuracy figures; router: no savings figure (before, the strong model took everything), model names garbled in the captions. Batching of the 60 chunks is not shown: 30k fits the 64k per-request budget, but chunks in state would sit near the 32k state-plus-longest-question budget (State: what you send Jev); TypeSafe's rerank recipe sends one pair per request (Cookbook: Re-ranking). Citation check: 88% on "partially", and 75% vs 25% between "fully" and "partially" (summing to 100), plus his remark that a separate confidence also comes back, suggest these are probabilities (consistent with Confidence vs probability); what the UI's 82% and 57% confidence figures are is unverified; the cookbook's quote-substring step and confidence >= 0.8 gate are not mentioned. Proposed, not built: ask every turn whether each piece of context is still relevant or can be dropped; grade every thread from the previous day in Langfuse (retrieval relevant? answer grounded? sources cited?) instead of sampling, since per-call cost stops forcing a sample; tag each chunk before indexing as filterable metadata (footer? table of contents? signature block? language? document type? topic? personal data? sensitive?), with language, type and topic as Choices over your own lists (inferred); a newer-version check needs code to fetch the candidate older chunk and ask pairwise (Cookbook: Knowledge graph entity alignment); keep deterministic detectors beside a PII Noul (inferred: adversarial text, Model jaggedness: known failure modes §6) Patterns: agent internals, routing, gates, context and memory P02, P07, Patterns: judging, search, documents, real-time and markets P15, P16, P17, P18 video
jevsearch (@kylemclaren) Reranks React site-search results none in the capture Patterns: judging, search, documents, real-time and markets P17 post, repo
JevPDF (@kylemclaren) Find-by-meaning in a PDF: one Noul per line ("does this line answer the query?"), 16 lines per request, 16 requests in flight, ranked by probability; highlights stream in page by page; no embeddings or index README: $0.042/Mtok input, output free (verified), "well under a cent" to check every line of a 15-page paper; no accuracy figures Patterns: judging, search, documents, real-time and markets P17, Patterns: browser, computer use, voice and product UI P34 repo
jev-research-pipeline (shimo4228) Daily research monitor: code owns the loop; Jev screens each (source, open question) pair (on-topic check, then yes/no gates and weighted scores → keep / review / drop), then picks the verbatim sentences that advance or contradict a question; an LLM writes only the prose, and Jev scores each section against its claims (one rewrite, then a template) Moving the judgments off an Opus agent loop cut the daily cost and stopped it circling one theme; prose stays on an LLM. Author's pilot, notes in Japanese; before/after figures and caveats on Builder lessons: what changed the result Patterns: judging, search, documents, real-time and markets P18, Patterns: marketing, sales, GTM, content, support and ops P31 repo, dev.to
jev-experiments (dabit3, Nader Dabit) Latency demos built by the Devin agent (README). Turbo Rerank: BM25 top 50 → one request, a relevance Score per candidate plus an "answer exists?" Noul; code sorts by expected level. NL Palette: 66 editor commands and typed argument slots, six questions per keystroke pause; top probability ≥ 0.55 runs on Enter, else the top 3; an is_destructive Noul asks for confirmation Turbo Rerank beat BM25 order on the author's small hand-labelled set in well under a second per request, and results vary between runs. NL Palette beat fuzzy matching on the author's phrasings. Author-made small sets; no licence. Numbers: Head-to-head: Jev against other models and methods Patterns: judging, search, documents, real-time and markets P17, Patterns: browser, computer use, voice and product UI P34 repo
Internal RAG reranker (@franalgaba_, 2026-09-28; quoting @Vtrivedy10's "Jev for RAG") Jev reranks retrieved passages Reports a 12% gain on his internal RAG evals and low cost; no setup, data or code. unverified Patterns: judging, search, documents, real-time and markets P17 post
Wrangle recruiting search (@johncrown, wrangle.ai; vendor) Jev reranks candidates in the main app's search @johncrown reports 30.64% more matches per 200 candidates on searches recruiters actually run, at 75% of the previous reranking cost; reranker replaced and metric definition not stated; full benchmark promised "soon". unverified Patterns: judging, search, documents, real-time and markets P17 post
Rox sales-context retrieval (@rox_ai, 2026-09-28; vendor) Grades each retrieved chunk (transcripts, emails, CRM notes, news, documents) on a four-level relevance Score (not / slightly / mostly / directly relevant), compared with LLM reranking by GPT-5 Mini 75 production-realistic queries, scored against labels set by GPT-6 Astra at high reasoning: Jev "approached the benchmark at 96% accuracy", "12% higher" than GPT-5 Mini (points or relative not stated); 0.3 s vs 6.2 s average per query; "10x cheaper" (absolute cost only in an image, not captured). Agreement with Astra, not human ground truth; no data or code. unverified Patterns: judging, search, documents, real-time and markets P17 method, results
IntentSQL (Amine-LG, @AmineLargou; MIT; alpha) Natural language to SQLite as a chain of small decisions: code inspects the database and offers real tables, columns, values and operators; Jev picks among them; each answer narrows the next question, later coverage checks can refuse or trigger a fixed repair; code compiles parameterised SQL. Writes: preview on a copy, a semantic check, explicit confirmation, 100-change cap, undo README, recorded snapshots on jev-latest (run dates and resolved model not recorded): internal suite 80/80 in one historical run (19 expected refusals); frozen 40-case unseen Spider subset 30/40 passed, 5 safe refusals, 4 wrong results, 1 plan mismatch (not an official Spider score; not rerun after later safety fixes). Out of scope: multi-hop joins, joined aggregates, subqueries, top-N per group (refused, not guessed). Developer's 10-day spend: Cost ledger: published cost per Jev decision. Same author's Decision Lab: local playground comparing Jev with OpenJEV (Open replicas: servers and logit readers over other models) and Laya (Open replicas and Jev-compatible servers), neither of them Jev, on one input Patterns: judging, search, documents, real-time and markets P17 repo
jev-doc-search (VectifyAI, makers of PageIndex; @PageIndexAI, 2026-10-02, quoting a TypeSafe post; Apache-2.0; vendor) Which page of a long PDF answers a question, no vectors: PageIndex turns the PDF into a tree of sections (title, summary, page range); Jev walks it with one Choice per level, a beam of 3 paths scored by the geometric mean of step probabilities (Cookbook: Hierarchical classification); one Choice over the pages of each final section (split into windows when too long); then one Noul per candidate page (up to 16) with the page text in state, keeping ≥ 0.5 or the best 2 README: 2 questions, both right (NVIDIA 10-K revenue, 93 pages; Citigroup 10-K net income, 318 pages); no accuracy set, no cost. Why a tree: one option per page outgrows the 32k state-plus-longest-question budget after a few dozen pages, and the 255-option cap (both verified, Models, aliases, pricing, rate limits, context, Choice questions); a Choice always names a winner, hence the Noul check (consistent with Choice questions). Needs a PageIndex key Patterns: judging, search, documents, real-time and markets P17 repo, post
Jevbox (Extend, extend-hq; built by @andrewlu0 on Extend Parse; @kushalbyatnal, 2026-10-02; vendor; no licence file at capture) Self-hosted document drive: Jev files each upload into the existing folder tree (an uncertain document stays in a suitable parent; a cheap LLM may propose a new folder name, validated); search is a permission-filtered Jev beam search through categories, documents and sections, then an independent usefulness score per source passage against a threshold; weak first-pass evidence widens to other routes; a chat LLM answers, citing page and section. No keyword or embedding fallback. Permissions enforced at retrieval (revoked source → dependent chats blocked); MCP server No numbers (launch video not transcribed). README: real parsing and model acceptance must be checked with your own keys Patterns: judging, search, documents, real-time and markets P17, P26 repo, post
Deterministic Text-to-SQL (Satoshi Nagayasu, @snaga; Apache-2.0; Japanese) No SQL is generated. A question becomes slots (base table, joined tables, group-by dimension, metric, ranking, filters), each a choice built from an Apache Ossie semantic model (tables, fields, metrics, synonyms, allowed values), plus nouls for business rules such as "exclude returns". Code finds joins by shortest path over the foreign-key graph (NetworkX), drops a parent filter a child filter implies, and builds the SQL as a sqlglot AST; category values bind only from the whitelist, numbers come from a regex. When the top two options are within 0.35 it stops and asks (one example: two count metrics at 51% and 49%) Article: three worked TPC-H queries (8 tables, up to 5 joins), no syntax errors or invented columns, ~180-220 ms each (one: Jev ~213 ms + AST 7 ms); no test set or rate. Its sample payload (text, options, statement, value, distribution) contradicts docs; the repo's builder sends state, model: "jev-latest" and questions with instructions and criteria as documented (HTTP API: POST /v1/systemone and GET /v1/models), and the article's 30-50 ms per request is not what its own examples show. The author's limit: pulling literals and numbers (is 1995 a year or an amount?); plans an LLM fallback for literal slot-filling only, and per-domain routing for big warehouses Patterns: judging, search, documents, real-time and markets P17, P01 article (2026-10-25), repo
postgres-search (Alex Forman, @alex_forman1, 2026-10-08) Algolia-style product search in plain Postgres over 440,302 USDA Branded Foods products, then two Jev calls in parallel. Keep or sink: one Noul per top-10 result ("is this what the user meant?"), all in one request; low scorers sink. Did you mean: one Choice over up to seven spellings, each option stating its edit distance and how many products use it. A no-match line when every result scores low. Typeahead stays in Postgres and never calls Jev README, jev-1.13.0 pinned, 5 runs over 5,067 queries, each test set frozen in a commit before any model saw it. Right product first on 500 one-edit misspellings and controls: plain full-text 35%, this SQL 63%, plus both Jev questions 82%, a Norvig-style dictionary corrector 80%. Real misspellings from Wikipedia's list 71% vs 64% for the corrector; words cut short 75% vs 28%. Against: Jev respells 8 of 200 correct words one edit from a commoner word (krab → crab), the corrector none. Version 1 left edit distance and product counts out of the options and lost to the corrector (78% vs 85%); putting that evidence in the options is what closed the gap. Spelling answers calibrated (ECE 0.010 over 12,870 answers; 97.7% right at ≥ 0.9); keep-or-sink AUC 0.726 against noisy string-match labels. "Nothing matches" on 12 of 15 queries with no answer vs 145 of 5,052 with one. Answers vary between identical runs (no-match line on 29% of 388 queries, first result 6%): the author suggests a cache keyed by query. Adds 164 ms to the page median when it calls. 1,937 input tokens per search → $0.081 per 1,000 searches (at $0.042/Mtok, our arithmetic: verified); at 100,000 searches a month $8.14 vs $181–381 on Algolia's Grow plan, whose record charge alone is $136 for 440k records (Algolia prices as read by the author on 2026-10-07, unverified). The post's "$8" is that 100,000-search case, not a flat fee. Database hosting is left out, and Algolia's ranking, scaling and tooling were not measured Patterns: judging, search, documents, real-time and markets P17 repo, post
Enterprise RAG with RBAC (Irfan, @Youcanbet111065, 2026-10-10; live demo; no licence) Two Jev jobs (through OpenRouter); an LLM (Groq gpt-oss-120b) answers. Guard: after a regex blocklist, a topic Choice (company / off_topic / unclear); fails open (on error or doubt the question goes through). Router: retrieval strategy per question: BM25 lookup for a named person, hybrid search, or multi-query hybrid. Access control is not Jev's: the JWT role limits which departments retrieval searches README: topic check right on 14 of 15 labelled questions, router right on every labelled question (count not stated); the answer model is also the judge and the sets are small (author says so); latency and cost not measured. The README's feature list calls the topic check an LLM check; its guardrail section names Jev Intent routing, Patterns: judging, search, documents, real-time and markets P16 repo, post
Parallax (@moecoding, 2026-10-19; about 820k arXiv CS papers since 2020) Literature search for agents: several keyword and semantic recalls, union and dedupe, then Jev screens every candidate in batches and sorts the keepers into research directions; each inclusion keeps its probabilities and criteria. Agents then read papers and write surveys and digests with linked citations (by an LLM, inferred). Replaced a DeepSeek Flash pipeline Builder: 36,000+ papers screened in 5 minutes, 10x+ faster at under 1/5 the cost of DeepSeek Flash; the home page says 10,000 papers in about 2 minutes, 10x faster and 5x cheaper than a general LLM. No accuracy or recall figures, no comparison of what each pipeline kept; unverified; videos not transcribed Patterns: judging, search, documents, real-time and markets P17, P18 post, site
RAG hallucination filter (Harsh Mishra, Analytics Vidhya, 2026-10-16; @AnalyticsVidhya) An LLM drafts five answers from the same source text; one Jev request per answer asks four Nouls (every fact supported? answers the question? adds a fact not in the source? contradicts the source?). Code sums a weighted score and applies a hard gate on all four (≥ 0.85, ≥ 0.80, ≤ 0.15, ≤ 0.10; the author's demo values, not production settings), then sends the best passing answer; when none passes, a stronger LLM picks one of the top two or rejects. jev-latest before the alias move, so 1.13 (inferred from the date) Author's test, 8 cases, 2 hand-planted wrong answers each: all 16 planted answers blocked, but 13 of the 40 correct LLM answers blocked too; one case whose source held an old and a new rule failed every correct answer and escalated. Best-of-5 added nothing: the first draft was already right in 8 of 8. 56 calls, 0.60 s average, 25,648 input tokens, about $0.001 (verified at list price). Planted errors are easier to catch than natural ones (author) Patterns: judging, search, documents, real-time and markets P16; Cookbook: Double-checking citations article, post
Festival event search (@torayaki61, 2026-10-24; Tohoku University festival app; Japanese) Jev inside the app's event search (how is not stated) "Very fast and cheap"; on day 2 the builder guesses total spend is still under ¥1,000. unverified; the app returned only a script shell to our fetch Patterns: judging, search, documents, real-time and markets P17 (inferred) post
Shoply filter gate (@yfno1, article, 2026-10-27; Shopify search app; vendor; part 2 of a series) Which filters a shopper's search sets, over ~35 candidate product attributes. Round 1: one Noul per attribute ("does the search ask for a specific value?"; for price, whether it sets a price limit, with deals, sales, rentals and model numbers spelled out as not limits); round 2: one Noul per listed value of each kept attribute. DeepSeek V4.1 Flash starts at the same moment on number attributes only, and the search waits for its price limit only when Jev's price answer is above 50%. Catalog search phrases are matched by code, not Jev. A Jev round still unanswered at 0.5 s is raced against GPT-6 Luna 200 real searches from four stores; answer key written by Claude Opus 5.5, not checked by a person (authors say so). Filter F1 69% vs 59% for DeepSeek alone (precision 71% vs 58%); Luna alone 55%. Slowest searches (9 of 10 finished within) 0.52-0.56 s vs 0.72-0.83 s; typical search about equal, under half a second. $0.44 vs $1.00 per 1,000 searches. The gate opened on 9 of 200: all 9 real price limits at 0.87-0.97, "deals" and "on sale" searches at 0.11 and 0.08; the tenth number limit ("bestsellers") was missed by both. Same filters on a rerun: 96% vs 87%. A first design (Jev for listed values, an LLM writing numbers on nearly every search) reached 68% but was slower (0.58 s typical). Against Jev: asked one phrase at a time whether the shopper typed it, it said yes far too often (275 phrases picked vs DeepSeek's 38; "bikini" for "jorts"), so code now does that string match. Luna race started on 4-11 of 397 rounds and won twice. Model version not stated Patterns: judging, search, documents, real-time and markets P17, Confidence-gated routing post, article
Revi (Juan Vargas, @emersoftware; blog, Spanish, 2026-10-23; Chilean building-permit review agent; code not public) Kept: (1) which file in a permit case file is the application form, and which the municipal observations record: a Choice over the file names plus "none" (replaced a Gemini call, which stays as the fallback); (2) after each regulation search (vector RAG over planning law, circulars and local plans), a relevance probability per retrieved chunk, keeping those ≥ 0.2. Tried and dropped: picking the agent's next tool among ~50, a "can this be answered without searching?" gate, and choosing which old tool output sub-agents can forget Author: document pick ~0.4 s vs ~3 s before, US$0.00003 per case file. Chunk filter on 20 questions about one city's plan: input tokens −43%, cost −38%, pass rate 18.5 vs 19 of 20; local plans usually keep about 2 of 10 chunks; LLM input was nearly all their spend (4.3M input vs 45k output tokens in their evals). Did not work: tool choice matched the agent's own pick 4 in 10 times, worse than always answering "no tool", and under 2 in 10 when a tool was needed; the no-search gate, set never to skip a needed search, found 12 of 71 answerable messages; pruning saved no tokens, cost +7%, and matched the municipal reviewer's findings 21 of 45 times vs 24 without it. Model version and the chunk check's question type not stated Patterns: judging, search, documents, real-time and markets P26, P17; Patterns: agent internals, routing, gates, context and memory P07, P08 blog, post
Sapling RCM search router (Ruhan Khandakar, @KhandakarRuhan, dev.to, 2026-10-29; medical billing app) Before a ⌘K search over eight tables, one Noul per kind of text (person name, claim number, payment ID and others; six kinds), with the app's real ID formats in each question; code maps kinds to tables, searching every kind ≥ 0.3, else the top one. typesafe/jev-1.14 pinned through OpenRouter's alpha Decisions endpoint with zero-data-retention routing; on any failure Gemini, then all tables; an empty result searches the remaining tables vs Gemini 3.5 Flash-Lite (JSON schema, temperature 0): median 466 vs 900 ms, p95 648 vs 1,588 ms; $0.000023 vs $0.00017 per search (~$23 vs ~$170 per million); searches that missed a result 3 of 27 vs 6 of 30; one Gemini timeout. Ten test queries, three runs each; surnames went only to Gemini (no BAA with OpenRouter yet). $0.000023 ≈ 550 input tokens at list price (our arithmetic, plausible). Author's notes: Phoenix shows sub-cent costs as <$0.01; search text can leak into traces and ORM error logs. The endpoint is OpenRouter's (TypeSafe's docs cover the OpenRouter SDK route, not this endpoint: unverified) Intent routing, Patterns: judging, search, documents, real-time and markets P17 article, post
RAG no-answer check (every, Inc., every Tech Blog, intern kazu-gor, 2026-11-02, Japanese; @yuyu0037) Recipe search for an AI agent over DELISH KITCHEN content on Databricks AI Search: hybrid search orders candidates, a vector-distance threshold drops far ones (the hybrid score comes from ranks, so it cannot say "no answer"); then one Noul per top-10 document, in parallel, with its top 3 chunks and their headings in state ("do these blocks answer the question, at least in part?"; the false criterion names same-topic text that does not answer; broad questions count as answered). ≥ 0.5 (provisional) passes; the top 3 passing documents go on; all "no" → "no answer"; a failed call is retried once, and unresolved failures return an error, never "no answer". Via Cloudflare Workers AI (typesafe/jev) Local test on made-up questions: no-answer questions that still got documents 5 of 5 without Jev → 1 of 5 with it; answerable questions that lost every document 0 of 12 → 1 (vector only) or 2 (hybrid plus vector). A "difference between A and B" question failed because each document covers one side: per-document checks miss answers spread over documents. Median 247 ms per call; changing which 3 chunks were sent moved the probability by a median 0.01 over 58 documents (max 0.61; two flipped across 0.5). The article says 15 questions; its table rows add to 17. Vector threshold was 0 in this test. Model version not stated Patterns: judging, search, documents, real-time and markets P16, P17; Cookbook: Classifying RAG passages article, post

Sources

Files in frontmatter sources:, captured by 2026-11-02; original URLs inline.