Agents: read the raw Markdown of this page, or start at llms.txt.
Cookbook: Verifying LLM answers against their source
TL;DR Send the question, the LLM's instruction, the retrieved paragraphs and the LLM's answer as one
state, with threeNoulquestions in the same request:answerable,declines,supported. In code: ifdeclines >= 0.5, risk =answerable; otherwise risk =1 - supported. Risk>= 0.2goes to review. On 1,000gpt-6-lunaanswers (TypeSafe's run,jev-1.14.0, 2026-11-07) the check passed 727 answers, 95.7% of them right (88.5% across all), and held back 84 of the 115 wrong ones. It also held back 189 right answers, and 31 wrong answers passed.
Goal
An LLM answers questions from paragraphs you retrieved for it, and most answers are right. The cookbook targets two kinds of mistake that, in its words, don't look "any different from a right answer":
- the LLM gives an answer the paragraphs never state, "filling a missing step with a guess";
- the LLM says the paragraphs don't answer the question when they do.
You end up with check_answer(). It takes a question, its paragraphs and an answer, and returns the three probabilities, the risk, and pass or review. The cookbook's diagram sends a review answer to "retry, a stronger model, or a person".
Upstream lists it under LLMs and agents, level Beginner (raw/docs/cookbooks.md).
Inputs / state shape
Data. Questions from MuSiQue (Trivedi et al., CC BY 4.0). Each joins two to four simple questions about Wikipedia into one, so the answer to an earlier part is needed to ask a later one. Each comes with 20 paragraphs, most of them on nearby topics that don't help. MuSiQue has two versions of each question: one where the paragraphs hold every needed fact, and one where a needed paragraph was removed, "though the remaining paragraphs still look as if they might" answer it. The cookbook's file musique_questions.json (downloaded from typesafe-public-examples, cookbooks/verify_llm_outputs) is a fixed random sample of 1,000 questions from MuSiQue's development set, one version each; about half can't be answered. The run prints 1,000 questions, 473 answerable from their paragraphs.
Paragraphs are joined into one string as ## {title}\n{text} blocks.
The answers being checked come from gpt-6-luna with this instruction, which allows a refusal (the source of the refusals the check reads):
ANSWER_INSTRUCTION = (
"Answer the question using only the paragraphs. Return only the shortest answer that fully "
"answers the question. Do not explain your reasoning. If the paragraphs do not answer the "
"question, say so."
)
To check your own LLM, the cookbook says to replace answer_prompt() and ask_llm().
The state sent to Jev, one request per answer:
def check_state(question: str, paragraphs: str, answer: str) -> dict:
return {
"request": question,
"response_requirement": ANSWER_INSTRUCTION,
"paragraphs": paragraphs,
"candidate_response": answer,
}
All 20 paragraphs go in, not just the relevant ones.
Questions asked
Three Noul questions in the same request, verbatim:
def noul(instructions: str, yes: str, no: str) -> Noul:
return Noul(instructions=instructions, criteria=NoulCriteria(true=yes, false=no))
CHECK_QUESTIONS = {
"answerable": noul(
"Do the paragraphs state every fact needed to answer the question?",
yes="each step from the question to its answer is stated in the paragraphs",
no="at least one step from the question to its answer is not stated in the paragraphs",
),
"declines": noul(
"Does the candidate response decline to answer, or say the information is missing?",
yes="the candidate says it cannot answer or that the paragraphs do not say",
no="the candidate gives an answer",
),
"supported": noul(
"Following the paragraphs step by step from the question, do you arrive at the "
"candidate's answer?",
yes="the paragraphs connect the question to the candidate's answer through stated facts",
no="the paragraphs do not connect the question to the candidate's answer",
),
}
| key | type |
what true means |
|---|---|---|
answerable |
noul |
every step from question to answer is stated in the paragraphs |
declines |
noul |
the LLM said it cannot answer or the paragraphs don't say |
supported |
noul |
the paragraphs connect the question to the LLM's answer through stated facts |
Combining logic in code
The risk formula, verbatim from check_answer():
- the LLM declined (
declines >= 0.5): risk =answerable("risky if the paragraphs do answer"); - the LLM answered: risk =
1 - supported("risky if the paragraphs don't lead there"); status="review"if risk>= REVIEW_AT(0.2), else"pass".
Complete check, condensed from the cookbook's Setup and "Ask Jev three questions" blocks (the dataset download, the LLM answering step, the chart and the labelling step are left out; the check logic is verbatim; the last two lines are our placeholder call).
import json
import os
import urllib.request
from pathlib import Path
from cooksafe import JsonCache
from typesafe_sdk import Noul, NoulCriteria, TypeSafeClient
TYPESAFE_MODEL = "jev-1.14.0" # the page's model; "jev-latest" runs the newest, live
REVIEW_AT = 0.2 # a risk at or above this sends the answer to review
client = TypeSafeClient(
api_key=os.environ.get("TYPESAFE_API_KEY", "cache-only"), # keyless kernels replay the cache
base_url=os.environ.get("TYPESAFE_ENDPOINT"),
timeout=120.0,
)
EXAMPLES = (
"https://raw.githubusercontent.com/typesafe-ai/typesafe-public-examples/main/"
"cookbooks/verify_llm_outputs"
)
def download(name: str) -> Path:
"""Fetch one of this cookbook's files from typesafe-public-examples unless it is on disk."""
path = Path(name)
if not path.exists():
urllib.request.urlretrieve(f"{EXAMPLES}/{name}", path)
return path
CACHE = Path("verify_llm_outputs_cache.json")
# True fills CACHE with the published answers, so the run makes no requests. To run
# live, set it to False and delete CACHE; a CACHE already on disk is never overwritten.
DOWNLOAD_CACHE = True
if DOWNLOAD_CACHE:
download(CACHE.name)
json_cache = JsonCache(CACHE, indent=None)
ANSWER_INSTRUCTION = (
"Answer the question using only the paragraphs. Return only the shortest answer that fully "
"answers the question. Do not explain your reasoning. If the paragraphs do not answer the "
"question, say so."
)
def noul(instructions: str, yes: str, no: str) -> Noul:
return Noul(instructions=instructions, criteria=NoulCriteria(true=yes, false=no))
CHECK_QUESTIONS = {
"answerable": noul(
"Do the paragraphs state every fact needed to answer the question?",
yes="each step from the question to its answer is stated in the paragraphs",
no="at least one step from the question to its answer is not stated in the paragraphs",
),
"declines": noul(
"Does the candidate response decline to answer, or say the information is missing?",
yes="the candidate says it cannot answer or that the paragraphs do not say",
no="the candidate gives an answer",
),
"supported": noul(
"Following the paragraphs step by step from the question, do you arrive at the "
"candidate's answer?",
yes="the paragraphs connect the question to the candidate's answer through stated facts",
no="the paragraphs do not connect the question to the candidate's answer",
),
}
def check_state(question: str, paragraphs: str, answer: str) -> dict:
return {
"request": question,
"response_requirement": ANSWER_INSTRUCTION,
"paragraphs": paragraphs,
"candidate_response": answer,
}
def as_json(questions: dict) -> str:
return json.dumps(
{k: q.model_dump(mode="json", exclude_none=True) for k, q in questions.items()},
sort_keys=True,
)
@json_cache
def ask_jev(
question: str, paragraphs: str, answer: str, questions_json: str, model: str
) -> dict:
response = client.system_one(
check_state(question, paragraphs, answer), json.loads(questions_json), model=model
)
return {
"nouls": {k: response.answers[k].noul for k in sorted(response.answers)},
"input_tokens": response.usage.input_tokens or 0,
}
def check_answer(question: str, paragraphs: str, answer: str) -> dict:
nouls = ask_jev(
question, paragraphs, answer, as_json(CHECK_QUESTIONS), model=TYPESAFE_MODEL
)["nouls"]
if nouls["declines"] >= 0.5:
risk = nouls["answerable"] # it declined: risky if the paragraphs do answer
else:
risk = 1 - nouls["supported"] # it answered: risky if the paragraphs don't lead there
return {**nouls, "risk": risk, "status": "review" if risk >= REVIEW_AT else "pass"}
paragraphs = "## Title A\nText of paragraph A.\n\n## Title B\nText of paragraph B."
print(check_answer("Your question?", paragraphs, "The LLM's answer."))
The questions travel as JSON (as_json) so the serialized questions are part of the JsonCache key: editing a question forces a fresh call. Since the 2026-11-11 docs the model is an argument too (model on the Jev calls, answering_model on ask_llm, recheck_model on recheck), so changing a model also forces fresh calls. Install: pip install openai ipython matplotlib 'cooksafe>=0.3.0,<0.4.0' (since the 2026-11-11 capture; 'cooksafe>=0.2.0,<0.3.0' in the 2026-11-08 one), then set TYPESAFE_API_KEY and OPENAI_API_KEY; cooksafe 0.3.0 (uploaded to PyPI 2026-11-10T23:44:13Z) requires typesafe-sdk>=0.7.0,<0.8.0, msgspec>=0.18 and lzstring. With DOWNLOAD_CACHE = True (the default, since 2026-11-11) the code downloads the published verify_llm_outputs_cache.json, so a run makes no requests; to run live set it to False and delete the file (until then the cache was a local json_cache.json). The cookbook runs the 1,000 checks with ThreadPoolExecutor(max_workers=4); its comment: "2,000 Jev calls; more workers hits the public rate limit". Rate limits: HTTP status codes, rate limits, retry semantics.
Results the cookbook reports
Numbers from jev-1.14.0 on 2026-11-07, checking answers gpt-6-luna wrote the same day (TypeSafe's run).
Four example answers (printed output):
| question | gpt-6-luna answer |
answerable | declines | supported | risk | status |
|---|---|---|---|---|---|---|
| Where did the painter of The Mud Bath study? | The Slade School of Art. | 0.97 | 0.02 | 0.98 | 0.02 | PASS |
| Where is Assomada located? | The paragraphs do not say where Assomada is located. | 0.07 | 0.99 | 0.97 | 0.07 | PASS |
| What was the population density of Tammy Leitner's city of birth in 2009? | 182.5/km² | 0.12 | 0.04 | 0.08 | 0.92 | REVIEW |
| When did the father of Dahyabhai Patel die? | The paragraphs do not say when Dahyabhai Patel’s father died. | 0.86 | 0.81 | 0.09 | 0.86 | REVIEW |
In the cookbook's reading: Tammy Leitner was born in San Diego, but no paragraph gives San Diego's 2009 population density, so supported is low. The Patel question can be answered (one paragraph names Sardar Vallabhbhai Patel as the father, another gives his death date), so the refusal goes to review on answerable 0.86.
Labels. Each answer is labelled right or wrong. A separate Jev request with one Noul (matches_reference) compares the answer with MuSiQue's reference answer, without the paragraphs. A non-match (< 0.5) goes to gpt-6.1-sol (reasoning_effort="high", JSON schema right / wrong / unclear), which reads the paragraphs, because "the reference is sometimes wrong about the paragraphs". Output: 739 matched the reference; of the other 261, gpt-6.1-sol labelled 146 right, 115 wrong and 0 unclear. The cookbook says the 146 are cases where MuSiQue counts a question answerable but the paragraphs skip a step, or the "unanswerable" version still has the answer in another paragraph.
The measurement (an answer passes when its risk is below the threshold):
| answers used | how many | right | wrong held back |
|---|---|---|---|
| every answer | 1,000 (100%) | 88.5% | 0 of 115 |
| risk below 0.2 | 727 (73%) | 95.7% | 84 of 115 |
| risk below 0.3 | 825 (82%) | 94.2% | 67 of 115 |
| risk below 0.5 | 917 (92%) | 91.9% | 41 of 115 |
| risk below 0.7 | 965 (96%) | 90.3% | 21 of 115 |
At 0.2 the check holds back 273 answers: 84 wrong, 189 right, "so most of the review work goes to confirming answers that were fine". The chart (risk histogram, right vs wrong answers) shows right answers concentrated near zero risk and wrong answers "spread across the whole range", which is why the threshold has to be low.
The cookbook says the setup makes about 1,250 OpenAI calls and 2,000 Jev calls (code comments); it reports no cost or latency.
Adapting it
- Your LLM: replace
answer_prompt()andask_llm(); put your own instruction inresponse_requirementsodeclinesandsupportedread the answer against what the LLM was actually asked. - Your threshold: the cookbook's advice is to run the labelling step on your own questions with reference answers and "pick the threshold from the table". 0.2 is tuned to this sample; it is not the docs' general Noul starting point (0.7 / 0.3 on a single Noul, Noul (yes/no) questions).
- Review route: the diagram names retry, a stronger model or a person; which one is your call.
- Your source: anything that can be pasted as paragraphs works the same way (inferred; the cookbook tests only MuSiQue's Wikipedia paragraphs).
- Playground:
make_playground_link(check_state(...), CHECK_QUESTIONS)opens the Patel refusal with its state and the three questions, to edit in the browser.
Gotchas
- Wrong answers that pass. 31 wrong answers passed at 0.2. 18 answer a related question instead of the one asked (the cookbook's examples: "Leftover Salmon" when asked who is in the band, "the NFL" when asked what kind of thing the NFL is); the paragraphs do lead there, so
supportedcomes back high. The other 13 are refusals of answerable questions whereanswerablealso came back low, 0.10 to 0.19: "Jev missed the same connection between paragraphs that the LLM missed". The check is not independent of the LLM's own blind spots. Multi-hop questions are the indirection mode in Model jaggedness: known failure modes (our link, not the cookbook's). - The branch is a hard cut on
declines.declines >= 0.5decides which Noul the risk reads; the other is ignored (the Tammy Leitner row readssupported, notanswerable). A half-refusal near 0.5 flips branches (inferred). - Labels come from models. "A stricter or looser re-check would change the 88.5%." The first pass of labelling is itself a Jev
Noul(inferred concern: the check and part of its scoring share a model). - Selection on the sample. TypeSafe picked these three questions over two other sets by trying them on the first 100 questions of this sample, so "the results on them are not a fair test".
- Review load. At 0.2, 189 of the 273 held-back answers were right; budget review for mostly-fine answers.
- Rate limit. More than 4 parallel Jev workers hit the public rate limit in TypeSafe's run.
Related
- Cookbooks overview — the cookbook catalog
- Cookbook: Double-checking citations — the other "check an LLM against its source" cookbook: one Choice per citation
- Cookbook: Classifying RAG passages — gating the passages before the LLM answers
- Cookbook: Guardrails for LLMs — screening what goes into and out of an LLM
- Speculative fan-out — all three questions in one request
- Companion Noul — one Noul deciding how to read another answer in the same request
- Noul (yes/no) questions — the yes/no primitive and its thresholds
- Testing and evaluating a Jev workflow — labelling and threshold-picking as an eval method
Sources
- raw/docs/cookbooks__verify_llm_outputs.md (https://docs.typesafe.ai/cookbooks/verify_llm_outputs); re-fetched 2026-11-11 (install pin
cooksafe>=0.3.0,<0.4.0, publishedverify_llm_outputs_cache.jsondownloaded viaDOWNLOAD_CACHE = True, every cached call takes its model; questions, threshold, models, date and every number unchanged) - raw/docs/cookbooks.md (https://docs.typesafe.ai/cookbooks.md) — section and level
- raw/docs/primitives__noul.md (https://docs.typesafe.ai/primitives/noul) — the 0.7 / 0.3 starting point
- raw/docs/model-jaggedness.md (https://docs.typesafe.ai/model-jaggedness) — indirection