$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people

Agents: read the raw Markdown of this page, or start at llms.txt.

~/wiki/cookbooks

Cookbook: Reading a scanned vaccination record

[ cookbook ][ updated 2026-11-03 ][ confidence high ][ jev-1.14.0 ][ python sdk 0.7.5 ]#cookbook · images · box · extraction · documents · noul

TL;DR Jev does not extract a list from a scan; it answers questions about it. Request 1 asks, for each of the 16 vaccines the clinic gives, a Noul (is it in the table, with exactly this name?) and a Box (where is its row?), plus one Noul for "any vaccine not on this list?". Request 2 asks, per found row, a year Choice whose options end with other and unreadable, and a legibility Noul. Request 3 sends the readings as text, no image, with the year-difference worked out in code, and asks a Noul per dose: will it still protect on the departure date? Code flags rows for a nurse. On the sample record (jev-1.14.0) Jev answered unreadable for the rabies date under a sticker and scored the five-year-old typhoid dose 0.38; both rows were flagged, nothing else was. Images and Box are experimental and need the TypeSafe-Danger-Zone-Images: true header.

Goal

Scanned vaccination records with no text layer need checking before a trip: which vaccines are recorded, when each dose was given, and whether each dose still covers the travel date. The cookbook reads one travel vaccination record into one row per vaccine and flags the rows a nurse should check (raw/docs/cookbooks__reading_scanned_documents.md). The sample record has two such rows: the rabies date is under a batch sticker, and the typhoid dose is from June 2021, which the cookbook says protects "for about three years", so it will not cover a trip in November 2026.

Upstream lists it under "Image processing", level Intermediate, next to Cookbook: Feeder-cam visit log and Cookbook: OCR confidence and bounding boxes (raw/docs/cookbooks.md; Cookbooks overview).

When to use it: documents where the possible answers are known in advance (the items a clinic, insurer or school can record; a range of plausible years) and an uncertain reading should go to a person. When not to: open-ended transcription of whatever is on the page, or a list of candidate items so long the questions will not fit one request (the cookbook flags this for "a list of thousands"; see Models, aliases, pricing, rate limits, context for context length). Scanning in, and filing the result in the traveller's record, are out of scope.

Inputs / state shape

Requests 1 and 2 carry the scan as {"page": Image(PAGE)}; every question names `page`, and all questions in a request see the same image, so each step's questions go in one request. Request 3 carries no image (and so needs no images header): a short context string and one entry per readable row.

The scan (vaccination_record.jpg) is in the typesafe-ai/typesafe-public-examples repo; since the 2026-11-03 docs the script fetches it itself (download(), below) unless it is already on disk. Responses are cached in a local json_cache.json, so a re-run replays them; until the 2026-11-03 docs the cookbook also published that cache in the same repo, to replay the published answers without calling the API. Accepted images are PNG or JPEG up to 16 MiB, 280 to 1568 pixels per side, so scale a high-resolution scan down first (Images and assets (experimental)).

The cookbook pins TYPESAFE_MODEL = "jev-1.14.0" and passes it on every call; it does not give a date for its run. jev-latest has pointed to jev-1.14.0 since 2026-10-24 (Models, aliases, pricing, rate limits, context).

Setup, verbatim except the plotting imports:

import hashlib
import json
import os
import urllib.request
from datetime import date
from pathlib import Path
from time import perf_counter

from cooksafe import JsonCache
from typesafe_sdk import Image, TypeSafeClient

TYPESAFE_MODEL = "jev-1.14.0"
IMAGES_ON = {"TypeSafe-Danger-Zone-Images": "true"}  # opt in to the image preview
READ_CUTOFF = 0.90  # below this, a person checks the reading against the paper
LEGIBLE_CUTOFF = 0.50  # the legibility Noul gets its own, lower cutoff
IN_DATE_CUTOFF = 0.50  # below this, a nurse checks whether the dose still covers the trip

client = TypeSafeClient(
    api_key=os.environ.get("TYPESAFE_API_KEY", "cache-only"),  # keyless kernels replay
    base_url=os.environ.get("TYPESAFE_ENDPOINT"),
    timeout=120.0,
)
json_cache = JsonCache(Path("json_cache.json"))


EXAMPLES = (
    "https://raw.githubusercontent.com/typesafe-ai/typesafe-public-examples/main/"
    "cookbooks/reading_scanned_documents"
)


def download(name: str) -> Path:
    """Fetch one of this cookbook's files from typesafe-public-examples unless it is on disk."""
    path = Path(name)
    if not path.exists():
        path.parent.mkdir(parents=True, exist_ok=True)
        urllib.request.urlretrieve(f"{EXAMPLES}/{name}", path)
    return path


PAGE = download("vaccination_record.jpg")
PAGE_SHA = hashlib.sha256(PAGE.read_bytes()).hexdigest()[:16]  # cache key for the scan


@json_cache
def ask(page_sha: str, questions_json: str) -> dict:
    """One request: the scan as the state, plus a set of questions about it."""
    started = perf_counter()
    response = client.system_one(
        state={"page": Image(PAGE)},
        questions=json.loads(questions_json),
        model=TYPESAFE_MODEL,
        extra_headers=IMAGES_ON,
    )
    return {
        "answers": {k: a.model_dump(mode="json") for k, a in response.answers.items()},
        "seconds": round(perf_counter() - started, 2),
        "input_tokens": response.usage.input_tokens or 0,
        "output_tokens": response.usage.output_tokens or 0,
    }


def ask_page(questions: dict) -> dict:
    if not questions:  # nothing to ask, such as a page with no rows found
        return {"answers": {}}
    return ask(PAGE_SHA, json.dumps(questions, sort_keys=True))

Install upstream: pip install ipython matplotlib 'typesafe-sdk>=0.7.5,<0.8.0' 'cooksafe>=0.2.0,<0.3.0', then set TYPESAFE_API_KEY. TYPESAFE_ENDPOINT is a cookbook convention, not an SDK variable (TYPESAFE_* environment variables across SDKs). Questions are plain dicts here, not SDK classes.

Questions asked

Request 1: find the rows (33 questions). The list deliberately includes look-alikes (the combined hepatitis A and B vaccine next to Hepatitis A) to check that Jev matches the exact name. The unlisted question uses structured instructions (a dict with question and list; Structured instructions, options, levels, criteria) to catch a vaccine no other question would.

VACCINES = [
    "Hepatitis A", "Hepatitis B", "Hepatitis A and B combined", "Typhoid",
    "Hepatitis A and typhoid combined", "Yellow fever", "Rabies", "Japanese encephalitis",
    "Tick-borne encephalitis", "Cholera", "MenACWY", "MenB",
    "Tetanus, diphtheria and polio (Td/IPV)", "MMR", "Varicella", "Influenza",
]
PRESENT_CUTOFF = 0.5  # P(recorded) at or above this counts the vaccine as on the page


def find_questions() -> dict:
    questions = {"unlisted": {
        "type": "noul",
        "instructions": {
            "question": "Does the vaccines table in `page` record any vaccine that is not "
                        "in this list?",
            "list": VACCINES,
        },
    }}
    for i, vaccine in enumerate(VACCINES):
        questions[f"vac_{i:02d}"] = {
            "type": "noul",
            "instructions": f"Is {vaccine} recorded in the vaccines table in `page`, with "
                            "exactly this name?",
        }
        questions[f"vac_{i:02d}|box"] = {
            "type": "box",
            "instructions": f"the whole {vaccine} row of the vaccines table in `page`",
        }
    return questions

Request 2: read the year (two questions per found row). The in-date check only needs roughly how old a dose is, so the Choice asks for the year alone. A Choice always picks an option, so without unreadable the date under the sticker "would otherwise get whichever year looked closest". The legibility Noul is a second, separate signal that something covers the date. The record writes day/month/year, so the instruction says the year is the last part.

YEAR = {
    **{f"y{n}": str(n) for n in range(2016, 2027)},
    "other": "A year before 2016 or after 2026",
    "unreadable": "The year is covered, stained, smudged or damaged and can't be read",
}


def read_questions(row: dict) -> dict:
    where = f"the DATE GIVEN cell of the {row['vaccine']} row of the vaccines table in `page`"
    i = row["id"]
    return {
        f"{i}|year": {
            "type": "choice", "criteria": YEAR,
            "instructions": "Dates on this record are written day/month/year, so the year "
                            f"is the last part. What year is written in {where}?",
        },
        f"{i}|legible": {
            "type": "noul",
            "instructions": f"Is the date in {where} fully legible, with nothing covering, "
                            "staining, smudging or blurring any of it?",
        },
    }

Request 3: does each dose cover the trip (text only, one Noul per readable row). The reading stays with the image questions; this request judges only the dose. Date arithmetic is "a job for code", so code turns each year into a rough "given before departure" phrase, and rows without a year are left out.

DEPARTURE = date(2026, 11, 12)  # from the traveller's booking


def before_departure(year: int) -> str:
    years = DEPARTURE.year - year
    return "less than a year" if years == 0 else f"about {years} year{'s' * (years != 1)}"


CHECK_CONTEXT = (
    "Each entry is one line of an adult traveller's vaccination record, as read from a "
    f"scan. The traveller leaves on {DEPARTURE.day} November {DEPARTURE.year}. Judge "
    "only whether the dose will still protect the traveller on that date, by usual "
    "travel-health guidance for adults."
)
IN_DATE_QUESTION = "Will this dose still protect the traveller on the departure date?"
IN_DATE_CRITERIA = {
    "true": "The dose is within the usual protection period for this vaccine on the "
            "departure date.",
    "false": "The dose is past the usual protection period for this vaccine by the "
             "departure date.",
}


def check_questions(readable: list[dict]) -> dict:
    return {
        row["id"]: {
            "type": "noul",
            "instructions": {"question": IN_DATE_QUESTION, "entry": row["id"]},
            "criteria": IN_DATE_CRITERIA,
        }
        for row in readable
    }


def check_state(readable: list[dict]) -> dict:
    return {
        "context": CHECK_CONTEXT,
        "entries": [{"id": r["id"], "vaccine": r["vaccine"], "year_given": as_text(r),
                     "given_before_departure": before_departure(year_given(r))}
                    for r in readable],
    }

(as_text and year_given turn the Choice key back into a year: None for other or unreadable, else int(key[1:]) on keys such as y2021.) Request 3 goes through a separate cached check(state_json, questions_json) that calls client.system_one(state=..., questions=..., model=TYPESAFE_MODEL) with no header. Until the 2026-11-03 docs ask() and check() also took the model as their first argument (so it was part of the cache key); they now read TYPESAFE_MODEL.

Combining logic in code

A high in-date score does not cancel a doubtful reading, because it was judged on a year that might be wrong. The cutoffs are "a starting point"; tune them on records a person has already checked.

def reading_score(row: dict) -> float:
    """The weakest of the answers that say the row was found and read."""
    return min(row["present"], row["year_confidence"])


def review_reasons(row: dict) -> list[str]:
    reasons = []
    if (unresolved(row) or reading_score(row) < READ_CUTOFF
            or row["legible"] < LEGIBLE_CUTOFF):
        reasons.append("Check the reading")
    if row["in_date"] is not None and row["in_date"] < IN_DATE_CUTOFF:
        reasons.append("Check the dose")
    return reasons

For each flagged row the nurse gets a crop of the page from request 1's box: the edges are fractions of the image's height and width, so multiply by the pixel size; pad by 0.02 of the page, or 0.05 when the box confidence is below 0.6.

Results / what the cookbook reports

All from jev-1.14.0 (raw/docs/cookbooks__reading_scanned_documents.md).

Request 1: the six vaccines on the page scored 0.97 or more; the ten not on it, look-alikes included, 0.02 or less; unlisted 0.02. Sorted by box top, the six rows came out in page order. The boxes of absent vaccines still came back, all with a top between 0.33 and 0.36.

Vaccine Recorded Year read Legible In date Flag
Hepatitis A 0.98 2026 0.96 0.94 –
Hepatitis B 0.98 2026 0.86 (highlighter stroke across the row) 0.83 –
Typhoid 0.98 2021 0.96 0.38 Check the dose
Yellow fever 0.98 2025 0.96 0.95 –
Tetanus, diphtheria and polio (Td/IPV) 0.98 2024 0.96 0.93 –
Rabies 0.97 unreadable 0.05 not asked Check the reading

Five years were read correctly. The legibility Noul scored clear dates 0.86 to 0.96, lower than the Choice answers, which is why it gets its own 0.50 cutoff. The lowest reading score in the flag table is 0.97 (Rabies), so READ_CUTOFF did not trigger anything; the Rabies flag came from unreadable and legibility 0.05. Crops: Typhoid box top 0.48, bottom 0.53, confidence 0.88; Rabies top 0.69, bottom 0.75, confidence 0.77; each crop shows the whole row. The page was not flagged.

Appendix: why reading and checking are separate requests. Asking the in-date question straight off the image (departure date read from the page) gave Hepatitis A 0.88, Hepatitis B 0.85, Typhoid 0.50, Yellow fever 0.96, Td/IPV 0.84 and Rabies 0.88. Both bad rows passed: Rabies at 0.88 with its date under a sticker, and Typhoid at 0.50, "right on the cutoff" (the check is < 0.50, so 0.50 passes). With one answer covering two steps "you can't tell whether Jev failed to read the date or judged the dose wrongly".

Adapting it to a new domain

Gotchas

Sources