$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people

Agents: read the raw Markdown of this page, or start at llms.txt.

~/wiki/cookbooks

Cookbook: Infinite cardinality choice

[ cookbook ][ updated 2026-11-11 ][ confidence high ][ jev-1.14.0 ][ python sdk 0.7.5 ]#cookbook · search · shortlist · noul · choice

TL;DR A Choice takes at most 255 options, so to pick one item out of thousands, narrow first: one Noul per item ("is this the one they mean?") on a short description, sent in parallel chunks; a second Noul round on a long description for the top 80; then one Choice over the top 20, whose confidence decides "show one" (0.8 or more) or "show three". On 100 real "what was that film?" requests over 5,000 films (TypeSafe's run, jev-1.14.0, November 2026) the right film came first for 78%, against 33% when an embedding search built the shortlist. All 48 answers at 0.8 or more were right.

Goal

Someone describes a film they half-remember ("a man and a boy running from these sharp metal eyeball things with blades on the side"); the catalog holds it among 5,000 films, and its plot tells the same story in other words. One Choice could pick it from a list, but upstream notes it "takes at most 255 options, and only about forty when each option is a long plot summary". So the search runs in three steps, "and each step reads more about fewer films".

You end up with find(): it takes a request and returns the films ranked by probability, in three rounds of parallel calls. Upstream lists it under Search and matching, level Intermediate (raw/docs/cookbooks.md).

The 255 cap is the API's ("You can have a maximum of 255 options per Choice", raw/docs/api.md; the same in raw/docs/primitives__choice.md); the cookbook's code adds that "256 options is rejected outright". See Choice questions.

Inputs / state shape

State is the request text alone. Each question carries its own film, so the state stays short and every question is judged against it.

Catalog films.jsonl (downloaded from typesafe-public-examples, cookbooks/infinite_cardinality_choice): title, year and two cuts of the plot section of each film's Wikipedia article, fetched 2026-11-07. plot is "the first hundred or so words" (step 1 reads it); long_plot is "the first six hundred or so" (steps 2 and 3). The catalog holds the film behind every request; the other films, the ones a search must rule out, are those whose Wikipedia articles exist in the most languages.

Requests requests.jsonl: 100 from Microsoft's tip-of-the-tongue dataset, collected from the forum iRememberThisMovie.com, each with the film that answered it; a random sample of the 709 whose film has a plot section on Wikipedia. The worked example is request 388 (a mother and two kids haunted by ghosts that "turn out to be time travelers"); the film is The Diabolical (2015).

Questions asked

Steps 1 and 2: one Noul per film, no criteria, the film named in the instructions with its plot attached ("so Jev compares the request with the plot instead of recalling the film from its title"):

def proposition(film_id: str, plot: str = "plot") -> str:
    return (
        f"The film this person is trying to remember is {label(film_id)}. "
        f"Its plot: {FILMS[film_id][plot]}"
    )

Step 1 uses plot; step 2 the same function with "long_plot".

Step 3: one Choice, options keyed by film id, each option's description the title plus the long plot:

DECISION = "Which film is this person trying to remember?"


def decision_question(options: tuple[str, ...]) -> dict:
    return {
        "film": Choice(
            instructions=DECISION,
            criteria={key: f"{label(key)}: {FILMS[key]['long_plot']}" for key in options},
        )
    }

Why a Choice at the end: each Noul judges its film alone, so two films that both fit can both get 0.4 or more (here 0.60 and 0.40), and the top five "alone add up to more than 1". A Choice weighs the twenty against each other, so "its top probability says how far ahead of the rest the top film is".

Combining logic in code

The upstream steps put together into one script (setup condensed: the plotting imports and chart code dropped; the step functions are verbatim; decide() packs the cookbook's threshold printout into a function). It replays the published cache unless you set DOWNLOAD_CACHE = False and delete the cache file.

# pip install openai ipython matplotlib 'cooksafe>=0.3.0,<0.4.0'
import json
import os
import urllib.request
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path
from time import perf_counter

from cooksafe import JsonCache, make_playground_link
from typesafe_sdk import Choice, Noul, RetryPolicy, TypeSafeClient

TYPESAFE_MODEL = "jev-1.14.0"  # the page's model; "jev-latest" runs the newest, live
CHOICE_LIMIT = 255  # enforced by the API; 256 options is rejected outright
CHUNK = 300  # films per filter call: ~150 tokens each keeps a call well under 64k
LONG_CHUNK = 40  # films per re-read call: ~700 tokens each, ~28k a call
FIRST_CUT = 80  # films read again with their long plot
CANDIDATES = 20  # films the Choice question sees
THRESHOLD = 0.8  # show one film at or above this Choice confidence

client = TypeSafeClient(
    api_key=os.environ.get("TYPESAFE_API_KEY", "cache-only"),  # keyless kernels replay the cache
    base_url=os.environ.get("TYPESAFE_ENDPOINT"),
    timeout=300.0,  # 300 questions is a big request
    # a request this size occasionally draws an overload response, so retries are not optional
    retry=RetryPolicy(max_retries=5, backoff_initial=1.0, backoff_max=30.0),
)

EXAMPLES = (
    "https://raw.githubusercontent.com/typesafe-ai/typesafe-public-examples/main/"
    "cookbooks/infinite_cardinality_choice"
)


def download(name: str) -> Path:
    """Fetch one of this cookbook's files from typesafe-public-examples unless it is on disk."""
    path = Path(name)
    if not path.exists():
        path.parent.mkdir(parents=True, exist_ok=True)
        urllib.request.urlretrieve(f"{EXAMPLES}/{name}", path)
    return path


CACHE = Path("infinite_cardinality_choice_cache.json")
# True fills CACHE with the published answers, so the run makes no requests. To run
# live, set it to False and delete CACHE; a CACHE already on disk is never overwritten.
DOWNLOAD_CACHE = True
if DOWNLOAD_CACHE:
    download(CACHE.name)
json_cache = JsonCache(CACHE)

FILMS = {f["id"]: f for f in map(json.loads, download("films.jsonl").read_text().splitlines())}
REQUESTS = [json.loads(line) for line in download("requests.jsonl").read_text().splitlines()]


def label(film_id: str) -> str:
    film = FILMS[film_id]
    return f"{film['title']} ({film['year']})" if film["year"] else film["title"]


def proposition(film_id: str, plot: str = "plot") -> str:
    return (
        f"The film this person is trying to remember is {label(film_id)}. "
        f"Its plot: {FILMS[film_id][plot]}"
    )


def as_json(questions: dict) -> str:
    """The questions as API JSON, so the cache key changes when their wording does."""
    return json.dumps(
        {key: q.model_dump(mode="json", exclude_none=True) for key, q in questions.items()}
    )


IDS = sorted(FILMS)
CHUNKS = [
    as_json({i: Noul(instructions=proposition(i)) for i in IDS[start : start + CHUNK]})
    for start in range(0, len(IDS), CHUNK)
]


# Step 1: one Noul per film on the short plot, every chunk sent at once
@json_cache
def filter_chunk(text: str, questions_json: str, model: str) -> dict:
    started = perf_counter()
    response = client.system_one(
        state=text, questions=json.loads(questions_json), model=model
    )
    return {
        "p": {key: answer.noul for key, answer in response.answers.items()},
        "seconds": round(perf_counter() - started, 2),
    }


def filter_films(text: str) -> dict:
    """Every chunk at once; one probability per film."""
    with ThreadPoolExecutor(max_workers=len(CHUNKS)) as pool:
        parts = list(
            pool.map(lambda chunk: filter_chunk(text, chunk, model=TYPESAFE_MODEL), CHUNKS)
        )
    return {key: p for part in parts for key, p in part["p"].items()}


def by_probability(request: dict) -> list[str]:
    p = filter_films(request["text"])
    return sorted(p, key=lambda key: (-p[key], key))


# Step 2: the top 80 again, with the long plot
def reread(request: dict) -> dict:
    """The filter's top films, asked again with the long plot; one probability per film."""
    first = by_probability(request)[:FIRST_CUT]
    chunks = [
        as_json({i: Noul(instructions=proposition(i, "long_plot"))
                 for i in first[start : start + LONG_CHUNK]})
        for start in range(0, FIRST_CUT, LONG_CHUNK)
    ]
    with ThreadPoolExecutor(max_workers=len(chunks)) as pool:
        parts = list(
            pool.map(
                lambda chunk: filter_chunk(request["text"], chunk, model=TYPESAFE_MODEL), chunks
            )
        )
    return {key: p for part in parts for key, p in part["p"].items()}


def candidates(request: dict) -> tuple[str, ...]:
    p = reread(request)
    return tuple(sorted(p, key=lambda key: (-p[key], key))[:CANDIDATES])


# Step 3: one Choice over the top 20
DECISION = "Which film is this person trying to remember?"


def decision_question(options: tuple[str, ...]) -> dict:
    return {
        "film": Choice(
            instructions=DECISION,
            criteria={key: f"{label(key)}: {FILMS[key]['long_plot']}" for key in options},
        )
    }


@json_cache
def rank(text: str, questions_json: str, model: str) -> dict:
    response = client.system_one(
        state=text, questions=json.loads(questions_json), model=model
    )
    answer = response.answers["film"]
    return {"film": answer.choice, "confidence": answer.confidence,
            "probabilities": dict(answer.probabilities)}


def find(request: dict, options: tuple[str, ...] | None = None) -> dict:
    """Rank a shortlist with one Choice; by default, the top twenty on the long plots."""
    options = candidates(request) if options is None else options
    ranked = rank(request["text"], as_json(decision_question(options)), model=TYPESAFE_MODEL)
    p = ranked["probabilities"]
    return {"film": ranked["film"], "order": sorted(p, key=lambda key: (-p[key], key)),
            "confidence": ranked["confidence"], "probabilities": p}


def decide(result: dict) -> list[str]:
    """One film when the Choice is sure enough, otherwise the top three."""
    return result["order"][:1] if result["confidence"] >= THRESHOLD else result["order"][:3]


example = next(r for r in REQUESTS if r["id"] == "388")
result = find(example)
print([label(key) for key in decide(result)])
print(make_playground_link(example["text"], decision_question(candidates(example))))

Per search that is 17 filter calls of up to 300 Nouls, 2 re-read calls of 40 and 1 Choice call: 20 requests and 5,081 questions (counted from the constants, inferred). Upstream's sizing: about 150 tokens per film in step 1, about 700 in step 2 ("~28k a call"), to stay under the 64k per-request context (Models, aliases, pricing, rate limits, context: "64k per-request / 32k per-question"). The chunks don't depend on each other, so they all go out at once, and "the wait stays about the same as the catalog grows, as long as your rate limit allows that many calls at a time". The measurement runs WORKERS = 4 searches side by side.

Results the cookbook reports

TypeSafe's run: "Numbers below came from jev-1.14.0, run in November 2026" (no exact day; the plots were fetched 2026-11-07).

The worked example (request 388). Step 1: "5,000 answers from 17 calls sent together; the slowest took 0.4s". The filter ranks Ghostbusters: Afterlife first at 0.64 (mother, two kids, haunted house, money trouble); The Diabolical is at position 6, because its time-traveller twist "comes after the first hundred words, so the filter never sees it". Step 2 on long plots: The Diabolical 0.60, Ghostbusters: Afterlife 0.40, The Others 0.30, Thirteen Ghosts 0.26, Knowing 0.17. Step 3: the Choice puts 1.00 on The Diabolical.

Across 100 requests, the filter's top 80 hold the right film for 93; after the re-read the top 20 hold it for 88, against 82 for the filter's own top 20. Three shortlists go to the same Choice:

embeddings no re-read all 3 steps
reaches the Choice 35% 82% 88%
film in the top 1 33% 73% 78%
film in the top 3 34% 76% 82%

embeddings = OpenAI text-embedding-3-small over each film's label: plot (short plot) and the request, twenty nearest; upstream calls this "the standard first try". no re-read skips step 2. "The Choice question can only rank the films it is given."

Show one film or a list. How often the top answer was right, by its probability:

top answer long-plot Nouls Choice question
under 0.5 30/49 right 9/30 right
0.5 to 0.8 33/40 right 21/22 right
0.8 to 0.95 11/11 right 18/18 right
0.95 and up 0/0 right 30/30 right

"Choice at 0.8 or more: 48 requests, right for 48"; "below 0.8: 52 requests, the film in the top 3 for 34". The re-read's probability "can't sort requests into sure and unsure": none reaches 0.95, only 11 reach 0.8, and its answers under 0.5 are still right 30 times in 49. Upstream's rule: at 0.8 or more show the one film; below, show the top three.

The playground link holds step 3 only (the request and the Choice over the re-read's twenty), because "the playground can't comfortably show thousands of questions".

Adapting it

Gotchas

Sources