Agents: read the raw Markdown of this page, or start at llms.txt.

Cookbook: Twitch chat filter

TL;DR Send each chat message in its own request with the stream title and game, and ask three questions: a Choice over question / hype / conversation / backseat / spam, a Noul for toxic and a Noul for spoiler. A plain decide() function turns the answers into show, highlight, dim, hide or blur: toxic at ≥ 0.7 hides, a hide category hides, spoiler at ≥ 0.7 blurs. On 29 sample messages from an Elden Ring stream, 7 were hidden, 2 blurred and 6 highlighted. Changing a rule re-runs only decide(), with no new request. Median about 0.4 s per request, so a busy chat needs concurrency and a policy for lines shown before their answer arrives.

Goal

Sort fast-moving chat as it arrives: highlight questions, hide spam, insults and gameplay tips the streamer did not ask for ("backseat"), blur spoilers. The cookbook's point: a keyword list catches spam links, but "Go left, there's a treasure box behind the tree" has no bad words in it, and neither does a plot twist. Whether a line is a spoiler or backseat advice depends on the game, so the stream goes into every request.

Upstream lists it under Classification, level Beginner (Cookbooks overview). It is also an official worked example of content moderation with rules kept in code; a Jev-powered Chrome extension filtering live Twitch chat is shown in the cookbook's demo video.

When to use it: short messages, labels you can describe in a sentence each, and actions you can change without re-asking. When not to: a channel that is mostly non-English (see Gotchas), or a spoiler policy that depends on how far the streamer has played, unless you put that progress in state.

Inputs / state shape

29 messages the authors wrote for an Elden Ring stream, so every kind of message appears at least twice; two are emote-only lines (PogChamp, LUL). The state for one request:

{"stream": {"title": "Elden Ring first playthrough! No spoilers please", "game": "Elden Ring"},
 "message": {"user": "backseat_bert", "text": "go left, there's a site of grace behind the tree"}}

Numbers below came from jev-1.13.0 on 2026-09-25.

Questions asked

All three in one request, against the same state:

conversation is the fallback when no other category fits; the descriptions carry the boundary cases (a link alone is not spam, repetition alone is not spam, instructions phrased as a question are backseat).

Combining logic in code

import os
from concurrent.futures import ThreadPoolExecutor

from typesafe_sdk import Choice, Noul, NoulCriteria, TypeSafeClient

TYPESAFE_MODEL = "jev-1.13.0"
client = TypeSafeClient(api_key=os.environ["TYPESAFE_API_KEY"], timeout=60.0)

STREAM = {"title": "Elden Ring first playthrough! No spoilers please", "game": "Elden Ring"}
QUESTIONS = {
    "category": Choice(
        instructions="Which category best describes the main purpose of `message.text`?",
        criteria={
            "question": "A genuine request for information from the streamer or chat, "
            "not gameplay instructions phrased as a question or a promotional pitch",
            "hype": "Cheering, reactions, or repeated emotes (PogChamp, LUL, GG, W, "
            "lets gooo); repetition alone does not make cheering spam",
            "conversation": "Ordinary chatter, opinions, replies, useful shared links, "
            "or any message whose main purpose does not fit the other categories",
            "backseat": "Unrequested gameplay advice telling the streamer what to do, "
            "including instructions phrased as a question; not a request for information",
            "spam": "Unsolicited advertising, self-promotion, follower selling, or "
            "promotional links; not ordinary cheering, repeated emotes, or a useful link",
        },
    ),
    "toxic": Noul(
        instructions="Is this message insulting, hateful or harassing towards someone?",
        criteria=NoulCriteria(
            true="Insults, slurs, threats or personal attacks on the streamer or another viewer",
            false="Friendly, neutral, or light banter with no target",
        ),
    ),
    "spoiler": Noul(
        instructions="Does this message reveal a plot point or ending of the game being played?"
    ),
}


def classify(user: str, text: str, stream: dict | None = None) -> dict:
    state = {"stream": STREAM if stream is None else stream, "message": {"user": user, "text": text}}
    response = client.system_one(state=state, questions=QUESTIONS, model=TYPESAFE_MODEL)
    category = response.answers["category"]
    return {
        "category": category.choice,
        "probabilities": dict(category.probabilities),
        "toxic": response.answers["toxic"].noul,
        "spoiler": response.answers["spoiler"].noul,
    }


RULES = {"question": "highlight", "hype": "show", "conversation": "show",
         "backseat": "hide", "spam": "hide"}
TOXIC_AT = 0.7    # hide at or above this chance of toxic
SPOILER_AT = 0.7  # blur at or above this chance of a spoiler


def decide(result: dict, rules: dict = RULES) -> str:
    if result["toxic"] >= TOXIC_AT:
        return "hide (toxic)"
    action = rules[result["category"]]
    if action == "hide":
        return f"hide ({result['category']})"
    if result["spoiler"] >= SPOILER_AT:
        return "blur (spoiler)"
    return action


CHAT = [("mira_plays", "how many hours have you played so far?"),
        ("backseat_bert", "go left, there's a site of grace behind the tree")]
with ThreadPoolExecutor(max_workers=8) as pool:  # one request per message, 8 at once
    results = list(pool.map(lambda message: classify(*message), CHAT))
for (user, text), result in zip(CHAT, results):
    print(decide(result), user, text)

Adapted from the cookbook: the upstream version wraps the call in cooksafe's JsonCache (keyed on state plus questions, not the model, so move the cache aside when you change TYPESAFE_MODEL), passes TYPESAFE_ENDPOINT as base_url, and also records response.model, elapsed seconds and response.usage.input_tokens. Precedence in decide(): toxic first, then a hide category, then spoiler, then the category's own rule.

Results / what the cookbook reports

Three messages up close:

The spoiler Noul catches the Night of Black Knives line although its category is a near tie; the last line reads like an insult but stays low on toxic, fitting the "light banter" description.

Whole chat (29 messages, one request each, eight at a time): show 14, highlight 6, hide (backseat) 3, hide (spam) 2, blur (spoiler) 2, hide (toxic) 2. On these samples every toxic probability was ≥ 0.97 or ≤ 0.24 and every spoiler ≥ 0.94 or ≤ 0.20, so any threshold between gives the same result; the cookbook says your own channel's chat will land in between, so check both thresholds on it.

Changing a rule reuses the stored answers: RULES | {"hype": "hide", "conversation": "dim"} changed 14 messages with 0 new requests.

Adapting it to a new domain

• Replace STREAM and the messages, or pass a different stream to classify() per channel.
• Rewrite the Choice options for your community's kinds of message; keep a fallback option like conversation, and put the boundary cases in each description (Writing instructions and criteria that Jev reads correctly).
• Keep actions in RULES and thresholds in code, so a policy change never re-asks Jev (Cookbook: Guardrails for LLMs uses the same split).
• In a live integration, keep the platform's message ID (or the page element) next to each request so answers update the right line; pool.map only preserves order within a batch.

Gotchas

• Each message waits for its answer. Requests took a median of about 0.4 s, so a new line appears before its answer returns: hide each new line until it is classified, or accept a toxic line on screen for that long.
• Concurrency. One request at a time cannot keep up with a busy chat; the cookbook runs eight at once. On a rate limit, lower max_workers and re-run (current limits: Models, aliases, pricing, rate limits, context).
• Non-English chat. At publication Jev was trained mainly on English and answered other languages less accurately; expect more misses on a non-English channel.
• A winning category can still be uncertain. The example always applies the winner's rule. Decide what to do with close category probabilities and Noul values near 0.5 (flag for review, or leave visible rather than hide), and test that policy on your own chat (Confidence vs probability).
• The spoiler check does not know the streamer's progress. It uses the game title and the model's knowledge; to filter only unseen plot points, put the current progress and relevant plot facts in state and adapt the question.
• Install line. pip install 'typesafe-sdk>=0.7.1,<0.8.0' ipython 'cooksafe>=0.2.0,<0.3.0'; set TYPESAFE_API_KEY. Every response is cached in json_cache.json in typesafe-ai/typesafe-public-examples; download it beside the script to replay the published numbers. The cookbook also builds a playground link with make_playground_link for the backseat message and the three questions.

Related

• Cookbook: Guardrails for LLMs — the same "Jev assesses, code decides" split for LLM inputs and outputs
• Cookbook: Self-consistency — choices — an uncertain outcome for moderation decisions
• Speculative fan-out — several questions in one request
• Choice questions, Noul (yes/no) questions — the two primitives used
• Patterns: judging, search, documents, real-time and markets — community moderation patterns (P16)
• Cookbooks overview — the full cookbook catalog

Sources

• raw/docs/cookbooks__twitch_chat_filter.md (https://docs.typesafe.ai/cookbooks/twitch_chat_filter.md), first captured 2026-10-13
• raw/docs/cookbooks.md (https://docs.typesafe.ai/cookbooks.md): section and level