---
title: "Cookbook: Twitch chat filter"
type: cookbook
tags: [cookbook, moderation, choice, noul, concurrency]
created: 2026-10-13
updated: 2026-10-15
confidence: high
sources:
  - raw/docs/cookbooks__twitch_chat_filter.md
  - raw/docs/cookbooks.md
jev_version: "jev-1.14.0"
sdk_python: "0.7.5"   # the cookbook pins typesafe-sdk>=0.7.1,<0.8.0 and cooksafe>=0.2.0,<0.3.0
summary: "Filter live chat: one request per message asks its kind (Choice), toxic and spoiler (Nouls); plain code shows, highlights, dims, hides or blurs. 29 sample messages, 8 requests at once."
---

# Cookbook: Twitch chat filter

> **TL;DR** Send each chat message in its own request with the stream title and game, and ask three questions: a `Choice` over `question` / `hype` / `conversation` / `backseat` / `spam`, a `Noul` for toxic and a `Noul` for spoiler. A plain `decide()` function turns the answers into show, highlight, dim, hide or blur: toxic at ≥ 0.7 hides, a `hide` category hides, spoiler at ≥ 0.7 blurs. On 29 sample messages from an Elden Ring stream, 7 were hidden, 2 blurred and 6 highlighted. Changing a rule re-runs only `decide()`, with no new request. Median about 0.4 s per request, so a busy chat needs concurrency and a policy for lines shown before their answer arrives.

## Goal

Sort fast-moving chat as it arrives: highlight questions, hide spam, insults and gameplay tips the streamer did not ask for ("backseat"), blur spoilers. The cookbook's point: a keyword list catches spam links, but "Go left, there's a treasure box behind the tree" has no bad words in it, and neither does a plot twist. Whether a line is a spoiler or backseat advice depends on the game, so the stream goes into every request.

Upstream lists it under Classification, level Beginner ([[cookbooks/overview]]). It is also an official worked example of content moderation with rules kept in code; a Jev-powered Chrome extension filtering live Twitch chat is shown in the cookbook's demo video.

When to use it: short messages, labels you can describe in a sentence each, and actions you can change without re-asking. When not to: a channel that is mostly non-English (see Gotchas), or a spoiler policy that depends on how far the streamer has played, unless you put that progress in `state`.

## Inputs / state shape

29 messages the authors wrote for an Elden Ring stream, so every kind of message appears at least twice; two are emote-only lines (`PogChamp`, `LUL`). The state for one request:

```python
{"stream": {"title": "Elden Ring first playthrough! No spoilers please", "game": "Elden Ring"},
 "message": {"user": "backseat_bert", "text": "go left, there's a site of grace behind the tree"}}
```

Numbers below came from `jev-1.13.0` on 2026-09-25.

## Questions asked

All three in one request, against the same state:

| key | type | `instructions` | `criteria` |
|---|---|---|---|
| `category` | `Choice` | `Which category best describes the main purpose of \`message.text\`?` | `question`: "A genuine request for information from the streamer or chat, not gameplay instructions phrased as a question or a promotional pitch"; `hype`: "Cheering, reactions, or repeated emotes (PogChamp, LUL, GG, W, lets gooo); repetition alone does not make cheering spam"; `conversation`: "Ordinary chatter, opinions, replies, useful shared links, or any message whose main purpose does not fit the other categories"; `backseat`: "Unrequested gameplay advice telling the streamer what to do, including instructions phrased as a question; not a request for information"; `spam`: "Unsolicited advertising, self-promotion, follower selling, or promotional links; not ordinary cheering, repeated emotes, or a useful link" |
| `toxic` | `Noul` | `Is this message insulting, hateful or harassing towards someone?` | `true`: "Insults, slurs, threats or personal attacks on the streamer or another viewer"; `false`: "Friendly, neutral, or light banter with no target" |
| `spoiler` | `Noul` | `Does this message reveal a plot point or ending of the game being played?` | none (instructions only) |

`conversation` is the fallback when no other category fits; the descriptions carry the boundary cases (a link alone is not spam, repetition alone is not spam, instructions phrased as a question are `backseat`).

## Combining logic in code

```python
import os
from concurrent.futures import ThreadPoolExecutor

from typesafe_sdk import Choice, Noul, NoulCriteria, TypeSafeClient

TYPESAFE_MODEL = "jev-1.13.0"
client = TypeSafeClient(api_key=os.environ["TYPESAFE_API_KEY"], timeout=60.0)

STREAM = {"title": "Elden Ring first playthrough! No spoilers please", "game": "Elden Ring"}
QUESTIONS = {
    "category": Choice(
        instructions="Which category best describes the main purpose of `message.text`?",
        criteria={
            "question": "A genuine request for information from the streamer or chat, "
            "not gameplay instructions phrased as a question or a promotional pitch",
            "hype": "Cheering, reactions, or repeated emotes (PogChamp, LUL, GG, W, "
            "lets gooo); repetition alone does not make cheering spam",
            "conversation": "Ordinary chatter, opinions, replies, useful shared links, "
            "or any message whose main purpose does not fit the other categories",
            "backseat": "Unrequested gameplay advice telling the streamer what to do, "
            "including instructions phrased as a question; not a request for information",
            "spam": "Unsolicited advertising, self-promotion, follower selling, or "
            "promotional links; not ordinary cheering, repeated emotes, or a useful link",
        },
    ),
    "toxic": Noul(
        instructions="Is this message insulting, hateful or harassing towards someone?",
        criteria=NoulCriteria(
            true="Insults, slurs, threats or personal attacks on the streamer or another viewer",
            false="Friendly, neutral, or light banter with no target",
        ),
    ),
    "spoiler": Noul(
        instructions="Does this message reveal a plot point or ending of the game being played?"
    ),
}


def classify(user: str, text: str, stream: dict | None = None) -> dict:
    state = {"stream": STREAM if stream is None else stream, "message": {"user": user, "text": text}}
    response = client.system_one(state=state, questions=QUESTIONS, model=TYPESAFE_MODEL)
    category = response.answers["category"]
    return {
        "category": category.choice,
        "probabilities": dict(category.probabilities),
        "toxic": response.answers["toxic"].noul,
        "spoiler": response.answers["spoiler"].noul,
    }


RULES = {"question": "highlight", "hype": "show", "conversation": "show",
         "backseat": "hide", "spam": "hide"}
TOXIC_AT = 0.7    # hide at or above this chance of toxic
SPOILER_AT = 0.7  # blur at or above this chance of a spoiler


def decide(result: dict, rules: dict = RULES) -> str:
    if result["toxic"] >= TOXIC_AT:
        return "hide (toxic)"
    action = rules[result["category"]]
    if action == "hide":
        return f"hide ({result['category']})"
    if result["spoiler"] >= SPOILER_AT:
        return "blur (spoiler)"
    return action


CHAT = [("mira_plays", "how many hours have you played so far?"),
        ("backseat_bert", "go left, there's a site of grace behind the tree")]
with ThreadPoolExecutor(max_workers=8) as pool:  # one request per message, 8 at once
    results = list(pool.map(lambda message: classify(*message), CHAT))
for (user, text), result in zip(CHAT, results):
    print(decide(result), user, text)
```

Adapted from the cookbook: the upstream version wraps the call in `cooksafe`'s `JsonCache` (keyed on state plus questions, not the model, so move the cache aside when you change `TYPESAFE_MODEL`), passes `TYPESAFE_ENDPOINT` as `base_url`, and also records `response.model`, elapsed seconds and `response.usage.input_tokens`. Precedence in `decide()`: toxic first, then a `hide` category, then spoiler, then the category's own rule.

## Results / what the cookbook reports

Three messages up close:

| message | top `category` | `toxic` | `spoiler` |
|---|---|---|---|
| "go left, there's a site of grace behind the tree" | `backseat` 1.00 | 0.02 | 0.09 |
| "wait till you find out Ranni planned the Night of Black Knives" | `conversation` 0.57 (`backseat` 0.42) | 0.03 | 0.94 |
| "you're so bad at this boss lmao, love you tho" | `conversation` 0.81 (`hype` 0.18) | 0.21 | 0.02 |

The spoiler `Noul` catches the Night of Black Knives line although its category is a near tie; the last line reads like an insult but stays low on `toxic`, fitting the "light banter" description.

Whole chat (29 messages, one request each, eight at a time): show 14, highlight 6, hide (backseat) 3, hide (spam) 2, blur (spoiler) 2, hide (toxic) 2. On these samples every `toxic` probability was ≥ 0.97 or ≤ 0.24 and every `spoiler` ≥ 0.94 or ≤ 0.20, so any threshold between gives the same result; the cookbook says your own channel's chat will land in between, so check both thresholds on it.

Changing a rule reuses the stored answers: `RULES | {"hype": "hide", "conversation": "dim"}` changed 14 messages with 0 new requests.

## Adapting it to a new domain

- Replace `STREAM` and the messages, or pass a different `stream` to `classify()` per channel.
- Rewrite the `Choice` options for your community's kinds of message; keep a fallback option like `conversation`, and put the boundary cases in each description ([[guides/writing-instructions-and-criteria]]).
- Keep actions in `RULES` and thresholds in code, so a policy change never re-asks Jev ([[cookbooks/llm-guardrails]] uses the same split).
- In a live integration, keep the platform's message ID (or the page element) next to each request so answers update the right line; `pool.map` only preserves order within a batch.

## Gotchas

- **Each message waits for its answer.** Requests took a median of about 0.4 s, so a new line appears before its answer returns: hide each new line until it is classified, or accept a toxic line on screen for that long.
- **Concurrency.** One request at a time cannot keep up with a busy chat; the cookbook runs eight at once. On a rate limit, lower `max_workers` and re-run (current limits: [[reference/models-and-pricing]]).
- **Non-English chat.** At publication Jev was trained mainly on English and answered other languages less accurately; expect more misses on a non-English channel.
- **A winning category can still be uncertain.** The example always applies the winner's rule. Decide what to do with close category probabilities and `Noul` values near 0.5 (flag for review, or leave visible rather than hide), and test that policy on your own chat ([[concepts/confidence]]).
- **The spoiler check does not know the streamer's progress.** It uses the game title and the model's knowledge; to filter only unseen plot points, put the current progress and relevant plot facts in `state` and adapt the question.
- **Install line.** `pip install 'typesafe-sdk>=0.7.1,<0.8.0' ipython 'cooksafe>=0.2.0,<0.3.0'`; set `TYPESAFE_API_KEY`. Every response is cached in [`json_cache.json`](https://github.com/typesafe-ai/typesafe-public-examples/blob/main/cookbooks/twitch_chat_filter/json_cache.json) in `typesafe-ai/typesafe-public-examples`; download it beside the script to replay the published numbers. The cookbook also builds a playground link with `make_playground_link` for the backseat message and the three questions.

## Related

- [[cookbooks/llm-guardrails]] — the same "Jev assesses, code decides" split for LLM inputs and outputs
- [[cookbooks/consistency-choice]] — an `uncertain` outcome for moderation decisions
- [[patterns/fan-out]] — several questions in one request
- [[concepts/choice]], [[concepts/noul]] — the two primitives used
- [[ideas/patterns-data]] — community moderation patterns (P16)
- [[cookbooks/overview]] — the full cookbook catalog

## Sources

- raw/docs/cookbooks__twitch_chat_filter.md (https://docs.typesafe.ai/cookbooks/twitch_chat_filter.md), first captured 2026-10-13
- raw/docs/cookbooks.md (https://docs.typesafe.ai/cookbooks.md): section and level