$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people
~/wiki/patterns

Speculative fan-out

[ pattern ][ updated 2026-09-21 ][ confidence high ][ jev-1.13.0 ]#patterns · fan-out · speculative · latency · cost

TL;DR Send all the questions your system could need in a single POST /v1/systemone call — including ones that only matter on branches you have not taken yet — then branch in code. "All questions are evaluated in parallel, so adding more questions usually has little effect on response time."

Problem

A decision tree has dependent branches: you classify a support ticket, and only if it is a bug report do you need its severity and whether it has reproduction steps; only if it is billing do you need to know whether a refund was requested.

The obvious implementation asks the classifying question first, waits, then issues a second call for the branch-specific questions. That serializes two round trips for what is logically one decision, and the second call re-sends the same state.

Pattern

From raw/docs/patterns__fan-out.md:

Because TypeSafe supports sending many questions in a single API call, we recommend putting all of the questions your system needs in a single request, and then using code to decide what is relevant after the fact. All questions are evaluated in parallel, so adding more questions usually has little effect on response time.

Wording changed 2026-09-21. Until this refresh raw/docs/patterns__fan-out.md said adding questions "typically doesn't add any latency to the response", and the speculative-questions note said extra questions cost "no speed cost". Both were softened to "usually has little effect on response time". The pattern is unchanged; the guarantee is not as absolute as it read before, so measure end-to-end latency on your own question set rather than assuming extra questions are free.

A speculative question is one you ask before you know whether its answer will be used. The source's definition:

Speculative questions: bug_severity and has_reproducible_steps only matter if the ticket is a bug report. refund_requested only matters for billing. We include all upfront because additional questions usually have little effect on response time. If the ticket turns out to be a feature request, the bug severity result will be irrelevant, in which case your code path simply ignores it.

Two steps: fan out, then route with code.

Implementation

Step 1: speculative fan-out

The documented example is support ticket triage. State and questions, verbatim from the source (the source renders these as a playground example; state and questions are the two top-level fields of the request body — add "model": "jev-latest" to make it a complete HTTP request, see HTTP API: POST /v1/systemone and GET /v1/models):

{
  "state": "Hi, I placed an order (#98423) last Thursday and was charged twice. I also can't log in after the site update, and adding Apple Pay would be really helpful. This is getting frustrating.",
  "questions": {
    "category": {
      "type": "choice",
      "instructions": "Determine the broad category of this support ticket",
      "criteria": {
        "bug_report": "The user is reporting something that is broken or producing errors",
        "billing": "Charges, invoices, refunds, subscriptions",
        "feature_request": "The user is requesting new functionality",
        "account": "Login, permissions, profile, security"
      }
    },
    "bug_severity": {
      "type": "score",
      "instructions": "How severe is the reported issue",
      "criteria": [
        "Cosmetic; no impact to functionality",
        "Broken or degraded feature; workaround exists",
        "Blocking issue; no workaround exists"
      ]
    },
    "has_reproducible_steps": {
      "type": "noul",
      "instructions": "The user describes specific steps to reproduce the issue"
    },
    "refund_requested": {
      "type": "noul",
      "instructions": "The user is explicitly asking for a refund or credit"
    },
    "frustration": {
      "type": "score",
      "instructions": "How frustrated the user appears",
      "criteria": ["Calm, matter-of-fact", "Frustrated but civil", "Very angry"]
    }
  }
}

Note the mix: one Choice, two Scores, two Nouls — all in one call. See Choice questions, Score questions, Noul (yes/no) questions.

Step 2: route with code

Your code decides what is relevant based on the classification result:

category = response.answers["category"]
bug_severity = response.answers["bug_severity"]
bug_repro = response.answers["has_reproducible_steps"]
refund = response.answers["refund_requested"]
frustration = response.answers["frustration"]

if category.choice == "bug_report":
    if bug_severity.score > 1.5 and bug_repro.noul > 0.6:
        escalate_to_engineering(ticket_id, severity="high")
    else:
        add_to_bug_backlog(ticket_id)

elif category.choice == "billing":
    if refund.noul > 0.7:
        route_to_billing_with_flag(ticket_id, refund_likely=True)
    else:
        route_to_billing(ticket_id)

elif category.choice == "feature_request":
    log_feature_request(ticket_id)

# Frustration is useful regardless of category
if frustration.score > 1.5:
    flag_for_priority_response(ticket_id)

The source's closing point: "Everything needed for the full decision tree comes from one call. Speculative questions are ignored when irrelevant and save a round trip when they are not."

Note that frustration is not speculative — it is consumed on every branch. Fan-out mixes both kinds freely.

Sending it

The example above is JSON for POST /v1/systemone. To run the same fan-out through an SDK, build the same questions map with Choice / Score / Noul (Python) or choice() / score() / noul() (JS) — see Quickstart: first call in HTTP, Python, JS for a complete runnable call, and Python SDK: install, clients, system_one() / JavaScript/TypeScript SDK: install, client, choice/score/noul for signatures.

When it fails

Variants

Related

Sources