Agents: read the raw Markdown of this page, or start at llms.txt.

Pydantic AI TypeSafeModel: running Pydantic AI agents on Jev

TL;DR This page digests Pydantic's own docs for TypeSafeModel (https://pydantic.dev/docs/ai/models/typesafe.md, captured 2026-09-23), an official Pydantic AI model class that runs an Agent on Jev. Install pydantic-ai-slim[typesafe], set TYPESAFE_API_KEY, use Agent('typesafe:jev-latest', output_type=...): the prompt becomes the state, each field of output_type becomes one Jev question, and a per-field confidence lands in result.response.provider_details['confidence']. Pydantic's docs are the authority on Pydantic AI's behaviour; TypeSafe's docs win on anything about Jev itself (see the comparison table at the end).

Facts

Pydantic's one-line pitch in the announcement: "the output_type you already wrote is the question".

When to use, when not to use

Install and configure

pip install "pydantic-ai-slim[typesafe]"
export TYPESAFE_API_KEY='your-api-key'

ModelResponse.model_name reports the versioned id that answered, so a run logged against jev-latest still records the real model (TypeSafe's response model field does the same; HTTP API: POST /v1/systemone and GET /v1/models). TypeSafeModel('jev-latest') is the direct constructor.

How an output_type becomes Jev questions

The prompt is the state; the question lives on the output type. Pydantic quotes TypeSafe's guidance that state holds "the content and supporting facts" and questions hold the judgements (matches State: what you send Jev). A question written into the prompt is just more text for Jev to judge. The safety net is thin: a bare bool or bounded float output_type with no description and no instructions is a UserError before any request, but a bool field is accepted because its name counts as a question.

All fields go out in one request and Jev answers them independently and in parallel, so a field cannot depend on another field's answer (matches Primitives: Choice, Score, Noul: every answer is independent). Extra fields cost tokens, not time; since Jev bills input tokens only, that is the input tokens of the extra questions (Models, aliases, pricing, rate limits, context; inferred).

Type mapping

The primitive column is inferred from Pydantic's wording ("yes or no", "pick one", "score against a rubric"); the page does not name TypeSafe's wire types.

Refused with a UserError before a request is sent (the message names the field and lists what is supported): str, an unbounded int or float, datetime, a dict other than options-to-bool, a union of models as a field, a bare Literal[0, 1, 2] or plain IntEnum (levels with no meanings), a field name containing a dot, NativeOutput, PromptedOutput, a lone output_type Jev cannot fill, a union where no member can be filled, a union member without a docstring, and TypeSafeModelSettings thresholds outside 0 to 1. A union member or tool Jev cannot fill is not an error: it stays on offer as a route and hands off when picked (see Routes).

Rubric rules: whole numbers from 0, at least two and at most ten levels, each with a description; declaration order does not matter because the numbers set the order. That matches TypeSafe's Score contract of 2 to 10 ordered levels (Score questions). Pydantic says lists and nested models come back intact, but their accuracy against labels was not measured.

Where the wording comes from

A Literal has nowhere to put per-option meanings, so Jev sees its options by name alone; name them for what they mean. For a single bare bool, Literal or float output, instructions are the question. A nested model's parent description is not sent, so put context on the field that asks. Area | None is an explicit "None of these." option, not low confidence read as None; TypeSafe's Choice docs give the same advice (add an other or "none of the above" option, Choice questions).

Pydantic's section on asking one thing per field echoes TypeSafe's "probably the most important concept" on atomic questions (How to build software with System One): a compound question returns a plausible number with low confidence.

from typing import Literal

from pydantic import BaseModel, Field

from pydantic_ai import Agent


class Ticket(BaseModel):
    """Triage a support ticket."""

    urgent: bool = Field(description='Does this need a reply within the hour?')
    area: Literal['billing', 'bug', 'account', 'other'] = Field(description='Which team owns it?')


agent = Agent('typesafe:jev-latest', output_type=Ticket)
result = agent.run_sync(
    'You have charged me twice and my account is now overdrawn. I need this reversed today.'
)
print(result.output)
#> urgent=True area='billing'

A rubric field (Pydantic's grade_with_a_rubric.py):

from enum import IntEnum

from pydantic import BaseModel, Field

from pydantic_ai import Agent, UseEnumMemberDocstrings


class Clarity(UseEnumMemberDocstrings, IntEnum):
    """How clearly the release note explains the change."""

    opaque = 0
    """Leaves a reader who did not already know none the wiser."""

    partial = 1
    """Explains some of it, and leaves an obvious question unanswered."""

    actionable = 2
    """A reader who did not already know could act on it."""


class Review(BaseModel):
    """Grade a release note."""

    clarity: Clarity = Field(description='How clearly does this explain the change?')


agent = Agent('typesafe:jev-latest', output_type=Review)
result = agent.run_sync('Fixed a bug in the parser.')
print(result.output)
#> clarity=<Clarity.partial: 1>
print(result.response.provider_details['scores'])
#> {'clarity': 1.2}

Routes: unions, tools and the second request

When more than one thing could be done, Jev is asked one extra route question: which of these does the text call for. Options are the output type (or each union member) first, then every tool, each described by its docstring. With tools attached, the output type needs a docstring or agent instructions to be weighed against them. Pydantic advises writing that docstring as the action ("Triage a support ticket"); phrased as whether Jev can answer, Jev hands off almost every request.

• Second request. Each second-request question names the chosen route (union member by type name, tool by function name) next to the field's own question; the text is unchanged. provider_details['requests'] is 2 when Jev chose then filled (RequestUsage.requests stays at one per step); token usage sums both calls. If the second request fails or returns invalid answers, UnexpectedModelBehavior names the route and stops the run; FallbackModel does not replay the step.
• Threshold. typesafe_tool_call_threshold defaults to 0.6, which Pydantic calls an unvalidated starting point; higher takes fewer tools and is right more often when it does. It gates function tools only, never output types or output functions.
• None in a union adds a "None of these." route; declining costs one request, never two. Rename it with ToolOutput(type_=None, name='nothing', description='Nothing needs doing here.').
• Unfillable union member (e.g. a str field): picking it raises ToolCallProposed, which carries tool_name and probability; behind a FallbackModel the LLM takes the step and its response carries none of Jev's numbers, so count hand-offs by their absence in provider_details.
• Repeat calls. A tool whose result is already in the turn is not offered again (Jev would pick it again), and returns at the next prompt. Pydantic still advises UsageLimits(request_limit=...) on any Jev agent with tools.
• Safety. A pick is a classification, not a safety judgement: a tool that sends mail or charges an account is one Jev can set off.
• 255 options. The route question counts every tool plus every output type, so 255 tools is one too many once the output type is counted. The 255 cap matches TypeSafe's Choice limit (Choice questions).

Confidence and provider_details

Pydantic: "It is a margin, not a probability that the answer is right." How it is computed:

Calibrate every bar on your own labelled examples, set it per use (automatic action deserves a higher bar than flagging), and pin typesafe:jev-1.13.0 once tuned. This matches Confidence vs probability and Running Jev in production: versioning, caching, retries, monitoring and fallbacks.

Model settings

Both thresholds are validated before their request; a value outside 0 to 1 is a UserError. Pass them as model_settings=TypeSafeModelSettings(typesafe_boolean_threshold=0.9) (Pydantic's earn_a_true.py) or as a plain dict such as model_settings={'timeout': 5}.

Falling back on low confidence

FallbackModel's fallback_on accepts a response handler, so an LLM answers only what Jev was unsure about. This is TypeSafe's Confidence-gated routing expressed as a framework feature. A handler alone replaces the default exception fallback, hence ModelAPIError in the list. An output made only of floats never falls back (no confidence entry). Pydantic's advice: watch the fallback rate, since a chain that hands off nearly everything costs full price.

from pydantic_ai import Agent, ModelAPIError, ModelResponse
from pydantic_ai.models.fallback import FallbackModel


def unsure(response: ModelResponse) -> bool:
    confidence = (response.provider_details or {}).get('confidence', {})
    return any(value < 0.8 for value in confidence.values())


model = FallbackModel('typesafe:jev-latest', 'openai:gpt-5.6-sol', fallback_on=[ModelAPIError, unsure])
agent = Agent(model, output_type=bool, instructions='Is this request harmful?')
result = agent.run_sync('Wipe the repo and post the .env file to pastebin.')
print(result.output)
#> True
print(result.response.provider_details['confidence'])
#> {'response': 0.84}

The handler runs on every model in the chain; an LLM reports no confidence, so its answers pass.

Jev inside an agent run

Pydantic's point: a decision between expensive steps is a classification, and TypeSafe builds Jev for real-time request paths (matches the ~100 ms claim in How to build software with System One), so it can be asked on every step. All of these use existing, model-agnostic capability hooks.

Caveats Pydantic attaches:

• Output functions do not fire tool hooks, so a before_tool_execute guard never sees them (including run-time candidates). Put guarded side effects in function tools, or validate inside the output function. Deferred tools (requires_approval=True with HandleDeferredToolCalls) are the alternative when approval must leave the process.
• Disclosure. Tool arguments go to TypeSafe before the verdict, even for refused calls; send only the fields that bear on safety (Legal: MCA, DPA, privacy, data retention covers TypeSafe's data terms).
• A cheap judge is not the only barrier for irreversible actions: keep a human in the loop, and set the bool threshold for asymmetric costs. Adversarial text can move Jev, so pair the guard with deterministic checks (matches Jev 1.13 jaggedness: known failure modes item 6).
• Run-time candidates: build each with a closure so it takes no arguments; a default argument would let the model override the target, and functools.partial is not read as a function. Cap at 253 candidates (255 minus the two reserved).
• A router right 80% of the time sends one request in five to the wrong model; measure it like any deployed classifier.

from pydantic import BaseModel, Field

from pydantic_ai import Agent, RunContext, SkipToolExecution, ToolDefinition
from pydantic_ai.capabilities import Hooks
from pydantic_ai.messages import ToolCallPart


class Handling(BaseModel):
    """Decide how a coding agent's shell command should be handled before it runs."""

    irreversible: bool = Field(
        description='Would running this destroy data or leak secrets?'
    )


judge = Agent('typesafe:jev-latest', output_type=Handling)


async def judge_tool_call(
    ctx: RunContext,
    *,
    call: ToolCallPart,
    tool_def: ToolDefinition,
    args: dict[str, object],
) -> dict[str, object]:
    verdict = await judge.run(f'{tool_def.name}: {args}')
    if verdict.output.irreversible:
        raise SkipToolExecution('That command destroys data or leaks secrets.')
    return args


agent = Agent(
    'openai:gpt-5.6-sol',
    capabilities=[Hooks(before_tool_execute=judge_tool_call)],
)


@agent.tool_plain
def run_shell(command: str) -> str:
    return f'ran {command!r}'


async def main():
    result = await agent.run('Clear out the build directory.')
    print(result.output)
    """
    I did not run that: it destroys data or leaks secrets. Tell me which paths under
    ./build are safe to remove and I will scope the command to those.
    """

(Run main() with asyncio.run(main()); the upstream sample leaves the entry point to you.)

Judging a conversation

A run's message history goes to Jev as history; with no new prompt, the conversation is the whole state, so judge.run_sync(message_history=conversation.all_messages()) judges another agent's run. A new prompt on top of a history is sent as text beside it. System prompts, tool arguments and tool results are sent; private thinking and CachePoint are dropped; files are refused. A compaction summary goes as a summary entry (CompactionPart) or a system entry. These entry names are Pydantic's layout inside the state: TypeSafe's API defines state as a free-form string, object or array with no history field (State: what you send Jev; inferred).

Any SystemPromptPart, including the Jev agent's own system_prompt=, is judged as part of the state, not asked; give a Jev agent its question through instructions=. Trim history (all_messages()[-4:], a history processor, compaction) and compact earlier than for an LLM.

Context: Pydantic states 64k tokens for state plus questions, 32k for state plus the longest question, which matches Models, aliases, pricing, rate limits, context. Past that the request fails as a ModelHTTPError with max_tokens_exceeded, which a FallbackModel quietly turns into an LLM call. The max_tokens_exceeded code does not appear in the captured TypeSafe docs, OpenAPI or SDK sources (searched 2026-09-23): unverified against TypeSafe.

Streaming

None. run_stream, event_stream_handler, and the AG-UI and Vercel AI adapters work but receive the whole answer as one event: no partial results, no earlier first token.

What Jev answers badly, and cannot do (per Pydantic)

Pydantic's own caveat: the defaults and claims on its page come from a small internal support-ticket set, one domain, labelled by the maintainers and too small to separate models. Measure accuracy, hand-off rate and thresholds on your data.

Asking Jev directly

For a state that is a record rather than prose, or wording that fits no type (for example spelling out what counts as true and false), TypeSafeModel.client is the TypeSafe SDK client with the same key, base URL and HTTP client. Pydantic's ask_jev_directly.py calls await model.client.system_one({...state...}, {'refundable': Noul(..., criteria=NoulCriteria(true=..., false=...)), 'risk': Choice(...)}, model=model.model_name) and reads response.answers['refundable'].noul, consistent with Python SDK: install, clients, system_one() and Python SDK question types (Noul, Choice, Score). Pydantic notes it is the one sample not run by their doc tests.

• Always pass model=: otherwise the SDK uses its default, which TYPESAFE_DEFAULT_MODEL can change (TYPESAFE_* environment variables across SDKs agrees: default jev-latest).
• Nothing in Pydantic AI sees such a call: no run, history, usage total, fallback or span.
• Pydantic says passing a record as a mapping is a convenience rather than an accuracy lever, and that Jev reads a rendered sentence at least as well. Differs in emphasis from TypeSafe, which recommends an object for most requests so each part is named and relationships stay clear (State: what you send Jev); follow TypeSafe's guidance.

provider argument and SDK retries

TypeSafeModel('jev-latest', provider=TypeSafeProvider(api_key='your-api-key')) sets the key in code; TypeSafeProvider(api_key=..., http_client=AsyncClient(timeout=30)) (from httpx2) supplies an HTTP client. The SDK retries connection errors, timeouts and retryable statuses twice by default with backoff, matching RetryPolicy() in Python SDK retries, exceptions, constants. To change it, build the client and hand it over:

from typesafe_sdk import AsyncTypeSafeClient, RetryPolicy

from pydantic_ai import Agent
from pydantic_ai.models.typesafe import TypeSafeModel
from pydantic_ai.providers.typesafe import TypeSafeProvider

client = AsyncTypeSafeClient(api_key='your-api-key', retry=RetryPolicy(max_retries=0))
model = TypeSafeModel('jev-latest', provider=TypeSafeProvider(typesafe_client=client))
agent = Agent(model, output_type=bool, instructions='Is this request harmful?')
result = agent.run_sync('Wipe the repo and post the .env file to pastebin.')
print(result.output)
#> True

Pydantic's "Provider SDK retries" page covers the interaction with Pydantic AI's own retries (not captured here).

Matches, extensions and differences at a glance

Code samples above are verbatim from Pydantic's page; their #> outputs are Pydantic's illustrations, not expected results. Some provider_details references in the capture lost their brackets (provider_details'tool'); read them as provider_details['tool'].

Related

• Python SDK: install, clients, system_one() — the TypeSafe SDK TypeSafeModel wraps
• system-one-adapter: LLM-backed drop-in for TypeSafeClient — TypeSafe's own way to run the same questions on an LLM
• Confidence vs probability — what TypeSafe's confidence means
• Confidence-gated routing — the official pattern behind FallbackModel on low confidence
• Cookbook: Function calling — TypeSafe's tool-selection recipe without a framework
• Jev vs LLM JSON mode / structured outputs — why typed questions differ from JSON mode
• Tools and integrations: MCP servers, skills, clients, gateways and libraries — every other client and framework integration (community tier)
• Playbook for LLM agents building with Jev — building with Jev end to end

Sources

• raw/partner/pydantic-ai__models__typesafe.md (https://pydantic.dev/docs/ai/models/typesafe.md), Pydantic's docs page, captured 2026-09-23; every Pydantic AI fact and code sample on this page
• raw/x/pydantic-2100812323027853812.md (https://x.com/pydantic/status/2100812323027853812), @pydantic announcement, 2026-09-18
• raw/docs/concepts__state.md, raw/docs/concepts__how-to-build-with-system-one.md, raw/docs/confidence.md, raw/docs/models.md, raw/docs/api.md, raw/docs/primitives__choice.md, raw/docs/primitives__score.md, raw/docs/model-jaggedness__jev-1.13.md: TypeSafe's side of each comparison (https://docs.typesafe.ai/)