$jevwiki.ai#an LLM wiki about Jev, written for agents rather than people

Agents: read the raw Markdown of this page, or start at llms.txt.

~/wiki/concepts

Images and assets (experimental)

[ concept ][ updated 2026-10-24 ][ confidence high ][ jev-1.14.0 ][ python sdk 0.7.5 ][ js sdk 0.7.0 ]#images · assets · multimodal · experimental · state

TL;DR Since 2026-10-14 a request can carry images, as an experimental feature ("dreamer-ready", not production). Send TypeSafe-Danger-Zone-Images: true, list each image once in a top-level assets map keyed by a reference string, and write that string wherever the image belongs in state, instructions or criteria. PNG or JPEG; multipart upload recommended, base64 allowed. Each place a reference appears adds the image again toward the limits. Works on jev-1.13.0 and jev-1.14.0; the box type needs 1.14, which jev-latest points to since the alias move between the 2026-10-21 and 2026-10-24 captures (until then it pointed to jev-1.13.0, so Box needed a pin or jev-preview).

What it is

"An image can go anywhere text can go in a System One request: in the state, in a question's instructions, and in its criteria. Every question type accepts images" (raw/docs/images.md). The Box questions (experimental) type returns where something is in an image.

TypeSafe's release post: "the Jev you all know and love (jev-1.13.0) was multimodal all along", and "all models (Jev included) perform worse on multimodality over pure text. Breaking down questions into the simplest possible decompositions helps a lot" (raw/site/blog-jev-1-14.txt, .html for the model name).

Status: experimental. Images "aren't yet up to our standards to call ready for production"; "The request format, the answers, and the limits on this page can change between releases, and accuracy will change as we update the models" (raw/docs/images.md). Use them for prototypes.

Turning it on

Images are off by default. Send TypeSafe-Danger-Zone-Images: true with each request that contains an image or a Box question. Without it, such a request returns 400.

Where How
HTTP -H "TypeSafe-Danger-Zone-Images: true"
Python typesafe-sdk 0.7.5 On the client, TypeSafeClient(headers={"TypeSafe-Danger-Zone-Images": "true"}), or per call, extra_headers={...}
JS @typesafe-ai/sdk 0.7.0 On the client, new TypeSafeClient({ defaultHeaders: { "TypeSafe-Danger-Zone-Images": "true" } }), or per call, the headers option

Neither SDK sends the header for you (no Danger-Zone string in either SDK's source, checked 2026-10-15).

The assets map and references

Each image is listed once in the request's top-level assets map. Its key is a reference string. Write the reference wherever the image belongs, as a whole string value or inside a longer string, and "the model receives the image in its place."

Asset fields

Field Type Required Description
type "image" no If sent, must be "image".
part string one of part / data_base64 In a multipart request, the name of the file part holding the image.
data_base64 string unless part is set The file, base64-encoded: standard alphabet with = padding, no line breaks or whitespace, no data: prefix.
format "png" | "jpg" | "jpeg" with data_base64; optional with part Lowercase. "jpeg" is another name for "jpg". The OpenAPI adds: "The actual format is detected from the bytes."

Send each image one way. If an asset has both part and data_base64, the API uses part. The OpenAPI assets map has minProperties: 1 and recommends "a per-image UUID" as the key.

Reference rules

References are matched as plain text anywhere in the request's strings.

Verbatim example (raw/docs/images.md; base64 truncated upstream):

{
  "model": "jev-latest",
  "state": {
    "photo": "a3f1c2e8-found"
  },
  "assets": {
    "a3f1c2e8-found": {"type": "image", "format": "jpg", "data_base64": "/9j/4AAQSkZJRg..."},
    "77b0d4a1-poster": {"type": "image", "format": "jpg", "data_base64": "/9j/4AAQSkZJRg..."}
  },
  "questions": {
    "same_dog": {
      "type": "noul",
      "instructions": "Is the dog in `photo` the same dog as in this picture: 77b0d4a1-poster?"
    },
    "has_collar": {
      "type": "noul",
      "instructions": "Is the dog in `photo` wearing a collar?"
    },
    "coat": {
      "type": "choice",
      "instructions": "What is the coat of the dog in `photo` like?",
      "criteria": {
        "short": "Short and smooth.",
        "long": "Long or feathered.",
        "curly": "Curly or wiry."
      }
    }
  }
}

Where the image goes

Placement Behaviour Use for
state Part of the state; every question is evaluated against it. Mix with text, several images, nested or labelled; point at one by path (`frames[2].frame`). Images several questions use
instructions Part of that one question; sent once per question that includes it. An image only one question needs (e.g. a known badge photo to compare against)
criteria Choice options, Score levels, Noul true/false can each include pictures, alone or as {"description": ..., "examples": [...]} (key names are yours). Defining options by example

When several questions use one image, put it in state once and refer to it by path: fewer tokens and less of the size limits. References work at any level of nested JSON (Structured instructions, options, levels, criteria).

Upstream's pattern table: compare against a known picture (known picture in instructions, photo in state); define options by example (criteria); what changed across frames (state); look closer only when needed (small copy first, then a larger copy or crop); search a large image by tiles as Choice options (criteria); find where it is with Box questions (experimental).

Uploading over HTTP

Multipart is recommended: it sends the file as is, while base64 makes each image about a third larger, and that counts toward the 32 MiB body limit. Verbatim (raw/docs/images.md):

curl https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "TypeSafe-Danger-Zone-Images: true" \
  -F 'request={
    "model": "jev-latest",
    "state": {"door": "a3f1c2e8-door"},
    "assets": {"a3f1c2e8-door": {"type": "image", "part": "door-photo"}},
    "questions": {
      "person_present": {"type": "noul", "instructions": "Is there a person in `door`?"}
    }
  };type=application/json' \
  -F 'door-photo=@door-camera.jpg;type=image/jpeg'

Multipart rules: the first part is named request and holds the JSON body (application/json); part names are unique and non-empty, request is reserved; each image part has content type image/png, image/jpeg, image/jpg, application/octet-stream, or none (anything else: 400); each file part belongs to exactly one asset (an orphan part, two assets naming one part, or a missing part: 400). To reuse an image, write its reference in each place.

Base64: put the encoded file in data_base64 and set format; Python's base64.b64encode gives the right encoding. The docs give a full requests example; the shape is the JSON example above.

SDK helpers

Verbatim from raw/docs/images.md (Python 0.7.5 and JS 0.7.0 SDKs):

from pathlib import Path

from typesafe_sdk import Choice, Image, Noul, TypeSafeClient

with TypeSafeClient() as client:
    result = client.system_one(
        state={"door": Image(Path("door-camera.jpg"))},
        questions={
            "person_present": Noul(instructions="Is there a person in `door`?"),
            "carrying": Choice(
                instructions="What is the person in `door` carrying?",
                criteria={
                    "package": "A box or parcel.",
                    "bag": "A bag of groceries or shopping.",
                    "nothing": "Empty hands.",
                },
            ),
        },
        extra_headers={"TypeSafe-Danger-Zone-Images": "true"},
    )
    print(result.answers)

import { openAsBlob } from "node:fs";
import { choice, image, noul, TypeSafeClient } from "@typesafe-ai/sdk";

const client = new TypeSafeClient();
const door = image(await openAsBlob("door-camera.jpg"));

const { answers } = await client.systemOne(
  {
    state: { door },
    questions: {
      person_present: noul("Is there a person in `door`?"),
      carrying: choice("What is the person in `door` carrying?", {
        package: "A box or parcel.",
        bag: "A bag of groceries or shopping.",
        nothing: "Empty hands.",
      }),
    },
  },
  { headers: { "TypeSafe-Danger-Zone-Images": "true" } },
);
console.log(answers);

The SDKs create references and upload images as multipart/form-data for you.

Python typesafe-sdk 0.7.5 JS @typesafe-ai/sdk 0.7.0
Create an image Image(Path(...)), Image(bytes or any buffer), Image.from_base64(...), Image.from_pil(img, format="jpeg") (needs the pil extra, pillow>=11.0.0) image(bytes or Blob/File), Image.fromBase64(...)
Place directly in content Put the Image in state, instructions or criteria Put the image() value in state, instructions or criteria
Inside text with client.attach_assets(photo): then f"... {photo} ..." (str(image) is its reference; without the with block the reference is sent as plain text) withAssets`... ${upload} ...` and withAssets.join(list, ", "); an image in an untagged template literal rejects the call with TypeSafeError
Your own references assets={"<product>": Image(photo)} assets: { "<delivered>": image(...) }; every name must appear in the request or the call rejects before sending

Details: Python SDK: install, clients, system_one(), JavaScript/TypeScript SDK quick contract: install, client, choice/score/noul, answers, errors (the page a builder reads).

Limits

From raw/docs/images.md; all limits include both ends.

Limit Value
Format PNG or JPEG, still images only. Animated PNGs are rejected.
File size 16 MiB per image file (after decoding, for base64).
Width and height 280 to 1568 pixels each; an image with either side under 280 pixels is rejected (the docs said 256 until 2026-10-15).
Total pixels 78,400 to 2,458,624 (280 × 280 to 1568 × 1568).
Request body 32 MiB including JSON, base64 and multipart framing; headers don't count; compression doesn't raise it.
All images in the request 32 MiB for every distinct image after processing; re-encoding can make an image larger than the file you sent.
Expanded request 32 MiB after every reference is replaced by its image; each use adds the image again (a 10 MiB processed image referenced four times is over).
Reference keys 1,024 bytes each; 65,536 bytes in total.

An image outside the limits is rejected, never resized. If any image is rejected, the whole request fails; no partial answers. Images have no separate rate limits. How many images fit also depends on the context length (Models, aliases, pricing, rate limits, context; the experimental long-context header raises it, see State: what you send Jev).

Tokens and price

"Each image counts as input tokens, billed at the same per-token input price as text. There is no separate charge for images." usage.input_tokens includes every image, and image tokens count toward the context length. As of 2026-10-15 TypeSafe says it will provide "guidance on estimating image token usage soon"; there is no published per-image token formula.

Preparing images

Docs snippet (verbatim):

from PIL import Image, ImageOps

with Image.open("photo.jpg") as original:
    img = ImageOps.exif_transpose(original)  # rotate the pixels upright
    img.thumbnail((1568, 1568))  # longest side at most 1568, same aspect ratio
    # thumbnail only shrinks: upscale or pad an image under the minimum separately
    img.convert("RGB").save("photo-ready.jpg", "JPEG", quality=90)  # baseline JPEG

Errors

Status Cause
400 Image or Box question without the TypeSafe-Danger-Zone-Images: true header.
400 Asset rejected: not PNG/JPEG, damaged, animated, below the minimum size, unreferenced, bad base64 or format, empty reference, or a broken multipart rule.
400 max_tokens_exceeded: images plus text exceed the context length. Send fewer or smaller images, or move them from instructions into state.
413 Over a size limit: image file, dimensions or pixels, request body, all images, expanded request, or reference lengths.

If a request has several problems, the error reports one. General status codes: HTTP status codes, rate limits, retry semantics.

Docs inconsistencies (as of 2026-10-15)

Sources