---
title: "Request mechanics: billing, limits, latency, calibration and stability"
type: community
source_tier: community
tags: [measurements, billing, limits, latency, calibration]
created: 2026-09-25
updated: 2026-09-25
confidence: medium
sources:
  - raw/community/archerhume-jevs-architecture-unmasked.md
  - raw/nate/jev-shaped-problems.md
  - raw/x-repos/blowxian__jev-fanout-bench.md
  - raw/x-repos/AboveColin__jevclient.md
  - raw/x-repos/teyhouse__jev-secret-detection.md
  - raw/x-repos/bitnovus__jev-spam-eval.md
  - raw/x-repos/Gaurav-Gosain__jev-sec-bench.md
  - raw/community/ayautomate-com-blog-jev-vs-llm-benchmark.md
jev_version: "jev-1.13.0"
summary: "How a Jev request is billed, capped and timed, measured by the community: exact $0.042/M billing, ~261-token overhead, 32k/64k limits by bisection, latency vs question count, calibration, stability."
---

# Request mechanics: billing, limits, latency, calibration and stability

> **TL;DR** Community probes of the request itself (2026-09-16→23, `jev-1.13.0`): billing is exactly input tokens × $0.042/M with output free, each request carries ~261 tokens of fixed overhead, the 32k/64k limits hold where the docs put them, latency stays flat to ~100 questions, and answers are stable but not bit-exact. Each figure is one person's run; official limits: [[reference/models-and-pricing]]. Split from [[ideas/measurements]] on 2026-09-25.

## Measured

| Number | Reported by | Setup | Caveat |
|---|---|---|---|
| 50 calls at 25 concurrent: 741 ms, no 429s; all 50 at once was slower (~1 s). Claim check on 16 labelled headlines: true 0.83-0.90, false 0.01-0.12 | [Nate B. Jones](https://unlock-ai.natebjones.com/guides/jev-shaped-problems) | run through OpenRouter | One run; 25-way concurrency is a starting point, not a limit |
| ~30k tokens in ~160 ms; server time barely moved up to ~100 questions, then rose steadily; 1,500 questions still returned in a few hundred ms; a 200-option question was as quick as a 2-option one | [Archer Hume](https://archerhume.com/posts/jevs-architecture-unmasked), `jev-1.13.0`, 2026-09-17 | 192 + 148 sequential requests, shuffled, 8 repeats per size | Times are the `x-envoy-upstream-service-time` header: shared service, queueing included; not a hardware benchmark |
| Token accounting is additive: 268 input tokens for one minimal Noul, 276 for two, 318 for a Noul + 2-option Choice + 2-level Score; a 23k-token state with 5,000 questions was accepted | same | 311 accounting + 35 limit-boundary requests | `verified` against the 64k-per-request / 32k-per-branch budgets, `state` counted once ([[reference/models-and-pricing]]) |
| Calibration: 10-bin ECE **0.031** over 1,200 MMLU items (990 in the 0.9–1.0 bin); MMLU-Pro 84.6%; fresh 3-digit multiplication 86.7% correct at mean top probability 0.83; two-step word problems 32% at 0.30; modular exponentiation 56% at 0.35 | same | 6,800 benchmark records + 190 generated maths items | Recomputed from recorded probabilities, not the API `confidence` field. Family averages hide bin-level error; maths is a documented weakness ([[concepts/jaggedness-jev-1-13]] §2) |
| Option-set effects: reversing a ticket's options moved the top probability 0.84–0.89 → 0.93–0.96; one added irrelevant option moved the log-odds between two existing options +0.38 → +0.11 (mean −0.28, paired 95% ≈ −0.36…−0.19, lower in all 10 blocks) | same | 181 order + 96 reference-card + 50 block-randomised requests | `unverified` against docs — one scenario per template; advisory reading in [[ideas/field-reports]] |
| `output_tokens` is billing arithmetic, not decoding: a Noul request costs 4 shared tokens + 15 per answer + the token length of each question id (which the docs say the model never sees); a 255-option response reported 2,714 | same | 311 requests | Output tokens are free ([[reference/models-and-pricing]]); never divide them by duration and call it speed |
| Jev's tokenizer matches none of 192 public tokenizers across 415 probes (closest Qwen at 348/415); every digit is a separate token; 8 `a`s count as 1 token, 16 as 4 | same | 445 requests | `unverified`. Local `tiktoken` or character estimates drift both ways — leave headroom under the caps |
| **Billing is input tokens × $0.042/M, exactly**: across 2,976 responses `usage.cost` matched to $0.000000000; 122,844 output tokens not charged. Fixed overhead **~261 input tokens per request** (one-word state). Batching N = 2 / 4 / 8 questions saved a median 43–49% / 65–74% / 76–86% of input tokens (tiny → ~3,000-token state); 8 questions in one call returned 7.7–8.6× faster than 8 serial calls; batched and single answers differed no more than identical repeats (Noul 0.005, Score 0.010, Choice top never changed) | blowxian, [jev-fanout-bench](https://github.com/blowxian/jev-fanout-bench) | via OpenRouter (provider TypeSafe, `jev-1.13-20260917`), 2026-09-23, $0.156; every request, response and checksum published | Price and free output `verified` ([[reference/models-and-pricing]]). 261 fits the docs' one-sentence Noul example (296 `input_tokens`) and Archer Hume's 268: a calculator counting only state + questions undercounts small requests. Serial timing; synthetic English tickets. Pattern: [[patterns/fan-out]] |
| Per-request formula (R² ≥ 0.999): ~260 tokens, plus per question ~8 framing, ~1 per instruction word, ~8 per Choice option (~21 with a six-word description), 8 per Score level. Characters per token: English 4.92, Spanish 4.44, Russian 1.78, Hindi 1.72, Arabic 1.49, Korean 1.44, Japanese 1.01, Chinese 1.00; JSON state ~2.35. **Limits by bisection**: state + longest question passed at 32,688, refused at ~32,813; the total passed at 65,388, refused at ~65,644. **1,000 questions in one call** succeeded; 16–120 unrelated extra questions moved four target answers no more than repeats (median 426 ms at 4 questions, 436 ms at 124) | same, round 2 | 479 billed requests + 8 over-limit probes; answers withheld, citing the MCA's competing-models term | Limits `verified`: the brackets contain 32,768 and 65,536 ([[concepts/state]]); "overhead counts toward neither" `unverified`. No question cap is documented (Archer Hume sent 5,000). His estimator at jevpricing.com claims 1.4% median error over 365 requests (`unverified`) |
| Translating the ticket moved yes/no answers 3–5× the English repeat noise; translating the questions too moved them further and cost 9–48% more tokens. **Reordering Choice options never changed the top option**; removing descriptions or paraphrasing moved probabilities 2–10 points. Over 960 reworded or translated answers, the lowest-confidence third of Choice answers moved 15× more than the top third (Spearman ρ −0.79) and held all five top-option changes (Score 7.5×, ρ −0.51) | same | same run | English-first `verified` ([[reference/models-and-pricing]]): keep questions in English. Order: Archer Hume (above) saw reversal move the top *probability* 0.84–0.89 → 0.93–0.96 and position change a reference option's hit rate. Stable winner, moving numbers: not a contradiction, but both `unverified` and the docs are silent. Re-check thresholds after any rewording |
| Latency flat to 100 questions: 3 / 25 / 100 / 400 questions → 521 / 1,242 / 4,017 / 15,417 input tokens and 712 / 690 / 714 / 1,336 ms (cold connection, Netherlands, 2026-09-17); warm 250–580 ms over 16 calls | Colin de Vries, [jevclient](https://github.com/AboveColin/jevclient) | README table, one location | (15,417 − 521) ÷ 397 ≈ 37.5 tokens per added question (our arithmetic), fitting the formula above and Archer Hume's flat-to-~100 curve |
| Server time p50 **75–90 ms** (`x-envoy-upstream-service-time`); warm round trip ~290 ms p50 from a machine 185 ms (TCP) away, ~1.2 s on a new connection. 16 requests at a time was fine; **all 100 at once** pushed round-trip p50 to ~2 s (server time still ~90 ms) and drew `529 Overloaded` retries | teyhouse, [jev-secret-detection](https://github.com/teyhouse/jev-secret-detection) (no licence) | 100-case set, `jev-1.13.0` | `verified` in direction: docs say "about 100 ms" ([[concepts/how-to-build]]); `529` is retryable ([[reference/rate-limits-and-errors]]), so cap client concurrency. His "AWS `us-west-2`" is `unverified`. Accuracy: [[ideas/failure-reports]] |
| 19,772 requests in ~252 s at concurrency 16, 32.06M input tokens, ~$1.35 at list price | bitnovus, [jev-spam-eval](https://github.com/bitnovus/jev-spam-eval) | full paired run, no API errors | ~4,700 requests/min, ~127k tok/s (our arithmetic): above the documented 1,200 rpm, under 250k tok/s. jev-sec-bench's 662 in 22.7 s is ~1,750/min. Limits "can change without notice" ([[reference/models-and-pricing]]); don't plan on headroom |
| Jev billed more input tokens than GPT-5.6 Terra for the same prompt: 360 vs 153 (8-way), 952 vs 828 (77-way), so per decision it was 40–49× cheaper than Terra, below the price-list ratio. Median 0.33 s; slowest of 791 calls 1.42 s | Adel Dahani, [AY Automate](https://www.ayautomate.com/blog/jev-vs-llm-benchmark) | via OpenRouter, 3,955 calls, $1.53 total; agency blog | Compare billed cost per decision, not price per token. Accuracy: [[ideas/head-to-head]] |

## How to measure and what not to assume

- **Don't assume bit-exact repeats.** Archer Hume's probes found small differences between identical requests, between duplicate questions in one request, and in key order. Measure on distributions, not equality; don't key a cache or golden test on exact probabilities.
- **Measure latency as Archer Hume did** for comparable numbers: server-reported upstream time (`x-envoy-upstream-service-time`), requests one at a time in shuffled order, several repeats per size. Local wall-clock adds your network and client.
- **Batch before you parallelise.** One request per question pays the ~261-token overhead each time and waits serially; ask everything about one `state` in one call ([[patterns/fan-out]], [[cookbooks/parallel-questions]]), then cap concurrency (16 was fine, 100 at once drew `529`s).

## Related

- [[ideas/measurements]] — workload cost and speed, replicas, access routes; [[ideas/field-reports]] — what these numbers mean when advising
- [[reference/models-and-pricing]], [[reference/rate-limits-and-errors]], [[concepts/confidence]] — the official contract

## Sources

Links are inline in each row. Raw captures (`raw/x-repos/`, `raw/nate/`, `raw/community/`) are in the private repo; Nate B. Jones's material is used with his permission and credited.