---
title: "Builds: desktop, mobile and voice computer use"
type: community
source_tier: community
tags: [ideas, builds, computer-use, voice, desktop]
created: 2026-09-25
updated: 2026-09-25
confidence: medium
sources:
  - raw/community/callstack-com-blog-exploring-jev-for-mobile-qa-with-agent-device.md
  - raw/x/milindlabs-2100631847155994852.md
  - raw/x/VersunPan-2102201927178371280.md
  - raw/community/yappy-biz-jev.md
  - raw/community/yappy-biz-research-yappy-vs-heyclicky.md
  - raw/x/Saccc_c-2100864907046768890.md
  - raw/x-repos/jcpsimmons__jev-macos-loop.md
  - raw/x-repos/jcpsimmons__jev-macos-loop__docs-performance.md
  - raw/x-repos/timpratim__macbrow.md
  - raw/x-repos/Sac-Y__Jev-cu.md
  - raw/x-repos/savka777__jev-use.md
  - raw/x-repos/kevinbadi__jev-voice.md
  - raw/x-repos/okinaaudio__live-jev.md
jev_version: "jev-1.13.0"
summary: "Community builds where Jev picks the next action on a Mac, Windows or phone screen, or turns a spoken command into an app action: what it decides, measured runs, documented failures, flags."
---

# Builds: desktop, mobile and voice computer use

> **TL;DR** 10 builds plus names-only lines (split from [[ideas/builds-browser-and-interface]] on 2026-09-25). Local code reads the screen (accessibility tree, OCR, on-device detection), Jev picks the next control or command from that list, and code acts and checks the result. The best evidence is small: Jev macOS Loop's 6/6 native tasks, with its failures written down. Policy lives in code: macbrow's safety file exists because an early build emptied a Desktop into one folder. Several builds here read other people's messages; see the flags.

**How to read.** *Builder reports* = the builder's own numbers, one run on their own machine, unverified. Pattern IDs: P12-P14 in [[ideas/patterns-interface]]; P02-P06 in [[ideas/patterns-agents]]; P38 in [[ideas/patterns-emerging]]. Browser agents and UI elements: [[ideas/builds-browser-and-interface]]. Point these agents only at apps and accounts you may automate (AUP note on [[ideas/patterns-interface]]).

## Desktop and mobile control

| Build | Jev decides | Builder reports | P |
|---|---|---|---|
| [Jev macOS Loop](https://github.com/jcpsimmons/jev-macos-loop) (Josh C. Simmons) | Local perception (ScreenCaptureKit, OmniParser CoreML, Apple Vision OCR, accessibility labels) turns one window into element ids; Jev picks the next click from that finite list, text only (pixels and coordinates stay on the Mac); code checks focus, occlusion and a fresh frame before input; a separate verifier, not Jev's DONE, decides success. Vercel Gateway (default), OpenRouter or direct | [Measured](https://github.com/jcpsimmons/jev-macos-loop/blob/1aadc01ef262c3b01460909c5dd102f0d9616ba5/docs/performance.md) 2026-09-17/18: 6/6 native GUI tasks on Vercel and 6/6 on OpenRouter; median decision 301.9 / 293.5 ms (p95 547.2 ms over 22 Vercel decisions); Finder: 9 files sorted in 7.39 s with one classification request and three group drags, vs 22.69 s one by one. **Documented failures:** a later Calculator regression typed an extra digit (3,968 instead of 448), caught by the independent verifier; OCR disagreed with accessibility labels or lagged a transition; an early suite scored 5/6 (action label vs checkbox state). Author: small samples, not reliability evidence; the direct TypeSafe route has no live test. No typing, one window. AGPL-3.0 | P14, P04 |
| [macbrow](https://github.com/timpratim/macbrow) (Pratim) | Voice (Gradium speech in and out, via LiveKit) → one `Choice` over the tools for the apps open now plus "chat" and "new action"; each candidate's enum arguments asked speculatively in the same request, free text as a span of the utterance; web tasks go to jev-ultrafast in your Chrome; follow-up, completeness and "did the script do it" are single `Noul`/`Choice` checks | ~300 ms routing; Amazon add-to-cart 4 steps in 11 s; a follow-up 6 steps in 3.7 s; a trailer playing 1.6 s after speech ends (author, own machine). **Failure that shaped it:** an early build asked to clean up the Desktop moved every Desktop file into a folder; `policy.py` now blocks file moves and deletes, settings, terminals, purchases and sign-ins, and state changes need a spoken yes (a misheard valid command is not covered). **Flag:** for unknown actions an LLM writes new AppleScript tools (compile, effect, policy and Jev-review gates), cached and run locally. Author: experimental, not for production. MIT | P14, P03, P02 |
| [Jev-cu](https://github.com/Sac-Y/Jev-cu) (Sac-Y; the code behind [@Saccc_c](https://x.com/Saccc_c/status/2100864907046768890)'s "Jev Use" post) | Codex Computer Use reads the accessibility tree; Jev picks element, action, done-ness and risk; dry-run by default; delete, send, pay, install and similar stop at `confirm`; per-app allowlist in `scripts/policy.mjs` | Post (video): smoother than Codex's own computer use at similar token use, `unverified`: an offline eval (`npm run p0`) ships without committed results. Goals written in English because Jev is most accurate there (matches [[reference/models-and-pricing]]: verified). MIT | P14, P03 |
| [jev-use](https://github.com/savka777/jev-use) (savka777; not shitianfang/jev-use on [[ideas/head-to-head]]) | macOS voice or typed commands over the accessibility tree: one request per step picks operation and target; typed text is a span of your sentence; low-confidence and destructive picks stop and ask; optional "Hey Jev" wake phrase | 0.3-1.5 s per step (author). The README lists exactly what is sent: no screenshots, secure fields excluded. MIT | P14, P38 |
| [agent-device QA](https://www.callstack.com/blog/exploring-jev-for-mobile-qa-with-agent-device) (Mike Grabowski, Callstack) | Mobile accessibility snapshot → `choice` over press/fill/scroll/wait plus `pass`/`fail`/`incomplete`; text to type is baked into the option | 14 s run, $0.0023. `systemOne` + `choice()` matches [[reference/javascript-sdk]] (verified) | P14 |
| [Screenshot-free computer use](https://x.com/milindlabs/status/2100631847155994852) (@milindlabs) | On-device segmentation + OCR turn the screen into a text list; Jev picks what to click, in a loop | ~90 ms per decision | P14 |
| [WeChat group bot](https://x.com/VersunPan/status/2102201927178371280) (reviewed by @VersunPan) | Jev + DeepSeek over the Android accessibility tree, screenshot + local OCR fallback, last 10 messages | No numbers; warns WeChat can detect it and ban the account | P14 |
| [Yappy](https://yappy.biz/jev/) (Mitosis Labs; closed, paid) | Mac computer-use agent: the front window as an accessibility table of controls (no screenshots) → one operation question and a target question per operation; a chat model types; hands off to the full agent below 0.25 confidence, after three no-op actions, or when a field value is unknown | **Vendor's own single run** (2026-09-18, one job-application form, n=1): 1m54s vs 13m14s and ~$0.24 vs ~$4.24 for a named competitor, whose time includes ~2m20s waiting on the user ([research page](https://yappy.biz/research/yappy-vs-heyclicky/)); five Jev decisions, 275-690 ms each, $0.003 in total | P14, P06 |

## Voice commands

| Build | Jev decides | Builder reports | P |
|---|---|---|---|
| [Jev Voice](https://github.com/kevinbadi/jev-voice) (kevinbadi) | Local whisper.cpp transcribes everything; only utterances naming the assistant reach Jev (a wake gate); one fan-out request picks the action and its arguments from code-made candidates; multi-step tasks run the jev-ultrafast loop on the accessibility tree or in Chrome | Jev fan-out 170-420 ms, whisper 80-130 ms (author's table); an AutoTrader search ~13 s, ~30 Jev calls, about one cent. Setup remaps Caps Lock with a LaunchAgent. **Flag:** the repo also drives rated chess.com games with Stockfish proposing and Jev picking; its README reports wins in rated games and itself notes engine play in rated games breaks chess.com's fair-play rules. Not a use to describe or recommend. MIT | P14, P38 |
| [Live Jev](https://github.com/okinaaudio/live-jev) (Okina Audio) | One English or Japanese sentence → an allow-listed Ableton Live action, candidates taken from your own set and plug-ins; fixed phrases handled locally without Jev; an optional Gemini rewrite is checked against the set and always confirmed | Chains of up to four commands are all checked before any runs, and earlier parts are rolled back if a later one fails; undo only with a receipt; a named track that does not exist writes nothing. Source only. MIT | P14 |

(jev-canvas, voice plus a pointing fingertip on a canvas, is on [[ideas/builds-browser-and-interface]] with the interface elements.)

Also seen, names only: [jev-cua](https://github.com/ronadin2002/jev-cua) (ronadin2002; floating macOS bar, accessibility controls → Jev choice → act → re-observe; demo, no licence), [jev-desktop](https://github.com/yikangy873-gif/jev-desktop) (yikangy873-gif; bounded Jev loop inside Codex Computer Use, every executable control allowlisted by the calling task; MIT). Messaging readers, flagged: [JevBystander](https://github.com/Nisaka520/JevBystander) (Android accessibility service reading private WeChat chats; ships a debug-signed APK with a public keystore: do not install), its sibling [JevIntent](https://github.com/Nisaka520/JevIntent) (Nisaka520; long-press a WeChat message for intent, emotion and urgency; **hooks WeChat through LSPosed and sends the other person's messages out**) and [Yanwai](https://github.com/YIRC99/yanwai) (YIRC99; emotion and subtext under WeChat bubbles; **sideloaded APK hooking WeChat, no licence, third-party messages sent out**).

## What these builds teach when advising

- **Verify outside the model.** Jev macOS Loop's verifier, not Jev's DONE, caught 3,968 ≠ 448; Jev Voice and jev-use also call DONE a claim, not proof.
- **Write the policy before the first run.** macbrow's Desktop incident; Jev-cu defaults to dry-run; Live Jev runs a chain all-or-nothing with rollback.
- **Gate what reaches Jev.** Wake words (Jev Voice, jev-use) and local fixed phrases (Live Jev) keep stray speech from becoming actions.
- **Vendor numbers are n=1.** Yappy's comparison is one run on one form, by the seller.
- **Other people's messages are not yours to send.** The WeChat readers send third-party chats to an API without that person's consent, and client hooking can get an account banned.

## Related

- [[ideas/patterns-interface]] — P14 with its fit verdict and the AUP note
- [[ideas/builds-browser-and-interface]] — browser agents and in-UI classifiers
- [[ideas/builds-agents]], [[ideas/builds-games-and-devices]] — coding agents; games, robots and devices
- [[ideas/failure-reports]] — where Jev broke

## Sources

Links are inline in each row; the raw captures are listed in the frontmatter. Names-only lines rest on our vetting at source where no capture exists: 2026-09-24 (docs/sweep/2026-09-24-awesome-jev/part1.md, part2.md) and 2026-09-25 (docs/sweep/2026-09-25-mrjev/partb.md).