Benchmarks

Figures below link to their available evidence and protocols. The Opus trial’s raw runs are not archived; its known setup and limits are stated alongside the result. Losses sit next to wins.

Reading code

Historical read-volume comparison on Pixel’s own repository (138K lines of Rust), first published at commit 632b3685 (exact binary version not recorded there). Counts are estimated tokens: UTF-8 bytes divided by four, not tokenizer or provider usage. No second model reads the files instead.

These are file/answer volumes, not an invoice or an end-to-end task comparison. The historical maintainer log reported 41 to 83% across 798 operations; its snippet-versus-candidate-pool ratio is a different baseline, not a measured reduction in session tokens or cost. The scenarios replay the shape of shunt’s benchmark, which reaches a similar ratio by rerouting reads through a paid second model. Method and figures, as first published

Well-known files

The same measurement on large files of popular projects, each pinned to a commit: the whole file against pixel list-signatures on it, with pixel 0.8.0 in October 2026. The home page’s token wall shows the Transformers row: read whole, that one file takes 57,058 tokens, 28% of a 200,000-token context window, before the agent has written anything. The signatures output is 79.7 to 97.1% less across the 8 kept files, median 94.5% in estimated tokens (UTF-8 bytes ÷ 4, rounded down); the files with the most signatures per line (VS Code’s text model, CPython’s typing.py) save the least.

React’s work loop is left out of the range: it is written in Flow, and the JavaScript grammar lists 20 of its 125 top-level functions, so its 99.4% measures a parse failure, not a saving. The Signatures column is the check for that on every row: signature counts provide a coarse coverage check, not proof of complete parsing; the Python counts are compared with module- and class-level definitions (Transformers: 90 for 89 def and class lines). Re-run it with scripts/bench/bench-read-savings.sh. Method and raw output

Cold start on a large repository

The first Pixel command on a repository builds everything from nothing: the text index from the committed files, then the code graph. On a private Rails application of 18,782 tracked files, from a fresh clone with an empty cache, Apple M2, September 2026, median of three runs:

Later commands reuse the index and read only what changed. One repository on one machine: re-run the command on yours. Method and raw runs

Against GitNexus

The jobs both tools do: 29 blast-radius cases on four repositories in Rust, TypeScript and Ruby, with callers found by grep as the ground truth. GitNexus 1.6.12 and Pixel 0.4.0, same machine, September 2026.

Recall is a tie at this sample size: Pixel finds every caller in Rust and TypeScript, GitNexus does better on Ruby. The index comparison covers one repository only. Full method and raw rows

On whole agent tasks

An Opus 5.5 medium-effort trial compared 11 runs without Pixel with 11 runs using Pixel install hooks on the same low-research scoping task:

No speed gain on this task; the hooks arm was slower and had more read volume and higher provider-reported cost. Pixel calls ranged from zero to four per hooked run; that range is not a median. This single low-research task does not establish a general slowdown, speedup, token or cost result. The raw runs are held outside the repository and are not archived, so their date, exact model and Pixel versions, sample size and cost, the per-run distributions, the exact CLI/harness versions beyond the v0.5.0 worktree tag, and the token-counting method cannot be independently checked. The reported API cost is not invoice-verified, and the token count is not necessarily billed-token usage. Known setup and provenance limits and the available protocol note

Read volume, estimated tokens, elapsed duration and provider-reported cost are different quantities. Adoption rates of the prompt-submit brief and the pixel skill are not quantified here; inspect pixel action-log (a command plugin) on your own sessions.

Where a specialist wins

• Natural-language search, the top 10: on 45 plain-English queries, semble has the right file in its top 10 for 100%, Pixel 0.6.0 for 96%. Pixel puts it first more often: 87%, against 64% for semble and 69% for WarpGrep, Morph’s search subagent, whose top 10 reaches 71%. Average answer time 0.7 s for Pixel, 1.6 s for semble, 6.8 s for WarpGrep. The queries are each repository’s own doc comments, which favours Pixel’s chunks, cut along symbols with their comments. WarpGrep’s figures are from its one run on the same queries, with Pixel 0.5.2; it is paid per search and sends the lines it reads to Morph’s API. Method and every row
• Compact repository map: stacklit covers more directories for fewer tokens on 3 of 4 repositories.
• Context cost: Pixel’s prompt, as 0.4.0 installed it, is 4.7× lighter than GitNexus, but heavier than semble (~980 tokens) and stacklit (~420). Later releases ship that guidance as a skill instead of an always-on prompt; its cost per turn was not re-measured.

They combine: semble for search and Pixel for the graph, history and Git costs about 5,100 always-on tokens, counting the prompt Pixel 0.4.0 installed. Full comparison

Coding decisions

pixel classify answers a bounded question with a decision model you configure. It returns one probability per label. The local Ollaya engine uses decision-model outputs; the remote engine asks a chat model to generate probabilities.

Published decision-model measurements

Source: Ollaya’s published benchmark, checked 2026-09-28. Ollaya describes the accuracy evaluation as the typed-decisions test split: 400 states and 2,000 questions, scored by argmax against the majority label. It reports its own model results and attributes Jev’s result to Winnow’s benchmark on the same questions. These are upstream reports, not a Pixel reproduction; no shared run artifact is archived here.

The latency figures use different environments and requests: local RTX 4090 inference versus a hosted API including network time. They do not establish a speed winner. The accuracy sample is separate from the five-question latency request. The laya latency is also an upstream measurement, not a CPU measurement by Pixel.