Vision tokens are context tokens

What I expected

My model was simple: the tokens follow the pixels. Twice the side would give four times the tokens and four times the wait. I expected one straight ratio with no fixed part. I expected that both models would obey this ratio.

I also expected that the peak RAM counter would increase with the KV of the extra tokens. One of the three expectations was correct.

The theory

The tiler is a grid, so N_v is a patch count

QWEN2.5-VL-3B AT 640x360 (arithmetic)

28 px per token: 14 px patch, 2x2 merge
(model card)
640 / 28 = 22.9 -> 23 across
360 / 28 = 12.9 -> 13 down
23 x 13 = 299 tokens
= +2 markers = 301   server says 301

SAME MODEL AT 320x180

320 / 28 = 11.4 -> 11 across
180 / 28 = 6.4 -> 6 down
11 x 6 = 66 tokens
299 / 66 = 4.5, not 4: the grid
snaps to 28 px, the image does not
= +2 markers = 68   server says 68

Both counts match exactly. I read the 2 as the image start marker and the image end marker. On that reading, a 28x28 image costs 3 tokens.

TTFT as a line through the two QWEN points (arithmetic)

slope: 5.5 ms/tok
intercept: 112 ms
prefill rate: 182 tok/s

(1935 - 656) / (332 - 99) = 1279 / 233 = 5.5 ms per prompt token; 656 - 99 x 5.5 = 112 ms that no resolution removes. So the wait is tokens x 5.5 + 112, and 4.4x the tokens turns into 2.9x the wait: the 112 and the 31 text tokens dilute it. The slope is encoder plus prefill together: the server's prompt_ms includes the image encode, and the run did not split them. 182 tok/s is 1 / slope, not the LLM's prefill rate.

Prefill: what the counters allow (arithmetic)

3B weights by name x 2 flop = 6 GFLOP / token
332 tokens = 2.0 TFLOP
2.0 TFLOP / 1.935 s = 1.0 TFLOP/s if prefill took all of it
Day 14 text prefill: 3.3 to 3.75 TF/s on this board
at 3.3 TF/s, 332 tokens = 0.6 s
= 1.3 s or more is encoder + launches (not split)

Decode, same model, one token

AI = 1 tok x 2 flop / 0.62 B per weight = 3 flop per byte
far below any ridge on this board: memory-bound
1 / 15 tok/s = 67 ms per token
Q4_K_M, 4.96 bits per weight (Day 14): 3B x 0.62 B = 1.9 GB
1.9 GB / 67 ms = 28 GB/s
= Day 2's 28 GB/s SLM pull

The image adds tokens to and work to the encoder. It does not change . The 15 at both sizes agrees with a weight stream near Chapter 2's 28 GB/s. Which side of the ridge prefill sits on, this run cannot say: prompt_ms holds the encoder and the LLM prefill together, and t_vit_ms is empty in results.json.

The setup

Run (board, ~10 min)

python3 ~/day19_vtokens.py
cat ~/day19/results.json | head -40

From results.json, the file the run wrote

"script": "day19_vtokens (llama-server)"
"prompt": "Describe what is in front of the robot in one sentence."
"res": { "hi": "640x360", "lo": "320x180" }
"rows": [ ... ]

The measurement

What	Expected	Measured	Note
SmolVLM-500M @ 640x360: N_v / TTFT / decode / peak	-	134 / 403 ms / 75 tok/s / 1914 MB	The image shows two rectangles placed s
SmolVLM-500M @ 320x180: N_v / TTFT / decode / peak	-	134 / 400 ms / 75 tok/s / 1942 MB	The image shows two rectangles placed s
Qwen2.5-VL-3B @ 640x360: N_v / TTFT / decode / peak	-	301 / 1935 ms / 15 tok/s / 5255 MB	The robot is in front of a green square
Qwen2.5-VL-3B @ 320x180: N_v / TTFT / decode / peak	-	68 / 656 ms / 15 tok/s / 5269 MB	In front of the robot, there is a red cu
robot default resolution	-	320x180 (Qwen2.5-VL-3B) (estimate)	re-decided on real frames

Measured on the board unless the row says estimate or predicted.

Fig 19.1Same image, two resolutions, two VLMs: vision tokens and time to first token.

Image to grid to tokens to wait

Image320 x 180
Snap to 28 px308 x 168
Grid 11 x 666 + 2 tokens
Prefill656 ms

At 640x360 the same chain reads 644 x 364, 23 x 13 = 299 + 2 = 301 tokens, 1935 ms. The 11 by 6 grid is the whole idea.

model	res	N_v	TTFT ms	tok/s	peak MB
SmolVLM-500M	640x360	134	403	75	1914
SmolVLM-500M	320x180	134	400	75	1942
Qwen2.5-VL-3B	640x360	301	1935	15	5255
Qwen2.5-VL-3B	320x180	68	656	15	5269

Measured: 15W, llama-server + mmproj, max 32 new tokens (Qwen stopped at 17 and 16), synthetic image with the words 'cup on table' drawn on it. Highlighted row: the default, a placeholder until real frames. The SmolVLM pair is my control: a resize onto a fixed grid, so the same 134 tokens and the same wait at both sizes. The wait follows the tokens, not the pixels.

Where it broke

The ratio came from the grid, not from the pixels. 320 rounds down to 11 columns and 640 rounds up to 23. The ratio is 4.5x by the count and 4.4x by the server. The wait increased only 2.9x, because 112 ms of the wait is fixed.

Peak RAM never showed the extra 233 tokens, and it could not. llama-server ran with -c 4096, which allocates the whole KV at start. At Chapter 3's 144 KiB per token, which is Qwen3-4B's figure and not this model's, 233 tokens would be 34 ; this model's own per-token KV was not read from its log. Both models had a higher peak on the SMALL image, by 14 and 28 MB. That is counter noise.

My prior was a ratio with no intercept. Each prefill pays the cost of the encoder and the launch one time. A line was the correct prior.

I set 320x180 as the default because its caption still named the object. But the synthetic image has the words 'cup on table' drawn on it (day19_vtokens.py, cv2.putText), and the 320x180 caption said 'a red cup on a table' while the 640x360 caption described squares. Most likely the model read the label, not the shapes. The default is a placeholder until real frames.

The 134 of SmolVLM is a floor, and no camera mode decreases it. Vision that costs less than this floor needs a different model. The number that affects the robot is 2.9x, not 4.4x.

Corrected 2026-10-04

I wrote a prefill on the peak, 1.25 TFLOP/s, with a 1.6 s floor and a ridge of 18, and called prefill compute-bound at 182 tok/s. Chapter 14 measured text prefill on tensor cores at 3.3 to 3.75 TF/s, and the model here is Qwen2.5-VL-3B-Instruct-Q4_K_M (day19_vtokens.py), not , so the AI line was wrong too. prompt_ms holds encoder and prefill together and t_vit_ms is empty, so the 5.5 ms/token slope is not the LLM's prefill rate. I wrote 32 new tokens; Qwen stopped at 17 and 16 (predicted_n). I wrote that counter noise hid the KV; -c 4096 allocates the KV at start, so peak RAM could not rise. 144 KiB per token is Qwen3-4B's figure. The 320x180 caption most likely read the label 'cup on table' that the script draws on the image.

Summary

N_v is the prompt token count of the server with the image, minus the count without the image. It is the number of prefill tokens that the image costs. The KB-per-token of the model, read from its log, changes it into KV bytes.
Not every scales. SmolVLM-500M resizes the image onto its fixed grid and costs 134 tokens at both sizes. Qwen2.5-VL-3B tiles the real pixels, and its count follows the image. Find which kind you have before you select a camera mode.
Here, includes the image encoder. A language benchmark never shows this part. It is the time that the robot waits before the first word.
The of Chapter 17 answered at 68 fps. At 640x360, the VLM waits 1.9 s for its first token.

Sources

These sources are not mine. Measured numbers came off my own board; specs, estimates and paper figures are labeled as such.

Qwen-VL / Gemma 4 multimodal cards (token / tile behavior)
TensorRT Edge-LLM VLM benches (ViT time column on Nano)
Your Day 3 KV math