The 8 GB Lab Book · Chapter 25
Packing STT + SLM + TTS + eyes
- Board
- Jetson Orin Nano Super
- Mode
- 15W
- Summary
- The floor and three residents, desktop up, took 5,178 of 7,620 MiB and left 2,442. RSS saw 21% of the SLM's cost and 90% of the eyes'. A tool call with the brain resident took 1,532 ms in the resident run; the first run read 4,689 with thinking left on. Two stacked reruns on 2026-10-04 read the pile at 5,174 and 5,017 MiB and the tool call at 1,530 and 1,534 ms.
What I expected
My prior for the pie was three guesses on top of a 1.5 GB floor. The guesses were approximately 3.0 GB for the SLMSmall language model: a language model small enough, usually after quantization, to run on edge hardware instead of a data-center GPU., approximately 0.5 GB for STT and approximately 0.4 GB for the eyes. The result is 5.4 GB used and 2.2 GB left. I trusted the SLM guess most, because I had arithmetic for that guess only.
I also expected that the three would share the bus without a problem. One decodeThe phase after prefill where a language model generates one token at a time and re-reads its weights for each one; it is usually limited by memory bandwidth. uses approximately 28 GB/s of a 68 GB/s limit (Chapter 2). In theory, a 30 Hz TensorRT engineThe compiled, GPU-specific plan TensorRT produces from a network; it has to be rebuilt for a different GPU or TensorRT version. fits beside the decode, and a tool call slows down very little.
The theory
The rent of one resident, and why RSS is not it
House SLM, my prior (arithmetic)
weights: 4.0 B x 4.5 bit / 8 = 2.25 GB KV at 4k: 4096 x 144 KB = 576 MiB (Day 3) CUDA context: call it 200 MB (a guess) sum = 3.0 GB = 2,861 MiB = measured +2,639 MiB: 222 MiB under
RSS against the delta (measured)
SLM 557 of 2,639: RSS sees 21% STT 167 of 456: RSS sees 37% eyes 520 of 576: RSS sees 90% same board, three different ratios = the delta is the rent. RSS is the CPU half
There is one DRAM and there are two allocators. The memory that a process requests from the GPU driver does not show in the RSSResident set size: the pages of a process's own memory that are in RAM right now, as ps and top count them; GPU allocations made through the driver are not included. of that process. The two server-style residents hide most of their cost there. The eyes process hides almost none of its cost.
The pie, prior against the board (prior is arithmetic in MB, the rest tegrastats MiB)
- prior, used
- 5,398 MB
- measured, used
- 5,178 MiB
- left
- 2,442 MiB
The setup
bash ~/day25_pack.sh
cat ~/day25/pile.txt"lines": [ ... ]
"slm": { "pile": 4137, "rss_mb": 557 }
"stt": { "pile": 4593, "rss_mb": 167 }
"eyes": { "pile": 5169, "rss_mb": 520 }
"idle": 1498
"all_resident_pile": 5178
"stt_ms": 1642
"tts_ms": 2708
"t_tool_ms": 4689The measurement
| What | Expected | Measured | Note |
|---|---|---|---|
| idle floor | - | 1,498 MiB | desktop up |
| + house SLM (4B Q4, 4k) | ~3.0 GB | +2,639 MiB | RSS 557 |
| + STT whisper.cpp base.en server | ~0.5 GB | +456 MiB | RSS 167; 5 s clip 1642 ms |
| + eyes (Day 11 engine, loop at 24.5 Hz in the stacked run, 2026-09-27) | ~0.4 GB | +576 MiB | RSS 520 |
| TTS piper per utterance | transient | 2708 ms / sentence | not resident |
| all resident / left | - | 5,178 / 2,442 MiB | before camera pool + gait |
| t_tool with three resident | - | 1,532 ms (first run 4,689) | stacked run 2026-09-27, thinking off; a 31-token 'reply with only this JSON' prompt, 14 tokens out; not Day 22's schema-in-system call |
| Rerun 2026-10-04, three resident, n=2 | - | pile 5,174 / 5,017 MiB, left 2,281 / 2,451 | idle 1,621 / 1,414 before the loads; the pile moves with the idle it starts from |
| Rerun 2026-10-04, t_tool with three resident | - | plain JSON 1,530 / 1,534 ms (2 runs of 3 calls); Day 22 schema call 2,458 ms, TTFT 972 (1 run of 5) | alone the schema call is 2,061 to 2,066 ms, TTFT 820: about 400 ms more beside STT and the eyes; 5 of 5 valid |
| Rerun 2026-10-04, eyes loop, n=2 | - | 19.6 / 24.3 Hz during the loads, 22.8 / 24.9 after the tool calls, 24.6 / 24.6 at the end | the loop sleeps 30 ms per frame |
Five bars, the four residents and what is left
One voice turn, end of speech to end of reply
- Speech ends5 s clip
- STT1,642 ms
- SLM tool call1,532 ms
- TTS2,708 ms
| resident | delta MiB | RSS MiB | latency |
|---|---|---|---|
| house SLM 4B Q4, 4k | 2,639 | 557 | t_tool 1,532 ms, three resident (first run 4,689) |
| STT base.en server | 456 | 167 | 5 s clip in 1,642 ms |
| eyes FP16 | 576 | 520 | 24.5 Hz in the stacked run (2026-09-27); the loop sleeps 30 ms per frame |
| TTS piper | transient | - | 2,708 ms per sentence |
| camera pool + gait | not measured yet | - | start each, read the delta |
| rerun 2026-10-04, n=2 | pile 5,174 / 5,017 | - | t_tool 1,530 / 1,534 ms plain JSON; 2,458 ms with the Day 22 schema; eyes 24.6 Hz at the end |
Where it broke
The eyes guess was the worst guess, and I thought that it was the safe guess. An FP1616-bit floating point (half precision): half the bytes of FP32, and tensor cores run it at much higher throughput than FP32. engine file is small. But the process that holds the file also holds a CUDA context, a stream and a private copy of each buffer that it binds. The file size was never the cost of the resident.
The second break was t_tool, and it was my harness. The first run read 4,689 ms with the brain resident, thinking on, cut at max_tokens 40. The stacked run of 2026-09-27 read 1,532 ms for the same request with thinking off: 31 prompt tokens, 14 tokens out, prompt eval 509 ms. That request is not Chapter 22's call. Chapter 22 carried the tool schema in the system prompt and returned a tool_calls field, so the two numbers do not compare, and neither says contention made anything faster. The like-for-like pair is in the stacked logs: the same request beside a waiting VLAVision-language-action model: a vision-language model extended to output robot actions, often as a short chunk of future commands. took 1,285 ms (pair log), so ears and eyes added about 247 ms. GPU slicing between the eyes' frames is my hypothesis for that; not tested. Two stacked reruns on 2026-10-04 read 1,530 and 1,534 ms for the same plain-JSON request. The Chapter 22 schema call beside the same three residents took 2,458 ms, TTFTTime to first token: the delay between sending a request and receiving the first generated token, mostly spent in prefill. 972, against 2,061 to 2,066 alone: the schema is the larger cost and the residents add about 400 ms.
I made the SLM prior from three terms, and it was 222 MiB vs MBA mebibyte (MiB) is 2^20 bytes and a megabyte (MB) is 10^6 bytes; some tools label MiB as "MB", so check which one a tool actually counts. too high once both sides are in MiB. A second load at 512 context, beside this one at 4k, gives the KV term its own number.
I read RSS first, and I almost wrote 557 MB as the cost of the SLM. On a board with shared memory, RSS is the CPU half of the cost. The tegrastatsNVIDIA's Jetson monitoring tool, which prints RAM, swap, CPU and GPU load, clock speeds, temperatures and power readings at a fixed interval. delta, in MiB, is the full cost.
Summary
- The pie has four slices. One slice must go, or become smaller, before I add the camera pool and the control HzHow often the low-level control loop reads sensors and updates motor commands; it normally runs much faster than the policy above it.. The slice to remove first is the desktop. It is on in every number above, and it does nothing for the robot.
- STT and TTS do not need to be resident at the same time. Voice is turn-based. The robot listens, then it speaks. It is possible to load one of them on demand.
- When I add slices, t_tool under contention is the number to monitor. The user hears this number as the robot "thinking".
Sources
These sources are not mine. Measured numbers came off my own board; specs, estimates and paper figures are labeled as such.
- Gemma 4 Jetson Nano Super voice+vision demo (E2B + Parakeet + Kokoro pattern)
- Memory-maximizing Jetson writeups (quant + OS + runtime)
- Your Day 3/5 budgets
- day27/rerun-2026-10-04/summary.json: day27_stacked.sh residents_first twice on 2026-10-04, plain-JSON and Day 22 schema tool calls beside the three residents (pile txt, ram csv, slm and eyes logs, tegrastats)