A fused Ampere preprocess kernel

What I expected

There are three kernels and three trips to DRAM. I wrote 3x on the whiteboard before I counted one byte. My unit was the trip, and a trip has no size.

I also expected that registers would decide occupancy, because that is the number that everyone quotes. I expected 32 registers for each thread. I set no limit in the code.

The theory

A byte ledger per pass, then a roofline floor for each path

k1 resize: 2.76 in + 0.92 out = 3.69 MB
k2 normalize: 0.92 in + 3.69 out = 4.61 MB
k3 layout: 3.69 in + 3.69 out = 7.37 MB
= = 15.67 MB through DRAM per frame

read the u8 frame once: 2.76 MB
write the f32 tensor once: 3.69 MB
the middle values live in registers
= = 6.45 MB through DRAM per frame

The predicted ratio is 15.67 / 6.45 = 2.43x, not 3x. The passes are not equal, because u8 is a quarter of f32. The first pass is cheap and the layout pass is the expensive pass. As a check, ledger / measured time = 32.7 and 31.4 GB/s. The board reported 32.6 and 31.4.

Why 6 blocks per SM, and why registers were never the limit

The SM holds 6 blocks of 256 = 1536 threads and calls that 1.00. 1536 is the thread cap. For registers, 64K per SM (spec) / (28 x 256 per block) = 9.1 blocks. Threads bind first. Registers could increase to 65536 / 1536 = 42 for each thread before they cost a block. My 32 was a number that people repeat.

For the work, there is one thread for each 4 outputs (one float4 write). 921,600 / 4 = 230,400 threads = 900 blocks. 8 SMs x 6 = 48 resident blocks, so 900 / 48 = 19 waves. Each thread does 4 x 4 = 16 scalar u8 reads for one 16 B write. The read side is the irregular side.

The setup

cd jetson-8gb-kernels/day07 && export PATH=/usr/local/cuda/bin:$PATH && bash run.sh
# builds preprocess.cu (nvcc -O3 -arch=sm_87), prints path A / path B mean p50 p99, GB/s, occupancy per kernel, writes results.json, then runs parity.py PASS/FAIL

cat jetson-8gb-kernels/day07/results.json
# mean_us A vs B, speedup. Registers and occupancy are in the stdout block above RESULT_JSON.

nvcc -O3 -arch=sm_87 -Xptxas -v -o /dev/null preprocess.cu 2>&1 | grep -E 'registers|spill'

"src_w": 1280
"src_h": 720
"dst_w": 640
"dst_h": 480
"warmup_iters": 50
"timed_iters": 1000
"path_a": { "mean_us": 479.893, "p50_us": 479.84, "p99_us": 483.744, "gbps": 32.647 }
"path_b": { "mean_us": 205.514, "p50_us": 205.472, "p99_us": 207.36, "gbps": 31.391 }
"speedup": 2.335

Environment and script

• Script: jetson-8gb-kernels/day07/run.sh (preprocess.cu, parity.py). Output: results.json.
• Board: Orin Nano Super, L4T R36.4.2, 15W (measured.json). Clocks not logged.
• Toolchain: CUDA 12.6, nvcc -O3 -arch=sm_87.
• Timing: cudaEvent, 50 warm-up and 1000 timed iterations per path.

The measurement

Every box is a full pass over DRAM

Where it broke

3x became 2.43x on paper and 2.34x on the board. The first decrease comes from a count of bytes and not of trips. The second decrease, 4%, is not the launches. Fewer launches would make B faster than the ledger, not slower. The board shows the opposite: B moved its bytes at 31.4 GB/s against 32.6 for A. The fused pass is a little less efficient for each byte. Most likely the 16 scalar u8 reads for each thread; not tested. Three reruns on 2026-10-04 held the order: 2.3402, 2.3290 and 2.3306, with B at 31.38 to 31.42 GB/s under A at 32.61 to 32.73 in every run.

The surprise is the thing that did not change. A and B each move bytes at 31 to 33 GB/s, which is less than half of the roof. The fusion removed passes, but it did not make the pass faster.

I counted trips and called the count a prediction. A trip has no size, but a byte has a size. The ledger has four lines. It could give me 2.43x before I compiled anything.

I monitored the incorrect limit. At 256 threads for each block, the thread cap of the SM binds much earlier than the registers.

2.34x is an improvement in bytes, not in the kernel. The two paths run at less than half of the roof: 31 GB/s against 68.

My notes from Day 7

Written by hand on 2026-09-07, before this chapter was typed. These are my original notes from that day. Where the chapter corrects them, the chapter is current.

Summary

• On a board with shared memory, the preprocess uses the same bus that the LLM uses to decode. Path A moves 15.67 MB for each frame. Path B moves 6.45 MB.
• The driver gives the occupancy number at no cost. cudaOccupancyMaxActiveBlocksPerMultiprocessor gave 6 blocks for each SM, occupancy 1.00.
• Bilinear is a contract. It is not a preference. Later, TensorRT input parity depends on a resize that is the same function that OpenCV computes.

Sources

These sources are not mine. Measured numbers came off my own board; specs, estimates and paper figures are labeled as such.

• CUDA C++ Programming Guide (occupancy, warps, shared memory)
• Ampere tuning (vectorized global loads)
• Nsight Compute occupancy calculator. sm_87 is the Orin ISA. Binaries built only for sm_90 do not run on it
• day07/rerun-2026-10-04/summary.json: day07/run.sh three times on 2026-10-04 (results_run1..3.json, tegrastats beside each)