A reproducible kernel artifact

What I expected

My plan was a day of packaging work: a README, three run scripts, an hour. I expected that each number from the three kernel days was reproducible by definition, because the code is the code.

I also expected that 'reproduces' had one meaning. It has three meanings, and I did not write any of them down: bit-exact, within noise, or not promised.

The theory

A small model: three classes of number, and a tolerance for each

parity 0.000000, days 07 and 09
registers 28, 6 blocks/SM (same nvcc)
the hash a23e893
input = a formula, so no dataset
= any drift here is a bug, mine or yours

fused 206 us, three kernels 480 us
hole +0.96 ms = 6.27 - 5.31
Rust host p50 967 us
same mode and clocks, or no comparison
= tolerance: a few %, derived below

The third class is weather: the Rust worst tick (1168 us, one event in 600 ticks, cause not isolated). It stays in results.jetson.json and out of the promise. The 1777 us p50 of Python is within noise as a number. A pageable copy sits inside its bracket (Chapter 9), so it is a host-plus-copy number, not a language number.

How wide is 'the same' for a timing row

This is the noise of a median. The fused kernel of Chapter 7 ran with 205.5 mean and 207.4 p99, which is less than 1% of spread. The standard error of a median over 1000 samples is approximately 1.25 x sigma / sqrt(1000) = 0.04 x sigma. The p50 is accurate to a tenth of a microsecond (arithmetic). The drift between runs on one board comes from thermal and clock state, not from sampling.

From board to board: the kernel ran at 31 of 68 GB/s (Chapter 7), under half of the roof. So I have not shown that it is memory-bound. Chapter 7 leaves open whether the scalar u8 loads or the wave tail limit it. If bandwidth is the limit, the 206 us follows the memory clock; untested. The mode is on each results file, because without it two numbers are not comparable, only different.

The hash has 7 hex digits = 28 bits = 268 M values. In a repo of n objects, the probability that two objects have the same prefix is approximately n^2 / 2^29. That is 0.2% at 1,000 objects and 1% at 2,300 objects. Seven digits are sufficient for years. I write eight digits when seven are not sufficient (arithmetic).

The setup

cd jetson-8gb-kernels && git add -A && git commit -m 'day08, day09: packages + board results' && git rev-parse --short HEAD

# github.com/Aryan95614/jetson-8gb-kernels
cd jetson-8gb-kernels && git checkout a23e893
cd day07 && bash run.sh
cd ../day08 && bash run.sh
cd ../day09 && bash run.sh
# compare results.json against results.jetson.json; mode line first

Environment and scripts

• Repo: jetson-8gb-kernels at a23e893 (public link to follow).
• Scripts: day07/run.sh, day08/run.sh, day09/run.sh. Each directory commits its board output as results.jetson.json.
• Board: Orin Nano Super, JetPack 6.2, 15W (measured.json). Clocks not logged.
• Boards tested: one.

The measurement

From a stranger's clone to a diff against my numbers

Where it broke

The README table has three rows, one for each day. In my tally, five entries are exact and three are statistical. Of the numbers that I liked to tell people, one is weather and one is a copy inside the bracket. A number that I could not promise had no column.

Also, 'the code is the code' was incorrect in two ways. The register count is a property of the nvcc version, not of the source. The 206 us is a property of the clocks and the power mode. The repo freezes the source, and the results file freezes the other items. A citation needs the two lines.

I thought that a freeze was packaging work. It was the first time that I had to write, for each number, what counts as the same. I did not have that list before today. The README is that list.

I wrote the table before the repo matched it. If a re-run drifts, the table gets the new number and a dated note. The table never changes the run.

Summary

• On the day that I froze the artifact, I sorted which numbers the README can promise. Parity and the occupancy readout are exact on this board. The timings are within noise. The Rust worst tick is not promised, and its cause is not isolated.
• The git hash is the citation. These three days refer to a23e893. They do not refer to "the kernel from last week".
• The README plus its results files is the packet that a second person needs to run the three days again.

Sources

These sources are not mine. Measured numbers came off my own board; specs, estimates and paper figures are labeled as such.

• The Day 4 harness (tools/bench.sh)
• Jetson-containers convention of tagging L4T