Reproducibility

Two things live in this repository, and they are at different stages. This page says plainly which commands have been run and which have not.

Path	Status
Documentation site	Verified. The commands below were run in this repository.
Agent, benchmark and tests	Verified. The commands below were run in this repository, on generated stand-in wafers.
A benchmark run against WM-811K	Not published here. The dataset is not committed; download it and the same commands produce a results file.

Documentation site

Verified on Node.js 22.17.1 / npm 11.5.1; the workflow runs the same commands on Node.js 20.

git clone https://github.com/geniuskey/reality-aware.git
cd reality-aware
npm ci
npm run docs:dev      # local server with hot reload
npm run docs:build    # static build into docs/.vitepress/dist
npm run docs:preview  # serve the built output

npm ci needs the committed package-lock.json, which is what the deployment workflow installs from as well.

Building for a different Pages base path

The base path is derived from GITHUB_REPOSITORY during CI, so project Pages (/<repo>/) and user or organisation Pages (/) both work without editing the config. Override it locally when you need to:

DOCS_BASE=/ npm run docs:build           # user or organisation Pages
DOCS_BASE=/my-fork/ npm run docs:build   # a fork with a different repo name

Publishing figures to the site

The canonical location for experiment output is results/. The site reads figures from docs/public/results/. Copy between them with the sync script rather than by hand:

npm run sync:assets

It copies known figure names from results/ into docs/public/results/, skipping anything absent, and reports what it did. Missing figures are not an error: pages render an explicit Run benchmark to generate results state instead of a broken image, and the build still succeeds.

Rebuilding the playground

The playground carries the twelve evaluation wafers inside the page, so it has to be rebuilt from a local copy of the dataset whenever its source or the published results change:

npm run build:playground   # python -m nano.playground: embeds the wafers, writes the page
npm run check:playground   # replays Python's decisions through the page's own engine

The first refuses to run on stand-in wafers, because the page tells its reader it is showing WM-811K. The second is what puts the N of M traces match Python count on the page, and it runs in CI on every deploy, so a divergence the browser port introduces fails the publish rather than shipping quietly.

Agent and benchmark

Verified on Python 3.11.

python -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

pytest                                        # test suite, no dataset needed
python -m nano.demo --wafer wm811k-645735 --seed 0    # the published episode, writes demo figures
python -m nano.benchmark --seeds 0 1 2 3 4 --ablation # full comparison, writes the results file

pip install -e . alone installs numpy and nothing else, which is enough to run the loop and the benchmark. The figures extra adds matplotlib and pillow, data adds pandas for reading LSWMD.pkl, and dev adds both plus pytest.

Running without the dataset

Every command takes --synthetic, which replaces WM-811K with wafer maps generated by nano.data.synthetic_wafers:

python -m nano.benchmark --synthetic --wafers 12 --seeds 0 1 2 3 4

This exercises the whole pipeline — loop, baselines, scoring, figures — without downloading anything. It is not the dataset. The run prints a warning, the results file records dataset.name as synthetic-wafers with a note, and the table on the benchmark page prints the wafer source, so a stand-in number cannot be read as a dataset number. Results from a stand-in run are not committed to this repository.

Getting the dataset

The WM-811K distribution is not committed here — see Dataset for the links and terms. The expected local layout is:

data/
└── raw/
    └── LSWMD.pkl        # downloaded, never committed

Derive the versioned evaluation index from it once:

python -m nano.subset --raw data/raw/LSWMD.pkl --wafers 12 --seed 0

The index is versioned (data/subsets/wm811k_eval.json), so an evaluation set can be reconstructed exactly without redistributing the source data. python -m nano.benchmark reads it by default.

Artifact locations

Artifact	Path
Results summary, single source of truth for the site	results/benchmark_summary.json
Error curve, wafer comparison, uncertainty maps	results/*.svg, results/*.webp
Demo recording	results/demo.gif
Site copies of published figures	docs/public/results/
Versioned evaluation index	data/subsets/wm811k_eval.json

Individual figures can be regenerated one at a time, without re-running the benchmark:

python scripts/plot_wafer_comparison.py --wafer wm811k-645735 --seed 0
python scripts/plot_uncertainty.py --wafer wm811k-645735 --seed 0
python scripts/plot_paired_improvement.py          # reads the results file
python scripts/plot_calibration.py                 # reads the results file
python scripts/record_demo.py --wafer wm811k-645735 --seed 0

--wafer takes a wafer id or a positional index. The id is the reliable form: an index means a different wafer as soon as the evaluation set changes, which is why the published figures name one. Which wafer the published figures show is not a choice made in this page — it is recorded in results/benchmark_summary.json under figures, next to the rule that selected it:

python -m nano.benchmark --seeds 0 1 2 3 4                          # picks both figure wafers
python -m nano.benchmark --seeds 0 1 2 3 4 --figure-wafer wm811k-19423   # override the first

The benchmark always also draws the wafer where the selection rule earned the least over Random and Grid, and --figure-wafer cannot change that one.

Figures follow the results file: --out pointed somewhere else writes that run's figures beside it, so a check run cannot leave its pictures next to someone else's numbers in results/.

Controlling seeds

Seeds are explicit arguments, never implicit global state:

python -m nano.benchmark --seeds 0 1 2 3 4     # published configuration
python -m nano.benchmark --seeds 7             # a single run, for a quick check

The seed set that produced a results file is recorded inside that file, and the site prints it under every table and chart. A summary whose seeds do not match the run that produced it is a bug worth reporting.

Verifying a result you did not produce

1. Read dataset.name, experiment.seeds, experiment.initial_measurements and experiment.measurement_budget from results/benchmark_summary.json. A dataset.note means the run did not use WM-811K.
2. Re-run the benchmark with exactly those values, against the subset index named in dataset.subset_file.
3. Compare strategies.*.final.mean against your own run. experiment.prior_bias and experiment.model record the parameters that produced them.

If the numbers do not reproduce, the results file is wrong and this site is wrong with it, because this site does nothing but read it.