Evaluation and rigour
Holdout dense MSE: fidelity to the teacher field on 800 unseen images. Sensitive to placement; not a semantic measure. Say the cohort: J at 64 on input512 is 0.0001137 on 800 images, 0.0001119 on the first 100. The per-image optimum is a finite Adam run (0.0000465 from a grid, 0.0000558 from Gridless64).
Linear probe: about 0.87 top-1 for every 16×16 arm; it reads a pooled summary that placement barely changes (probing practice).
Pet mIoU: +0.0031 and +0.0039 with image-bootstrap intervals excluding zero, then −0.0002 [−0.0023, +0.0021] on a third seed. Intervals over images miss run-to-run variation. “Two of three seeds; not robust.”
Seeds: compare effect with spread. Three-seed spreads under 1% against effects of 12 to 27%: safe decisions, but report with intervals over seeds (Agarwal et al.). A +0.10% change is “no detectable effect”.
Holdout reuse: checkpoint selection (best up to about 2% below final), design decisions, tuned quick-experiment settings. Remedy: a sealed 2,000-image test set, not yet created.
Baselines: the simplest compressor at equal budget is often hardest (downsampling result). Trained one-shot Gridless64 (0.000104) lost to per-image k-means (0.000086); clustering B (0.0000804) and J (0.0000587) beat it.
Threats to validity
- Holdout used for selection and decisions; no sealed test set.
- Quick experiments tuned and oracles computed in-sample.
- One seed for most arms and for everything on 32×32.
- Quick experiments in plain PyTorch with slightly different settings.
- Gridless64 and Anchored64 differ in capacity and compute.
- Dense MSE is fidelity, not semantics; probe blind; Pet gain withdrawn.
- 16×16 target saturates;
input512is a different field. - The per-image optimum depends on its start.
- One cohort, one teacher, one GPU; no standard dense benchmarks.
- Teacher needed at inference.
- No intervals over seeds; 1% differences unresolved.
- Cross-cohort comparisons (100 vs 800 images).
- Per-count comparison: one seed, final checkpoints, drift at 128 and 256.
- Closest work not read in full.