Summary · Chapter 8

Resolution

Problem: the 16×16 teacher field is a piecewise-constant mosaic; 256 Gaussians fit it exactly (6e-12), so the budget curve above about 96 is meaningless there.

CacheHow builtError 16 / 32 stepsProbe
16×16256 px, one pass0.0000587 / 0.00005310.869
shift8256 px, four passes shifted by 8 px, interleaved0.0001208 / 0.00011510.871
input512512 px, one pass0.0001137 / 0.00010620.889

64 Gaussians, 800 holdout images, one seed; each error is on its own target, so about twice the error is a harder target, not a worse model. Teacher CLS probe 0.921 at 512 px.

Roughness (stride 8 over stride 16 adjacent-token squared difference, 100 images): shift8 0.75, input512 0.49, smooth field about 0.25 (reference, not measured). Coarse placement: fitted on 16×16, re-solved on 32×32 gives 0.000147 against 0.000111 (32% worse). On shift8, the old sub-grid error equals the full-grid error (0.0001212 vs 0.0001208). Choice: input512.

Upsamplers (FeatUp, JAFAR, LoftUp, AnyUp) produce a denser feature field from the image and coarse features; this project reruns the teacher at 512 px and then compresses the field into Gaussians.

Defence answer: “The 16×16 target hid structure, so I moved to a 32×32 target from a 512 px pass, chosen on roughness, error and probe from one seed.”

Sources: roadmap.md section 2d and run log 14 November; docs/research/presentation/2026-11-20/results.md.