8.1 Why this matters for your claim
Your error numbers depend on the grid they are measured on. On the 16×16 target, 64 Gaussians reach a dense error near 0.0000587. On a 32×32 target the same model sits near 0.000114. A reviewer who sees both will ask whether the model got worse. It did not; the target got harder. This chapter explains why, and why 32×32 from a 512 px input is the target you chose.
input512, and contrast your approach with the feature upsamplers that other groups build.First, write what you expect a 16×16 grid to lose, and why a Gaussian model might care.
8.2 What a 16×16 grid hides
Chapter 6 showed that the teacher gives one vector per 16×16 pixel square. Between grid points there is no information. Read your target as a picture and it is piecewise constant: a mosaic of 256 tiles, each one flat.
That has two consequences for fitting Gaussians.
- The target cannot locate anything inside a tile. A Gaussian moved by half a patch changes nothing the target can see, so placement is only weakly identified. The roadmap (13 November) worried that the placement model was partly fitting the patch-grid quantisation.
- The budget curve saturates early. Quick experiment s6 found that 256 Gaussians fit the 256 patches exactly, an error of 6e-12. Above about 96 Gaussians the curve on this target says nothing.
A finer target fixes both, if it carries real structure. Chapter 6's remedy for a finer grid was: change the input, not the Gaussians. There are two ways to do that.
8.3 Two ways to a 32×32 field
Both caches were built by cache_frozen_patches_fine_20261113.py, using the runtime in frozen_dino_runtime_grid.py, which allows any square input that is a multiple of 16. Both cover the same 6,400 train and 800 holdout images.
input512. Render the whole image at 512 px and run the teacher once. The output is 32×32 tokens. Each token reads a 16 px window of the 512 image, which is an 8 px cell of the original 256 px frame. Windows tile the image without overlap.shift8. Keep the 256 px image and run the teacher four times, with the window shifted by (0,0), (8,0), (0,8) and (8,8) px. Interleave the four 16×16 outputs into one 32×32 grid. Each token still reads a 16 px window, but windows now start every 8 px, so neighbours overlap by half. The shifted windows read 8 px of reflection padding past the right or bottom edge.The drawing is one-dimensional and not to scale in pixels. The point is the geometry: input512 samples non-overlapping cells of a different, finer view of the image; shift8 samples overlapping windows of the same coarse view. In shift8 the even rows and columns (shift (0,0)) are exactly the old 16×16 cache, which the builder checks.
8.4 What the measurements said
The project's best placement model so far (Part III explains it; the roadmap calls it model J) was trained at 64 Gaussians on each target, one seed (20261112), scored on the 800 holdout images. Dense MSE after 16 and 32 optimisation steps (Chapter 11 explains them):
| Target | 16 steps | 32 steps | Probe top-1 |
|---|---|---|---|
| 16×16 | 0.0000587 | 0.0000531 | 0.869 |
32×32 shift8 | 0.0001208 | 0.0001151 | 0.871 |
32×32 input512 | 0.0001137 | 0.0001062 | 0.889 |
Sources: roadmap.md run log (14 November) and results.md, figure 3. The teacher's own CLS vector reaches 0.921 at 512 px.
Reading 1: about twice the error. Either 32×32 target gives about twice the error of the 16×16 target at the same Gaussian count. Do not call this a regression. Each number is measured on its own target, so it describes how hard the target is, not a like-for-like gain (results.md says the same).
Reading 2: sub-grid equality. On shift8, the error on the old 16×16 sub-grid (0.0001212) equals the error on the full grid (0.0001208). So the fine model is no worse at the old grid points; the extra difficulty sits in the new tokens between them. The roadmap's reading is that the coarse target was hiding roughly half the field's variation.
Reading 3: coarse placement costs. Quick check s7 (100 holdout images, plain PyTorch, clustering width tuned on the same images): 64 Gaussians fitted on the 16×16 sub-grid and then re-solved on the full grid give 0.000147, against 0.000111 for Gaussians fitted on the full grid. That is 32% worse, about the same as the unrefined starting layout. Placement trained on the coarse target does not transfer.
8.5 Why input512: roughness
If shift8 and input512 both give 32×32 grids, why prefer one? Quick check s7 measured roughness.
| Field | Roughness ratio |
|---|---|
| Smooth field (reference, not measured) | about 0.25 |
input512 | 0.49 |
shift8 | 0.75 |
The reference 0.25 is the explanatory note in the s7 file, not a measured value. The 0.49 and 0.75 come from 100 images.
A second test showed the same thing. Geometry fitted on two of the four shift positions fails on the other two at 256 Gaussians: 0.000306 against 0.000131. The conclusion in the roadmap is that much of the shift8 detail is per-window variation, not structure. A teacher window moved by 8 px gives a somewhat different answer because its attention context changed, even if the picture did not.
Decision (14 November). input512 was chosen: error is 6% lower than shift8 (0.0001137 against 0.0001208), the probe is higher (0.889 against 0.871), and the field is smoother. Present this as a choice made on a roughness argument plus one seed of results, not as a proof that input512 is the true fine field.
A caveat worth saying aloud: a teacher run at 512 px is a different input for the network than one at 256 px. Objects now span more tokens. Whether the 512 px field is a better description of the image, or only a finer one, is not settled by these numbers.
8.6 The other answer: feature upsamplers
Many groups noticed the same problem, a coarse feature grid, and built feature upsamplers: small networks that take the low-resolution features, usually together with the full-resolution image, and produce features at any resolution. Four to know:
- FeatUp restores spatial detail in deep features independent of task or model. It has two variants: one guided by a high-resolution signal in a single forward pass, and one that fits an implicit model to each image. Both use a multi-view consistency loss[Fu et al., 2024].
- JAFAR matches high-resolution queries, derived from low-level image features, with semantically rich low-resolution keys by attention. It is trained at low upsampling ratios yet generalises to much higher ones[Couairon et al., 2025].
- LoftUp uses a coordinate-based cross-attention transformer that takes the image, pixel coordinates and low-resolution features, trained against pseudo-ground-truth features built from class-agnostic masks and self-distillation[Huang et al., 2025].
- AnyUp is feature-agnostic at inference time, so one upsampler serves many encoders without retraining for each[Wimmer et al., 2025].
Also in the resource list: Upsample Anything, which optimises one anisotropic Gaussian kernel per low-resolution pixel for each image, with no training[Seo et al., 2025]. It is the nearest prior on Gaussians in this area.
How your approach differs
| Upsamplers | This project | |
|---|---|---|
| Output | A denser feature field, the product | A compact set of Gaussians; the field is the target |
| Fine field comes from | A learned guess from the image and coarse features | Re-running the teacher at 512 px |
| Stored size | A full dense grid | 64 Gaussians |
The two are not rivals at the same task. An upsampler could, in principle, produce the target your Gaussians then compress, and the roadmap's publication track plans to compare against AnyUp, JAFAR and NAF on the common probe suite. The roadmap notes from a literature check that learned upsamplers gain only about 1 to 4 mIoU over bilinear on DINOv3 (that figure is from the roadmap, not from the abstract pages read here). Chapter 17 returns to upsamplers as dense fields.
8.7 Check yourself
1. 64 Gaussians have error 0.0000587 on 16×16 and 0.000114 on 32×32. What is the right reading?
2. How does the teacher produce the input512 grid?
3. Why was shift8 judged worse than input512?
4. What does 32% worse for coarse-fitted placement show?
5. How do you contrast your work with a feature upsampler?
8.8 Close the loop
Without scrolling up, explain what the 16×16 grid hides, how the two 32×32 caches differ, why input512 was chosen, and how an upsampler differs from your approach. Then compare with your first answer.
A model answer
“A 16×16 grid is a piecewise-constant mosaic of 256 tiles; 256 Gaussians fit it exactly and placement inside a tile is invisible. I built two 32×32 targets: input512 runs the teacher once on a 512 px image, shift8 runs it four times with 8 px shifts and interleaves the results. At 64 Gaussians both give about twice the 16×16 error, which is a harder target and not a worse model. Shift8 is rougher (0.75 against 0.49, where a smooth field gives about 0.25), so I chose input512, which also had lower error and a higher probe on one seed. Feature upsamplers such as FeatUp, JAFAR, LoftUp and AnyUp learn to guess a dense field from the image and a coarse one; I rerun the teacher and then compress the field into 64 Gaussians.”
Something unclear, or want to rehearse how you would say this to a reviewer? Ask your teacher in the chat.