20.1 One sentence, defended
Everything in this book comes down to one sentence you will say early in the talk: what is new. Say too much and the first reviewer who knows GViT or DINOSAUR takes it apart. Say too little and the work sounds like k-means with extra steps. This chapter builds that sentence from the closest work, lists what you can claim, rehearses the five hardest questions, and lays out the talk.
Before reading on: write your one-sentence novelty statement as you would say it today.
20.2 The closest work, side by side
Chapters 15 to 17 met these methods one by one. Here they are on the two questions of Chapter 1, how the parameters are found and what they represent, plus the one difference you would name first. Details are in reference/closest-work.md, checked against the arXiv abstract pages.
| Work | Represents | Found by | Key difference from this project |
|---|---|---|---|
| GViT[Hernandez et al., 2025] | Image as 2D Gaussians used as ViT input tokens; RGB reconstruction plus classification | Feed-forward, fixed count | This project fits a frozen teacher's feature field, solves features in closed form, serves any count |
| GPSToken[Zhang et al., 2025] | Gaussian tokens with texture features for reconstruction and generation | Entropy-driven partition, then a trained transformer refines | Placement learned against the true error; entropy splitting lost to direct clustering here (Chapter 14) |
| Instant GaussianImage[Zeng et al., 2025] | RGB image as 2D Gaussians, count adapts to complexity | Network start plus short per-image fine-tune | Features, not RGB; one model for many counts; optimiser inside the model |
| Upsample Anything[Seo et al., 2025] | Gaussian kernels as a filter for upsampling features | Per-image test-time optimisation, one kernel per low-resolution pixel | A compact set that owns feature vectors, placed feed-forward |
| Feature 3DGS[Zhou et al., 2023] | 3D Gaussians carrying distilled 2D-teacher features | Per-scene optimisation | Single image in 2D, feed-forward, features by least squares |
| DINOSAUR[Seitzer et al., 2022] | Slots reconstructing frozen DINO features | Slot attention, feed-forward | Slots are explicit Gaussians with geometry and a closed-form feature solve |
| APT[Choudhury et al., 2025] | Adaptive-size square patches as ViT input | Entropy thresholds, a pre-pass | Gaussians, learned placement; the matched-count comparison is still missing |
Read the last column top to bottom and a pattern appears: each neighbour shares one or two ingredients, none shares the combination. That is both the novelty and its limit.
20.3 The novelty statement and its limits
Its limits, which you say if pressed, not before.
- Every ingredient exists elsewhere. Feature-carrying Gaussians (Feature 3DGS); soft assignment over frozen DINO features (DINOSAUR); solving the linear part in closed form inside a nonlinear fit, which is variable projection[Golub and Pereyra, 1973]; unrolling an optimiser as layers[Gregor and LeCun, 2010]; one model for many budgets[Kusupati et al., 2022]. The claim is the combination and what it measures, not any one part.
- “Not found” is a statement about a search, not a proof. GViT v2 and GPSToken have been checked on their abstract pages but not read in full; the roadmap requires that before submission.
- “Within about 5%” is narrow. It is model J at 64 Gaussians on the 32×32
input512target, run to 32 refinement steps, against a 200-step per-image optimum, on the first 100 holdout images, one seed. At 16 steps the gap is 13%; at 256 Gaussians it is 32%. - The features are the teacher's. They are least-squares combinations of DINOv3 features, so the semantics are inherited, and the teacher is needed at inference.
20.4 The contribution list
Order by strength of evidence, and give each its number and its weakness.
- A Gaussian feature field with closed-form features. Ridge least-squares features given the geometry lower holdout dense error by about 22% against weighted means (Gridless64 0.000134 to 0.000104), three seeds, probe unchanged.
- Amortised placement by unrolled clustering and unrolled optimisation. On the 16×16 target the ladder runs 0.000104 (one-shot) → 0.0000804 (8 clustering iterations, three seeds) → 0.0000587 (plus 16 Adam steps, three seeds) → 0.0000531 (run to 32 steps), against a per-image optimum of about 0.0000465 (100 images). Uniform deep supervision was the lever that made end-to-end training help; learned step sizes were dropped.
- One model for any count. Trained at 64, it serves 16 to 256 on the 32×32 target within about 4% of per-image k-means plus the same 16 steps, at every count. Weakness: the gap to the optimum grows to 32% at 256; one seed.
- A measured account of target resolution. The 16×16 target hid about half the field's variation (error about twice as high at 64 on 32×32), and 256 Gaussians fit it exactly;
input512was chosen overshift8by roughness and transfer measurements. - Negative results on hierarchical placement (Chapter 14): splitting, merging, information-gain criteria and per-image counts, at matched budget.
- Engineering, stated as such. Triton kernels with hand-written backward passes under a strict dispatch guard, and a blocked Cholesky beyond 128 Gaussians (Chapter 18). Enabling, not the scientific claim.
Per-count trained models (16, 32, 128 and 256 Gaussians on input512, one seed, 800 images, 16 steps) have finished: 0.000338, 0.000205, 0.0000743 and 0.0000390, against 0.000346, 0.000207, 0.0000642 and 0.0000374 for the model trained at 64. Training at the count gains 1 to 2% at 16 and 32 and loses at 128 and 256, where holdout error rose after early updates. That strengthens contribution 3: transfer from 64 is the lever, not training at each count.
20.5 The five hardest questions
Try each before opening the model answer.
1. “Isn't this just per-image k-means, or a feature upsampler, or GViT?”
Model answer
“Per-image k-means is our main baseline, and an early trained one-shot predictor lost to it (0.000104 against 0.000086). The unrolled clustering stage beat it in one pass (0.0000804), and with refinement the model matches per-image k-means plus the same refinement at every count from 16 to 256, within about 4%, without running k-means per image. So the claim is not that we beat clustering by a margin, but that a single trained network reproduces the strong per-image pipeline, is differentiable end to end, and serves any count. It is not an upsampler: it outputs a compact set of feature-carrying Gaussians, not a dense high-resolution map, and we have not evaluated it under upsampler protocols. GViT learns Gaussians as RGB input tokens for classification at a fixed count; we fit a frozen teacher's feature field with closed-form features at any count.”
2. “The teacher must run at inference. What is this for?”
Model answer
“That is correct and it is our main limitation. Today the method is a compact, continuous, queryable representation of features that already exist: 64 Gaussians store about 6% of the numbers in a 32×32 feature grid. Uses that follow are fewer tokens for a downstream model, storage, and querying the field at any location. A pixel encoder that predicts Gaussians without the teacher is the next step; we have not built it, so we do not claim independence.”
3. “Where is the evidence that this is semantic? Your probe is flat and your segmentation gain disappeared.”
Model answer
“We do not claim new semantics. The features are least-squares combinations of DINOv3 features, so the semantics are inherited; what we measure is how faithfully a small set of Gaussians reproduces the teacher's field, and placement improves that by more than half. The classification probe reads a pooled summary that placement does not change, so it cannot see the difference. An early Pet segmentation gain held in two seeds and not in a third, so we withdrew it. Dense benchmarks at matched token count, against pooling and k-means, are the most important missing experiment.”
4. “One seed, a reused holdout, settings tuned on the scored images. Why trust any number?”
Model answer
“The main effects have three seeds with spreads under 1% against effects of 22 to 27%, so selection on the holdout, which moved checkpoints by up to about 2%, cannot reverse them. Smaller effects, such as the 4% from end-to-end refinement training or the 1% from adaptive count, could be affected, and we say so. Final numbers will use at least three seeds, intervals over seeds and images, and a sealed 2,000-image test set that no decision has touched.”
5. “Your research question promised hierarchical placement by scene complexity. It failed. What is left?”
Model answer
“We tested it before building it, at matched count and refinement steps. Splitting lost to direct clustering by 33 to 39% at 64, merging down by 11 to 29%, information gain was barely better than random, and per-image counts gained about 1% even with an oracle. What is left is the stronger finding: the count matters a great deal, error roughly halving per doubling, while its allocation across images hardly matters on this metric. So we built one model that clusters directly at whatever count is requested. A learned split network was not tried, and a downstream task might value per-image counts differently.”
20.6 The venue plan
From the roadmap's publication track: CVPR 2027 closed on 16 November 2026 and is out of reach. ICML 2027 full papers are due 22 January 2027. ICCV 2027 (about March) and NeurIPS 2027 (about May) dates are not yet verified. TMLR accepts on correctness of evidence and has no deadline.
What a strong venue requires, in the roadmap's order of how much is missing:
| Item | What it needs | State |
|---|---|---|
| P1 Downstream evidence | Dense benchmarks with linear probes, accuracy against token count, baselines at the same count | Not started (k-means only in quick experiments) |
| P2 Scale | Larger training set, sealed protocol, a larger teacher | 32×32 caches exist; rest open |
| P3 Shared RGB and semantics | Gaussian field as a feature upsampler, upsampler protocols | Literature check only |
| P4 Teacher-free inference | A pixel encoder, or a clear tokeniser framing with an application | Open |
| P5 Analysis | Amortisation gap curve, when placement matters, split study | Partly measured (ladder, split study); not written up |
| P6 Rigour | Sealed test sets, three or more seeds, matched arms, full reading of the closest work, code release | Open |
A suggestion, not a decision. For ICML in January, P1 with its baselines and P6 are the minimum, and the analysis (P5) is the paper's backbone. If P1 shows no task gain at matched count, the honest framing is an analysis paper about amortised placement and budgets, for which TMLR suits.
20.7 The shape of the talk
A storyline that follows the evidence: problem, mechanism, surprise, honest limits. The figures are in docs/.
- The question (one slide). Can a few Gaussians carrying DINOv3 features represent an image's feature field? Show a feature map and an ellipse layout side by side.
- The representation. Gaussians, responsibilities, closed-form features: −22%, three seeds. Settle “is it a neural representation?” with Chapter 5's two questions in one line.
- Placement is the bottleneck. The amortisation ladder (
fig1): one-shot, clustering, refinement, per-image optimum. Name the amortisation gap. - Closing the gap. Unrolled clustering then unrolled Adam with uniform deep supervision (
fig2); what failed on the way (discounted supervision, learned step sizes). - The twist: the target was too coarse. 16×16 against 32×32 (
fig3); 256 Gaussians fit 16×16 exactly. - How many Gaussians. The budget curve from one model (
fig4), the gap growing with count, and models trained at each count doing no better. - What did not work. Split, merge, information gain, adaptive count (
fig5), one slide, matched budget. - Position and limits. The closest-work table in three rows (GViT, GPSToken, DINOSAUR), the novelty sentence, then the teacher at inference, the missing downstream evidence and the rigour plan. End on the next experiment, not on a claim.
Put the threats to validity on a slide near the end (Chapter 19). Said by you, they read as control; said by a reviewer, as discovery.
20.8 Check yourself
Answer from memory; each question gives its reason once you choose.
1. What exactly is new in the novelty statement?
2. “Within about 5% of per-image optimisation” holds under which conditions?
3. A reviewer says the method is “just per-image k-means”. What is the honest core of the reply?
4. Why end the talk on the next experiment rather than a claim?
5. Which item ranks lowest as a scientific contribution?
20.9 Close the loop
Rewrite your novelty sentence from the start of the chapter. Then add one sentence of limits you would say if pressed.
A model answer
“We represent a frozen DINOv3 feature field with a small set of 2D Gaussians whose features are solved in closed form, placed in one pass by an unrolled clustering-and-optimisation network that serves any count from 16 to 256 from one model and comes within about 5% of per-image optimisation at 64 Gaussians; we found no published method that combines these. Each ingredient exists separately, the 5% is one seed on 100 images, the semantics are the teacher's and the teacher is needed at inference, and downstream evidence at matched token count is the next experiment.”
Something unclear, or want to rehearse how you would say this to a reviewer? Ask your teacher in the chat.