17.1 Why this chapter matters
The phrase “Gaussians that carry features” has a famous cousin: feature 3D Gaussian splatting, where each Gaussian in a 3D scene carries a feature vector. If you present without placing yourself relative to it, a reviewer will do it for you, and probably as “this is Feature 3DGS without the 3D”. You also sit beside feature upsamplers, which attack the same coarse feature grid you do (Chapter 8).
Before reading: if a 2D teacher gives you a feature map for each of many photographs of one scene, how would you store one consistent 3D feature field?
17.2 Distilling a 2D teacher into a 3D field
2D foundation models (CLIP, DINO and similar, Chapter 7) give excellent features for single images but know nothing about 3D consistency. Feature-field work lifts them into a 3D scene so that one can ask “where in this room is the mug?” from any viewpoint.
- Distilled Feature Fields (Kobayashi et al.) distil CLIP-LSeg or DINO features into a 3D feature field optimised in parallel with a NeRF's radiance field. Users can query by text, an image patch or a click, which enables 3D segmentation and selective editing[Kobayashi et al., 2022].
- LERF grounds CLIP embeddings in a NeRF by volume-rendering them along training rays, giving a dense multi-scale language field and relevancy maps for text queries[Kerr et al., 2023].
Both use implicit neural fields (Chapter 1), which are slow to render and optimise per scene. Their idea is the same as yours at the highest level: a compact structure is trained so that a rendered feature map matches a teacher's feature map.
17.3 Putting the features on Gaussians
Swap the NeRF for 3D Gaussian splatting (Chapter 15) and the feature field becomes explicit.
- Feature 3DGS extends splatting to features of arbitrary dimension, distilled from SAM and CLIP-LSeg. Its abstract names two obstacles, mismatches in spatial resolution and in channel consistency between RGB images and feature maps, proposes changes to deal with them, and reports being significantly faster to train and render than NeRF-based feature fields, with no figure on the abstract page[Zhou et al., 2023].
- LangSplat puts CLIP-derived language features on 3D Gaussians and renders them by tile-based splatting. To limit memory it trains a scene-wise autoencoder and learns features in its latent space; it uses SAM to get hierarchical semantics. Its abstract reports being 199 times faster than LERF at 1440 × 1080[Qin et al., 2024].
Notice the resolution mismatch Feature 3DGS names: the teacher's feature map is much smaller than the image. That is your Chapter 8 problem again, in a different setting.
Same as yours: Gaussians that carry feature vectors; a rendered feature map compared with a teacher's feature map. Different:
- Dimension. 3D scenes seen from many cameras, against one 2D image.
- Features are optimised. A 3D Gaussian's feature must agree with many views, so it is found by gradient descent. Yours are one linear solve given geometry (Chapter 4).
- Placement. Per scene by densification, against a trained network that places Gaussians for a new image in one pass.
- Dimension of the feature. LangSplat compresses CLIP features with an autoencoder to save memory. Yours keep all 384 numbers, which is feasible because the Gaussian count is small and the solve is shared across the whole feature vector.
- Purpose. They want a 3D scene you can query and edit; you want a compact, queryable 2D representation of a teacher's tokens.
17.4 Feature Gaussians used in other ways
Three more papers show the range of uses.
- Feature Splatting uses object-centric feature Gaussians for language-driven editing and physics[Qiu et al., 2024].
- FiT3D lifts 2D features to 3D Gaussians and then fine-tunes the 2D model with them, so the Gaussians act as a training signal for better 2D features[Yue et al., 2024].
- Splat and Distill uses a feed-forward lifting of teacher features onto Gaussians and distils back into a 2D student; it is the feed-forward variant (method from the audit)[Shavin and Benaim, 2026].
The last is the closest in training style to your amortised placement, because lifting is a single network pass. It still lives in 3D, from several views. The point for your defence is that no one on this list represents one image's feature field with a small set of 2D Gaussians placed by a feed-forward model.
17.5 Feature upsamplers as dense 2D fields
The other community that looks at your target is feature upsampling. Chapter 8 introduced it in brief. In the language of this book: a ViT gives an explicit 16×16 grid; an upsampler produces a field that can be read at higher resolution.
- FeatUp has a feed-forward variant and a per-image implicit variant, trained with multi-view consistency[Fu et al., 2024]. The per-image implicit one is the nearest neural-field relative.
- LoftUp and JAFAR use coordinate-based cross-attention to answer queries at chosen positions[Huang et al., 2025][Couairon et al., 2025]. AnyUp is a feature-agnostic local attention upsampler with a full comparison table[Wimmer et al., 2025]. NAF uses RGB-guided neighbourhood attention[Chambon et al., 2025].
- Upsample Anything fits one anisotropic Gaussian kernel per low-resolution pixel for each image, by test-time optimisation, as a joint bilateral filter on features. Its abstract reports 0.419 seconds per 224 px image[Seo et al., 2025].
Upsample Anything is the only one that uses Gaussians, and the differences are real: its Gaussians are filter kernels at a fixed layout, one per low-resolution pixel, fit per image; the features stay on the grid. Your Gaussians own the features, their count is chosen freely, and a trained network places them.
Upsamplers guide with the full-resolution RGB image. Yours does not, yet: guiding geometry by RGB while taking features from the low-resolution teacher is a roadmap item (P3), not a result. The roadmap also records that learned upsamplers gain only about 1 to 4 mIoU over bilinear on DINOv3; that figure comes from the literature audit and was not re-verified here.
17.6 Where your field sits
Line the works up on the two questions of Chapter 1.
| Work | Space | Structure | Features found by | Placement |
|---|---|---|---|---|
| Distilled Feature Fields, LERF | 3D scene | Neural field | Gradient descent | None (implicit) |
| Feature 3DGS | 3D scene | 3D Gaussians | Gradient descent | Per scene, densification |
| LangSplat | 3D scene | 3D Gaussians, compressed CLIP | Gradient descent in latent space | Per scene |
| Splat and Distill | 3D, feed-forward | 3D Gaussians | Feed-forward (audit) | Feed-forward |
| FeatUp (implicit) | One image | Neural field | Per-image training | None |
| Upsample Anything | One image | Gaussian kernels on the grid | Filtering the teacher grid | Fixed, one per pixel |
| This project | One image | 2D Gaussians, soft regions | Closed-form ridge solve | Unrolled, amortised, any count |
Two honest caveats belong with this table. First, the features in your model are closed-form means of teacher patches (ridged towards the weighted mean), so the semantics are inherited from the teacher, and the teacher is still needed at inference (roadmap P4). Second, nobody has yet compared your field with an upsampler at matched protocol; the roadmap's resolution-free query test (train on 16×16, evaluate against 32×32 and 64×64) is the experiment that would make that comparison.
The search note from closest-work.md applies: no paper was found that has all four of 2D Gaussians with features, a closed-form feature solve, feed-forward placement and a frozen-DINOv3 dense evaluation. That is a finding about the search, not a proof, and a fresh arXiv search is recommended before submission.
17.7 Check yourself
Answer from memory; each question gives its reason once you choose.
1. Why can your Gaussian features be solved in closed form when Feature 3DGS must optimise them?
2. What does LangSplat do to the CLIP features before putting them on Gaussians?
3. Which statement about Upsample Anything is correct?
4. A reviewer says “this is Feature 3DGS without the 3D”. Which reply is strongest?
5. What would a fair comparison with feature upsamplers need?
17.8 Close the loop
Without scrolling up, explain in four sentences how your model differs from Feature 3DGS and from an upsampler.
A model answer
“A distilled feature field trains a structure so that its rendered features match a 2D teacher. In 3D (Feature 3DGS, LangSplat) the Gaussians are per scene with optimised features. Upsamplers make a denser grid; Upsample Anything uses Gaussian kernels as a filter. Mine is a single-image field of 2D Gaussians with closed-form features, placed by a trained network, at any count.”
Something unclear, or want to rehearse how you would say this to a reviewer? Ask your teacher in the chat.