Tokenization beyond patches
Five families of “fewer, smarter tokens”.
| Family | Example | Key fact | Versus this project |
|---|---|---|---|
| Merge or prune | ToMe | About 2x throughput, 0.2-0.3% drop (ViT-L at 512 px) | No geometry or read-out |
| Content patches | APT, SPiT | APT: +40% (ViT-L), +50% (ViT-H) throughput | Rectangles or superpixels, one token each |
| Variable tokenisers | FlexTok | 1 to 256 tokens, nested dropout | Abstract codes, pixel target |
| Gaussian tokens | GViT, GPSToken | GViT-B 76.9% vs ViT-B/16 79.8% (same recipe); GPSToken 128 tokens, rFID 0.65 | RGB targets, fixed or entropy-split layout |
| Slots over DINO | DINOSAUR | Reconstructs frozen DINO features | No geometry; objects, not a tiling |
Defence answer. “GViT is the closest: 2D Gaussians as tokens predicted feed-forward. Mine targets a frozen DINOv3 feature field, solves features in closed form, serves any count from 16 to 256, and places Gaussians by unrolled clustering and Adam. DINOSAUR shares the recipe of frozen features and soft competition, without geometry.”
Open item: equal-count baselines (pooling, per-image k-means, token merging) are not yet run.