Learning to cluster
Unrolled iteration: a fixed number of steps of an iterative algorithm written as network layers, with settings learned end to end. Precedents: Neural EM, Superpixel Sampling Networks, slot attention (softmax over slots, weighted-mean update, GRU, no geometry).
One iteration of free_gaussian_cluster.py (8 iterations, 64 Gaussians):
softmax over Gaussians per token; times residual importance ωp ∈ [0.5, 2.5] (detached); moment fit of positions → centre and Cholesky factor × width w[i], mass logit = log(occupancy + 0.01); blend with weight βi; slot-attention-style state update and a small geometry correction. In-loop features are weighted means; least squares after the last iteration.
Learned per iteration: blend, feature and geometry sharpness, width. Gridless64 is the same loop with fixed gentle knobs (blend 0.25, mass carried over); it is reproduced exactly.
| Arm | Dense MSE | Centre spread |
|---|---|---|
| Least-squares Gridless64 | 0.0001041 | 0.019 |
| A: blend 1, mass refit | 0.0000872 | 0.065 |
| B: all knobs learned | 0.0000804 (3 seeds 0.00008046, spread 0.24%) | 0.068 |
| C: B without coverage penalty | 0.0000806 | |
| Per-image k-means (reference) | 0.000086 |
16×16 target, 800 holdout images, 6,000 updates, seed 20261112 unless stated. Probe unchanged (0.868 to 0.874). B's schedule: blend 0.59 → 0.80, feature sharpness 19 → 60, width 0.76 → about 1.2. 12,000 updates: 0.0000787 (−2.0%).
Defence: “The skeleton is soft k-means, deliberately. Its steps are trained on the soft least-squares error and beat per-image k-means in one pass; every slot is an explicit Gaussian read by a fixed renderer and a closed-form feature solve. It stops short of the optimum because a clustering fixed point is not the error's minimum; Chapter 11 adds steps on the error itself.”
Source: Locatello et al., Object-Centric Learning with Slot Attention (2020).