Summary · Chapter 10

Learning to cluster

Unrolled iteration: a fixed number of steps of an iterative algorithm written as network layers, with settings learned end to end. Precedents: Neural EM, Superpixel Sampling Networks, slot attention (softmax over slots, weighted-mean update, GRU, no geometry).

One iteration of free_gaussian_cluster.py (8 iterations, 64 Gaussians):

akp = wf[i] ⟨ŝk, tp⟩ + wg[i] (−log(1 + d²kp) + log ωp)

softmax over Gaussians per token; times residual importance ωp ∈ [0.5, 2.5] (detached); moment fit of positions → centre and Cholesky factor × width w[i], mass logit = log(occupancy + 0.01); blend with weight βi; slot-attention-style state update and a small geometry correction. In-loop features are weighted means; least squares after the last iteration.

Learned per iteration: blend, feature and geometry sharpness, width. Gridless64 is the same loop with fixed gentle knobs (blend 0.25, mass carried over); it is reproduced exactly.

ArmDense MSECentre spread
Least-squares Gridless640.00010410.019
A: blend 1, mass refit0.00008720.065
B: all knobs learned0.0000804 (3 seeds 0.00008046, spread 0.24%)0.068
C: B without coverage penalty0.0000806
Per-image k-means (reference)0.000086

16×16 target, 800 holdout images, 6,000 updates, seed 20261112 unless stated. Probe unchanged (0.868 to 0.874). B's schedule: blend 0.59 → 0.80, feature sharpness 19 → 60, width 0.76 → about 1.2. 12,000 updates: 0.0000787 (−2.0%).

Defence: “The skeleton is soft k-means, deliberately. Its steps are trained on the soft least-squares error and beat per-image k-means in one pass; every slot is an explicit Gaussian read by a fixed renderer and a closed-form feature solve. It stops short of the optimum because a clustering fixed point is not the error's minimum; Chapter 11 adds steps on the error itself.”

Source: Locatello et al., Object-Centric Learning with Slot Attention (2020).