The 2D Gaussian
Shape: centre μ, covariance Σ (symmetric 2×2). Distance in widths ρ² = (x − μ)ᵀ Σ⁻¹ (x − μ); density φ(x) = exp(−ρ²/2) / (2π √det Σ). Contours are ellipses; at 1, 2, 3 widths the bump is 0.61, 0.14, 0.01 of its peak.
Reading Σ: eigenvectors are the principal axes, eigenvalues the squared axis lengths. Example: diagonal 0.05, off-diagonal 0.03 has axes along (1, 1) of length 0.283 and along (1, −1) of length 0.141.
Positive definite: all eigenvalues positive (2×2: top-left entry and determinant positive). Otherwise the density is undefined: entries 0.05, 0.06, 0.05 have eigenvalue −0.01.
Cholesky factor: Σ = LLᵀ, L = a0bd, Σ = a²ababb² + d². Any a, d > 0 gives a valid Σ; every valid Σ has exactly one. Density via u = δ₁/a, v = (δ₂ − bu)/d, ρ² = u² + v², √det Σ = ad.
Density and mass: log(mφ) = log m − log 2π − log a − log d − ½(u² + v²). Narrow means tall: halving both axes quadruples the peak.
| This project (6 numbers) | Rule | Code |
|---|---|---|
| Centre, in [−1, 1]², y down | tanh | _decode_free, free_gaussian_spatial.py |
| Cholesky a, b, d | a, d = 0.008 + 0.05·softplus; b = 0.05·raw | same; refinement uses QDecode (floor 0.004) |
| Mass logit | softmax over Gaussians, sums to 1 | last_axis_softmax |
| Density | formula above | gaussian_log_density, geometry.py |
Defence answer: “Each Gaussian is a centre, a Cholesky factor and a mass logit. The factor guarantees a valid covariance for any network output, with no checks or repairs.”
Source: Bishop, Pattern Recognition and Machine Learning (2006), section 2.3.