Summary · Chapter 2

The 2D Gaussian

Shape: centre μ, covariance Σ (symmetric 2×2). Distance in widths ρ² = (x − μ)ᵀ Σ⁻¹ (x − μ); density φ(x) = exp(−ρ²/2) / (2π √det Σ). Contours are ellipses; at 1, 2, 3 widths the bump is 0.61, 0.14, 0.01 of its peak.

Reading Σ: eigenvectors are the principal axes, eigenvalues the squared axis lengths. Example: diagonal 0.05, off-diagonal 0.03 has axes along (1, 1) of length 0.283 and along (1, −1) of length 0.141.

Positive definite: all eigenvalues positive (2×2: top-left entry and determinant positive). Otherwise the density is undefined: entries 0.05, 0.06, 0.05 have eigenvalue −0.01.

Cholesky factor: Σ = LLᵀ, L = a0bd, Σ = a²ababb² + d². Any a, d > 0 gives a valid Σ; every valid Σ has exactly one. Density via u = δ₁/a, v = (δ₂ − bu)/d, ρ² = u² + v², √det Σ = ad.

Density and mass: log(mφ) = log m − log 2π − log a − log d − ½(u² + v²). Narrow means tall: halving both axes quadruples the peak.

This project (6 numbers)RuleCode
Centre, in [−1, 1]², y downtanh_decode_free, free_gaussian_spatial.py
Cholesky a, b, da, d = 0.008 + 0.05·softplus; b = 0.05·rawsame; refinement uses QDecode (floor 0.004)
Mass logitsoftmax over Gaussians, sums to 1last_axis_softmax
Densityformula abovegaussian_log_density, geometry.py

Defence answer: “Each Gaussian is a centre, a Cholesky factor and a mass logit. The factor guarantees a valid covariance for any network output, with no checks or repairs.”

Source: Bishop, Pattern Recognition and Machine Learning (2006), section 2.3.