The Natural Geometry of Learning

The Natural Geometry of Learning is an ongoing expository essay series about the geometric ideas that recur across probability and machine learning. It begins with two complementary rulers—Fisher information for statistical distinguishability and Wasserstein distance for spatial displacement—before moving toward KL geometry, Schrödinger bridges, Hodge decomposition, Langevin dynamics, and their connections to learning.

Diagram showing parameter space mapping into distribution space, with tangent vectors represented logarithmically as score functions and the Fisher metric obtained as a pullback.

The First Ruler of Distinguishability

Why Fisher information is the local geometry of statistical models

This post introduces Fisher information as the first “ruler” for probability distributions: a local metric that measures how distinguishable nearby statistical models are from data. We will see why it arises naturally as a pullback metric, why Chentsov’s theorem makes it unique, and how it foreshadows the deeper connection between KL divergence, natural gradients, and learning.

Jun 5, 2026
Gaurav Khanal

Diagram comparing a Kantorovich transport plan with many faint mass-splitting connections to a Brenier deterministic map with one arrow from each source bin to a target bin.

The Second Ruler of Displacement

Why Wasserstein distance is the geometry of moving probability mass

This post introduces Wasserstein distance as the second “ruler” for probability distributions: a metric that measures the cost of moving probability mass through space. We will see why it arises from the Earth Mover’s problem, why the quadratic case has special geometric structure through Brenier’s theorem, and how the Benamou–Brenier formulation turns optimal transport into a dynamic geometry of probability flows.

Jun 13, 2026
Gaurav Khanal
Forward and reverse KL approximations of a bimodal probability distribution

The Asymmetry of Information

Why KL divergence remembers direction

This post examines KL divergence as a directed contrast rather than a defective distance. We will see how its two directions shape learning, how Fisher geometry reappears as its local second-order shadow, how exponential families reveal its Bregman structure, and why path-space KL leads naturally to the Schrödinger bridge.

Aug 22, 2026
Gaurav Khanal
Brownian reference paths and controlled Schrödinger bridge paths between two probability distributions

The Bridge

How entropy turns transport into stochastic inference

This post develops the Schrödinger bridge as the least-surprising stochastic evolution between two distributions. We will connect path-space KL to optimal transport, derive the bridge drift through its two-sided potential transform, recover Sinkhorn as alternating KL projection, and prepare the way for Hodge decomposition.

Oct 19, 2026
Gaurav Khanal
No matching items