logo

📂Probability Theory

Definition of Wasserstein Distance

Definition

For two random variables $X_{1} : \Omega \to \mathbb{R}$, $X_{2} : \Omega \to \mathbb{R}$ whose cumulative distribution functions are $F_{1}$ and $F_{2}$ respectively, the $p$-Wasserstein distance is defined as follows. $$ W_{p} \left( X_{1}, X_{2} \right) := \left( \int_{0}^{1} \left| F_{1}^{-1}(y) - F_{2}^{-1}(y) \right|^{p} dy \right)^{\frac{1}{p}} $$

Explanation

For $y \in [0, 1]$, $F_{1}^{-1}$ and $F_{2}^{-1}$ are a kind of $y$-percentile, so in the end this becomes a distance that measures how similar two distributions are. In practice, the case $p = 1$ is used most often, and in this case, for the probability density functions $f_{1}$, $f_{2}$ of $X_{1}$ and $X_{2}$, since $$ y = F(x) \implies dy = f(x) dx $$ holds, $W_{1}$ is no different from the common-sense comparison of two distributions $$ \int_{-\infty}^{\infty} \left| f_{1}(x) - f_{2}(x) \right| dx $$ However, when such probability distributions are not given exactly and only an empirical distribution, in other words only data, is given, it is far more convenient to compute the percentiles exactly as in the original definition than to compute the improper integral.

Code

The following is Julia code for the numerical computation of the $1$-Wasserstein distance.

using Statistics, StatsBase

function w1(a::AbstractVector, b::AbstractVector)
    a = filter(isfinite, a)
    b = filter(isfinite, b)
    if isempty(a) || isempty(b)
        return Inf
    end
    q = range(0.0, 1.0, length = 100)
    return mean(abs.(quantile(a, q) .- quantile(b, q)))
end