The Wasserstein distance seems to have been reinvented more often than the wheel. It pops up under various names, including “Earth Mover’s Distance”, and “Mallows Distance”. One reason people keep reinventing it is probably because it comes in quite handy when comparing two probability distributions.
There are two interpretations for the Wasserstein distance, and one involves putting stuff in a wheelbarrow and lugging it about. We’ll start with that one.
Say your company owns one mine and one factory. The mine is at location and the factory at location in Flatland, ie . You are VP for Logistics, and your job is to carry the ore from the factory to the mine. Your company kindly provided you with a wheelbarrow for that purpose.
How much work to you have to do, in the physical sense? If you carry tons of ore in a wheelbarrow, the total amount of work done is , i.e. distance times mass.
Now let’s say you have two mines (at and ) producing amount and , and two factories (at and ) that require amounts and , and assume . You need both factories to get the amount they want. You also want to minimise how much work you have to do. This latter quantity is given by a simple weighted sum:
in which is the amount of stuff going from mine to factory .
What we can already guess at this stage is that unless each factory is exactly opposite a mine, and consumes exactly what that mine produces, there’s going to be some work involved. The minimal amount of work you can do is given by:
subject to the following conditions
- (a mine can’t give more than it has)
- (a factory doesn’t take in more than it needs)
- for rather plain and boring reasons.
We immediately recognise a classical linear program (linear objective function, linear constraints), and we know that no closed-form solution is available in the general case.
Mines and factories are all well and good, but what does this have to do with probability distributions? Well, if we now assume that and , our mines and factories problem corresponds to having two discrete densities, and , such that:
The are Dirac “point-masses” at the locations of the factories and mines. At this stage there’s an obvious extension to continuous densities that we won’t get into.
The Wasserstein distance is the minimum work you have to do to “transport” onto . Why does this make sense as a distance between and ? One intuition is that before we noted that having no work to do, and therefore a distance of 0, meant that “each factory is exactly opposite a mine, and consumes exactly what that mine produces”. This just means …