logo

Zipf's Law 📂Algorithm

Zipf's Law

Law

Let $f_{k}$ denote the relative frequency of the $k$-th most frequently occurring word in a corpus. Then $$ f_{k} = {{C} \over {k}} $$

Explanation

Here $C$ is a normalization coefficient that makes $\displaystyle \sum_{k} f_{k} = 1$. When represented as a histogram, it roughly takes the shape above, with the scale adjusted so that the total area is exactly $1$.

LKfZK.png

The thick tail shape appearing on the right is called the long tail. Like Heaps’ law, it is an empirically obtained law, and it has the advantage of not only fitting well but also being remarkably concise.

20180509_152201.png

Meanwhile, taking the logarithm of both sides gives $\log f_{k} = \log C - \log k$, which appears to have a linear relationship. However, unlike in theory, in practice it may not hold well for words that do not appear frequently, so caution is needed.