Random typing produces the most famous law of language

Count every word in a book and sort them by how often they turn up. In English the winner is always “the”, at about seven percent of everything. Second is “of”, at roughly half that. Third is about a third of the first.

Frequency times rank comes out roughly constant. This is Zipf’s law, and the usual way to write it is that a word’s frequency goes as its rank raised to some power near minus one.

Zipf thought this proved something about meaning. His book is called Human Behavior and the Principle of Least Effort, and his claim was that the law reflects an economy between speaker and hearer, a language optimizing itself.

In 1957 George Miller pointed out that a monkey hitting typewriter keys at random produces the same law. Mandelbrot had shown it mathematically a few years earlier. If that is true, the most celebrated statistical fact about language might be a fact about nothing at all.

That is a dispute I could settle with my own hands, so I did.

Zipf’s law is real, and it is remarkably consistent

Five books from Project Gutenberg: Moby-Dick, Pride and Prejudice, War and Peace, Frankenstein, and the Sherlock Holmes stories. Different authors, three centuries, lengths from 75,000 words to 583,000.

Fitting the middle of the rank range, where the statistics are trustworthy:

Book	Words	Distinct	Exponent
Moby-Dick	219,054	16,956	1.088
Pride and Prejudice	128,565	6,729	1.112
War and Peace	583,368	17,445	1.096
Frankenstein	75,328	6,977	0.987
Sherlock Holmes	105,863	7,817	1.109

Mean 1.078, spread 0.046, and every one of those fits sits at an R-squared of 0.996 or better. One slope, five writers.

Worth noting alongside it: between a third and a half of every vocabulary appears exactly once in the whole book. In Moby-Dick 43.5 percent of Melville’s distinct words are used one time and never again.

The monkey really does produce it, and it is pure combinatorics

Now the counter-argument. Pick a keyboard with M letter keys and a space bar. Hit keys uniformly at random. Call everything between two spaces a word.

Every string of length L has exactly the same probability, and there are M^L strings of that length, so the number of possible words grows geometrically while their shared probability shrinks geometrically. That relation is a power law, and you can write its exponent down in closed form from the alphabet size and the probability of hitting space.

I measured it and it holds to about one percent. At two keys the predicted exponent is 1.286 and I measured 1.233. At five keys, 1.123 predicted and 1.113 measured. At seven, 1.102 and 1.095, with the fits sitting at an R-squared of 1.0000 to four decimal places.

So the deflationary camp is right about the thing they claim. You can get Zipf’s law with no meaning, no grammar, no speakers, and no optimization of anything. String lengths alone will do it.

But the monkey is not language, and there are three tells

Being right that a power law is cheap is not the same as being right that Zipf’s law is uninteresting. Three things separate the real thing from the mimic, and all three came out of my own data rather than out of the literature.

The shape is wrong. Because every word of a given length in monkey text has the same probability, the rank-frequency curve is not a curve. It is a staircase: enormous flat shelves, one per word length, with cliffs between them. Real text is smooth all the way down.

That is the whole dispute in one picture. Both lines obey a power law. Only one of them could have come from a language.

The exponent has a knob on it, and the knob is meaningless. The monkey’s exponent is a function of how many keys are on the keyboard: 1.29 at two keys, falling toward 1.05 at fifty. So the fact that a 26-key monkey lands near 1.08, which is about where English sits, is a coincidence of alphabet size rather than evidence of a shared cause. Surveys across a hundred languages report exponents clustered near 1, varying between languages but not with the size of the writing system. That part I read rather than measured: my own five books are all English.

The vocabulary economy is not close. This is the tell I find hardest to argue with. Reading exactly as many words as War and Peace contains, 583,368 of them:

• Tolstoy uses 17,445 distinct words.
• The monkey invents 335,853.

Real language reuses its words relentlessly. Random typing barely repeats itself, and would need astronomically more text to even sample the vocabulary it is generating.

The verdict, honestly in both directions

The skeptics are right that rank-frequency plots are treacherous and that the existence of a power law is worth very little on its own. Noise makes one. I would not now accept “it follows a power law” as evidence of anything at all without being told what the alternatives produce.

And they are wrong that this makes Zipf’s law empty. The specific exponent near one, the smoothness, and the tight reuse of a small vocabulary are real properties, and random typing reproduces none of them.

The lesson generalises past language, and it is the one I actually took away: the tell is never that order exists. It is the particular order. A mimic can produce the law. It cannot produce the shape of the law.

A second law, and a seam I left showing

While I had the corpora loaded I measured Heaps’ law, which says the number of distinct words in a text grows as the total raised to some power β. It does, cleanly: β between 0.50 and 0.62 across my five books, with R-squared above 0.99. Vocabulary grows roughly as the square root of length.

There is a textbook relation connecting the two laws, which says β should equal one over the Zipf exponent. With exponents near 1.08, that predicts about 0.92.

I measured about 0.55. That is not a small discrepancy.

The relation is not wrong, it is asymptotic: it holds in the limit, and it is most fragile exactly when the Zipf exponent sits just barely above 1, which is where every real text I measured sits. So a finite book is precisely the case where the connection between the two laws does not hold numerically.

I am leaving that in rather than dropping the measurement, because a clean piece with the awkward number removed would be a worse record of what the night found.

What the doing corrected

One methodological thing, because it cost me an hour and it was the interesting part.

My first fits of the monkey data were garbage. Fitting the full rank range gave nonsense, and so did the usual fix of throwing out the rarest words. The reason is that the staircase means thousands of words share each frequency, so rank within a shelf is arbitrary and the fit is dominated by whatever the tie-breaking did.

The fix was to stop fitting words and fit shelves: one point per word length, which is the natural unit of the monkey’s structure. That is when the exponent came out razor clean and matched the closed form.

And the thing that had been wrecking the earlier fits, the enormous flat shelf of words appearing exactly once, was not noise to be excluded. It was one of the tells. I had been treating the signal as a nuisance because it did not fit the instrument I had reached for first.