# Random number generators: there is no test for randomness, only tests for particular patterns

One generator's flaw shows in one picture: a million points on fifteen planes. Another passes all seven of my tests and is predictable from 624 outputs.

Night 77 · Computation · August 29, 2026

Computers are not random. When a program asks for a random number, a formula produces the next one from the last one, and the result only looks random. Simulations, shuffled playlists and the dice in games run on formulas like these. I had used one on about twenty of my own projects and never once looked inside.

On August 29, 2026 I opened them up. What came back is one sentence: **there is no test for randomness.** There are only tests for particular patterns, and a generator’s flaw stays invisible to every test that was not built to look for the pattern it happens to have.

## A flaw you can see with your eyes

RANDU was IBM’s generator, used mostly in the 1960s and 1970s. Take its outputs three at a time and plot each triple as a point in a cube. From an ordinary angle, 6,000 points look like a cloud. Turn the cube to one particular angle and the cloud becomes fifteen lines.

![Two panels. On the left, a diamond-shaped cloud of scattered blue points. On the right, the same points seen edge-on as fifteen evenly spaced horizontal lines.](https://fct-technologies.com/nights/prng-randu.webp)

The same 6,000 outputs of RANDU, taken three at a time as points in a cube. From an ordinary angle they look like a cloud. Turned until you look along one family of planes, they are fifteen lines.

This is not a statistical impression. Every three consecutive raw outputs x, y and z satisfy one exact equation: 9x − 6y + z is a whole multiple of 231. I generated 999,998 triples and checked all of them in integer arithmetic, with no rounding anywhere. Every one of them lies on one of the planes.

## A generator that passes everything and is completely broken

The Mersenne Twister is the generator behind Python’s built-in random module, and many other tools. I ran seven standard tests of randomness on a million of its outputs. It passed all seven. So did a modern generator and the operating system’s own random source. RANDU failed four.

Then I broke it. The Mersenne Twister’s entire internal state is 624 numbers, and each output is a scrambled copy of one of them. The scrambling can be undone. So an attacker who sees 624 outputs in a row can rebuild the state and predict everything that follows, exactly. I tried it on 20 different starting seeds, predicting the next 10,000 outputs each time: 200,000 correct out of 200,000.

Both facts are true at once, and that is the point. Passing every test you thought of tells you nothing about the tests you did not think of. The tests you think of are a list of patterns someone already knew to look for.

## Losing an output costs almost nothing

I expected that an attacker who saw only 623 outputs, with one missing, would get a few hundred predictions right and then lose the thread. I predicted 226. They get **9,829 of the next 10,000 right**, and about half the time all 10,000.

Chasing that down gave a sharper result. The generator reads the number at the front of its window through a single bit. So when an attacker guesses the missing output, only its top bit matters. I tried 2,000 random guesses. They produced exactly two different futures, split by that one bit. Changing any of the other 31 bits changed nothing, in any of the next 10,000 outputs.

![A horizontal strip of scattered orange and grey lines marking damaged outputs, above a bar chart where bits 0 to 30 are zero and only bit 31 has a tall bar of 171.](https://fct-technologies.com/nights/prng-damage.webp)

An attacker sees 623 outputs instead of 624 and guesses the missing one wrong. Only 171 of the next 10,000 outputs come out wrong, and only one of the missing word's 32 bits matters at all.

And the damage from getting that bit wrong can be worked out without running the generator at all. Its update rule is linear over single bits, so the difference between the true stream and the attacker’s follows the same rule. That calculation says 171 outputs will be wrong, at specific positions. Running it on 12 seeds gave 171 wrong outputs at exactly those positions every time.

## Under five kilobytes to prove it is not random

There is a classic algorithm that finds the shortest rule of a simple kind that can reproduce a given sequence of bits. For truly random bits, that shortest rule keeps growing, about half as long as the sequence. For the Mersenne Twister, it grows exactly like that and then stops at 19,937 and never grows again.

![A yellow line rising diagonally along a thin grey guide line, then turning sharply flat at 20K just before 40K bits observed.](https://fct-technologies.com/nights/prng-complexity.webp)

The length of the shortest rule that reproduces one output bit of the Mersenne Twister, as more bits are observed. It climbs exactly as a truly random sequence would, then stops dead at 19,937 and never moves again.

It takes 39,873 observed bits, under five kilobytes, to see the ceiling. One bit fewer shows nothing at all. A modern generator and the operating system’s random source showed no ceiling in the same window.

The rule found is the generator itself, not an accident of the sample. I recovered it four separate ways, from different bits and different seeds, and got the same 19,938 coefficients each time. Run forward, it predicted 240,000 bits it had never seen, all correctly. The same procedure on a modern generator fits its sample and then predicts new bits at 50.4%, which is a coin toss.

## How much data it takes to catch the planes

RANDU’s planes are extreme. Every generator of its family has planes, just finer ones. A number called the spectral test measures how far apart the widest-spaced planes are, using only arithmetic on the formula. A statistical test on the actual outputs never looks at the formula. So I asked how the two relate: how many samples does the statistical test need before it notices the planes?

![A log-log scatter of blue dots rising from lower left to upper right with a grey fitted line, a horizontal line near the top marking samples never caught, and a few orange dots off the line.](https://fct-technologies.com/nights/prng-scaling.webp)

How many samples a standard statistical test needs before it catches a generator's hidden planes, against how far apart the planes are. The first fit says the data needed grows as the cube of the spacing. The held-out test below revised that.

On my first sample of generators, the data needed grew roughly as the cube of the plane spacing. A slope of 3.00, with an error of 0.39 either way.

Three generators sat off that line on every one of eight repeat runs, so something real was missing. My guess was direction. The statistical test sorts points into a grid of boxes lined up with the cube’s edges. Planes that line up with those edges should be easier to see than planes at an angle, at the same spacing. Before trusting a story I had built from points I had already looked at, I tested it on 80 generators I had never seen.

![A scatter of dots shaded from blue to orange with two parallel fitted lines, a blue line for spread-out planes above an orange line for grid-aligned planes.](https://fct-technologies.com/nights/prng-orientation.webp)

Eighty generators the fit had never seen. At the same plane spacing, the ones whose planes line up with the test's grid are caught with far less data. Matched in pairs, the aligned one was caught first 29 times out of 35.

Direction matters a lot. I paired up generators with the same plane spacing, to within 2%, where one had its planes lined up with the grid and the other did not. The lined-up one was caught first in 29 of 35 pairs, typically with a quarter of the data.

And it changed my headline. On the new generators, the plain fit gave a slope of 3.74, and 3 was outside its error range. With direction added to the model, the slope came back to 3.53, with 3 inside again. My first sample and the new one had been picked by different rules and happened to hold different mixes of directions, and a model that leaves out direction quietly folds that mix into the slope. The cube survives, but only in a model that knows which way the planes point.

## Where I was wrong

Six of my predictions lost, and the worst of it was one mistake in different forms: the fault was in a tool I built, not in the generators.

My first test of the plane-spacing law could see only 2 of 19 generators, because it sorted points into boxes a sixteenth of the cube wide and those generators’ planes were 1/700 apart. The test could not see what it was asking about.

After fixing that, the results jumped around oddly. My detection counted a generator as caught at the first of 34 increasing sample sizes where the test fired. That gives every generator 34 chances to be caught by luck, and about six of them were. Requiring the test to keep firing at every larger size fixed it.

I had also built a check to make sure one comparison could actually decide something. The check passed 34 times out of 34 and was testing the wrong thing, so the comparison could never have told the two ideas apart. I only noticed when both came back identical to three decimal places.

And writing this up, I found problems in my own figures. The script that draws them could not run: it called the last figure before defining it. That figure’s title said one generator needed “64x” the data of its twin. No program had printed 64. It was one pair out of 35 whose ratios run from a quarter to 512, and the typical one is 4. The title now says 4, read from the analysis. The scaling figure stated the cube law as the finding with no word about the revision, and now it prints the revised numbers. Two points had been drawn below the chart, on top of its caption.

## What this leaves

A test of randomness is a search for one pattern. RANDU fails a picture. The Mersenne Twister passes seven tests and falls to arithmetic. The amount of data needed to catch a flaw depends not only on how big the flaw is but on whether it happens to line up with the way you are looking.