The Law of Large Numbers

“Another mistaken notion connected with the law of large numbers is the idea that an event is more or less likely to occur because it has or has not happened recently. The idea that the odds of an event with a fixed probability increase or decrease depending on recent occurrences of the event is called the gamblers fallacy. For example, if Kerrich landed, say, 44 heads in the first 100 tosses, the coin would not develop a bias towards the tails in order to catch up! Thats what is at the root of such ideas as “her luck has run out” and “He is due.” That does not happen. For what its worth, a good streak doesnt jinx you, and a bad one, unfortunately , does not mean better luck is in store.”

Leonard Mlodinow

 

 

Law of Large Numbers - Old Adding Machine


Photo by Maik Usinger from Pexels

 

Law of Large Numbers (LLN) states that as the number of observations of an event increases, the observed probability approaches to the expected value. For instance, if you roll a fair coin many times, the observed probability of getting Heads or Tails will converge to the true value, which is 0.5 in this experiment.

More formally, suppose that we have n i.i.d variables (a1, a2, …., an) from the same distribution with mean µ. Let ā be the average of these variables. Law of Large Numbers tells us that as n increases, the probability that ā converges to µ gets closer and closer to 1, or:

Pr (|ā – µ| < ε) = 1

The term Law of Large Numbers was first used by Simeon Denis Poisson in the 19th century, but the concept was well-known from the early 16th Century in the works of the Italian mathematician Gerolamo Cardano, but without proofs. 

It is clear … that he [Cardano] is aware of the so-called law of large numbers in its most rudimentary form. His mathematics belongs to the period antedating the expression by means of formulas, so that he is not able to express the law explicitly in this way, but he uses it as follows: when the probability for an event is p then by a large number n of repetitions the number of times it will occur does not lie far from the value m = np.

Øystein Ore – Cardano: The Gambling Scholar

 

Near two centuries later, the Swiss mathematician Jacob Bernoulli published the first formal proof for the Law of Large numbers in his book ‘Ars Conjectandi’, after devoting twenty years of his life to this issue:

“This is therefore the problem that I now wish to publish here, having considered it closely for a period of twenty years, and it is a problem of which the novelty as well as the high utility together with its grave difficulty exceed in value all the remaining chapters of my doctrine. Before I treat of this “Golden Theorem” I will show that a few objections, which certain learned men have raised against my propositions, are not valid.”

 

Law of Large Numbers - Big Bang Theory

Big Bang Theory – Season 1 – The Luminous Fish Effect

 

My favorite example of the law of large numbers in the pop culture is when Sheldon needs eggs for his breakfast and also his scientific experiment. Penny knocks and says she is going shopping and if he needs anything. Sheldon replies:

“Oh, well, this would be one of those circumstances that people unfamiliar with the law of large numbers would call a coincidence.”

 

For an example of the Law of Large Numbers, see the dice roll experiment with Python.