Okay, let’s try a game. Lebron James releases his jump shot and it goes in. Thousands of fans rise and exultation. The announcer confidently exclaims that Lebron shooting hand is on fire. Did Lebron make that shot because he had the hot hand? No, he made that shot because he is a very good shooter and a random process happen to go his way.
As a final example let’s move to the world of finance, specifically the world of forensic accounting. When an individual or corporation is suspected of financial fraud, a forensic accountant evaluates their financial records. In many cases those financial records have been doctored, the perpetrator just tries to destroy evidence by replacing the illicit transactions with random numbers representing seemingly innocuous transactions.
How can one determine whether a set of transactions might be fraudulent? One surprising tool comes from the nature of randomness itself. In the 1930s the physicist Frank Benford collected measurements of a wide range of phenomena: the populations of cities, the size of rivers even hitting statistics of baseball players.
For every measurement he wrote down the first digit, for example if a city had a population of 1000, 10,000, 1 million zero down one, if the population was 2000 etc. He wrote down 2. He repeated this for thousands and thousands of data points and then plotted how often each from 1 to 9 came up as a first digit. We might expect that each would be the first digit equally often or about 11% of the time, but that’s not what Benford found.
Instead the numeral 1 was the most frequent, about 31% of the time and each succeeding was less and less frequent with 9 only during the first digit about 5% of the time. This relationship has become known as Benford’s law to understand Benford’s law. Better let’s return to the financial domain.
A powerful tool for forensic accountants
Consider a large investment bank that holds the retirement funds for thousands of investors. Those investors differ in innumerable ways some are wealthier than others, some start investing earlier than others and so forth. Benford’s law predicts that more than six times as many accounts would begin with the digit one than would begin with the digit nine.
Why? Let’s assume, for simplification, that the average account increases in value by about 10% per year. An account with $10,000 will take about seven years to double the $20,000, but an account with $90,000 will grow to $100,000 in slightly more than one year. The same principle holds regardless of whether the account has hundreds of thousands or millions of dollars therein. Going from 1 to 2 requires doubling but going from 9 to 1 again requires only a little more than 10% gain.
The same reasoning applies to the populations of cities, the heights of mountains or to almost anything else that can be measured. Benford’s law has become a powerful tool for forensic accountants when people create sets of random numbers, as when an embezzler seeks to cover his tracks they try to use every numeral equally.
Fraudulent balances tend to begin with nine just as often as they begin with one but that attempt at randomness doesn’t match the sort of randomness in real account balances and thus it can be detected. Benford’s law illustrates the importance of thinking about the process the generate structure.
The process the generate structure
If events come from some well-defined process like the growth of money due to compounding interest then knowledge about that process can help us make decisions but if we don’t know or can’t know the generative process and patterns are unlikely to be helpful. Let me summarize today’s lecture in three short recommendations:
First, know when patterns are likely to be meaningful and when they aren’t. We often think that we can predict the future based on the patterns of the past, we are inveterate predictors we see patterns in the weather in the outcomes of sporting events in the fluctuations of the financial markets, but we humans keep trying to predict even when we shouldn’t.
We often get it exactly backwards the things that we think are random aren’t but the things that we think aren’t random are. There are several rough guidelines I can help you know when patterns are likely to be the result of a random process. In general if a pattern is very abstract, if you could arise because of many different reasons and if experts disagree about what will happen next than a pattern maybe nothing more than randomness.
You shouldn’t use it in your decision-making. When investing it’s very tempting to look for patterns in the fluctuations of market indices in stock prices, but it’s difficult to time the markets from month to month much less to find a meaningful signal in the day-to-day noise of the market. Don’t overestimate your own abilities it’s easy to see something that isn’t really there.
Think about what didn’t happen
Second, don’t think about a pattern without thinking about what didn’t happen. At the end of the previous lecture I talked about Bill Miller a mutual fund manager who became famous for beating the S&P 500 benchmark 15 years in a row and then stepped away from managing his firm central fund five years later after that same fund became one of the very worst in its class.
Did Bill Miller suddenly forget how to invest? This hot hand turn cold? No! Even this amazing streak seems to be just a pattern in randomness. What you think about what didn’t happen? There are approximately 8,000 mutual funds available to US investors, suppose that whether a fund is better or worse than average is not based on the skill of its manager but on randomness, like flipping a coin each year to see whether the fund beats the market.
Researchers have calculated the chance that any of those 8000 funds would have had a 15 year winning streak sometime in the past few decades: that chance is three in four. It might be that Bill Miller forgot how to invest, but a simpler explanation is that with so many funds to choose from there’s bound to be some very long streaks just by chance
And there’s an interesting second act in the story of Bill Miller’s investments, he subsequently co-managed a mutual fund that adopted a contrarian strategy, it sought out shares of companies whose stock prices have been hammered because of other investors doubts about their future viability, despite still having good fundamentals in the present.
We need to seek the right evidence
As the stock market rebounded strongly in 2013 Bill Miller was featured in news stories again this time because his new fund had the highest three-year returns in its class. Third and finally, look for evidence they get stronger as time passes. If a pattern is just due to randomness then as time passes it won’t recur. The evidence will not get any stronger, but is there something real the pattern will get clear and clear.
Let’s think back to that face on Mars. In the years after that first 1976 photo the NASA leadership realized that people were curious, they wanted to know more about the spot on another planet that looks so human. By the late 1990s a new spacecraft, the Mars Global Surveyor was again flying around Mars snapping photographs.
So, when the opportunity presented itself NASA adjusted the course of that spacecraft very slightly so that he could take a photo of the same rocky mesa at much much higher resolution than before. It was really an amazing photo a view of another planet with the resolution down to the meter level and it looks like a mesa not a face not an alien civilization, just rocky hills in the middle of a flat landscape.
We are naturally good at seeing patterns, even when they aren’t really there, but if we seek the right evidence we can learn what patterns are real and what aren’t and we can make better decisions as a result.