Lies, Damn Lies and Probabilities
That there is a 30% chance something happens makes perfect sense, until you start looking more closely.
Recently, Sean offered an interesting hypothesis in response to a post. He wondered whether “The thinking limit of an AI reflects the limit of statistics as a way of seeing the world.” I’m not sure if the idea is right, but it seems like a significant insight. For there are a range of reasons why statistics are inherently limited as a method for understanding the world. For example, in a post on scientific research, I summarised the argument that the reliance on statistics has slowed down scientific discovery. The key worry was that “statistics encourage researchers to play with numbers and data to look for different effects, rather than focusing on attempts to genuinely understand what the real dynamics, systems or factors are.”
However, there is a particular, more philosophical, worry that I want to highlight. A core engine of modern statistics is the calculation of probabilities, but statistical probabilities are often highly imprecise and conceptually vague when we apply them to the real world. The mathematical precision of our calculations often disguises significant conceptual fluffiness which means that probabilities and statistics can never be highly precise when applied to the real world. This will necessarily impose limits on what statistics, and possibly therefore AI, can tell us.
This fuzziness exists across a few categories, and we’ll look at three in turn.
What does a probability mean in the real world?
We all encounter and consider probabilities every day but we rarely have (or need) a precise idea of what they mean. Let’s take a weather forecast as a typical case. Imagine I’ve consulted the forecast and have read that there is a 50% chance of rain in Canberra tomorrow. But what does this actually mean for the real world? There are a wide range of possible interpretations.
For a start, it is unclear what it means when we consider geography. It could be any one of these:
There is a 50% chance that it will rain somewhere across Canberra tomorrow.
There is a 50% chance that it will rain at the designated weather station that provides the official record.
For every place within the Canberra region, there is a 50% chance it will rain tomorrow.
Within the Canberra region, it will rain in 50% of places and not in the other 50%.
It is also unclear what it means about time and timing, as these are both equally valid interpretations:
At any point in time tomorrow, there is a 50% chance that it will rain.
There is a 50% chance it will rain at some point in time tomorrow.
The weather forecasters can no doubt tell us exactly which of these definitions is the one they use. But as we ordinarily use these concepts (and weather is just one example), the concepts and phrasing are highly imprecise. While this bothers some people, for almost all practical purposes this imprecision doesn’t matter. The weather forecast gives us enough information to make decisions about what to do, what clothes to wear and whether to have a picnic regardless of the precise meaning.
However, this imprecision matters if we are trying to use probabilities and statistics as part of a research project or as a method of uncovering truth. For example, how we test and validate probabilities depends crucially on the exact interpretation we have in mind. But this becomes very difficult when we only have access to the openly available statements, which rarely include the precise definitions needed for accurate research.
What actually is a probability?
While the imprecisions around the real world interpretation of a probability are worth noting, there are deeper issues. There exists significant conceptual fuzziness and deep philosophical debate about what a probability actually is or actually measures.
For a start, some people read probabilities as objective statements about the world - referred to in the academic literature as chances. Using our weather forecast example above, on this view, based on the current state of the world, it isn’t set whether it will rain tomorrow. If we could create many universes and start them in exactly the same state as our current one, in 50% of them it would rain in Canberra tomorrow, and in 50% it wouldn’t.1 This seems like a robust way of thinking about probabilities in certain cases, like flipping a coin or rolling a dice.
A different view is to see probabilities primarily as statements about human knowledge rather than as objective statements about the world - in the literature often referred to as credences. On this view, the laws of physics and weather have already locked in whether it will rain in Canberra tomorrow, but based on our best knowledge and information, the best we can do is judge that there is a 50% chance of rain. This way of thinking clearly makes sense if we are assessing the probability, say, that a certain suspect committed a murder.
You might think that these different interpretations are only relevant for philosophical debates, but they underpin different mathematical theories. Roughly speaking, the world of statistics and probabilities is split into two camps based on these different interpretations: frequentists and Bayesians.2 These two camps have different foundational formulae for the calculation of probabilities, and therefore sometimes provide different answers to statistical questions.
So we don’t agree on what a probability actually means about the real world, nor on the correct way to calculate them. Any methods that rely on probabilities for research or to uncover truth therefore need to step carefully, and this includes generative AI.
What is the right way to calculate a probability?
The question of how to calculate a probability, or run a correct statistical analysis, quickly gets very complicated, technical and philosophical if we take it seriously. There are a range of statistical tests we can use on data sets, but these all make assumptions about the nature of the data that we often cannot know in advance. Moreover, we very often use probabilities to help us make predictions - and predictions open up more philosophical and mathematical questions.
Some methods base future probabilities on properties of existing datasets but, as the financial advice tells us, past performance is not a reliable predictor of future performance.
In other cases, we rely on expert judgement to assess the probability, often by relying on the ‘wisdom of the crowd’ and asking multiple experts. Exactly when expert judgement is reliable, and why we can trust it in certain circumstances, are usually unanswered questions.
More mathematically sophisticated approaches build models of the situation under consideration and use these to assess future probabilities. But in this case the reliability of the numbers depends entirely on the accuracy of the model to capture the relevant real world phenomena.
For information, this is the approach weather forecasters use. They have highly sophisticated weather models that they then run many, many times with slight variations on the data. Technically speaking, the statement that there is a 50% chance of rain tomorrow means that the forecasters have run their model many times (at least hundreds) and in 50% of the model runs it rains tomorrow. What this precisely means about the real world depends on your views about the match between models and reality, plus all the questions above.
Vague and imprecise, but still very useful
From my sketch of the issues here, we can see there are a range of reasons why statistics and probabilities are not immediately reliable as guides to the real world - and can often cover over gaps in knowledge or information. In one sense this is a typical philosopher’s complaint, as it doesn’t reduce their usefulness in many day to day situations. We don’t need to know exactly what the 50% chance of rain means, as the fuzzy statement gives us enough information to make better decisions. A more conceptually precise formulation would often be less useful if it isn’t as easy to calculate, understand or apply.
However, it does mean there are limits to what statistics can ever teach us about the real world. As we have seen, probabilities are highly sophisticated and detailed mathematics based on fuzzy and sometimes sloppy philosophical foundations. If we are looking to do serious work to discover truth, the fuzziness becomes a problem.
Given that generative AI is fundamentally built on statistical analysis, it seems likely that these same limitations will be inherited. Despite many claims that AI will help us uncover deep truth in a neutral, objective way - the fundamentally statistical approach may be forever limited by the fuzziness of the underlying probabilities and statistical theories.
If anyone wants to be more precise, this phrasing itself is a simplification that has a number of different interpretations. For example, some read it more as a statement of past observations, meaning that in past conditions analogous to the ones currently observed, it rained in Canberra the next day 50% of the time.
Again this is a simplification of a complex topic and debate. As always, the Stanford Encyclopedia of Philosophy has a detailed summary for anyone interested.
Loved this. Obviously.