What do I mean when I talk about uncertainty and likelihoods?

I talk a lot about uncertainty and likelihood, however I don’t often make clear what I mean. For example, if I offer a view on how likely a no-deal Brexit is, but what does my answer even mean? I decided to attempt to answer that question, hopefully in a way that makes sense.

Some philosophical groundwork:

Firstly, I am taking for granted that there is a physical reality, and real ‘mechanisms’ that take us from one state of the universe to later ones.

Secondly, I do not believe we are capable of perfectly knowing the current state or the mechanisms. However, that is not to say that what knowledge we have is useless, rather, I believe it can be very useful in forming expectations of what will happen.

I suspect these views accord fairly well with common-sense, so hopefully they won’t upset too many readers.

Fundamental uncertainty

Based on our lack of knowledge of the state of the universe and of the real mechanisms at work, I believe there is always some level of uncertainty as to what will happen or indeed what has happened. I call this fundamental uncertainty (my label — happy to hear if there’s a better term in the literature). This uncertainty is mostly related to our lack of knowledge (what philosophers call epistemological). I’m open to the existence of some degree of physical randomness, perhaps at the quantum level, but even if the world is deterministic, there would still be considerable fundamental uncertainty.

Unfortunately, fundamental uncertainty is extremely difficult to work with, as we never know how much we don’t know. There are two common approaches to overcoming this problem: making assumptions and accepting subjectivity.

Assumption-based uncertainty

The first way of coping with fundamental uncertainty is to make assumptions about the state of the universe or of the mechanisms at work.

These assumptions may be deterministic, in which case they will lead to definite predictions. For example, we can assume a certain starting point, and assume negligibility of certain influences, and then deduce a definite outcome.

Alternatively, the assumptions we make can be probabilistic, in which case we can calculate probabilistic predictions. For example, we can assume that a coin toss leads to heads with 50% probability irrespective of starting position, or that a hidden playing card has a 1/52 chance of being the Queen of Hearts.

If we agree on our assumptions, we should be able to agree on our conclusions. It is even possible that multiple people will have different assumptions which still lead to the same conclusions. In both of these cases, I believe we can talk about objective uncertainty, and to quantify it.

However, it is important to note that our predictions are only as good as the assumptions we make. It is also the case that if people cannot agree on assumptions, they may be unable to agree on quantifications of the uncertainties. Finally, in many cases it will be impossible to come up with a suitable set of assumptions, preventing us from coming up with objective probabilities.

Subjective uncertainty

The second approach is to allow subjectivity. Here, we accept that different people form their own view of how likely different outcomes are, without making any of the assumptions that were needed to calculate an objective assumption-based uncertainty. This method has its appeal in a lot of contexts where we have no idea what sort of assumptions to make, for example ones that depend on people’s actions.

However, I’m never sure how good people’s subjective estimates of uncertainty are. It seems entirely likely that they make assumptions just like in assumptions-based uncertainties, just without acknowledging them. Subjective estimates also seem to depend a lot how you think about the question. One common technique for eliciting subjective uncertainties is to observe what kinds of bets people are willing to make, however this leads to a blurring of people’s likelihood estimates with their attitude to risk and wealth.

Another concern with subjective estimates of uncertainty is that they limit our ability to judge whose estimates are best. We can perhaps highlight inconsistencies, or try to convince people to change their mind, but it is there is no objective way to judge a person’s subjective estimates as wrong.

Conclusion

In practice, most of the likelihood estimates I give are both subjective and based on assumptions. When I’m discussing politics or human behaviour, I am unlikely to accept any set of assumptions that would be simple enough to generate an objective estimate likelihood. And I can guarantee I don’t have the knowledge to make my subjective estimate free of implicit assumptions.

While it is important to acknowledge that your likelihood estimates are subjective, and therefore not right or wrong, they are still worthwhile exercises in thinking about the future. Sure, I sympathise with those that are sick of thinking about all the possible Brexit outcomes, but I’m not likely to stop.

And I would acknowledge the value of making assumptions explicit, even once you’ve accepted that your estimate is subjective. This gives me the chance to correct for any biases in my estimate. And it allows me to share the rationale for my subjective estimate, allowing the conversation that may result in improving my estimates or those of others.