There is no neutral way to choose between theories

We can only take a theory on its own terms and see how it holds up

I have previously argued that the foundations of knowledge are plural abstracted representations of the world - theories, stories, pictures, models, principles or systems - rather than individual facts. This explains some useful things, like the way that facts don't (immediately) change people's minds. But it raises a following question: how do we decide between competing representations? Or to narrow down a little, how do we choose between different scientific theories?

Common advice is that we should choose the theory that is most consistent with, or explains best, the known facts. If a theory can't explain the facts, then we discount it as a strong theory. However, as noted previously, there is a fundamental weakness with this approach: what we count as facts are statements that are typically theory-laden. The terms we use in expressing facts are often products of some theory, and we generally need to use some kind of theory to demonstrate they are facts - including in how we measure or analyse. There isn't a class of 'theory neutral' facts we can drawn on.

An, admittedly slightly extreme, example might make this clearer. In the book The Primacy of Doubt, that I looked at last year, Tim Palmer suggests that truly understanding the chaotic and stochastic nature of the world requires us to use a different number system: p-adic numbers. Let's take this suggestion seriously, without worrying about any of the details. It means that we have two competing theories to explain physics and various real world phenomena, each based on a different number system and a different way of doing the mathematics at the heart of physics. Depending on which theory we start with, many of our results, ways of measuring data, and how we do analysis will change. In other words, the scientific results we count as basic facts could be different in the two different theories (due to using different number systems).

These dynamics are common when we are looking at competing theories or schools of thought. Different theories often do not agree on what the facts are or take the same written statements of fact to mean very different things. So 'the facts' depend crucially on which set of theories and systems you adhere to. But that leaves us with difficult question: how do we decide between different theories or abstract representations if we can't rely on facts? How do we judge between them?

Predictions, not facts

Theorists of science like Thomas Kuhn, Paul Feyerabend and Bruno LaTour sought to answer these questions by looking at the history and practices of science. However, as is often the case when humans are involved, what we actually do is somewhat at odds with what we say we do or think we should do.

A more useful answer arises when we consider the broader logical structure that science works with - as refined over time and with experience. Over time, scientists moved from an inductive approach built on collecting demonstrable facts (as advocated by Francis Bacon) to postulating theories and then testing them against experiment and experience. On this way of looking at things, what happens when we have two (or more) competing theories?

The ideal, which is often advocated, is that we identify what the different theories predict about the world and then run experiments or tests to see which theory predicts correctly.¹ Obviously this requires the different theories to make different predictions - but that is a fairly trivial requirement. If two theories always make the same predictions, it is hard to argue that they are genuinely different theories.

It is often assumed that choosing between theories by running different experiments requires some neutral, theory-independent arena or the experiments produce some set of neutral facts. However, the logic structure of the approach doesn't require this and in many ways works better if it doesn’t hold.

Let's go back to the example where we have two competing number systems underpinning different theories of physics. In this example, there is no neutral number system from which we can run the calculations. However, the number system is part of the theory we are testing, so we can let each competing theory run its own calculations on the experiment and see whether the experiment confirms or disproves their prediction, using the resources or assumptions from within the theory. We can let each theory define the predicted outcomes and then test whether the experiment meets those outcomes.

If thinking about different number systems is too abstract, research in psychology offers another example. There are a range of different theories to describe and explain personality, from psycho-analytic approaches following Freud, to personality trait explanations like the ‘Big 5’, through behaviorism and social learning approaches. These theories define terms and concepts by which researchers identify and research personality effects. This means that all research has to be based on one of these various theories, which drives experiment design, survey questions, data collection and so on. There is no neutral set of data that is independent of these theories. Instead we have to consider how well each of the theories does at explaining what it intends to explain in the real world.

As an aside, it is possible that two theories can use different words or numbers to describe exactly the same real world situation or result. If we are generous, we often discover after arguing with someone that we actually agree but we were just using different words.

Testing a theory depends on that theory

This means that the tests by which we decide between theories are significantly determined by internal factors within each theory. Does the theory stack up as a good explanation and prediction of the world, according to its own structure, principles and assumptions? This means that, in the end, when we are comparing two theories, we are testing them in parallel. Each offers an abstract representation of the world. So how well does each work according to the logic and standards it sets for itself?

To see how this plays out, we can consider two competing theories of how to live well - based on self-help stereotypes. One approach argues that the world is hopelessly tragic and it will help you accept the world for what it is, become resigned to the realities of the world and therefore be at peace. Another approach claims that our unconscious desires get in our way and through exercise and meditation we can achieve ongoing happiness.

If someone was to say that the first approach is nonsense because it doesn't make them happy, that critique clearly misses the mark. The approach doesn't promise (or predict) happiness - it presumably argues that happiness is in fact impossible. However, if you looked at followers of the first approach and found that few of them had achieved the internal peace promised, then you have a valid reason to critique it. Likewise, the strongest critique of the second approach would be that it doesn't work, based on the promises (i.e. predictions) it itself makes.

We often hear critiques of different theories based on arguments like, to follow another self-help stereotype, ‘neither of these theories adequately account for the role of childhood trauma’. According to our account here, this type of critique isn’t directly showing either of the previous accounts is wrong. Instead, it is an argument that there is a better theory that depends on incorporating childhood trauma, and this third theory explains the world better.

Simple logic, but harder in practice

Even if this account of how knowledge formation and testing theories or ideas sounds fairly simple in theory, in our day-to-day lives it is far more complicated. None of us work with clearly logical and coherent theories or pictures of the world that apply universally. We all rely on a constellation of overlapping stories, worldviews, scientific theories and personal principles that fit together in complicated ways. And our societal and cultural approaches are just as jumbled.

In social settings, we often rely on a set of agreed facts that appear neutral and objective. This appearance is somewhat misleading. The facts arise because we have all accepted the relevant theories and background as trustworthy. This doesn't change the fact that there is no neutral, objective standpoint from which to judge whether the theories are true or not. Instead, these theories have been repeatedly tested (using their internal criteria) by our relevant social group against the world and judged as trustworthy.

These social dynamics, which I’ll explore more in the future, don't detract from the simple core principle: the only definitive tests of whether a theory or account of the world rely on factors internal to the theory. In other words, what matters is what the theory predicts about the world (on its own terms) and whether this matches up against experience, evidence and experiment.

To put it differently, the only real way to determine whether a theory or account is reliable or not is to take it seriously and see if it stacks up. As a practical point, this is impossible for anyone to do with every possible idea or theory, so it is another reminder that knowledge is hard and it is easy to get it wrong.

1. Predictions, in this sense, aren’t only about the future. You may predict something and then go through a range of existing evidence to see if it stacks up. Historians often use this approach when looking through evidence - they have predictions about what they expect to find if their account is correct, and then look to see whether the evidence supports it.