Informed about Information

When we talk about 'information' we aren't all talking about the same thing

Photo by Roman Kraft on Unsplash

Recently, I've been doing some reading about the history of information theory and cybernetics. This is a field that set many of the theoretical assumptions that lead to the development of modern digital computing and AI. In my reading, I was surprised at the standard definition of the concept of information, as it is noticeably different to the more classically philosophical ways I have thought about the concept.

The differences illuminate a number of attitudes I have taken issue with in previous articles. To simplify, there are varied fundamental attitudes towards the reliability of information writ large. The key question is whether the default assumption is that all information is reliable until proven otherwise, or whether we should not take anything as given. After this very issue came up in the comments recently, I decided an article explaining the differences was in order.

One word, two concepts

Different approaches to thinking about information have been explored in various places over the past few decades, and the Stanford Encyclopedia of Philosophy offers a very succinct summary of different theories of information. I want to focus on the first two listed, namely:

Information-A:

Knowledge, logic, what is conveyed in informative answers

Information-B:

Probabilistic, information-theoretic, measured quantitatively

The second of these, Information-B, derives from the field of Information Theory, which was founded by an article titled "A Mathematical Theory of Communication", published by Claude Shannon in 1948. This article laid many of the conceptual foundations that lead to modern computing and, accordingly, the assumptions embedded in Information Theory have framed the way many people today think about the concept of information.

Within the field, the focus is on the role of information within a broader system, thought of in terms of the transmission of discrete messages, and a primary concern is ensuring that messages are transmitted without error. So, for example, as the purpose of the messages is to inform another part of the system, it leads to an approach in which "information can be thought of as the resolution of uncertainty."¹ In other words, the role of transmitted information is to clarify something that we (or the system) want to know, or are uncertain about. Given this emphasis, the concept of information content turns into something quantifiable that measures "the level of 'surprise' of a particular outcome".²

However, it is important to understand the limits of the concept of information as defined with Information Theory. In short, "one aspect of information that Shannon’s definition explicitly does not cover is the actual content of the messages".³ By contrast, this is the explicit focus of Information-A, or what I would label the Traditional Philosophical approach to information. Importantly, it is the approach that has underpinned how I've written about information here at Humble Knowledge.

In this approach, the information content of a message is not related to surprise, uncertainty or any measure related to probability. Instead, it is equivalent to what the message tells us about the world or the way things are. If the message is a sentence, then the information content is the same as the meaning of the sentence. Information provides us with a claim as to what the world is like, to use an analogy I've used before, it describes part of a map of reality.

These two different approaches lead to strikingly different attitudes towards information if we are concerned with knowledge or truth. The Information-Theoretic framework explicitly does not deal with the content of messages, yet we cannot operate without some assumptions about the reliability of the messages we are working with. It therefore has to default to an attitude that treats all messages - all information - as equally reliable or true.

The Traditional Philosophical approach, by contrast, separates the information content from all questions of reliability or truth. On this approach, a message or statement makes a claim about how the world is, but we need to verify or judge separately whether it is an accurate or reliable description. The default attitude is to withhold any judgement about the reliability of a message until we have reason to decide one way or another.

The differences become clear when we consider a message such as "Snow is black." If we take it seriously, this is a highly surprising message and so presumably, on the Information-Theoretic approach, would have high information content. Yet we know that this message does not resolve uncertainty (which is the aim of information in this framework) as it is incorrect as a description of our world. The natural way to resolve this tension is to treat false messages as if they are not genuine information. In other words, the concept information is restricted to messages (of whatever form) that are reliable in terms of what is happening in the world.

The Traditional Philosophical approach ends in a very different place. In this framework, "Snow is black" expresses meaningful information, namely a particular form of frozen water that falls from the sky is (normally or by default) a particular colour (i.e. black). However, it happens that this information is not an accurate description of what happens on our Earth (although it could be true elsewhere). The message communicates meaningful information, but that information happens to be false.

Getting our attitudes right

These different theories of information embody two very different ways of thinking that turn on the question: when we talk about information does that mean it is reliable and genuinely informs us about the world? One challenge is that we tend to flip between these two ways of thinking in ordinary conversations without realising it. If I was to tell you that a source had given me some information about an issue I'm interested in, would you think that means the information is reliable or not? It would all depend on the context.

While the Information-Theoretic attitude to information has been highly influential in our modern world, it has limitations when questions of reliability of information or truth arise. As it deliberately puts to one side any questions of truth or reliability of the underpinning data or messages, it can describe information transmission or processing very accurately but not information reliability.

I have written previously about how more information isn't always better, yet this is an attitude we see often - particularly with data. More data is always assumed to be a good thing and big data sets are assumed to have significant economic value in and of themselves. To claim otherwise almost literally doesn't make sense if we take the Information-Theoretic definitions seriously. If information is defined in terms of the way it reduces uncertainty, then more information will always lead to a greater reduction of uncertainty. More information is always a positive. Under the Traditional Philosophical attitude, however, what matters is the quality or reliability of the information we include and we have to pay close attention to problems that arise when we keep looking at poor information.

One notable example of this distinction can be found in the way that modern Large Language Models have been trained as it underpins a weird disconnect in modern discourse. On the one hand, if a friend is doing deep research on the Internet we are likely to start worrying about what crazy ideas they will end up with. "I looked it up on the Internet" is just as often evidence someone has it wrong as they have it right. On the other hand, our strategy to create reliable Artificial Intelligence has typically involved letting an LLM go all out on the Internet and absorb everything that is available. Why is the Internet as a whole a reliable source for AI but not for humans?

The origins of these differing mindsets are in attitudes towards the reliability of information. Those who are training LLMs clearly think in Information-Theoretic terms. Information is, by default, reliable and more information always decreases uncertainties. As the Internet is the largest source of readable information ever, it is the best training source. The other attitude reflects more the Traditional Philosophical mindset. The Internet provides us with lots of information, but we need to think about whether it is reliable first, rather than just using it. Given the volume of information online, it is more likely that most of it is wrong than it is right.

As an aside, this offers a simple explanation of why LLMs regularly get information wrong - as their training data is not restricted to the subset of information that is reliable, there is no good reason to assume the outputs would always be reliable. Always adding more unreliable information doesn't automatically create something reliable.⁴

These different attitudes are also relevant to another relevant topic here: fact-checking and misinformation. I have always been intrigued by the linguistic construction of the term ‘misinformation’. It assumes that ‘information’ is good but when it gets twisted into something misleading then we can label it ‘misinformation’. This makes perfect sense if we have adopted the Information-Theoretic attitude. However, from my default Traditional Philosophical mindset around information, it is weird and unhelpful. ‘Information’ can be reliable or it can be completely false, but it is still information. There is no obvious line between information and misinformation.

Finally, and of obvious interest to me, is that the Information-Theoretic mindset implicitly assumes a particular epistemic attitude. It makes sense to assume that information is, by default, reliable if we have epistemic confidence or certainty. When we think that reliable knowledge is readily achieved, then our information will be normally reliable. On the other hand, if we adhere to epistemic humility, and believe that knowledge is hard and we often get it wrong, the Traditional Philosophical mindset is a natural fit. We have lots of information but we need to be wary about what we trust.

It is commonly accepted today that people need to learn to be more discerning and less trusting of what they find online. To put that in the terms I’ve laid out here, this means they need to drop the default Information-Theoretic mindset and think more along the lines of traditional philosophers: the information we find is never, by itself, trustworthy but always needs to be verified in some way. This, in turn, requires us all to be more humble about what we know.

1. This neat summary comes from the relevant Wikipedia article: https://en.wikipedia.org/wiki/Information_theory

2. This summary is also from Wikipedia: https://en.wikipedia.org/wiki/Information_content

3. As nicely summarised in the Stanford Encyclopedia of Philosophy: https://plato.stanford.edu/entries/information/

4. To be clear, this is a simplification and it only holds with respect to the reliability or truth of information. Adding more training data has clearly lead to improvements in language and reasoning sophistication.