More information isn't always better
We should aim to match the level of detail and data to the problem at hand
It is often argued that AI and digital techniques will be superior to human approaches because they can absorb and analyse vastly more information and data. This connects to a broader cultural assumption - that more information will always help us understand what is going on and make better decisions. This has been backed up by the way many fields, including various sports, have undergone data revolutions as they have pursued more granular and sophisticated information.
However, despite this seeming success, if we think seriously about the structure of knowledge and our decision-making processes, more information isn't always better. This is a short post to motivate the idea that sometimes we don't need more detail and therefore we should be thinking carefully about what information we need and when.
Beginning with an example: gravity
Let's motivate the idea with a simple example - our understanding of gravity. We all know that any unsupported object will fall towards the ground. This statement is a perfectly useful and correct theory of gravity for most regular, day-to-day situations. It is all I need to know to be able to successfully decorate a room, stop vases from breaking or try to throw something into the bin in the corner of the room.
Nevertheless, physics easily gives us a theory with more detail: all unsupported objects accelerate towards the centre of the earth at a rate of ~ 9.8 m/s2 (absent any significant air resistance effects). This is more detailed and more accurate, but is it more helpful? Often it isn’t and may even be unhelpful. For example, trying to incorporate this more detailed theory into my attempt to stop a glass smashing on the floor will slow me down and reduce my chances at success. The extra detail is hard to incorporate and won't give me any extra information that is relevant to achieving my goal.
However, if I'm designing an aeroplane or a bridge, the more detailed description of gravity that physics gives us is essential and needs to be incorporated. But, again, physics gives us extra layers of detail that aren't necessarily helpful, for example the General Theory of Relativity and the effect of gravity on the shape of space-time. That extra detail is, again, not useful in designing the wings on a plane, even if it is useful in other contexts.
So we have multiple layers of theories that are useful in different contexts. Moreover, as argued on multiple occasions before, all of these theories are incomplete and are more like sketches of reality than perfectly detailed and accurate descriptions. The different sketches are accurate at different scales of resolution - and therefore we need to match the resolution with the question or task at hand. More detailed theories, at a higher scale of resolution, are often not helpful in understanding or solving things at a less granular scale.
Knowledge isn’t built on details
This example involving gravity shows that, at the very least, there can be pragmatic reasons why more information and detail isn't better. There are many cases where including the extra information makes the analysis and decision-making far more onerous and complex, without giving any real benefits. Perhaps the extra work might give us more confidence, but is more likely to slow everything down and get in the way of practical achievement.
There are, however, more foundational or structural reasons why more information is not always better.
As noted in a previous post, we do not routinely or systematically build knowledge from the bottom up via facts and evidence. Instead, there is a dynamic process by which we formulate higher level theories or representations and test them against the data and evidence. Our theories might be inspired by the data, but they only become knowledge once a theory is confirmed by further evidence. In other words, simply collecting more information, evidence or data does not directly lead to more knowledge. It is only happens when that information reveals some flaws in our existing theories and then we identify a way to improve or replace our theories with better ones.
In practice, it often happens that extra information and data falls into a category that means it does not affect our knowledge. Sometimes the extra data simply conforms to our expectations and our theories, which changes nothing in our knowledge except perhaps our confidence. Other times the extra information is actually irrelevant, even if it might appear important. More seriously, if we don't have a framework or conceptual approach that enables us to incorporate the extra data and update our thinking, then it just ends up in a kind of limbo and tends to be ignored. Adding any more data from these categories, and there is typically a lot, adds nothing of value to your analysis.
The nature of many of the systems we are trying to understand also matters. As noted in my post on complexity science, there are many systems that can't be understood by breaking them down into their constituent parts. The technical term is that these systems have 'emergent' properties that can't be understood by analysing all the details and parts of the system - you have to understand the system as a whole.
To pick one example, there are well known crowd dynamics in which groups of people very often behave in the same way - especially in emergency situations. The individual characteristics of the individuals in the crowd - education level, psychological profiles, fitness, etc - are rarely relevant in understanding how the whole crowd with act. Chasing more information and more detail about these systems is typically unhelpful and counter-productive as the detail is at the wrong level to explain the situation you are interested in.
The point here that more information and more detail isn't always good is not a novel one. The old saying that someone can't see the wood (or forest) for the trees captures much of the same idea. However, we live in an information and data obsessed world that often assumes the opposite: once we have all the data and information about all the trees, then we will truly understand the wood (or forest). However, there are solid reasons why more detail and more information isn't always helpful, including the structure of knowledge and the nature of many of the systems we want to understand. Instead we should think carefully about matching the level of detail and scale of the system to what we want to understand or solve.