ENTROPY AND TRUTH

Why is simplicity the measure of truth in science? 

On the face of it, it's obvious. Of two theories, both equal in the accuracy of its predictions, the simpler is obviously preferable. 

Okay, but why is the simpler preferable? A friend says, "fewer parts". That's question-begging: "fewer parts" is one definition of "simpler". The simpler is preferable because it's simpler!! But why should fewer parts be preferable? Because it's simpler. I want to show that the "fewer parts", though it can't be the answer, is a key to the answer, but it's important first to recognize that there really is a problem with simplicity=true. Short answer: scientific theories are about probability, not truth. 

So let's get started. 

"Simple" and "true" are not synonyms. They appear to be quite distinct properties. And yet Occam's Razor and theoretical economy, sometimes called theoretical beauty, parsimony -- these are all expressions of simplicity in science, and in choosing between two theories, one complex, the other simple, we scientists insist that the simpler must be true or closer to truth than the complex one even if the two theories have the same predictions. Bu' Y?

There are other metrics as well -- the relation to other theories, that is, does the theory fit smoothly into other well-established theories? -- but even this is a kind of simplicity measure. Can the different theories be reduced to a comprehensive one or do we have to tolerate many sources of prediction for many categories of phenomena? 

Looking at emergent or higher-level sciences like psychology or semiology, where there really are many theories unrelated to the reductionist science of physics, the need for the emergent laws is also driven by simplicity. Reductionist physics -- like quantum physics -- can describe the shape of a statue's nose, but not explain why it has that shape. To explain how that nose came to be we need higher-level sciences like psychology and anthropology and art history to discuss and explain the role of the artist and why an artist would sculpt a nose-like shape, a question that quantum physics can't answer but that these higher level sciences can, sciences that are in many ways independent of physics (upcoming post "information faster than light"). The emergent science -- whether psychology or the anthropology of arts -- is not adding complexity, it's winnowing away the irrelevant complications of the reductionist details. 

A reductionist physical explanation might work if it started with the origin of the universe, detailing every moment until it arrived at the sculpting of the nose, but such a description would not only be a huge task, it would have to have developed all the emergent sciences along the way, and it will be those emergent properties that will explain the nose, not the physics, so the entire project would be useless. We already have those emergent sciences, so it's just simpler to cut to the chase. It's the quandary of reductionism described in Borges "On the Exactitude in Science" where the map is as large as the city it maps. It's useless. Reductionism isn't useful in understanding emergent properties like statues and cities. It's mere description. We want explanation. 

And there's actually no reason to believe that physics alone would be able reconstruct the emergent properties that explain the form of a statue. After all, historically the sciences of psychology or neurology did not begin with physics. They began with observing emergent entities like human behaviors and brains and cities and economies. Again, if we lived in a simplistic LaPlacean dream, we could retrace the world to its original state, but that would take us many billions of years before we'd provide an explanatory chain of the entire development of that bronze nose and we'd still need those emergent sciences.

Emergent sciences, though less accurate and less predictive than an entire reductionist history of the world, are vastly simpler. There it is again.

Notice that here simplicity overrides accuracy (truth), since the emergent sciences don't explain or even describe the microstates of the nose, just its emergent properties, the most important of which is that viewers will recognize that it's a representation of a nose. In fact, the microstates are often irrelevant. Whether the bronze has a high or low nickel content is irrelevant to the viewer.  The value of an emergent science is the simplicity of its understandings (commonly called "explanations") at the expense of truth. IOW, simplicity in emergent sciences is preferable to truth. lol

So I was puzzled, in my stupid, slow way, by this prima facie obvious equivalence between truth and simplicity in the sciences -- Occam's Razor, parsimony, elegance, beauty. It seems so obvious and natural, yet when you think about it, why should it be so? The more you think about it, the more you realize that truth shouldn't have any relation to simplicity. These two, simple and true, are clearly not synonyms, so why the relation? 

I like asking stupid questions. They often lead to the most surprising answers. Why does gravity work over distances? Space-time. Why does time go in one direction? Probability in an entropic universe. There are stupid questions about politics, and stupid questions about psychology and stupid questions about language Their answers reveal the most wonderful surprises about political views, psychology and language. Judging crowds out learning, and so does assuming the obvious. 

It occurred to me recently that there is an obvious answer to the simplicity metric in science: simple things are more probable than complicated things, all other things equal. In an entropic world, where even time's arrow is nothing more than the probability of increasing entropy is vastly higher than decreasing (though I have questions about this in a system being fed by an energy source like the sun; why isn't the direction of time reversed in natural selection evolution from simpler high entropy to lower entropic complexity? -- seems like the time's arrow theory is question begging too), a likelihood metric for the sciences makes perfect sense. Coordinating complexes is vastly much less likely than coordinating fewer elements. The math is exponential for just the number of elements, and if there's an ordering of elements, the possibilities are even greater and if there's any recursion in the elements, the possibilities are infinite. The simplicity metric is really just a probability metric in an entropic universe. It's not just fewer parts. It's that fewer parts are more likely to be coordinated, given entropy. This answer is not question-begging. It's a practical result. 

I should re-emphasize that the reason this simplicity metric works is not because simpler theories predict better. The question is, why prefer a simple theory in cases where a more complex one predicts just as well, exactly as well? That's the puzzle question. Now, it's also true that simple means or theories are more likely to predict better because nature prefers simplicity, but that is a different stupid puzzle, though the answer is the same: entropy.

And remember, you can always jimmy a theory to make it work. It'll get more complex, but it'll work. What's wrong with the theory, then, if it really does work? It's less likely that in an entropic world such complexity would would be its foundation. That doesn't refute the complex theory. It just makes the theory less likely -- the theory will be less likely,  not its predictions, which will be the same. 

This explanation of why theoretical simplicity is preferable has several interesting implications. For one, it implies that we're not assessing truth in science at all. We're assessing likelihoods. I believe that those of us in the sciences already hold this view, but it might be good to be more explicit about it. Scientists are well aware that science is not a body of knowledge but an ongoing investigation, and any scientific theory is not The Truth but merely the best approximation to The Truth currently available. Maybe we should shift that a little bit: a successful scientific theory is the most likely theory currently available. No mention of truth is needed. It's just likelihood -- its predictions are more likely, but even when the predictions are the same as more complicated theories, its terms are simpler, so the theory itself is more likely than an equally predictive but more complex theory. It's all just probability. 

Another interesting implication: it aligns theoretical value with two other basics of physics, time and the path of least resistance. All three can be subsumed under probability: time, path of least resistance, theoretical simplicity. 

All three of these share a kind of puzzling obviousness. Why should time progress in only one direction? Why should nature's path be the simplest and least obstructive? Why should truth be simple? Three stupid-sounding questions. How could these phenomena be any different that they obviously are? 

Well, they are not stupid questions and their answers seem to be the same or closely related. There is no actual physical law of time's direction, no law of path of least resistance and nothing specially truer about simpler theories. It's all just the probabilities of entropy or the probabilities of paths converging. The mathematics that explain the convergence to the path of least resistance is over my head, but I can understand enough of Feynman to see that this is again a reductionist probability answer. And for time, the probabilities don't require any math to understand. It is vastly more likely that a dropped egg will break into a mostly random mess than a broken egg will jump up and put itself together into the highly ordered egg state that took millions of years of natural selection to develop. As I learned when I was teaching undergrads, the possible wrong answers are infinite, right answers are few, so two identical wildly wrong answers in submissions of two students can be a red flag of cheating, if the exam question was not a trick. Entropy explains all, and it's all probability.

Talk about tricks, the title of this post is one. The post's actual content is about simplicity and entropy. It's intent is not to explain truth at all, but to replace truth in the sciences with probability. That's not to abandon all truth. There are two uses of truth (or "true"), one metaphysical used explicitly in the sciences, and another tacitly used in communication -- in conversation. There are many posts on this blog dealing with the communication function of truth and "true", in machine learning, in the logic of speech and the logic of conversation. The subtitle of this blog (de-perplex) mentions "interactive cognition" and the communication function of truth is one case of it. The metaphysical or scientific sense of "true" can be replaced with probability, but the interactive, communicative use of truth cannot be abandoned. 

Probably.