Occam's razor is the principle that one should prefer the simplest explanation that fits the data. It is one of the foundations of the scientific method. Occam's razor has turned out to be a flawed way of managing hypotheses - but it's still not too bad an idea.Occam's razor is a special case of a more general principle: that the least useful may be profitably discarded. This idea is associated with evolution via natural selection. In honor of Darwin, I propose that we call it "Darwin's razor".
Definition of "Darwin's razor": the least useful is best discarded.
Occam famously discarded all but the shortest hypothesis. However, from a more Darwinian perspective, that's often a bad idea. For one thing you lose a diverse breeding population of hypotheses. For another, it is harder to adapt if new observations no longer fit the previously-selected hypothesis. Lastly, if you only consider the shortest hypothesis, it turns out that you can't estimate probabilities correctly.
"Darwin's razor" represents the winnowing, destructive side of Darwin's theory. I've previously referred to this as being "natural elimination". Of course, Darwinism also has a creative side: the production of new forms. However, that is hardly well described as being like the operation of a razor. "Darwin's razor" is all about eliminating the old, the unwanted and the unfit.
Occam's razor famously had only one blade. Darwin's razor is more like one of those modern multi-blade razors. This analogy works in two ways:
- Occam's razor said to only keep one hypothesis, while Darwin's razor allows you to keep a breeding population of hypotheses;
- Occam's razor only applied to one thing: hypotheses, while Darwin's razor is a universal razor that can cut through practically anything.
- Wallet full of small coins? Apply Darwin's razor.
- Too much spam? Darwin's razor can help.
- Fridge full of useless jars? Darwin's razor works there too.
- Garden full of things you never planted? Use Darwin's razor.
Occam's razor has turned out to be a somewhat-inaccurate approach. However, Darwin's razor does a much better job of surviving critical scrutiny. It isn't possible to keep all unrefuted hypotheses around (as Solomonoff induction might suggest) because there isn't enough space in the universe. You have to discard something. Darwin's razor recommends discarding the least useful stuff. It's simple and pragmatic advice that's hard to disagree with. Darwin's razor.
Posts
As you have noted the principle of Occam's razor does not seem quite right. Simple explanations may be only simple minded and wrong; some additional criteria is required. Your suggestion of Darwin's razor may be more to the point. In my view Darwinian processes follow the logic of Bayesian inference, which has been called the 'logic of science'. Instead of selecting 'simple' explanations Bayesian inference selects 'plausible' ones or perhaps, in your terminology, 'useful' ones in the sense that a plausible explanation is more likely to be useful. As E.T. Jaynes wrote below the useful aspects of Occam's razor are implied by Bayesian inference:
ReplyDeleteTo understand this, let us ask: 'How do we all decide these things intuitively?" Having observed some facts, what is the real criterion that leads us to prefer one explanation of them over another?
Suppose that two explanations, A and B, could account for some proven historical facts equally well. But A makes four assumptions, each of which seemed to us already highly plausible; while B makes only two assumptions, but they seem strained, far-fetched, and unlikely to be true. Every historian finds himself in situations like this; and he does not hesitate to opt for explanation A, although B is intuitively simpler. Thus our intuition asks, fundamentally, not how simple the hypotheses are; but rather how plausible they are.
But there is a loose connection between simplicity and plausibility, because the more complicated a set of possible hypotheses, the larger the manifold of conceivable alternatives, and so the smaller must be the prior probability of any particular hypothesis in the set.
Your comments illustrate another flaw associated with Occam's razor - that the idea of a theory being compatible with the facts is too black-and-white. In the real world, facts have some associated uncertainty. Instead of the shortest explanation that fits the facts, one generally needs to consider an ensemble of possible explanations - and consider both their simplicity and how well they are in accord with both existing observations - and reasonable inferences from those observations. Rather ironically, Occam's razor turned out to be too simple.
Delete