Home

Occam’s Razor/Ockham’s Razor

Occam’s razor (also spelled Ockham's razor) is a law of parsimony: the principle gives precedence to simplicity; of two competing theories, the simplest explanation of an entity is to be preferred. The principle is also expressed as ‘Entities are not to be multiplied beyond necessity’.

We can prove the validity of Occam’s razor using Bayesian reasoning. Bayesian inference inherently prefers simpler models because the likelihood, P(data|hypothesis), of more complex models is more spread out. For example, suppose that I throw n dice and tell you that the spots sum to 3. How many dice did I throw? P(3 spots|1 dice) = 1/6 = 0.1667, P(3 spots|2 dice) = 2*(1/6)*(1/6) = 0.0556, P(3 spots|3 dice) = (1/6)*(1/6)*(1/6) = 0.0046. Plugging this into Bayes formula shows that, even with equal priors, the simpler model (one die) is to be preferred.

Top 10 Papers

  1. BLUMER, A., et al., 1987. Occam's razor. Information Processing Letters. [Cited by 437] (20.89/year)
  2. FORD, A.P., et al., 1994. Alpha 1-adrenoceptor classification: sharpening Occam's razor.. Trends Pharmacol Sci. [Cited by 106] (7.62/year)
  3. JEFFERYS, W. and J. BERGER, 1992. Ockham's razor and Bayesian analysis(statistical theory for systems evaluation). American Scientist. [Cited by 81] (5.09/year)
  4. DOMINGOS, P., 1999. The Role of Occam's Razor in Knowledge Discovery. Data Mining and Knowledge Discovery. [Cited by 119] (13.35/year)
  5. MURPHY, P.M. and M.J. PAZZANI, 1997. Exploring the Decision Forest: An Empirical Investigation of Occam's Razor in Decision Tree …. Computational Learning Theory and Natural Learning Systems. [Cited by 94] (8.61/year)
  6. WEBB, G.I., 1996. Further Experimental Evidence against the Utility of Occam's Razor. Arxiv preprint cs.AI/9605101. [Cited by 78] (6.55/year)
  7. BALASUBRAMANIAN, V., 1997. Statistical Inference, Occam's Razor, and Statistical Mechanics on the Space of Probability …. Neural Computation. [Cited by 80] (7.33/year)
  8. HOLLAND, M.A., 1997. Occam's Razor Applied to Hormonology (Are Cytokinins Produced by Plants?).. Plant Physiology. [Cited by 61] (5.59/year)
  9. GLIICK, R. and A.V. KLIMOV, Workshop on Static Analysis. Occam's razor in metacomputation: the notion of a perfect process tree. [Cited by 43] (?/year)
  10. THORBURN, W.M., 1918. The Myth of Occam's Razor. Mind. [Cited by 41] (0.46/year)

Links

Bibliography

Webmaster: Martin Sewell