In mathematical modeling, overfitting is "the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably". An overfitted model is a mathematical model that contains more parameters than can be justified by the data. In a mathematical sense, these parameters represent the degree of a polynomial. The essence of overfitting is to have unknowingly extracted some of the residual variation as if that variation represented underlying model structure.
A photograph of Anne Graham Lotz included in the training set of Stable Diffusion, a text-to-image model
In philosophy, Occam's razor is the problem-solving principle that recommends searching for explanations constructed with the smallest possible set of elements. It is also known as the principle of parsimony or the law of parsimony. Attributed to William of Ockham, a 14th-century English philosopher and theologian, it is frequently cited as Entia non sunt multiplicanda praeter necessitatem, which translates as "Entities must not be multiplied beyond necessity", although Occam never used these exact words. Popularly, the principle is sometimes paraphrased as "The simplest explanation is usually the best one."
Part of a page from John Duns Scotus's book Commentaria oxoniensia ad IV libros magistri Sententiarus, showing the words: "Pluralitas non est ponenda sine necessitate", i.e., "Plurality is not to be posited without necessity"
Manuscript illustration of William of Ockham
Possible explanations can become needlessly complex. It might be coherent, for instance, to add the involvement of leprechauns to any explanation, but Occam's razor would prevent such additions unless they were necessary.
Andreas Cellarius's illustration of the Copernican system, from the Harmonia Macrocosmica (1660). Future positions of the sun, moon and other solar system bodies can be calculated using a geocentric model (the earth is at the centre) or using a heliocentric model (the sun is at the centre). Both work, but the geocentric model arrives at the same conclusions through a much more complex system of calculations than the heliocentric model. This was pointed out in a preface to
