Complexity of models

The pictures below show least squares fits (found using Fit in Mathematica) to polynomials with progressively higher degrees and therefore progressively more parameters. Which fit should be considered best in any particular case must ultimately depend on external considerations. But since the 1980s there have been attempts to find general criteria, typically based on maximizing quantities such as -Log[p] - d (the Akaike information criterion), where p is the probability that the observed data would be generated from a given model (-Log[p] is proportional to variance in a least squares fit), and d is the number of parameters in the model.