Orthogonal bases

The defining feature of a set of basic forms is that it is complete, in the sense that any piece of data can be built up by adding the basic forms with appropriate weights. Most sets of basic forms used in practice also have the feature of being orthogonal, which turns out to make it particularly easy to work out the weights for a given piece of data. In 1D, a basic form a〚i〛 is just a list. Orthogonality is then the property that a〚i〛 . a〚j〛  0 for all i ≠ j. And when this property holds, the weights are given essentially just by data . a.

The concept of orthogonal bases was historically worked out first in the considerably more difficult case of continuous functions. Here a typical orthogonality property is Integrate[f[r, x] f[s, x], {x, 0, 1}]  KroneckerDelta[r, s]. As discovered by Joseph Fourier around 1810, this is satisfied for basis functions such as Sin[2 n π x]/√2.