A
vector space is spanned by a
basis: a set of
orthogonal vectors: $e_1, e_2, \ldots e_n$, such that
$$e_i \cdot e_j = 0\ \text{for}\ i \neq j$$
This rule says that all the basis vectors are perpendicular.
That is, no basis vector $e_i$ has a component in the direction of
any other basis vector, $e_j$.
A vector $v$ in the vector space can be projected onto each
basis vector to find the contribution of that basis vector. The
contribution of $e_i$ to $v$ is
$${ v \cdot e_i \over e_i \cdot e_i }\ e_i$$
and all the contributions add up to $v$:
$$v = \sum_i { v \cdot e_i \over e_i \cdot e_i }\ e_i$$
Why is it important that $e_i \cdot e_j = 0$ for $i \neq j$?
Often the basis vectors are unit length: $|e_i| = 1$. In this
case, the denominators in the sum above are all equal to 1 and we
get
$$v = \sum_i (v \cdot e_i)\ e_i$$
The coordinates of $v$ are $( (v \cdot e_1), (v \cdot e_2), \ldots, (v \cdot e_n) )$.
A basis of a function space is a set of orthogonal functions:
$f_1(x), f_2(x), \ldots, f_n(x)$ such that
$$ f_i(x) \cdot f_j(x) = 0\ \text{for}\ i \neq j $$
In other words,
$$ \int f_i(x) f_j(x) dx = 0\ \text{for}\ i \neq j $$
Example
Let $f_1(x) = \cos x$ and $f_2(x) = \sin x$ be two basis functions.
Note that $f_1 \cdot f_2 = 0$ since $\int \cos x \sin x\ dx = 0$.
Note the distinction between the coordinates $v =
(a,b)$ and the function represented by those
coordinates, $v(x) = a \cos x + b \sin x$. (This is a bit
confusing, as we think of the coordinates in vector
spaces to be the same as the vector represented by those
coordinates ... but this is not correct.)
Suppose $v(x) = A \cos(x - \theta)$ where $A$ is the
amplitude and $\theta$ is the phase shift ($\theta > 0$ is a
right shift).
What are the coefficients of $v(x)$ in the $\langle \cos x, \sin x \rangle$ basis?
$$\begin{array}{rl}
v(x) & = a \cos x + b \sin x \\
\\
a & = \displaystyle { v(x)\ \cdot\ \cos x \over \cos x\ \cdot\ \cos x } \\
& = \displaystyle { \int_0^{2 \pi} v(x)\ \cos x\ dx \over \int_0^{2 \pi} \cos^2 x\ dx } \\
& = \displaystyle { A\ \pi \cos\theta \over \pi } \\
& = A \cos\theta \\
\\
b & = \displaystyle { v(x)\ \cdot\ \sin x \over \sin x\ \cdot\ \sin x } \\
& = \displaystyle { \int_0^{2 \pi} v(x)\ \sin x\ dx \over \int_0^{2 \pi} \sin^2 x\ dx } \\
& = \displaystyle { A\ \pi \sin\theta \over \pi } \\
& = A \sin\theta \\
\end{array}$$
$v(x) = (A \cos\theta) \cos x + (A \sin\theta) \sin x$
Here's what $A \cos(x - \theta)$ looks like in the $\langle
\cos x, \sin x \rangle$ basis:
