Suppose a problem can be reduced to finding a "least squares" solution to a system of equations. Then the problem is to find the solution $x$ to

$A x = b$

that minimizes the error $|A \; x - b \:|$.

The system is usually overdetermined (i.e. has more rows than columns) because each row corresponds to an observation and there are many observations. For example, when fitting a plane to a set of points, each point is an observation and the unknown $x$ contains the coefficients of the plane equation.

So $A$ does not have an inverse and we cannot compute $x = A^{-1} \; b$.

Instead, we can do this:

$\begin{eqnarray} A \; x & = & b \\ A^T \; A \; x & = & A^T \; b \\ x & = & (A^T \; A)^{-1} \; A^T \; b \\ x & = & A^+ \; b \end{eqnarray}$

$A^+$ is the "pseudoinverse": $A^+ = (A^T \; A)^{-1} \; A^T$.

If $A$ is overdetermined (with more rows than columns) there is no solution. In this case, $x = A^+ \; b$ is the $x$ that minimizes $|A \; x - b \:|$. $A^+$ exists if the columns of $A$ are of full rank.

If $A$ is underdetermined (with more columns than rows) there are infinitely many solutions. In this case, $x = A^+ \; b$ is the solution of minimum length, $|x|$. The pseudoinverse exists if the rows are of full rank.

Problem: Find the plane $\{ x | n \cdot x = 1 \}$ that best fits a set of points, $x_1, x_2, \ldots, x_k$.

Requiring all of the $x_i$ to be on the plane yields a system of equations:

$\begin{bmatrix} \cdots x_1 \cdots \\ \cdots x_2 \cdots \\ \\ \cdots x_k \cdots \end{bmatrix} \begin{bmatrix} \vdots \\ n \\ \vdots \end{bmatrix} = \begin{bmatrix} 1 \\ 1 \\ \\ 1 \end{bmatrix}$

This is of the form $A \; n = b$, so the pseudoinverse can be used to find the $n$ that minimizes $|A \; n - b \:|$ (i.e. that minimizes the sum of squared distances between the points $x_i$ and the plane).

Problem: Find the sphere with radius $r$, centre $c$, and equation $(x - c) \cdot (x - c) = r^2$ that best fits a set of points, $x_1, x_2, \ldots, x_k$.

The sphere equation can be rewritten as

$\begin{eqnarray} x \cdot x - 2 \; x \cdot c + c \cdot c & = & r^2 \\ 2 \; x \cdot c + (r^2 - c \cdot c) & = & x \cdot x \end{eqnarray}$

Requiring all of the $x_i$ to be on the sphere yeilds a system of equations:

$\begin{bmatrix} \cdots 2 \: x_1 \cdots & & 1 \\ \cdots 2 \: x_2 \cdots & & 1 \\ \\ \cdots 2 \: x_k \cdots & & 1 \end{bmatrix} \begin{bmatrix} \vdots \\ c \\ \vdots \\ \\ r^2 - c \cdot c \end{bmatrix} = \begin{bmatrix} x_1 \cdot x_1 \\ x_2 \cdot x_2 \\ \\ x_k \cdot x_k \end{bmatrix}$

Note that the last entry in the $4 \times 1$ solution vector doesn't explicitly encode $r$. Rather, $r$ can be derived from it and the three components of $c$ in the solution vector.

Again, this is of the form $A \; x = b$, so the pseudoinverse can used to find the solution that minimizes the error between the points and the sphere.