Eigenvectors and Eigenvalues

The eigenvectors of a matrix ${\bf A}$ are those special vectors ${\bf v}$ for which ${\bf A v} = \lambda {\bf v}$, where $\lambda$ is an associated constant (possibly complex) called the eigenvalue. Let us rearrange the eigenvalue equation to the form $({\bf A} - \lambda) {\bf v} = {\bf0}$, where ${\bf0}$ represents a vector of all zeroes (the zero vector). We may rewrite this expression using the identity matrix ${\bf I}$ to yield $({\bf A} - \lambda {\bf I}) {\bf v} = {\bf0}$, which will be more convienient for the next step. Now to solve for ${\bf v}$, multiply the left and right sides of the equation by $({\bf A} - \lambda {\bf I})$, if it exists. This yields ${\bf v} = ({\bf A} - \lambda {\bf I})^{-1} {\bf0}$.

This last equation presents a challenge. Anything multiplied by the zero vector yields the zero vector, but clearly that is a trivial solution for ${\bf v}$ that we aren't interested in. Thus, our assumption that $({\bf A} - \lambda {\bf I})^{-1}$ exists must be wrong. The eigenvalue equation ${\bf A v} = \lambda {\bf v}$ therefore has non-trivial solutions only when $({\bf A} - \lambda {\bf I})^{-1}$ does not exist. In our previous discussion of determinants, we noted that a matrix ${\bf A}$ does not have an inverse if its determinant $\vert {\bf A} \vert$ is zero. Thus, we can satisfy the eigenvalue equation for those special values of $\lambda$ such that $\vert{\bf A} - \lambda {\bf I}\vert = 0$. This is called the secular determinant, and expanding the $n \times n$ determinant gives an $n$-th degree polynomial in $\lambda$ called the secular equation or the characteristic equation. Once the roots of this equation are determined to give $n$ eigenvalues ${\lambda}$, these eigenvalues may be inserted into the eigenvalue equation, one at a time, to yield $n$ eigenvectors.

As an example, let us find the eigenvalues and eigenvectors for the $3 \times 3$ matrix

$${\bf A} = \left( \begin{array}{ccc} 1 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array} \right).$$ (25)

We begin with the secular determinant $\vert{\bf A} - \lambda {\bf I}\vert = 0$, which in this case becomes

$$\left\vert \begin{array}{ccc} 1 - \lambda & 0 & 1 \\ 0 & 1-\lambda & 0 \\ 1 & 0 & 1-\lambda \end{array} \right\vert = 0.$$ (26)

Expanding out this determinant using the rules given above for the determinants of $3 \times 3$ matrices, we obtain the following characteristic equation:

$$(1 - \lambda)^3 - (1 - \lambda) = 0$$ (27)

$$(1 - \lambda) [ (1 - \lambda)^2 - 1 ] = 0$$

$$(1 - \lambda) (\lambda) (\lambda - 2) = 0,$$

which has solutions $\lambda = 1, 0, 2$. These are the three eigenvalues of ${\bf A}$. What are the corresponsing eigenvectors? We substitute each of these eigenvalues, one at a time, into the eigenvalue equation, and solve for the $3 \times 3$ system of equations that result.

Let us begin with the eigenvalue $\lambda = 1$. Substituting this into $({\bf A} - \lambda {\bf I}) {\bf v} = {\bf0}$, we obtain

$$\left( \begin{array}{ccc} 0 & 0 & 1 \\ 0 & 0 & 0 \\ 1 & 0 & 0 \end{array} \right) \left( \begin{array}{c} v_1 \\ v_2 \\ v_3 \end{array} \right) = \left( \begin{array}{c} 0 \\ 0 \\ 0 \end{array} \right).$$ (28)

This is three equations in three unknowns, which we may rewrite as

$$v_3 = 0$$ (29)

$$0 = 0$$ (30)

$$v_1 = 0.$$ (31)

The middle equation is, of course, not particularly useful. However, we know that the components $v_1$ and $v_3$ of the eigenvector corresponding to $\lambda = 1$ are both zero, and there is no equation governing the choice of $v_2$. We are therefore free to chose any value for $v_2$, and a valid eigenvector will result. Note that any eigenvector times a constant will yield another valid eigenvector. Most frequently, we chose normalized eigenvectors by convention (such that ${\bf v} \cdot {\bf v} = 1$), so in this case we will choose $v_2 = 1$. This gives the final eigenvector

$${\bf v} = \left( \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right).$$ (32)

We can verify that this is indeed an eigenvector corresponding to the eigenvalue $\lambda = 1$ by multiplying this eigenvector by the original matrix ${\bf A}$:

$$\left( \begin{array}{ccc} 1 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 1 \end{array} \right) \left( \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right) = 1 \left( \begin{array}{c} 0 \\ 1 \\ 0 \end{array} \right),$$ (33)

that is, multiplication of ${\bf A}$ times the eigenvector yields the eigenvector again times a constant (the eigenvalue, $\lambda = 1$).

By a similar procedure, one can obtain the other two eigenvectors, which, when normalized, are

$${\bf v} (\lambda = 0) = \frac{1}{\sqrt{2}} \left( \begin{array}{c} 1 \\ 0 \\ -1 \end{array} \right) , {\bf v} (\lambda = 2) = \frac{1}{\sqrt{2}} \left( \begin{array}{c} 1 \\ 0 \\ 1 \end{array} \right).$$ (34)

Although an $n \times n$ matrix ${\bf A}$ has $n$ eigenvalues, they are not necessarily distinct. That is, one or more of the roots $\lambda$ of the characteristic equation may be identical. In this case, we say that those eigenvalues are degenerate. Determination of eigenvectors is somewhat more complicated in such a case, because there will be additional flexibility in selecting them. Consider the matrix

$$\left( \begin{array}{ccc} 1 & 0 & 1 \\ 0 & 2 & 0 \\ 1 & 0 & 1 \end{array} \right),$$ (35)

which has the characteristic equation

$$(\lambda - 2)^2 \lambda = 0,$$ (36)

with solutions $\lambda = 0, 2, 2$. Although it is no difficulty to find the eigenvector corresponding to $\lambda = 0$, the doubly-degenerate eigenvalue $\lambda = 2$ presents an additional complication. Upon substituting this value into the eigenvalue equation, we obtain

$$\left( \begin{array}{ccc} -1 & 0 & 1 \\ 0 & 0 & 0 \\ 1 & 0 & -1... ...{array} \right) \left( \begin{array}{c} v_1 \\ v_2 \\ v_3 \end{array} \right) = \left( \begin{array}{c} 0 \\ 0 \\ 0 \end{array} \right),$$ (37)

or

$$- v_1 + v_3 = 0$$ (38)

$$0 = 0$$

$$v_1 - v_3 = 0.$$

In this case, the first and third equations are equivalent (one equation is just the minus of the other), and so they are not independent. Additionally, the second equation gives no information. We therefore have only one equation to determine the three coefficients of the eigenvector. Recall that only two valid equations resulted above for our previous non-degenerate case, because any multiple of an eigenvector still yields a valid eigenvector. Here, a double degeneracy has lost us one of our equations. Hence, we only know that $v_1 = v_3$ and $v_2$ is arbitrary. Any eigenvector satisfying these rules will be satisfactory, and clearly there are an infinite number of ways to chose them, even if we require them to be normalized. Let us, somewhat arbitrarily, pick the normalized vector

$${\bf v} (\lambda = 2, {\rm first}) = \frac{1}{\sqrt{3}} \left( \begin{array}{c} 1 \\ 1 \\ 1 \end{array} \right).$$ (39)

Then, it is traditional to try to pick the second eigenvector for $\lambda = 2$ as orthogonal to the first (there are reasons for doing this, most commonly because we might wish to use these vectors as a new orthonormal basis). In that case, the following (normalized) vector will be suitable:

$${\bf v} (\lambda = 2, {\rm second}) = \frac{1}{\sqrt{6}} \left( \begin{array}{c} 1 \\ -2 \\ 1 \end{array} \right).$$ (40)

You can verify that both of these vectors are (a) orthonormal, and (b) satisfy the eigenvalue equation for $\lambda = 2$.