Matrices

A matrix is a rectangular collection of numbers (again, possibly complex). It might represent a collection of row/column vectors, or a transformation that changes a vector into a different vector. For example, the rotation of a real 2D vector by an angle $\theta$ about the $z$ axis could be represented as a 2x2 matrix,

$$\left( \begin{array}{cc} cos \theta & - sin \theta \\ sin \theta & cos \theta \end{array}\right).$$

The effect of this rotation on a 2D vector in the $x,y$ plane can be determined by multiplying the vector by this transformation matrix,

$$\left( \begin{array}{c} x' \\ y' \end{array} \right) = \le... ...} \right) \left( \begin{array}{c} x \\ y \end{array} \right),$$ (9)

where $x' = x cos \theta - y sin \theta$ and $y' = x sin \theta + y cos \theta$.

More generally, the product of a matrix and a column vector may be written as

$$\left( \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n}... ...gin{array}{c} c_1 \\ c_2 \\ \vdots \\ c_n \end{array} \right),$$ (10)

where

$$c_1 = a_{11} b_1 + a_{12} b_2 + \cdots + a_{1n} b_n$$ (11)

$$c_2 = a_{21} b_1 + a_{22} b_2 + \cdots + a_{2n} b_n$$

$$\vdots$$

$$c_n = a_{n1} b_1 + a_{n2} b_2 + \cdots + a_{nn} b_n.$$

Notice that each element of the product, $c_i$, is just the dot product of the $i$th row of the matrix with the vector. It is also possible to think of the column vector ${\bf c}$ as being a linear combination of the columns of the matrix ${\bf A}$, with each column $i$ having a weight $b_i$.

It is also possible to ``left-multiply'' a matrix by a row vector, like this:

$$\left( b_1 b_2 \cdots b_n \right) \left( \begin{array}{cccc}... ... a_{nn} \end{array}\right) = \left(c_1 c_2 \cdots c_n \right),$$ (12)

where

$$c_1 = b_1 a_{11} + b_2 a_{21} + \cdots + b_n a_{n1}$$ (13)

$$c_2 = b_1 a_{12} + b_2 a_{22} + \cdots + b_n a_{n2}$$

$$\vdots$$

$$c_n = b_1 a_{1n} + b_2 a_{2n} + \cdots + b_n a_{nn}.$$

This time, each element of the product, $c_i$, is the dot product of the vector on the left with the $i$th column of the matrix. The row vector ${\bf c}$ may be thought of as a linear combination of the rows of the matrix ${\bf A}$, with each row $i$ having a weight of $b_i$.

From the information given, it should be obvious that a row vector times a matrix times a column vector yields a number (scalar).

Now consider the multiplication of two matrices ${\bf A}$ and ${\bf B}$, which yields a new matrix, ${\bf C}$:

$$\left( \begin{array}{cccc} a_{11} & a_{12} & \cdots & a_{1n} \\ ... ...dots & \vdots & \vdots \\ b_{n1} & b_{n2} & \cdots & b_{nn} \end{array}\right) = \left( \begin{array}{cccc} c_{11} & c_{12} & \cdots & c_{1n} \\ ... ...ots & \vdots & \vdots \\ c_{n1} & c_{n2} & \cdots & c_{nn} \end{array}\right),$$ (14)

where

$$c_{ij} = a_{i1} b_{1j} + a_{i2} b_{2j} + \cdots + a_{in} b_{nj}.$$ (15)

That is, each element $c_{ij}$ of the product matrix ${\bf C}$ is the result of a dot product between row $i$ of the matrix ${\bf A}$ and column $j$ of the matrix ${\bf B}$. It is possible to think of matrix multiplication as a generalization of the product of a matrix times a vector, where now instead of one column vector on the right, we have a series of them. It is also possible to think of matrix multiplication as a generalization of the product of a row vector times a matrix, where now instead of one row vector on the left, we have a series of them.

The transpose of a matrix ${\bf A}$, typically denoted ${\bf A}^T$, is obtained simply by swapping elements across the diagonal, $a_{ij} \rightarrow a_{ji}$. For example,

$$\left( \begin{array}{cc} a & b \\ c & d \end{array} \right)^... ...= \left( \begin{array}{cc} a & c \\ b & d \end{array} \right).$$ (16)

For complex numbers, usually the complex conjugate transpose, or adjoint, is more useful. The adjoint of a matrix ${\bf A}$ is often denoted ${\bf A}^{\dagger}$. For example,

$$\left( \begin{array}{cc} a & b \\ c & d \end{array} \right)^... ... \begin{array}{cc} a^* & c^* \\ b^* & d^* \end{array} \right).$$ (17)

If a matrix ${\bf A}$ is equal to its adjoint ${\bf A}^{\dagger}$, it is said to be a Hermitian matrix. Clearly, this can only happen if the diagonal elements are real (otherwise, $a$ will never equal $a^*$, for example). One can also take the adjoint of a vector, which is effectively what one does in order to take a dot product of two complex vectors (see above):

$$\left( \begin{array}{c} a_1 \\ a_2 \\ \vdots \\ a_n \end{... ...ray}{c} a_1^* \\ a_2^* \\ \vdots \\ a_n^* \end{array} \right).$$ (18)