Cramer's Rule, Inverse Matrix, and Volume

A formula for the inverse

Two lectures of hard work produced the formula for the determinant and its properties. This final determinant lecture is about the applications — “always this determinant packs all this information into a single number,” and that number now gives us formulas for things we’ve been computing all along without formulas.

Start with the one inverse we know by heart. For a 2×2 matrix,

$$\begin{bmatrix} a & b \\ c & d \end{bmatrix}^{-1} = \frac{1}{ad - bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}.$$

What are we dividing by? The determinant — and that makes good sense, because the formula is perfect exactly as long as the determinant isn’t zero, which is exactly when the inverse exists. But can we recognize the entries $d, -b, -c, a$? Strang’s hint: think cofactors. The $d$ is the cofactor of $a$ — strike out row one and column one and $d$ is what’s left. The $-b$ sitting in the $(1,2)$ position is the cofactor of $c$: strike out $c$‘s row and column and you’re left with $b$, and the sign is minus because $c$ sits in position $(2,1)$ and $2 + 1$ is odd.

What kind of formula is this? For a 3×3 matrix, $\det A$ is a sum of products of three entries ($aei$, $bfg$, $cdh$, $-ceg$, and so on), while each cofactor is a determinant one size smaller — products of two entries. In general: the determinant involves products of $n$ entries, the cofactors products of $n-1$. We never noticed any of this when we computed inverses by Gauss–Jordan — tuck the identity next to $A$, eliminate until $A$ becomes $I$, and the identity has become $A^{-1}$. That was great numerically, “but we never knew what was going on.” Now we have the algebra instead of the algorithm.

Why the formula is true

It’s a pretty magic formula, so it needs checking. The claim to verify is

$$A\, C^{\mathsf{T}} = (\det A)\, I.$$

Multiply row one of $A$ against column one of $C^{\mathsf{T}}$ — which, because of the transpose, holds the cofactors from row one. The $(1,1)$ entry of the product is

$$a_{11} c_{11} + a_{12} c_{12} + \cdots + a_{1n} c_{1n}.$$

What does that sum up to? That is exactly the cofactor formula for the determinant from last lecture — entries of a row times their own cofactors give $\det A$. The same happens down the whole diagonal: row two times the cofactors from row two is $\det A$ again, and so on to row $n$. The cofactor expansion is a sum of products, so of course it’s secretly a matrix multiplication.

The remaining question is the off-diagonal entries: why does a row of $A$ times the cofactors from a different row automatically give zero? Check it in the 2×2 case: row one is $(a, b)$; the cofactors from row two are $(-b, a)$, and sure enough $a(-b) + b(a) = 0$. But we want a reason that works for $n \times n$.

So $A C^{\mathsf{T}} = (\det A) I$ is an identity — “a beautiful identity” — and dividing by $\det A$ gives the inverse. One thing the formula buys us that Gauss–Jordan never could: sensitivity. If you nudge the $(1,1)$ entry of a nonsingular matrix — add one to it, say — what happens to the inverse? The formula answers: watch what happens to the determinant and to the cofactors. It’s all there.

Cramer’s Rule

Second application: solving $A\mathbf{x} = \mathbf{b}$. Of course the solution is $\mathbf{x} = A^{-1}\mathbf{b}$, and now there’s a formula for $A^{-1}$:

$$\mathbf{x} = \frac{1}{\det A}\, C^{\mathsf{T}} \mathbf{b}.$$

This is the very same answer elimination produced on the first day of class — but now it’s a formula, and Cramer’s Rule is a way of looking at this formula. Every component of $\mathbf{x}$ has $\det A$ underneath; no surprise. The question is what sits on top. The first entry of $C^{\mathsf{T}}\mathbf{b}$ is $b_1 c_{11} + b_2 c_{21} + \cdots + b_n c_{n1}$ — cofactors times numbers. And any time you multiply cofactors by numbers, you’re computing the determinant of something. Cramer’s good idea was to realize which matrix.

Why does $\det B_1$ equal the first entry of $C^{\mathsf{T}}\mathbf{b}$? Expand $\det B_1$ in cofactors down its first column — the column that is now $\mathbf{b}$. The entry $b_1$ multiplies its cofactor, which is a determinant one size smaller and is exactly $c_{11}$ of the original $A$ (the rest of $B_1$ is $A$). Then $b_2$ gets multiplied by $c_{21}$, and so on: precisely the sum we wanted. The book gives another, cuter proof that leans less on cofactors, but with cofactors already on the board, this one is free.

Is the rule any good in practice? Cramer might say: all you have to do is compute $n+1$ determinants — the $n$ matrices $B_j$ and $A$ itself. Strang remembers a popular math book from his childhood, Mathematics for the Million, which explained elimination in a complicated way and then presented Cramer’s Rule as the way to go. “Actually, Cramer’s Rule is a disastrous way to go, because to compute these determinants, it takes, like, approximately forever” — he now thinks of the title as Mathematics for the Millionaire, since you’d have to pay for a hopelessly long calculation that elimination does in an instant. MATLAB would never use it. The value of the formula is algebraic, not computational: it lets you do algebra with letters instead of running algorithms. “They’re nice formulas, but I just don’t want you to use them.”

The determinant is a volume

Third application, and the amazing one: the determinant equals the volume of a box. Take a 3×3 matrix $A$. Draw the vector to the point $(a_{11}, a_{12}, a_{13})$ — row one — as one edge of a box starting at the origin. Row two gives a second edge, row three a third. Complete the figure: the proper word is parallelepiped, but “for obvious reasons, I wrote box.” Every face is a parallelogram.

The elegant proof: show that box volume satisfies the same three defining properties as the determinant — then it must be the determinant.

Property 1. If $A = I$, the rows are the coordinate vectors and the box is the unit cube, volume one. Determinant one. ✓

Go up one level, and bring back the previous chapter: let $A = Q$, an orthogonal matrix — orthonormal columns, perpendicular unit vectors. The box is again a cube, just rotated in space, so its volume is one. Is $\det Q = \pm 1$? Start from the defining equation $Q^{\mathsf{T}} Q = I$ and take determinants of both sides. Rule 9 (the product rule) pays off: $\det Q^{\mathsf{T}} \cdot \det Q = 1$. Rule 10 pays off: $\det Q^{\mathsf{T}} = \det Q$. So $(\det Q)^2 = 1$ and $\det Q = \pm 1$. ✓

Property 2. Exchanging two rows reverses the sign of the determinant — but it doesn’t change the box at all, and it doesn’t change the absolute value. No problem.

Property 3a. Stretch one edge — double row one, keeping the other edges fixed. The volume doubles: the new box is two identical copies stacked together. And doubling a row of the matrix doubles the determinant — that’s rule 3a, factoring $t$ out of a row. ✓

Property 3b — linearity in a single row, $\det$ of a matrix with first row $(a + a', b + b')$ splitting into two determinants — is the one that forces us away from right angles, and it’s where the lecture’s clock runs out. Strang leaves it: “the book has a carefully drawn figure to show why that works.” With all the defining properties matched, volume equals determinant.

Areas without lengths or angles

In two dimensions, the box is a parallelogram and volume means area. Take the parallelogram with one edge to $(a, b)$, another to $(c, d)$, and fourth vertex at $(a+c, b+d)$. Last year you’d have computed base times height: the base is $\sqrt{a^2 + b^2}$, and the height is “other revolting stuff” — square roots everywhere. Now:

$$\text{area of parallelogram} = \left| \det \begin{bmatrix} a & b \\ c & d \end{bmatrix} \right| = |ad - bc|.$$

No square roots. Totally rememberable — it’s exactly the formula we’ve studied for three lectures. This, more than Cramer’s Rule, is the formula that impresses Strang: if you know the coordinates of the corners, you have a great formula for area or volume with no lengths, no angles, no heights.

Next lectures are about eigenvalues — “so we’re really into the big stuff.”

Problems

Work these before revealing the solutions. Everything you need is in the lecture’s three formulas.

Problem 20.1 Inverse by cofactors

Use the cofactor formula $A^{-1} = \frac{1}{\det A} C^{\mathsf{T}}$ to invert

$$A = \begin{bmatrix} 2 & 1 & 0 \\ 0 & 3 & 1 \\ 0 & 0 & 4 \end{bmatrix}.$$

Show solution

The matrix is triangular, so $\det A = 2 \cdot 3 \cdot 4 = 24$.

Compute the nine cofactors $c_{ij} = (-1)^{i+j} M_{ij}$:

$$c_{11} = 12,\quad c_{12} = 0,\quad c_{13} = 0,$$$$c_{21} = -4,\quad c_{22} = 8,\quad c_{23} = 0,$$$$c_{31} = 1,\quad c_{32} = -2,\quad c_{33} = 6.$$

So

$$A^{-1} = \frac{1}{24} C^{\mathsf{T}} = \frac{1}{24} \begin{bmatrix} 12 & -4 & 1 \\ 0 & 8 & -2 \\ 0 & 0 & 6 \end{bmatrix} = \begin{bmatrix} \tfrac{1}{2} & -\tfrac{1}{6} & \tfrac{1}{24} \\ 0 & \tfrac{1}{3} & -\tfrac{1}{12} \\ 0 & 0 & \tfrac{1}{4} \end{bmatrix}.$$

Check the $(1,1)$ entry of $AA^{-1}$: $2 \cdot \tfrac{1}{2} + 1 \cdot 0 + 0 = 1$. ✓ (Notice the inverse of an upper triangular matrix is upper triangular — the formula shows why: the cofactors $c_{12}$, $c_{13}$, $c_{23}$ below the diagonal of $C^{\mathsf{T}}$ all vanish.)

Problem 20.2 Cramer's Rule, 2×2

Solve by Cramer’s Rule:

$$\begin{aligned} 3x + 2y &= 7 \\ x + 4y &= 9. \end{aligned}$$

Show solution

$$\det A = \det \begin{bmatrix} 3 & 2 \\ 1 & 4 \end{bmatrix} = 12 - 2 = 10.$$

Replace column one by $\mathbf{b}$ for $B_1$, column two for $B_2$:

$$\det B_1 = \det \begin{bmatrix} 7 & 2 \\ 9 & 4 \end{bmatrix} = 28 - 18 = 10, \qquad \det B_2 = \det \begin{bmatrix} 3 & 7 \\ 1 & 9 \end{bmatrix} = 27 - 7 = 20.$$

So

$$x = \frac{\det B_1}{\det A} = \frac{10}{10} = 1, \qquad y = \frac{\det B_2}{\det A} = \frac{20}{10} = 2.$$

Check: $3(1) + 2(2) = 7$ ✓ and $1 + 4(2) = 9$ ✓.

Problem 20.3 Why off-diagonals vanish

Let

$$A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}.$$

Write down the cofactor matrix $C$, compute the product $A C^{\mathsf{T}}$ in full, and explain — using the “matrix with two equal rows” argument from lecture — why the off-diagonal entries had to be zero.

Show solution

The cofactors: $c_{11} = 4$, $c_{12} = -3$, $c_{21} = -2$, $c_{22} = 1$, so

$$C = \begin{bmatrix} 4 & -3 \\ -2 & 1 \end{bmatrix}, \qquad C^{\mathsf{T}} = \begin{bmatrix} 4 & -2 \\ -3 & 1 \end{bmatrix}.$$

Multiply:

$$A C^{\mathsf{T}} = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}\begin{bmatrix} 4 & -2 \\ -3 & 1 \end{bmatrix} = \begin{bmatrix} 4 - 6 & -2 + 2 \\ 12 - 12 & -6 + 4 \end{bmatrix} = \begin{bmatrix} -2 & 0 \\ 0 & -2 \end{bmatrix} = (\det A)\, I,$$

since $\det A = 4 - 6 = -2$.

The $(1,2)$ entry is row one of $A$ times the cofactors from row two of $A$: $1 \cdot (-2) + 2 \cdot 1$. That is exactly the cofactor expansion of the determinant of the “screwed up” matrix

$$A_S = \begin{bmatrix} 1 & 2 \\ 1 & 2 \end{bmatrix}$$

— row one of $A$ appearing twice, expanded along its second row. A matrix with two equal rows is singular, so the expansion must give zero.

Problem 20.4 Volume of a box

Find the volume of the parallelepiped with edges from the origin to $(1, 1, 0)$, $(0, 2, 1)$, and $(1, 0, 3)$. Is the ordering of these three edges right-handed or left-handed?

Show solution

Put the edges in the rows and take the determinant, expanding along row one:

$$\det \begin{bmatrix} 1 & 1 & 0 \\ 0 & 2 & 1 \\ 1 & 0 & 3 \end{bmatrix} = 1 \det \begin{bmatrix} 2 & 1 \\ 0 & 3 \end{bmatrix} - 1 \det \begin{bmatrix} 0 & 1 \\ 1 & 3 \end{bmatrix} + 0 = 1(6) - 1(-1) = 7.$$

The volume is $|7| = 7$. Since the determinant is positive, the three edges in this order form a right-handed box. (Exchanging any two of the edges would flip the sign to $-7$ — same volume, left-handed orientation.)

Problem 20.5 Triangle from three corners

Find the area of the triangle with vertices $(1, 2)$, $(4, 3)$, and $(2, 6)$, using the lecture’s determinant formula with the column of ones. Then verify by translating the triangle so one vertex is at the origin.

Show solution

The formula:

$$\text{area} = \frac{1}{2} \left| \det \begin{bmatrix} 1 & 2 & 1 \\ 4 & 3 & 1 \\ 2 & 6 & 1 \end{bmatrix} \right|.$$

Subtract row one from rows two and three (determinant unchanged):

$$\det \begin{bmatrix} 1 & 2 & 1 \\ 3 & 1 & 0 \\ 1 & 4 & 0 \end{bmatrix} = 1 \cdot \det \begin{bmatrix} 3 & 1 \\ 1 & 4 \end{bmatrix} = 12 - 1 = 11,$$

expanding down the third column. So the area is $\tfrac{11}{2}$.

Verification by translation: shift every vertex by $(-1, -2)$, giving $(0,0)$, $(3, 1)$, $(1, 4)$. Then

$$\text{area} = \frac{1}{2} |ad - bc| = \frac{1}{2} |3 \cdot 4 - 1 \cdot 1| = \frac{11}{2}. \checkmark$$

The row subtractions in the 3×3 determinant were exactly this translation to the origin.