The Geometry of Linear Equations

One problem, three pictures

The fundamental problem of linear algebra is to solve a system of linear equations. Strang starts with the nicest case: $n$ equations, $n$ unknowns, and one solution. But the real content of this lecture isn’t the solving — it’s that the same system can be seen three different ways:

1. The row picture — one equation at a time. Each equation draws a line (or a plane), and the solution is where they all meet. This is the picture you already know from school.
2. The column picture ★ — one column at a time. The system asks you to combine vectors in the right amounts. Strang stars this one: it may be new to you, and it’s the picture the whole course is built on.
3. The matrix form — the algebra: $A\mathbf{x} = \mathbf{b}$, where $A$ holds the coefficients, $\mathbf{x}$ the unknowns, $\mathbf{b}$ the right-hand side.

A 2×2 example, written as a matrix

Take two equations in two unknowns:

$$\begin{aligned} 2x - y &= 0 \\ -x + 2y &= 3 \end{aligned}$$

Before any pictures, Strang can’t resist writing the matrix form. The coefficient matrix $A$ collects the numbers, and the system becomes

$$\underbrace{\begin{bmatrix} 2 & -1 \\ -1 & 2 \end{bmatrix}}_{A} \underbrace{\begin{bmatrix} x \\ y \end{bmatrix}}_{\mathbf{x}} = \underbrace{\begin{bmatrix} 0 \\ 3 \end{bmatrix}}_{\mathbf{b}}$$

Every linear system in this course will be written this way: $A\mathbf{x} = \mathbf{b}$.

The row picture: where lines meet

Take one row at a time and plot every point $(x, y)$ that satisfies it.

The first equation, $2x - y = 0$, passes through the origin (put $x = 0, y = 0$ — it works), and through $(1, 2)$. All its solutions form a line — that’s what linear means.

The second equation, $-x + 2y = 3$, does not pass through the origin ($0 \neq 3$). Setting $y = 0$ gives $x = -3$; setting $x = -1$ gives $y = 1$. Connect the points: a second line.

The two lines meet at exactly one point: $(x, y) = (1, 2)$. Check it in both equations: $2(1) - 2 = 0$ ✓ and $-1 + 2(2) = 3$ ✓. That intersection point is the solution.

The column picture: combining vectors

Now the key move. Instead of reading the system row by row, read it column by column:

$$x \begin{bmatrix} 2 \\ -1 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \end{bmatrix}$$

The columns of $A$ are vectors, and the equation asks: in what amounts should we combine them to produce $\mathbf{b}$?

Geometrically: draw column one, $(2, -1)$. From its tip, attach column two, $(-1, 2)$ — and since we need two of it, attach it twice. You land exactly at $(0, 3) = \mathbf{b}$. One of column one plus two of column two: the coefficients are precisely the solution $x = 1, y = 2$ found in the row picture.

Don’t take the lecture’s word for it — drive both pictures yourself. Slide $x$ and $y$ and watch the same pair of numbers move a point across two lines and steer a chain of column vectors toward $\mathbf{b}$. They agree only at the solution.

Three equations, three unknowns

Now a 3×3 system:

$$\begin{aligned} 2x - y \phantom{{}+ 0z} &= 0 \\ -x + 2y - z &= -1 \\ -3y + 4z &= 4 \end{aligned} \qquad A = \begin{bmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -3 & 4 \end{bmatrix}, \quad \mathbf{b} = \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix}$$

Row picture: each equation is now a plane in $xyz$-space. Two planes meet in a line; the third plane cuts that line in a point — the solution. But Strang admits the art is the weak point: three planes meeting is already hard to see, and in four dimensions it’s hopeless. The row picture runs out of steam.

Column picture: combine three column vectors to produce $\mathbf{b}$:

$$x \begin{bmatrix} 2 \\ -1 \\ 0 \end{bmatrix} + y \begin{bmatrix} -1 \\ 2 \\ -3 \end{bmatrix} + z \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix} = \begin{bmatrix} 0 \\ -1 \\ 4 \end{bmatrix}$$

This right-hand side was chosen kindly: $\mathbf{b}$ is the third column. So the combination is obvious — none of column one, none of column two, one of column three: $(x, y, z) = (0, 0, 1)$. That’s the point where the three planes met, and the column picture found it by inspection.

And if $\mathbf{b}$ changes? Set $\mathbf{b} = (1, 1, -3)$, the sum of columns one and two — then the solution flips to $(1, 1, 0)$. Same columns, new combination. The column picture handles a change of right-hand side gracefully; the row picture would redraw all three planes.

Strang apologizes for his board drawing here (“your lecturer is not Rembrandt”). Fair enough — here is the picture he was reaching for, in actual 3D. Drag it around until you can see that $\mathbf{b}$ sits exactly on the tip of column 3, then switch $\mathbf{b}$ to column 1 + column 2 and watch the tip-to-tail chain land on it:

Can we solve Ax = b for every b?

Here’s the big question the example raises, asked two ways:

• Algebra: does $A\mathbf{x} = \mathbf{b}$ have a solution for every right-hand side $\mathbf{b}$?
• Geometry: do the linear combinations of the columns fill the whole space?

Same question, different words. For the matrix above the answer is yes — it’s what we’ll call a non-singular, invertible matrix, the kind we like best.

Strang then dares nine dimensions: nine equations, nine unknowns, nine column vectors in $\mathbb{R}^9$. You can’t visualize it — “pretend you do.” A random 9×9 matrix is virtually certain to be invertible. But if the ninth column repeats the eighth, it adds nothing new, and the combinations fill only an eight-dimensional plane inside nine-dimensional space. Getting comfortable with that sentence is learning linear algebra.

How to multiply A times x

One piece of unfinished business: what does $A\mathbf{x}$ actually mean? Two ways to compute it.

Problems

Work these before revealing the solutions. The lecture gives you everything you need.

Problem 1.1 Row & column pictures

Draw the row picture and the column picture for the system

$$\begin{aligned} x + 2y &= 3 \\ 2x + y &= 3 \end{aligned}$$

and solve it both ways.

Show solution

Row picture: the lines $x + 2y = 3$ and $2x + y = 3$ intersect at $(1, 1)$.

Column picture:

$$x \begin{bmatrix} 1 \\ 2 \end{bmatrix} + y \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 3 \end{bmatrix}$$

One of each column: $\begin{bmatrix} 1 \\ 2 \end{bmatrix} + \begin{bmatrix} 2 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 3 \end{bmatrix}$, so $x = y = 1$. Both pictures give $(1, 1)$.

Problem 1.2 Ax by columns and by rows

Compute $A\mathbf{x}$ for

$$A = \begin{bmatrix} 3 & 1 \\ 1 & 2 \end{bmatrix}, \qquad \mathbf{x} = \begin{bmatrix} 2 \\ -1 \end{bmatrix}$$

first as a combination of the columns of $A$, then row by row as dot products.

Show solution

By columns: $2\begin{bmatrix} 3 \\ 1 \end{bmatrix} - 1\begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 6 - 1 \\ 2 - 2 \end{bmatrix} = \begin{bmatrix} 5 \\ 0 \end{bmatrix}$

By rows: $(3)(2) + (1)(-1) = 5$ and $(1)(2) + (2)(-1) = 0$. Same result: $A\mathbf{x} = \begin{bmatrix} 5 \\ 0 \end{bmatrix}$.

Problem 1.3 When does it go wrong?

For which number $c$ is the matrix

$$A = \begin{bmatrix} 1 & 2 \\ 3 & c \end{bmatrix}$$

singular — that is, for which $c$ do the two columns lie on the same line? For that $c$, describe which right-hand sides $\mathbf{b}$ make $A\mathbf{x} = \mathbf{b}$ solvable.

Show solution

The columns $\begin{bmatrix} 1 \\ 3 \end{bmatrix}$ and $\begin{bmatrix} 2 \\ c \end{bmatrix}$ are parallel when $\begin{bmatrix} 2 \\ c \end{bmatrix} = 2\begin{bmatrix} 1 \\ 3 \end{bmatrix}$, i.e. $c = 6$.

Then every combination of the columns is a multiple of $\begin{bmatrix} 1 \\ 3 \end{bmatrix}$, so $A\mathbf{x} = \mathbf{b}$ is solvable exactly when $\mathbf{b}$ lies on that line — when $\mathbf{b} = \begin{bmatrix} b_1 \\ 3b_1 \end{bmatrix}$. Every other $\mathbf{b}$ is unreachable.

Problem 1.4 Thinking bigger

Suppose $A$ is a 3×3 matrix whose third column equals the sum of its first two columns. Without computing anything, give one right-hand side $\mathbf{b} \neq \mathbf{0}$ for which $A\mathbf{x} = \mathbf{b}$ definitely has a solution, and explain geometrically why some right-hand sides have none.

Show solution

Any combination of the columns works as a reachable $\mathbf{b}$ — the simplest is $\mathbf{b} = \mathbf{c}_1$ (take $x = 1, y = z = 0$), or $\mathbf{b} = \mathbf{c}_1 + \mathbf{c}_2$, which even has two visible solutions: $(1, 1, 0)$ and $(0, 0, 1)$.

Geometrically, all three columns lie in one plane (the third adds no new direction), so every combination $A\mathbf{x}$ stays in that plane. Any $\mathbf{b}$ off the plane is unreachable — the matrix is singular.