Column Space and Nullspace

Subspaces: the two requirements

We’re at the start of chapter three, “which is really getting to the center of linear algebra.” First, the recap from lecture five. A vector space is a bunch of vectors where you can add any two vectors and stay in the space, and multiply any vector by any constant and stay in the space. Combine the two requirements into one: all linear combinations $c\mathbf{v} + d\mathbf{w}$ stay in the space.

The whole space $\mathbb{R}^3$ certainly qualifies. But Strang is after subspaces — some vectors inside $\mathbb{R}^3$ that still make up a vector space of their own. “It’s a vector space inside a vector space.” The examples:

• A plane through the origin. Add two vectors in the plane, the sum stays in the plane; multiply by $-2$, still in the plane.
• A line through the origin. Same checks pass.
• The zero vector alone. Yes — absolutely a subspace.

Every subspace has to contain the zero vector; a plane or line that misses the origin fails immediately.

Unions and intersections

Call the plane $P$ and the line $L$ (with $L$ not lying in $P$). Two quiz questions probe whether the requirements have sunk in.

Is $P \cup L$ — all vectors in $P$ or $L$ or both — a subspace? No. Because you can’t add. Take one vector from the plane and one from the line and add them: normally you land off somewhere else, outside the union. The addition requirement fails.

Is $P \cap L$ a subspace? For this picture the intersection is only the zero vector — which is a subspace. And the answer stays yes in general:

The column space of A

Now the first of today’s two stars. Take the matrix

$$A = \begin{bmatrix} 1 & 1 & 2 \\ 2 & 1 & 3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix}$$

Which space does it live in? The columns have four components — they’re vectors in $\mathbb{R}^4$, because $A$ is 4 by 3. So $C(A)$ is a subspace of $\mathbb{R}^4$. And how big is it? Does combining three vectors fill all of four-dimensional space? The class’s feeling — no — happens to be right. It’s a smaller space. How much smaller is the real question, and it’s coming.

The connection to Ax = b

Behind the abstract definition there is a purpose: to understand $A\mathbf{x} = \mathbf{b}$. Written out,

$$\begin{bmatrix} 1 & 1 & 2 \\ 2 & 1 & 3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ x_3 \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \\ b_3 \\ b_4 \end{bmatrix}$$

Does this have a solution for every $\mathbf{b}$? No — four equations, only three unknowns. As Strang puts it, “anybody is going to say, no you dope, you can’t usually solve four equations with only three unknowns.” But sometimes you can, and which right-hand sides allow it is the question for today — the one that gets two question marks on the board.

Partial answers first. $\mathbf{b} = \mathbf{0}$ always works: take all the $x$‘s zero. $\mathbf{b} = (1, 2, 3, 4)$? That’s the first column, so $\mathbf{x} = (1, 0, 0)$ solves it by inspection. $\mathbf{b} = (1, 1, 1, 1)$? That’s the second column: $\mathbf{x} = (0, 1, 0)$. In fact the easy way to generate good right-hand sides is backwards — think of an $\mathbf{x}$ first and see what $\mathbf{b}$ turns out to be.

That’s why we care about the column space: “it’s the central guy. It says when we can solve.”

The nullspace

Now a completely different subspace, from the same matrix. It contains not right-hand sides $\mathbf{b}$ but solutions $\mathbf{x}$ — and the only $\mathbf{b}$ in sight is all zeroes.

The official machinery for finding nullspaces is elimination, coming next lecture. But the beauty of a small example is that you can see it by eye. One solution needs no looking at the matrix at all: $\mathbf{x} = \mathbf{0}$, since $A$ times the zero vector is certainly zero. Another? You’re hunting for a combination of the columns that gives zero — and since column three is column one plus column two, take $\mathbf{x} = (1, 1, -1)$. Another: $(2, 2, -2)$. Might as well take them all:

$$N(A) = \left\{ c \begin{bmatrix} 1 \\ 1 \\ -1 \end{bmatrix} \right\} = \text{all vectors } \begin{bmatrix} c \\ c \\ -c \end{bmatrix}$$

The perfect description: $c = 0$ gives the zero vector automatically, $c = 1$ gives the solution we found, and every multiple like $(7, 7, -7)$ or $(-11, -11, 11)$ is in there too. Drawn, it’s a line through the origin in $\mathbb{R}^3$, going both ways.

Are we entitled to the word “space”? Strang never uses it without the two requirements holding, so check them. If $A\mathbf{v} = \mathbf{0}$ and $A\mathbf{w} = \mathbf{0}$, is $A(\mathbf{v} + \mathbf{w}) = \mathbf{0}$? Yes — the distributive law of matrix multiplication splits it into $A\mathbf{v} + A\mathbf{w} = \mathbf{0} + \mathbf{0}$. (A law we’re using without proving: “we only live so long, we just skip that proof.”) And $A(12\mathbf{v})$? The scalar moves outside: $12\,A\mathbf{v} = 12 \cdot \mathbf{0} = \mathbf{0}$. It checks. The solutions to $A\mathbf{x} = \mathbf{0}$ always form a subspace.

When the right-hand side isn’t zero

To see the point of all this, change the right-hand side: solve $A\mathbf{x} = (1, 2, 3, 4)$. That’s a very special $\mathbf{b}$ — it’s in the column space — so solutions exist. Do they form a subspace? No. The zero vector is not a solution, so we never even got started: no subspace without $\mathbf{0}$.

What do the solutions look like instead? One solution is $(1, 0, 0)$. Another is $(0, -1, 1)$: minus the second column plus the third is the first column. There’s a whole bunch of solutions — and the picture is a line that does not pass through the origin. Subspaces have to go through the origin; if you’re looking at $\mathbf{x}$‘s, they’d better solve $A\mathbf{x} = \mathbf{0}$.

Problems

Work these with the lecture’s tools: inspection, the two subspace requirements, and the solvability criterion.

Problem 6.1 Union vs. intersection

Let $L_1$ and $L_2$ be two different lines through the origin in $\mathbb{R}^2$. Is $L_1 \cap L_2$ a subspace? Is $L_1 \cup L_2$ a subspace? Justify each answer from the two requirements.

Show solution

Intersection: two different lines through the origin meet only at $\mathbf{0}$, and the zero vector alone is a subspace — so yes. (In general, any intersection of subspaces is a subspace.)

Union: no. Take a nonzero $\mathbf{v}$ on $L_1$ and a nonzero $\mathbf{w}$ on $L_2$. Their sum $\mathbf{v} + \mathbf{w}$ lies on neither line (if it were on $L_1$, then $\mathbf{w} = (\mathbf{v}+\mathbf{w}) - \mathbf{v}$ would be on $L_1$ too — contradiction). The addition requirement fails, so the union is not a subspace.

Problem 6.2 Describing a column space

Let

$$A = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 2 & 1 & 3 \\ 1 & 2 & 3 \end{bmatrix}$$

The column space $C(A)$ is a subspace of which $\mathbb{R}^n$? Can a column be thrown away without changing $C(A)$? What is the dimension of $C(A)$?

Show solution

The columns have four components, so $C(A)$ is a subspace of $\mathbb{R}^4$.

Column three is the sum of columns one and two: $(1,0,2,1) + (0,1,1,2) = (1,1,3,3)$. So every combination that uses column three can be rewritten using only columns one and two — throw column three away and $C(A)$ is unchanged.

Columns one and two are not multiples of each other, so each contributes a new direction. $C(A)$ is a two-dimensional subspace (a plane) inside $\mathbb{R}^4$.

Problem 6.3 Which b are solvable?

For the matrix $A$ of Problem 6.2, decide by inspection whether $A\mathbf{x} = \mathbf{b}$ is solvable for each right-hand side, and give a solution when it is:

$$\mathbf{b}_1 = \begin{bmatrix} 0 \\ 1 \\ 1 \\ 2 \end{bmatrix}, \qquad \mathbf{b}_2 = \begin{bmatrix} 1 \\ 1 \\ 3 \\ 3 \end{bmatrix}, \qquad \mathbf{b}_3 = \begin{bmatrix} 1 \\ 0 \\ 0 \\ 0 \end{bmatrix}$$

Show solution

$A\mathbf{x} = \mathbf{b}$ is solvable exactly when $\mathbf{b}$ is in the column space.

$\mathbf{b}_1$ is column two itself: solvable, with $\mathbf{x} = (0, 1, 0)$.

$\mathbf{b}_2$ is column one plus column two (it equals column three): solvable, with $\mathbf{x} = (1, 1, 0)$ — or $(0, 0, 1)$.

$\mathbf{b}_3$: a combination $x_1 \mathbf{c}_1 + x_2 \mathbf{c}_2 + x_3 \mathbf{c}_3$ has first two components $x_1 + x_3$ and $x_2 + x_3$. Matching $\mathbf{b}_3$ forces $x_1 + x_3 = 1$ and $x_2 + x_3 = 0$. The third component is then $2x_1 + x_2 + 3x_3 = 2(x_1 + x_3) + (x_2 + x_3) = 2$, but $\mathbf{b}_3$ has third component $0$. No solution — $\mathbf{b}_3$ is not in $C(A)$.

Problem 6.4 Finding a nullspace by eye

Find the nullspace of

$$B = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 3 & 5 \\ 4 & 1 & 5 \end{bmatrix}$$

without elimination, and describe it geometrically.

Show solution

Look for a combination of the columns giving zero. Column three is column one plus column two: $(1,2,4) + (2,3,1) = (3,5,5)$. So $\mathbf{x} = (1, 1, -1)$ solves $B\mathbf{x} = \mathbf{0}$, and so does every multiple:

$$N(B) = \left\{ c \begin{bmatrix} 1 \\ 1 \\ -1 \end{bmatrix} \right\}$$

Since columns one and two are independent (not multiples of each other), no further directions of solutions exist. Geometrically $N(B)$ is a line through the origin in $\mathbb{R}^3$, running both ways through $(1, 1, -1)$.

Problem 6.5 Why b ≠ 0 breaks the subspace

Suppose $\mathbf{b} \neq \mathbf{0}$ and the equation $A\mathbf{x} = \mathbf{b}$ has solutions. Show in two different ways that the set of solutions is not a subspace: once using the zero vector, and once using addition.

Show solution

Zero vector: every subspace contains $\mathbf{0}$. But $A\mathbf{0} = \mathbf{0} \neq \mathbf{b}$, so the zero vector is not a solution. The solution set misses the origin — not a subspace.

Addition: take two solutions $\mathbf{v}$ and $\mathbf{w}$, so $A\mathbf{v} = \mathbf{b}$ and $A\mathbf{w} = \mathbf{b}$. By the distributive law, $A(\mathbf{v} + \mathbf{w}) = A\mathbf{v} + A\mathbf{w} = 2\mathbf{b} \neq \mathbf{b}$. The sum of two solutions is not a solution, so the set is not closed under addition.

(The picture: the solutions form a line or plane parallel to the nullspace but shifted off the origin.)