Linear Transformations and Their Matrices

Transformations without coordinates

Linear algebra courses used to begin with this lecture — so in a sense, Strang says, he’s beginning the course again. The idea of a linear transformation makes sense without any matrix at all, and physicists like it better that way: no coordinates, no numbers, just what happens to the whole space. But for most of us, if we’re going to compute anything, we introduce coordinates — and then every linear transformation leads us to a matrix, and everything we’ve done with null spaces, determinants, and eigenvalues comes from that matrix. Behind it all, though, is the coordinate-free idea.

Example 1: projection. Take all of $\mathbb{R}^2$ and a line through the origin. The transformation $T$ projects every vector onto that line: give it an input $\mathbf{v}$, it produces the output $T(\mathbf{v})$, the shadow of $\mathbf{v}$ on the line. A transformation is exactly like a function — the words map or mapping are also used — written

$$T : \mathbb{R}^2 \to \mathbb{R}^2.$$

No coordinates were needed to say that. Strang draws axes out of habit, then apologizes and erases them — “I’m not a physicist” — because the whole point is that the projection is described without them.

Not every transformation earns the name linear. The requirement involves exactly the two operations we can do on vectors — adding and multiplying by scalars:

Projection passes the test: double $\mathbf{v}$ and its projection doubles; flip $\mathbf{v}$ to $-\mathbf{v}$ and the projection flips. Any $c$ works.

Two non-examples

Shift the whole plane. Fix a vector $\mathbf{v}_0$ and let $T(\mathbf{v}) = \mathbf{v} + \mathbf{v}_0$ — every vector gets the same fixed vector added on. Perfectly reasonable, perfectly simple, and not linear. Double the input: the output starts at $2\mathbf{v}$ but only goes one $\mathbf{v}_0$ out, not two — so $T(2\mathbf{v}) \neq 2\,T(\mathbf{v})$.

There’s a quick test hiding here, a special case worth noticing. Take $\mathbf{v} = \mathbf{0}$ and $c = 3$ in the scaling rule: $T(\mathbf{0}) = 3\,T(\mathbf{0})$, which forces

$$T(\mathbf{0}) = \mathbf{0}.$$

The zero vector can’t move under a linear transformation. The shift moves it to $\mathbf{v}_0$ — disqualified immediately.

Take the length. Let $T(\mathbf{v}) = \|\mathbf{v}\|$, mapping $\mathbb{R}^3$ to the real numbers $\mathbb{R}^1$. Zero does go to zero, and doubling a vector does double its length. But multiply $\mathbf{v}$ by $-2$: the length gets multiplied by $+2$, not $-2$. The requirement fails for negative $c$, so length is not linear.

Example 2: rotation by 45°. Back to a genuine example. Rotate every vector in the plane by $45°$. Described with no coordinates at all — and it’s linear: doubling $\mathbf{v}$ doubles the rotated vector, and rotating $\mathbf{v} + \mathbf{w}$ gives the same answer as rotating each and then adding.

Watching a whole house transform

The book’s cover has the right picture: instead of tracking one vector at a time, take a house in $\mathbb{R}^2$ — a whole outline of points — and watch where all of it goes at once. Under the $45°$ rotation, the whole house rotates and sits there tilted.

Example 3: transformations from a matrix. Every matrix $A$ produces a linear transformation by the simplest possible rule:

$$T(\mathbf{v}) = A\mathbf{v}.$$

Linearity is automatic — $A(\mathbf{v} + \mathbf{w}) = A\mathbf{v} + A\mathbf{w}$ and $A(c\mathbf{v}) = cA\mathbf{v}$. Check and check. Take the particular matrix

$$A = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}$$

and apply it to every point of the house. It’s diagonal and easy to read: the $x$-component stays, the $y$-component reverses sign. The tip of the roof keeps its $x$ but drops below the axis — the house lands upside down, door and all.

So the idea of a linear transformation is “kind of the abstract description of matrix multiplication.” The goal of the lecture: understand linear transformations by finding the matrix that lies behind them. And to do that, we have to bring in coordinates — we have to choose a basis.

How much information determines T?

Suppose $T$ maps $\mathbb{R}^3$ to $\mathbb{R}^2$. (What’s a quick example of such a $T$? Multiply by any $2 \times 3$ matrix — a $2 \times 3$ matrix eats a vector in $\mathbb{R}^3$ and produces one in $\mathbb{R}^2$. The claim of this lecture is that there are essentially no other examples.)

Now the key question: how much do I have to tell you before you know $T(\mathbf{v})$ for every input $\mathbf{v}$? If I tell you $T(\mathbf{v}_1)$ for one vector, you know one output. Tell you $T(\mathbf{v}_2)$ as well — now you know much more than two outputs. By linearity you know $T$ on every combination of $\mathbf{v}_1$ and $\mathbf{v}_2$: the whole plane they span. (Provided they’re independent — if $\mathbf{v}_2 = 6\mathbf{v}_1$, the second answer told you nothing new; you already knew it would be $6\,T(\mathbf{v}_1)$.)

This is exactly what makes coordinates possible. Coordinates come from a basis. The coordinates of $\mathbf{v}$ are the numbers $c_1, \dots, c_n$ — how much of each basis vector is in $\mathbf{v}$. When you write $\mathbf{v} = (3, 2, 4)$ you’ve been silently assuming the standard basis: $3$ of the first coordinate vector, $2$ of the second, $4$ of the third. But nothing forces that choice — the basis might instead be the eigenvectors of some matrix. Change the basis and the coordinates change.

Building the matrix: projection two ways

To turn $T : \mathbb{R}^n \to \mathbb{R}^m$ into a matrix we need two bases: an input basis $\mathbf{v}_1, \dots, \mathbf{v}_n$ and an output basis $\mathbf{w}_1, \dots, \mathbf{w}_m$. The plan in words: take a vector, find its coordinates in the input basis, multiply by the right matrix $A$, and out come the coordinates of $T(\mathbf{v})$ in the output basis.

Now do the same projection in the standard basis, projecting onto the $45°$ line with $\mathbf{v}_1 = (1,0)$, $\mathbf{v}_2 = (0,1)$. The matrix is the projection matrix from Chapter 4, $P = \dfrac{\mathbf{a}\mathbf{a}^T}{\mathbf{a}^T\mathbf{a}}$, which works out to

$$P = \begin{bmatrix} \tfrac12 & \tfrac12 \\[2pt] \tfrac12 & \tfrac12 \end{bmatrix}.$$

Input $(1, 0)$ gives output $(\tfrac12, \tfrac12)$ — the correct projection — and likewise for $(0,1)$. Same transformation, different basis, different matrix: not diagonal, because we chose the handiest basis rather than the great one. It’s still a good matrix — symmetric, with $P^2 = P$ — but in the eigenvector basis those properties are visible at a glance.

The rule for the matrix, and one more transformation

Here is the general recipe, given the bases.

Why is it right? Check column one. If the input coordinates are $(1, 0, \dots, 0)$, the input is $\mathbf{v}_1$ itself, and multiplying $A$ by that coordinate vector picks out the first column — which is, by construction, exactly the output coordinates of $T(\mathbf{v}_1)$. Right on the first basis vector, right on the second, right on all of them — and then, by linearity, right on every vector.

Problems

Work these before revealing the solutions — the lecture’s rule for building matrices is all you need.

Problem 30.1 Linear or not?

For each transformation of the plane, decide whether it is linear. Justify with the two rules or a specific counterexample.

(a) $T(\mathbf{v}) = \mathbf{v} + \begin{bmatrix} 1 \\ 1 \end{bmatrix}$ (b) $T\!\left(\begin{bmatrix} x \\ y \end{bmatrix}\right) = \begin{bmatrix} y \\ x \end{bmatrix}$ (c) $T\!\left(\begin{bmatrix} x \\ y \end{bmatrix}\right) = \begin{bmatrix} x^2 \\ y \end{bmatrix}$

Show solution

(a) Not linear. $T(\mathbf{0}) = (1,1) \neq \mathbf{0}$, and a linear transformation must send zero to zero. This is the shift from lecture.

(b) Linear. Swapping components respects addition — $T(\mathbf{v} + \mathbf{w})$ swaps the sums, which equals the sum of the swaps — and $T(c\mathbf{v}) = c\,T(\mathbf{v})$ since $c$ multiplies both components. In fact $T(\mathbf{v}) = A\mathbf{v}$ with $A = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$.

(c) Not linear. Take $\mathbf{v} = (1, 0)$ and $c = 2$: $T(2\mathbf{v}) = (4, 0)$ but $2\,T(\mathbf{v}) = (2, 0)$. Squaring breaks the scaling rule (zero going to zero is not enough).

Problem 30.2 Matrix from a basis rule

A linear transformation $T : \mathbb{R}^2 \to \mathbb{R}^2$ satisfies

$$T\!\left(\begin{bmatrix} 1 \\ 0 \end{bmatrix}\right) = \begin{bmatrix} 2 \\ 1 \end{bmatrix}, \qquad T\!\left(\begin{bmatrix} 0 \\ 1 \end{bmatrix}\right) = \begin{bmatrix} -1 \\ 3 \end{bmatrix}.$$

Find the matrix of $T$ in the standard basis (inputs and outputs), and compute $T\!\left(\begin{bmatrix} 4 \\ 5 \end{bmatrix}\right)$.

Show solution

Column $j$ of $A$ is $T$ applied to the $j$-th basis vector, expressed in the output basis — here the standard basis, so the outputs go straight into the columns:

$$A = \begin{bmatrix} 2 & -1 \\ 1 & 3 \end{bmatrix}.$$

Then by linearity, $T(4, 5) = 4\,T(1,0) + 5\,T(0,1)$:

$$A \begin{bmatrix} 4 \\ 5 \end{bmatrix} = 4\begin{bmatrix} 2 \\ 1 \end{bmatrix} + 5\begin{bmatrix} -1 \\ 3 \end{bmatrix} = \begin{bmatrix} 3 \\ 19 \end{bmatrix}.$$

Knowing $T$ on a basis determined $T$ everywhere.

Problem 30.3 Eigenvector basis vs standard basis

Let $T$ be the transformation of the plane that reflects every vector across the $45°$ line $y = x$.

(a) Choose the basis $\mathbf{v}_1 = \frac{1}{\sqrt{2}}(1, 1)$ (on the line) and $\mathbf{v}_2 = \frac{1}{\sqrt{2}}(-1, 1)$ (perpendicular to it), for both inputs and outputs. What is the matrix of $T$?

(b) What is the matrix of $T$ in the standard basis?

Show solution

(a) Reflection leaves the line alone and reverses the perpendicular direction: $T(\mathbf{v}_1) = \mathbf{v}_1$ and $T(\mathbf{v}_2) = -\mathbf{v}_2$. So column one is $(1, 0)$ and column two is $(0, -1)$:

$$A = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix} = \Lambda.$$

These basis vectors are the eigenvectors of the reflection, with eigenvalues $1$ and $-1$ — the matrix is diagonal because the basis is the good one.

(b) In the standard basis, apply $T$ to each coordinate vector: reflecting $(1,0)$ across $y = x$ gives $(0,1)$, and reflecting $(0,1)$ gives $(1,0)$. Columns:

$$A = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}.$$

Same transformation, two different matrices — one diagonal, one not — depending on the basis.

Problem 30.4 Derivative-style matrix

Let $T = \dfrac{d}{dx}$ act on the space of cubics $c_1 + c_2 x + c_3 x^2 + c_4 x^3$, with input basis $1, x, x^2, x^3$ and output basis $1, x, x^2$.

(a) Find the matrix $A$ of $T$.

(b) Describe the null space of $A$ and say what it means about derivatives.

Show solution

(a) Apply $T$ to each input basis vector and expand in the output basis $1, x, x^2$: $T(1) = 0$, $T(x) = 1$, $T(x^2) = 2x$, $T(x^3) = 3x^2$. Those expansions fill the columns of the $3 \times 4$ matrix

$$A = \begin{bmatrix} 0 & 1 & 0 & 0 \\ 0 & 0 & 2 & 0 \\ 0 & 0 & 0 & 3 \end{bmatrix}.$$

Check on coordinates: $A(c_1, c_2, c_3, c_4)^T = (c_2, 2c_3, 3c_4)^T$, the coordinates of $c_2 + 2c_3 x + 3c_4 x^2$. Correct.

(b) $A\mathbf{c} = \mathbf{0}$ forces $c_2 = c_3 = c_4 = 0$ with $c_1$ free: the null space is the line of vectors $(c_1, 0, 0, 0)$, i.e. the constant polynomials. That is exactly the calculus fact that the derivative kills constants — a function has derivative zero exactly when it is constant. A one-dimensional null space out of a four-dimensional input space, matching rank $3$.