Differential Equations and exp(At)

From powers of A to exponentials of A

“Lift-off on differential equations.” This lecture solves systems of first-order, constant-coefficient linear equations — and done right, “it turns directly into linear algebra.” The key idea: the solutions to constant-coefficient linear equations are exponentials. Look for an exponential, and all you have to find is what goes up in the exponent and what multiplies it — and that’s the linear algebra.

The result is completely parallel to the last lecture, which computed high powers like $A^{100}$ for the difference equation $\mathbf{u}_{k+1} = A\mathbf{u}_k$. Now it’s not powers but exponentials, the natural thing for a differential equation:

$$\frac{d\mathbf{u}}{dt} = A\mathbf{u}$$

An example, mechanics first

Take two coupled equations, matrix and initial condition:

$$\frac{d\mathbf{u}}{dt} = \begin{bmatrix} -1 & 2 \\ 1 & -2 \end{bmatrix} \mathbf{u}, \qquad \mathbf{u}(0) = \begin{bmatrix} 1 \\ 0 \end{bmatrix}$$

At the start everything is in $u_1$. But $du_2/dt$ will be positive because of that $u_1$ term, so flow moves into the $u_2$ component and out of $u_1$. We follow that movement by doing the first job for any matrix: before anything else, find the eigenvalues and eigenvectors.

Can you spot the eigenvalues? The matrix is singular — the second column is $-2$ times the first, the determinant is zero — so $\lambda_1 = 0$ is an eigenvalue immediately. The trace is $-1 + (-2) = -3$, and the trace equals the sum of the eigenvalues, so the other one had better be $\lambda_2 = -3$. As a check, the characteristic polynomial is

$$\det(A - \lambda I) = \lambda^2 + 3\lambda = \lambda(\lambda + 3)$$

confirming $\lambda_1 = 0$, $\lambda_2 = -3$.

Already the eigenvalues are telling us the most important information about the answer. The negative eigenvalue will contribute an $e^{-3t}$, which becomes very, very small as time goes on — that part disappears. The zero eigenvalue contributes $e^{0t} = 1$, a constant. So the solution will have two parts, and as $t \to \infty$ the second part vanishes and the first part is a steady state — it won’t move. A zero eigenvalue means a steady state.

Now the eigenvectors. For $\lambda_1 = 0$, the eigenvector is the guy in the null space of $A$: give the free variable the value $1$ and you need a $2$ on top, so $\mathbf{x}_1 = (2, 1)$. For $\lambda_2 = -3$, add $3$ to the diagonal:

$$A + 3I = \begin{bmatrix} 2 & 2 \\ 1 & 1 \end{bmatrix}$$

singular, as it should be, with $\mathbf{x}_2 = (1, -1)$ in its null space (or $(-1, 1)$ — either is perfectly good).

Building the solution from pure exponentials

The general solution is a combination of two pure exponential solutions, one for each eigenvalue:

$$\mathbf{u}(t) = c_1 e^{\lambda_1 t}\mathbf{x}_1 + c_2 e^{\lambda_2 t}\mathbf{x}_2$$

Check that each piece really solves the equation. Plug $e^{\lambda_1 t}\mathbf{x}_1$ into $d\mathbf{u}/dt = A\mathbf{u}$: the time derivative brings down a $\lambda_1$, the right side is $A e^{\lambda_1 t}\mathbf{x}_1$, the exponential appears on both sides, and what remains is $A\mathbf{x}_1 = \lambda_1 \mathbf{x}_1$ — check.

For our matrix, $e^{\lambda_1 t} = e^{0t} = 1$, so the solution is

$$\mathbf{u}(t) = c_1 \begin{bmatrix} 2 \\ 1 \end{bmatrix} + c_2\, e^{-3t} \begin{bmatrix} 1 \\ -1 \end{bmatrix}$$

Everything so far was immediate from the eigenvalues and eigenvectors. The constants come from the initial condition. At $t = 0$:

$$c_1 \begin{bmatrix} 2 \\ 1 \end{bmatrix} + c_2 \begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}$$

Spot the answer: if $c_1$ and $c_2$ were both $1$, the sum would be $(3, 0)$ — so take one third of that. $c_1 = c_2 = \frac{1}{3}$, and

$$\mathbf{u}(t) = \frac{1}{3}\begin{bmatrix} 2 \\ 1 \end{bmatrix} + \frac{1}{3}\, e^{-3t} \begin{bmatrix} 1 \\ -1 \end{bmatrix}$$

You can see what’s actually happening with this flow. It started at $(1, 0)$; as time goes on, some fraction moves from the first component into the second; and in the limit the $e^{-3t}$ part disappears, leaving the steady state

$$\mathbf{u}(\infty) = \begin{bmatrix} 2/3 \\ 1/3 \end{bmatrix}$$

That’s total insight into the behavior of the solution — what the differential equation does — read off from two eigenvalues and two eigenvectors.

One more observation about the step that found $c_1, c_2$: as a matrix equation it is $S\mathbf{c} = \mathbf{u}(0)$, where $S$ is the eigenvector matrix — the eigenvectors sit in its columns. You figure out how much of each pure exponential is present in $\mathbf{u}(0)$, and then off they go, each on its own. “By getting it right at the start, it stays right forever.”

Stability: it’s the real part that matters

We won’t always have a steady state. Sometimes the solution approaches zero; sometimes it blows up. The eigenvalues tell us which.

Stability — $\mathbf{u}(t) \to 0$ for every initial condition — needs every $e^{\lambda t}$ going to zero, which asks for negative eigenvalues. But eigenvalues can be complex. Suppose $\lambda = -3 + 6i$. How big is $e^{(-3+6i)t}$? The factor $e^{6it} = \cos 6t + i\sin 6t$ has absolute value one — $\cos^2 + \sin^2 = 1$ — it just runs around the unit circle. The magnitude is $e^{-3t}$: it’s the real part of $\lambda$ that matters. The imaginary part only oscillates the solution between components.

So the three behaviors:

• Stability: $\mathbf{u}(t) \to 0$ when every eigenvalue has $\operatorname{Re}\lambda$ negative.
• Steady state: one eigenvalue $\lambda_1 = 0$ and all other eigenvalues have negative real part — exactly our example.
• Blow-up: if any eigenvalue has positive real part. Reverse all the signs of our matrix $A$ and every eigenvalue changes sign: the zero stays at zero, but the $-3$ becomes $+3$, an $e^{3t}$ appears, and the solution takes off.

Uncoupling the system: u = Sv

The matrix $A$ couples $u_1$ and $u_2$; the whole point of eigenvectors is to uncouple — to diagonalize. Set $\mathbf{u} = S\mathbf{v}$, where $S$ is the eigenvector matrix (a constant). Then

$$S\frac{d\mathbf{v}}{dt} = AS\mathbf{v} \qquad\Longrightarrow\qquad \frac{d\mathbf{v}}{dt} = S^{-1}AS\,\mathbf{v} = \Lambda \mathbf{v}$$

In the eigenvector basis the system is diagonal — no coupling anymore. Each equation stands alone: $dv_1/dt = \lambda_1 v_1$, and so on for all $n$ of them. Each is trivially solved, $\mathbf{v}(t) = e^{\Lambda t}\mathbf{v}(0)$, and transforming back,

$$\mathbf{u}(t) = S e^{\Lambda t} S^{-1}\, \mathbf{u}(0)$$

That matrix $S e^{\Lambda t} S^{-1}$ is the exponential we’re after: it is $e^{At}$, and $\mathbf{u}(t) = e^{At}\mathbf{u}(0)$ is the answer. But what does it mean to raise $e$ to a matrix?

What e^At means

Strang can’t resist the side comment: there are two beautiful Taylor series in this world. The exponential series, and the geometric series $1 + x + x^2 + x^3 + \cdots = \frac{1}{1-x}$ — the nicest power series of all, with no factorials. The matrix version gives

$$(I - At)^{-1} = I + At + (At)^2 + (At)^3 + \cdots$$

a reasonable way to approximate an inverse when $t$ is small: $(I - At)^{-1} \approx I + At$. But without the factorials this series can fail — if $At$ has an eigenvalue larger than one in magnitude, squaring and cubing blow it up. It converges only when all eigenvalues of $At$ are below one; the exponential series is safe always.

Now connect the definition to $S$ and $\Lambda$. Use the fundamental formula of this whole chapter, $A = S\Lambda S^{-1}$, in every term. Squaring makes the inner $S^{-1}S$ cancel: $A^2 = S\Lambda^2 S^{-1}$, then $A^3 = S\Lambda^3 S^{-1}$, and so on. Write the identity as $SS^{-1}$ too, and pull $S$ out on the left and $S^{-1}$ out on the right of the whole series:

$$e^{At} = S\left(I + \Lambda t + \frac{(\Lambda t)^2}{2} + \cdots\right)S^{-1}$$

And the exponential of the diagonal matrix $\Lambda$ is completely decoupled — diagonal, as the whole point demands:

$$e^{\Lambda t} = \begin{bmatrix} e^{\lambda_1 t} & & \\ & \ddots & \\ & & e^{\lambda_n t} \end{bmatrix}$$

just ordinary scalar exponentials down the diagonal, zeros elsewhere. From this formula you can see the stability statement again: $S$ and $S^{-1}$ aren’t moving, so $e^{At} \to 0$ exactly when every $e^{\lambda_i t} \to 0$ — every eigenvalue with negative real part.

Second order becomes first order

One final example: a single second-order equation,

$$y'' + by' + ky = 0$$

How do we make one second-order equation into a $2 \times 2$ first-order system? Just the way we did for Fibonacci: let the vector unknown be $\mathbf{u} = (y', y)$ and add the trivial extra equation $y' = y'$. Then

$$\mathbf{u}' = \begin{bmatrix} y'' \\ y' \end{bmatrix} = \begin{bmatrix} -b & -k \\ 1 & 0 \end{bmatrix} \begin{bmatrix} y' \\ y \end{bmatrix}$$

The first row is the differential equation ($y'' = -by' - ky$); the second row is the trivial one. In the general case — a fifth-order equation becoming a $5 \times 5$ first-order system — the coefficients of the equation go across the top row, and four trivial equations put ones below the diagonal. The eigenvalues of this matrix come out naturally connected to the differential equation.

That’s differential equations: a parallel lecture to powers of a matrix. We can now do exponentials.

Problems

Work these before revealing the solutions — the method is exactly the one from the lecture.

Problem 23.1 Solve du/dt = Au

Solve

$$\frac{d\mathbf{u}}{dt} = \begin{bmatrix} -2 & 1 \\ 1 & -2 \end{bmatrix}\mathbf{u}, \qquad \mathbf{u}(0) = \begin{bmatrix} 3 \\ 1 \end{bmatrix}$$

and describe the behavior as $t \to \infty$.

Show solution

Eigenvalues: trace $= -4$, determinant $= 4 - 1 = 3$, so $\lambda^2 + 4\lambda + 3 = (\lambda + 1)(\lambda + 3) = 0$: $\lambda_1 = -1$, $\lambda_2 = -3$.

Eigenvectors: for $\lambda_1 = -1$, $A + I = \begin{bmatrix} -1 & 1 \\ 1 & -1 \end{bmatrix}$ gives $\mathbf{x}_1 = (1, 1)$. For $\lambda_2 = -3$, $A + 3I = \begin{bmatrix} 1 & 1 \\ 1 & 1 \end{bmatrix}$ gives $\mathbf{x}_2 = (1, -1)$.

Constants: $c_1(1,1) + c_2(1,-1) = (3,1)$ means $c_1 + c_2 = 3$ and $c_1 - c_2 = 1$, so $c_1 = 2$, $c_2 = 1$.

$$\mathbf{u}(t) = 2e^{-t}\begin{bmatrix} 1 \\ 1 \end{bmatrix} + e^{-3t}\begin{bmatrix} 1 \\ -1 \end{bmatrix}$$

Both eigenvalues are negative, so the system is stable: $\mathbf{u}(t) \to \mathbf{0}$. The $e^{-3t}$ part dies faster, so for large $t$ the solution approaches zero along the direction $(1,1)$.

Problem 23.2 Trace–determinant test

Without computing any eigenvalues, decide whether each system $d\mathbf{u}/dt = A\mathbf{u}$ is stable:

$$A_1 = \begin{bmatrix} -1 & 3 \\ -2 & -1 \end{bmatrix}, \qquad A_2 = \begin{bmatrix} -5 & 2 \\ 1 & 1 \end{bmatrix}$$

Then verify your verdict on $A_1$ by finding its eigenvalues.

Show solution

$A_1$: trace $= -2$ (negative ✓), determinant $= (-1)(-1) - (3)(-2) = 1 + 6 = 7$ (positive ✓). Stable.

$A_2$: trace $= -4$ (negative), but determinant $= (-5)(1) - (2)(1) = -7$ (negative ✗). Not stable — a negative determinant means the eigenvalues are real with opposite signs, so one is positive and the solution blows up.

Check on $A_1$: $\lambda^2 + 2\lambda + 7 = 0$ gives $\lambda = -1 \pm i\sqrt{6}$. Complex eigenvalues — but the real part is $-1$, negative, so $|e^{\lambda t}| = e^{-t} \to 0$. Stable, as the test predicted; the imaginary part $\pm\sqrt{6}\,i$ only makes the solution oscillate on the way to zero.

Problem 23.3 Computing a matrix exponential

Find $e^{At}$ for

$$A = \begin{bmatrix} 1 & 1 \\ 0 & -1 \end{bmatrix}$$

using $e^{At} = S e^{\Lambda t} S^{-1}$, and check your answer at $t = 0$.

Show solution

$A$ is triangular, so the eigenvalues are on the diagonal: $\lambda_1 = 1$, $\lambda_2 = -1$.

Eigenvectors: for $\lambda_1 = 1$, $A - I = \begin{bmatrix} 0 & 1 \\ 0 & -2 \end{bmatrix}$ gives $\mathbf{x}_1 = (1, 0)$. For $\lambda_2 = -1$, $A + I = \begin{bmatrix} 2 & 1 \\ 0 & 0 \end{bmatrix}$ gives $\mathbf{x}_2 = (1, -2)$.

$$S = \begin{bmatrix} 1 & 1 \\ 0 & -2 \end{bmatrix}, \qquad S^{-1} = \begin{bmatrix} 1 & \tfrac{1}{2} \\ 0 & -\tfrac{1}{2} \end{bmatrix}$$

Then

$$e^{At} = \begin{bmatrix} 1 & 1 \\ 0 & -2 \end{bmatrix} \begin{bmatrix} e^{t} & 0 \\ 0 & e^{-t} \end{bmatrix} \begin{bmatrix} 1 & \tfrac{1}{2} \\ 0 & -\tfrac{1}{2} \end{bmatrix} = \begin{bmatrix} e^{t} & \tfrac{1}{2}(e^{t} - e^{-t}) \\ 0 & e^{-t} \end{bmatrix}$$

Check at $t = 0$: $e^{0} = 1$ and $\tfrac{1}{2}(1 - 1) = 0$, so $e^{A\cdot 0} = I$ — exactly as the series $I + At + \cdots$ requires.

Problem 23.4 Second order to first order

Convert the second-order equation

$$y'' + 4y' + 3y = 0$$

into a $2 \times 2$ first-order system $\mathbf{u}' = A\mathbf{u}$ with $\mathbf{u} = (y', y)$. Find the eigenvalues of $A$ and write the general solution $y(t)$. Is the equation stable?

Show solution

The first row of $A$ comes from the equation, $y'' = -4y' - 3y$; the second row is the trivial equation $y' = y'$:

$$A = \begin{bmatrix} -4 & -3 \\ 1 & 0 \end{bmatrix}$$

Eigenvalues: $\det(A - \lambda I) = \lambda^2 + 4\lambda + 3 = (\lambda + 1)(\lambda + 3) = 0$, so $\lambda = -1, -3$ — the characteristic polynomial of the matrix is exactly the characteristic polynomial of the original differential equation.

General solution: each eigenvalue contributes a pure exponential, so

$$y(t) = c_1 e^{-t} + c_2 e^{-3t}$$

Both eigenvalues are negative (equivalently: trace $-4$ negative, determinant $3$ positive), so the equation is stable — every solution decays to zero.