★ Unit 3 checkpoint

Exam 3

The Unit 3 exam covers sessions 25 through 32 — symmetric matrices and their real eigenvalues, positive definite and semidefinite matrices, similar matrices, the singular value decomposition, and linear transformations. Three multi-part problems test classifying matrices by their SVD, diagonalizing and taking powers and exponentials of a matrix, and solving equations by expanding in an orthonormal eigenvector basis.

▶ Review lecture Exam on OCW ↗

Taking this exam

This exam covers all of Unit 3 (sessions 25–32): symmetric matrices, positive definite matrices and their tests, similar matrices and diagonalization, the singular value decomposition $A = U\Sigma V^T$ , and linear transformations. Attempt it closed-book under exam conditions before revealing any solutions — every problem here rewards thinking about eigenvalues and matrix classes rather than brute computation. If you get stuck, or once you’re done, watch the unit review video linked above, where a course TA works through problems of exactly this kind.

Problems

Problem 1 Exam 3

(a) If a square matrix $A$ has all $n$ of its singular values equal to $1$ in the SVD, what basic classes of matrices does $A$ belong to? (Singular, symmetric, orthogonal, positive definite or semidefinite, diagonal)

(b) Suppose the (orthonormal) columns of $H$ are eigenvectors of $B$ :

H = \frac{1}{2} \begin{bmatrix} 1 & 1 & -1 & -1 \\ 1 & -1 & -1 & 1 \\ 1 & 1 & 1 & 1 \\ 1 & -1 & 1 & -1 \end{bmatrix} \qquad H^{-1} = H^T

The eigenvalues of $B$ are $\lambda = 0, 1, 2, 3$ . Write $B$ as the product of 3 specific matrices. Write $C = (B + I)^{-1}$ as the product of 3 matrices.

(c) Using the list in question (a), which basic classes of matrices do $B$ and $C$ belong to? (Separate question for $B$ and $C$ )

Show solution

(a) If $\Sigma = I$ then $A = U\Sigma V^T = UV^T$ = product of orthogonal matrices = orthogonal matrix.

Second proof: all $\sigma_i = 1$ implies $A^TA = I$ . So $A$ is orthogonal.

( $A$ is never singular, and it won’t always be symmetric — take $U = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$ and $V = I$ , for example. This also shows it can’t be diagonal, or positive definite or semidefinite.)

(b) Diagonalization with the orthonormal eigenvector matrix $H$ :

B = H \Lambda H^{-1} \quad \text{with} \quad \Lambda = \begin{bmatrix} 0 & & & \\ & 1 & & \\ & & 2 & \\ & & & 3 \end{bmatrix}

For $C$ , adding $I$ shifts every eigenvalue by $1$ and keeps the same eigenvectors, and inverting reciprocates them:

(B + I)^{-1} = H (\Lambda + I)^{-1} H^{-1} \quad \text{with} \quad (\Lambda + I)^{-1} = \begin{bmatrix} 1 & & & \\ & 1/2 & & \\ & & 1/3 & \\ & & & 1/4 \end{bmatrix}

(c) $B$ is singular, symmetric, positive semidefinite (its eigenvalues $0, 1, 2, 3$ are real and nonnegative, with one zero, and it has orthonormal eigenvectors).

$C$ is symmetric positive definite (its eigenvalues $1, \tfrac{1}{2}, \tfrac{1}{3}, \tfrac{1}{4}$ are all positive).

Problem 2 Exam 3

(a) Find three eigenvalues of $A$ , and an eigenvector matrix $S$ :

A = \begin{bmatrix} -1 & 2 & 4 \\ 0 & 0 & 5 \\ 0 & 0 & 1 \end{bmatrix}

(b) Explain why $A^{1001} = A$ . Is $A^{1000} = I$ ? Find the three diagonal entries of $e^{At}$ .

(c) The matrix $A^TA$ (for the same $A$ ) is

A^TA = \begin{bmatrix} 1 & -2 & -4 \\ -2 & 4 & 8 \\ -4 & 8 & 42 \end{bmatrix}.

How many eigenvalues of $A^TA$ are positive? zero? negative? (Don’t compute them but explain your answer.) Does $A^TA$ have the same eigenvectors as $A$ ?

Show solution

(a) The eigenvalues are $-1, 0, 1$ since $A$ is triangular (they sit on the diagonal).

\lambda = -1 \text{ has } \mathbf{x} = \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix}, \qquad \lambda = 0 \text{ has } \mathbf{x} = \begin{bmatrix} 2 \\ 1 \\ 0 \end{bmatrix}, \qquad \lambda = 1 \text{ has } \mathbf{x} = \begin{bmatrix} 7 \\ 5 \\ 1 \end{bmatrix}.

Those vectors $\mathbf{x}$ are the columns of $S$ (upper triangular!):

S = \begin{bmatrix} 1 & 2 & 7 \\ 0 & 1 & 5 \\ 0 & 0 & 1 \end{bmatrix}

(b) $A = S\Lambda S^{-1}$ and $A^{1001} = S\Lambda^{1001}S^{-1}$ . Notice $\Lambda^{1001} = \Lambda$ , since $(-1)^{1001} = -1$ , $0^{1001} = 0$ , and $1^{1001} = 1$ — so $A^{1001} = S\Lambda S^{-1} = A$ .

But $A^{1000} \neq I$ : $A$ is singular, and $0^{1000} = 0 \neq 1$ , so $\Lambda^{1000} \neq I$ .

$e^{At}$ has $e^{-t}$ , $e^{0t} = 1$ , $e^{t}$ on its diagonal. Proof using series: $\sum_{n=0}^{\infty} (At)^n / n!$ has triangular matrices, so the diagonal entries are $\sum (-t)^n/n! = e^{-t}$ , $\sum 0^n/n! = 1$ , and $\sum t^n/n! = e^{t}$ .

Proof using $S\Lambda S^{-1}$ :

e^{At} = S e^{\Lambda t} S^{-1} = \begin{bmatrix} 1 & \times & \times \\ 0 & 1 & \times \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} e^{-t} & & \\ & 1 & \\ & & e^{t} \end{bmatrix} \begin{bmatrix} 1 & \times & \times \\ 0 & 1 & \times \\ 0 & 0 & 1 \end{bmatrix}.

All three factors are triangular, so the diagonal of the product is $e^{-t}, 1, e^{t}$ .

(c) $A^TA$ has 2 positive eigenvalues (it has rank 2, and eigenvalues of $A^TA$ can never be negative). One eigenvalue is zero because $A^TA$ is singular. And $3 - 2 = 1$ , so none are negative.

(Or: $A^TA$ is symmetric, so the eigenvalues have the same signs as the pivots. Do elimination: the pivots are $1$ , $0$ , and $42 - 16 = 26$ .)

$A^TA$ does not have the same eigenvectors as $A$ : since $A^TA$ is symmetric, its eigenvectors are orthogonal, while the eigenvectors of $A$ (the columns of $S$ above) are not.

Problem 3 Exam 3

Suppose the $n$ by $n$ matrix $A$ has $n$ orthonormal eigenvectors $\mathbf{q}_1, \ldots, \mathbf{q}_n$ and $n$ positive eigenvalues $\lambda_1, \ldots, \lambda_n$ . Thus $A\mathbf{q}_j = \lambda_j \mathbf{q}_j$ .

(a) What are the eigenvalues and eigenvectors of $A^{-1}$ ? Prove that your answer is correct.

(b) Any vector $\mathbf{b}$ is a combination of the eigenvectors:

\mathbf{b} = c_1\mathbf{q}_1 + c_2\mathbf{q}_2 + \cdots + c_n\mathbf{q}_n \,.

What is a quick formula for $c_1$ using orthogonality of the $\mathbf{q}$ ‘s?

(c) The solution to $A\mathbf{x} = \mathbf{b}$ is also a combination of the eigenvectors:

A^{-1}\mathbf{b} = d_1\mathbf{q}_1 + d_2\mathbf{q}_2 + \cdots + d_n\mathbf{q}_n \,.

What is a quick formula for $d_1$ ? You can use the $c$ ‘s even if you didn’t answer part (b).

Show solution

(a) $A^{-1}$ has eigenvalues $\dfrac{1}{\lambda_j}$ with the same eigenvectors $\mathbf{q}_j$ . Proof — start from the eigenvalue equation and multiply by $A^{-1}$ :

A\mathbf{q}_j = \lambda_j \mathbf{q}_j \;\longrightarrow\; \mathbf{q}_j = \lambda_j A^{-1}\mathbf{q}_j \;\longrightarrow\; A^{-1}\mathbf{q}_j = \frac{1}{\lambda_j}\,\mathbf{q}_j \,.

(All $\lambda_j > 0$ , so dividing by $\lambda_j$ is legal and $A^{-1}$ exists.)

(b) Multiply $\mathbf{b} = c_1\mathbf{q}_1 + \cdots + c_n\mathbf{q}_n$ by $\mathbf{q}_1^T$ . Orthogonality kills every term but the first: $\mathbf{q}_1^T\mathbf{b} = c_1\mathbf{q}_1^T\mathbf{q}_1$ , so

c_1 = \frac{\mathbf{q}_1^T\mathbf{b}}{\mathbf{q}_1^T\mathbf{q}_1} = \mathbf{q}_1^T\mathbf{b}

(the last step because the $\mathbf{q}$ ‘s are orthonormal, so $\mathbf{q}_1^T\mathbf{q}_1 = 1$ ).

(c) Multiplying $\mathbf{b}$ by $A^{-1}$ multiplies each $\mathbf{q}_i$ by $\dfrac{1}{\lambda_i}$ (part (a)). So each $c_i$ becomes $c_i/\lambda_i$ , and

d_1 = \frac{c_1}{\lambda_1} \qquad \left( = \frac{\mathbf{q}_1^T\mathbf{b}}{\lambda_1\,\mathbf{q}_1^T\mathbf{q}_1} \;\text{ or }\; \frac{\mathbf{q}_1^T\mathbf{b}}{\lambda_1} \right).