★ Unit 2 checkpoint

Exam 2

The Unit 2 exam covers sessions 14 through 24 — orthogonality of vectors and subspaces, projections onto subspaces, least squares approximations, Gram-Schmidt and orthonormal bases, and the properties and formulas for determinants. Four multi-part problems test determinants of orthogonal matrices, fitting a line by least squares, projection matrices from Gram-Schmidt, and the determinant as a polynomial via the big formula.

▶ Review lecture Exam on OCW ↗

Taking this exam

This exam covers Unit 2 (sessions 14–24): orthogonal vectors and subspaces, projections onto subspaces and projection matrices, least squares, orthonormal bases and Gram-Schmidt, and the properties, formulas, and applications of determinants. Attempt it closed-book under exam conditions before revealing any solutions — every problem here rewards reasoning from the properties, not brute computation. If you get stuck, or once you’re done, watch the unit review video linked above, where a course TA works through problems of exactly this kind.

Problems

Problem 1 Exam 2

Suppose $q_1, q_2, q_3$ are orthonormal vectors in $\mathbb{R}^3$ . Find all possible values for these 3 by 3 determinants and explain your thinking in 1 sentence each.

(a) $\det \begin{bmatrix} q_1 & q_2 & q_3 \end{bmatrix} =$

(b) $\det \begin{bmatrix} q_1 + q_2 & q_2 + q_3 & q_3 + q_1 \end{bmatrix} =$

(c) $\det \begin{bmatrix} q_1 & q_2 & q_3 \end{bmatrix}$ times $\det \begin{bmatrix} q_2 & q_3 & q_1 \end{bmatrix} =$

Show solution

(a) The determinant of any square matrix with orthonormal columns (an “orthogonal matrix”) is $\pm 1$ .

(b) Here are two ways you could do this.

(1) The determinant is linear in each column:

\begin{aligned} \det \begin{bmatrix} q_1 + q_2 & q_2 + q_3 & q_3 + q_1 \end{bmatrix} &= \det \begin{bmatrix} q_1 & q_2 + q_3 & q_3 + q_1 \end{bmatrix} + \det \begin{bmatrix} q_2 & q_2 + q_3 & q_3 + q_1 \end{bmatrix} \\ &= \det \begin{bmatrix} q_1 & q_2 + q_3 & q_3 \end{bmatrix} + \det \begin{bmatrix} q_2 & q_3 & q_3 + q_1 \end{bmatrix} \\ &= \det \begin{bmatrix} q_1 & q_2 & q_3 \end{bmatrix} + \det \begin{bmatrix} q_2 & q_3 & q_1 \end{bmatrix} \end{aligned}

(a repeated column kills a determinant, so the extra terms drop out). Both of these determinants are equal — see part (c) — so the total determinant is $\pm 2$ .

(2) You could also use row reduction (column operations don’t change the determinant, except that a swap flips the sign and a scaling factors out):

\begin{aligned} \det \begin{bmatrix} q_1 + q_2 & q_2 + q_3 & q_3 + q_1 \end{bmatrix} &= \det \begin{bmatrix} q_1 + q_2 & -q_1 + q_3 & q_3 + q_1 \end{bmatrix} \\ &= \det \begin{bmatrix} q_1 + q_2 & -q_1 + q_3 & 2q_3 \end{bmatrix} \\ &= 2 \det \begin{bmatrix} q_1 + q_2 & -q_1 + q_3 & q_3 \end{bmatrix} \\ &= 2 \det \begin{bmatrix} q_1 + q_2 & -q_1 & q_3 \end{bmatrix} \\ &= 2 \det \begin{bmatrix} q_2 & -q_1 & q_3 \end{bmatrix} \\ &= 2 \det \begin{bmatrix} q_1 & q_2 & q_3 \end{bmatrix} \end{aligned}

Again, whatever $\det \begin{bmatrix} q_1 & q_2 & q_3 \end{bmatrix}$ is, this determinant will be twice that, so $\pm 2$ .

(c) The second matrix is an even permutation of the columns of the first matrix (swap $q_1/q_2$ , then swap $q_2/q_3$ ), so it has the same determinant as the first matrix. Whether the first matrix has determinant $+1$ or $-1$ , the product will be $+1$ .

Problem 2 Exam 2

Suppose we take measurements at the 21 equally spaced times $t = -10, -9, \ldots, 9, 10$ . All measurements are $b_i = 0$ except that $b_{11} = 1$ at the middle time $t = 0$ .

(a) Using least squares, what are the best $\widehat{C}$ and $\widehat{D}$ to fit those 21 points by a straight line $C + Dt$ ?

(b) You are projecting the vector $\mathbf{b}$ onto what subspace? (Give a basis.) Find a nonzero vector perpendicular to that subspace.

Show solution

(a) If the line went exactly through the 21 points, then the 21 equations

\begin{bmatrix} 1 & -10 \\ 1 & -9 \\ \vdots & \vdots \\ 1 & 0 \\ \vdots & \vdots \\ 1 & 10 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ \vdots \\ 1 \\ \vdots \\ 0 \end{bmatrix}

would be exactly solvable. Since we can’t solve this equation $A\mathbf{x} = \mathbf{b}$ exactly, we look for a least-squares solution $A^{\mathrm{T}}A\hat{\mathbf{x}} = A^{\mathrm{T}}\mathbf{b}$ :

\begin{bmatrix} 21 & 0 \\ 0 & 770 \end{bmatrix} \begin{bmatrix} \widehat{C} \\ \widehat{D} \end{bmatrix} = \begin{bmatrix} 1 \\ 0 \end{bmatrix}

So the line of best fit is the horizontal line

\widehat{C} = \frac{1}{21}, \qquad \widehat{D} = 0.

(b) We are projecting $\mathbf{b}$ onto the column space of the matrix $A$ above — a basis is the two columns

\begin{bmatrix} 1 & 1 & \cdots & 1 \end{bmatrix}^{\mathrm{T}}, \qquad \begin{bmatrix} -10 & -9 & \cdots & 10 \end{bmatrix}^{\mathrm{T}}.

There are lots of vectors perpendicular to this subspace; one is the error vector

\mathbf{e} = \mathbf{b} - P_A\mathbf{b} = \frac{1}{21}\begin{bmatrix} (\text{ten } -1\text{'s}) & 20 & (\text{ten } -1\text{'s}) \end{bmatrix}^{\mathrm{T}}.

Problem 3 Exam 2

The Gram-Schmidt method produces orthonormal vectors $q_1, q_2, q_3$ from independent vectors $a_1, a_2, a_3$ in $\mathbb{R}^5$ . Put those vectors into the columns of 5 by 3 matrices $Q$ and $A$ .

(a) Give formulas using $Q$ and $A$ for the projection matrices $P_Q$ and $P_A$ onto the column spaces of $Q$ and $A$ .

(b) Is $P_Q = P_A$ and why? What is $P_Q$ times $Q$ ? What is $\det P_Q$ ?

(c) Suppose $a_4$ is a new vector and $a_1, a_2, a_3, a_4$ are independent. Which of these (if any) is the new Gram-Schmidt vector $q_4$ ? ( $P_A$ and $P_Q$ from above)

\textbf{1.}\ \ \frac{P_Q a_4}{\lVert P_Q a_4 \rVert} \qquad \textbf{2.}\ \ \frac{a_4 - \dfrac{a_4^{\mathrm{T}} a_1}{a_1^{\mathrm{T}} a_1}a_1 - \dfrac{a_4^{\mathrm{T}} a_2}{a_2^{\mathrm{T}} a_2}a_2 - \dfrac{a_4^{\mathrm{T}} a_3}{a_3^{\mathrm{T}} a_3}a_3}{\lVert\ \text{norm of that vector}\ \rVert} \qquad \textbf{3.}\ \ \frac{a_4 - P_A a_4}{\lVert a_4 - P_A a_4 \rVert}

Show solution

(a) The general projection formula gives

P_A = A(A^{\mathrm{T}}A)^{-1}A^{\mathrm{T}} \qquad \text{and} \qquad P_Q = Q(Q^{\mathrm{T}}Q)^{-1}Q^{\mathrm{T}} = QQ^{\mathrm{T}},

since $Q^{\mathrm{T}}Q = I$ for a matrix with orthonormal columns.

(b) $P_A = P_Q$ , because both projections project onto the same subspace — Gram-Schmidt doesn’t change the column space. (Some people did this the hard way, by substituting $A = QR$ into the projection formula and simplifying. That also works.)

$P_Q Q = Q$ : the columns of $Q$ are already in the column space, so projecting them changes nothing.

$\det P_Q = 0$ , because $P_Q$ is singular (like all non-identity projections): all vectors orthogonal to the column space of $Q$ are projected to $\mathbf{0}$ .

(c) Answer: choice 3,

q_4 = \frac{a_4 - P_A a_4}{\lVert a_4 - P_A a_4 \rVert}.

Subtracting the projection onto the column space leaves the component of $a_4$ perpendicular to $q_1, q_2, q_3$ ; dividing by its length normalizes it. Choice 1 fails because $P_Q a_4$ lies in the subspace rather than perpendicular to it. Choice 2 is tempting, and would be correct if the $a_i$ were replaced by the $q_i$ — but the $a_i$ are not orthogonal!

Problem 4 Exam 2

Suppose a 4 by 4 matrix has the same entry $\times$ throughout its first row and column. The other 9 numbers could be anything like $1, 5, 7, 2, 3, 99, \pi, e, 4$ :

A = \begin{bmatrix} \times & \times & \times & \times \\ \times & & & \\ \times & & \text{any numbers} & \\ \times & & & \end{bmatrix}

(a) The determinant of $A$ is a polynomial in $\times$ . What is the largest possible degree of that polynomial? Explain your answer.

(b) If those 9 numbers give the identity matrix $I$ , what is $\det A$ ? Which values of $\times$ give $\det A = 0$ ?

A = \begin{bmatrix} \times & \times & \times & \times \\ \times & 1 & 0 & 0 \\ \times & 0 & 1 & 0 \\ \times & 0 & 0 & 1 \end{bmatrix}

Show solution

(a) Every term in the big formula for $\det A$ takes one entry from each row and each column. The $\times$ ‘s occupy row 1 and column 1, and a term can use at most one entry from row 1 and one from column 1 — so each term contains at most two $\times$ ‘s, and the determinant has degree 2.

(b) You can find this by cofactor expansion; here’s another way. Factor $\times$ out of the first column, then subtract each of columns 2, 3, 4 from column 1 — that removes the three $1$ ‘s below the top and changes the corner entry to $1 - 3\times$ , leaving a triangular matrix:

\det A = \times \det \begin{bmatrix} 1 & \times & \times & \times \\ 1 & 1 & 0 & 0 \\ 1 & 0 & 1 & 0 \\ 1 & 0 & 0 & 1 \end{bmatrix} = \times \det \begin{bmatrix} 1 - 3\times & \times & \times & \times \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} = \times(1 - 3\times) \det \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}

\det A = \times(1 - 3\times).

This is zero when $\times = 0$ or $\times = \tfrac{1}{3}$ .