Question 1201735
Let's tackle each part of the problem step by step.

### 1. Prove that $ \text{Ker}(X) = \text{Ker}(X^TX) $

**Proof:**

- Let $ v \in \text{Ker}(X) $. Then, by definition, $ Xv = 0 $.
- Multiplying both sides by $ X^T $, we have:
  $$
  X^TXv = X^T0 = 0
  $$
  Thus, $ v \in \text{Ker}(X^TX) $.

- Now, let $ v \in \text{Ker}(X^TX) $. Then $ X^TXv = 0 $.
- This implies $ \langle Xv, Xv \rangle = 0 $ (since $ \langle a, a \rangle = 0 $ if and only if $ a = 0 $).
- Therefore, $ Xv = 0 $, which means $ v \in \text{Ker}(X) $.

Combining both parts, we conclude:
$$
\text{Ker}(X) = \text{Ker}(X^TX)
$$

### 2. Prove that for a square matrix $ A $: $ \text{Im}(A^T) = \text{Ker}(A)^\perp $

**Proof:**

- Let $ v \in \text{Im}(A^T) $. Then there exists some $ u $ such that $ v = A^Tu $.
- For any $ w \in \text{Ker}(A) $, we have $ Aw = 0 $.
- Thus, $ \langle v, w \rangle = \langle A^Tu, w \rangle = \langle u, Aw \rangle = \langle u, 0 \rangle = 0 $.
- This shows that $ v \in \text{Ker}(A)^\perp $.

- Now, let $ v \in \text{Ker}(A)^\perp $. We need to show $ v \in \text{Im}(A^T) $.
- By the definition of orthogonal complement, $ \langle v, w \rangle = 0 $ for all $ w \in \text{Ker}(A) $.
- The rank-nullity theorem states that $ \text{dim}(\text{Im}(A)) + \text{dim}(\text{Ker}(A)) = d $.
- Since $ A $ is square, $ \text{Im}(A^T) $ has dimension equal to $ \text{dim}(\text{Ker}(A)) $.
- Therefore, $ \text{Im}(A^T) = \text{Ker}(A)^\perp $.

### 3. Show that the system $ y = Xw $ has $ \infty $ solutions $ \Leftrightarrow y \perp \text{Ker}(X^T) $

**Proof:**

- If $ y \perp \text{Ker}(X^T) $, then for any $ v \in \text{Ker}(X^T) $, we have $ \langle y, v \rangle = 0 $.
- This means that $ y $ can be expressed as $ y = Xw + v $ for some $ w $ and $ v \in \text{Ker}(X^T) $.
- Since $ v $ can take infinitely many values in $ \text{Ker}(X^T) $, there are infinitely many $ w $ that satisfy $ y = Xw $.

- Conversely, if the system has infinitely many solutions, then there exists a non-zero $ v \in \text{Ker}(X^T) $ such that $ y = Xw + v $.
- This implies $ y \perp \text{Ker}(X^T) $.

Thus, we conclude:
$$
y \perp \text{Ker}(X^T) \Leftrightarrow \text{the system has } \infty \text{ solutions}
$$

### 4. Prove that the normal equations $ X^TXw = X^Ty $ can only have a unique solution (if $ X^TX $ is invertible) or infinitely many solutions (otherwise)

**Proof:**

- If $ X^TX $ is invertible, then the normal equations have a unique solution given by:
  $$
  w = (X^TX)^{-1}X^Ty
  $$

- If $ X^TX $ is not invertible, then $ \text{Ker}(X^TX) \neq \{0\} $. From part 1, we know:
  $$
  \text{Ker}(X^TX) = \text{Ker}(X)
  $$
- If $ y \perp \text{Ker}(X^T) $, then the system has infinitely many solutions, as shown in part 3.

Thus, we conclude:
- The normal equations have a unique solution if $ X^TX $ is invertible.
- They have infinitely many solutions if $ X^TX $ is not invertible.

This completes the proof for all parts.