Problem 25, Section 6.3

We'll do this one as practice for the sorts of things you'd do for problem 26 from the same section. As always, the path of least resistance is to follow the instructions.

In [7]:
from sympy import *
init_printing(use_latex='mathjax')

They refer me back to exercise 36 in section 6.2, where I'm supposed to find this matrix $U$. Looking back there, you'll see that the construction of $U$ takes as its starting point another matrix:

In [8]:
A=Matrix([[-6,-3,6,1],
          [-1,2,1,-6],
          [3,6,3,-2],
          [6,-3,6,-1],
          [2,-1,2,3],
          [-3,6,3,2],
          [-2,-1,2,-3],
          [1,2,1,6]])
A
Out[8]:
$\displaystyle \left[\begin{matrix}-6 & -3 & 6 & 1\\-1 & 2 & 1 & -6\\3 & 6 & 3 & -2\\6 & -3 & 6 & -1\\2 & -1 & 2 & 3\\-3 & 6 & 3 & 2\\-2 & -1 & 2 & -3\\1 & 2 & 1 & 6\end{matrix}\right]$

Then, $U$ is supposed to be obtained from $A$ by "normalizing each column of $A$". What this means is that we need to scale each column by its norm, so as to ensure the new columns have norm $1$. First, let's see what the lengths (or norms) of the columns are to begin with; they'll be the square roots of the diagonal entries of $A^TA$:

In [9]:
A.T*A
Out[9]:
$\displaystyle \left[\begin{matrix}100 & 0 & 0 & 0\\0 & 100 & 0 & 0\\0 & 0 & 100 & 0\\0 & 0 & 0 & 100\end{matrix}\right]$

OK, so that tells me that

  • the columns of $A$ were orthogonal (i.e. the angle between each two is $\frac {\pi}2$);
  • and the squared length of each column is $100$, meaning that its length is $10$.

In short, to get myself the matrix $U$ all I have to do is take $A$ and scale it down by $10$:

In [10]:
U=A/10
U
Out[10]:
$\displaystyle \left[\begin{matrix}- \frac{3}{5} & - \frac{3}{10} & \frac{3}{5} & \frac{1}{10}\\- \frac{1}{10} & \frac{1}{5} & \frac{1}{10} & - \frac{3}{5}\\\frac{3}{10} & \frac{3}{5} & \frac{3}{10} & - \frac{1}{5}\\\frac{3}{5} & - \frac{3}{10} & \frac{3}{5} & - \frac{1}{10}\\\frac{1}{5} & - \frac{1}{10} & \frac{1}{5} & \frac{3}{10}\\- \frac{3}{10} & \frac{3}{5} & \frac{3}{10} & \frac{1}{5}\\- \frac{1}{5} & - \frac{1}{10} & \frac{1}{5} & - \frac{3}{10}\\\frac{1}{10} & \frac{1}{5} & \frac{1}{10} & \frac{3}{5}\end{matrix}\right]$

And to double check that indeed now the columns are orthonormal:

In [17]:
U.T*U
Out[17]:
$\displaystyle \left[\begin{matrix}1 & 0 & 0 & 0\\0 & 1 & 0 & 0\\0 & 0 & 1 & 0\\0 & 0 & 0 & 1\end{matrix}\right]$

Next, I have to work with this column-vector ${\bf y}$ that's all $1$. The easy way to define it:

In [18]:
y=Matrix([1]*8)
y
Out[18]:
$\displaystyle \left[\begin{matrix}1\\1\\1\\1\\1\\1\\1\\1\end{matrix}\right]$

Finally, we are asked for "the closest point to ${\bf y}$ in $\mathrm{Col}~U$". That's the orthogonal projection of ${\bf y}$ on the column space of $U$. According to Theorem 10 in the book, since I already have an orthonormal basis for $\mathrm{Col}~U$ as the columns of $U$, that projection is nothing but $UU^T{\bf y}$:

In [19]:
proj_y=U*U.T*y
proj_y
Out[19]:
$\displaystyle \left[\begin{matrix}\frac{6}{5}\\\frac{2}{5}\\\frac{6}{5}\\\frac{6}{5}\\\frac{2}{5}\\\frac{6}{5}\\\frac{2}{5}\\\frac{2}{5}\end{matrix}\right]$

Let's also check, for added certainty, that this vector proj_y is indeed contained in the columns space of $U$. There are many ways to do this. You could, for instance, check whether the linear system

$$ U{\bf x} = \mathrm{projy} $$

is consistent. I'll do something else though: I'll simply check that $U$ and the matrix obtained to adding proj_y to it as a column have the same rank or, equivalently, the same column space. Here:

In [20]:
U.rank()
Out[20]:
$\displaystyle 4$

Now add proj_y to $U$ as the first column, say (it doesn't matter which one it is, since we're just interested in the dimension of the column space).

In [21]:
U.col_insert(0,proj_y)
Out[21]:
$\displaystyle \left[\begin{matrix}\frac{6}{5} & - \frac{3}{5} & - \frac{3}{10} & \frac{3}{5} & \frac{1}{10}\\\frac{2}{5} & - \frac{1}{10} & \frac{1}{5} & \frac{1}{10} & - \frac{3}{5}\\\frac{6}{5} & \frac{3}{10} & \frac{3}{5} & \frac{3}{10} & - \frac{1}{5}\\\frac{6}{5} & \frac{3}{5} & - \frac{3}{10} & \frac{3}{5} & - \frac{1}{10}\\\frac{2}{5} & \frac{1}{5} & - \frac{1}{10} & \frac{1}{5} & \frac{3}{10}\\\frac{6}{5} & - \frac{3}{10} & \frac{3}{5} & \frac{3}{10} & \frac{1}{5}\\\frac{2}{5} & - \frac{1}{5} & - \frac{1}{10} & \frac{1}{5} & - \frac{3}{10}\\\frac{2}{5} & \frac{1}{10} & \frac{1}{5} & \frac{1}{10} & \frac{3}{5}\end{matrix}\right]$

And its rank:

In [16]:
U.col_insert(0,proj_y).rank()
Out[16]:
$\displaystyle 4$

Yes: the same number!