Unsupervised Learning : PCA

Unsupervised Learning : PCA

Let's say we have a data set like this. So, this is a data set of examples x and R2 and let's say I want to reduce the dimension of the data from two-dimensional to one-dimensional. In other words, I would like to find a line onto which to project the data. So what seems like a good line onto which to project the data, it's a line like this, might be a pretty good choice.

SO In the above picture we will project the data onto some line such that their distance(which is shown in blue line) is min we selected red line.Now make some other line like magente color in picture so the distance is high in magenta so we will not select this 

One thing I didn't do was give a mathematical proof that the U1 and U2 and so on and the Z and so on you get out of this procedure is really the choices that would minimize these squared projection error. Right, remember we said What PCA tries to do is try to find a surface or line onto which to project the data so as to minimize to square projection error.

Suppose we run PCA with k = n, so that the dimension of the data is not reduced at all. (This is not useful in practice but is a good thought exercise.) Recall that the percent / fraction of variance retained is given by: $\displaystyle\frac{\sum_{i=1}^kS_{ii}}{\sum_{i=1}^nS_{ii}}$ Which of the following will be true? Check all that apply.

Ans

 U

${}_{\text{reduce}}$ will be an $n \times n$ matrix.

$x_{\text{approx}}=x$ for every example $x$.

The percentage of variance retained will be 100%.