-1

I have the following question:

Given 3 points (-1, 1), (0, 0), (1, 1). What's the first principal component and what are the coordinates of the projected data points? What would be the variance of the projected data? How to reconstruct the original data points and what would be the reconstruction error (squared)?

Am I right that the first PC would just be a horizontal line at y = 0.5 because the PC is the best fit line where the variance of the data is the maximum? And the variance would be 2 in that case?

Any help much appreciated.

1 Answers1

2

I think you may have forgotten to subtract the mean. As far as I know you have to center the data, otherwise you will compute variance with respect to the origin, rather than the variance within the data.

Your data points have a mean vector

$$\mu = \left[0, \frac{2}{3}\right] $$

Let's subtract the mean from the data and put it into a matrix

$$ \tilde{X} = \begin{bmatrix} -1 & \frac{1}{3} \\ 0 & \frac{-2}{3} \\ 1 & \frac{1}{3} \end{bmatrix}$$

The covariance matrix $\Sigma$ is now given by

$$ \Sigma = \frac{1}{3} \tilde{X}^T \cdot \tilde{X} = \begin{bmatrix} \frac{2}{3} & 0 \\ 0 & \frac{2}{9} \end{bmatrix}$$

The $\frac{1}{3}$ is because the covariance matrix involves an expectation value. This is neat, $\Sigma$ is alread diagonal, so the first primary component is the x-axis with a variance of $\frac{2}{3}$. That is because the direction of largest variance is the eigenvector with the largest eigenvalue of $\Sigma$ (for why that is, see here).

matthiaw91
  • 1,545
  • 5
  • 17