0

My professor during the class mentioned (page 35) that for two vectors in high-dimensional space (say dimension $d$), we typically have $$ \left|\boldsymbol{u}^T \boldsymbol{v}\right| \approx \frac{\|\boldsymbol{u}\| \cdot\|\boldsymbol{v}\|}{\sqrt{d}} $$ Yet I am not sure how is it true. I have two somewhat conflicting understanding for this equation:

  1. For high-dimensional space, any two randomly drawn vectors are nearly perpendicular. This makes $\boldsymbol{u}^T \boldsymbol{v}$ almost 0, whereas the norm of the vector can be large. I am not sure how it can be that the absolute value of a small number (LHS) can equate to RHS.

  2. If both vectors are independently drawn from distributions of 0 mean and 1 variance (say Gaussian), then I can see that dot product $\boldsymbol{u}^T \boldsymbol{v}$ should be approximately Gaussian with 0 mean and $d$ variance. (Consider $\mathbb{E}[(\boldsymbol{u}^T \boldsymbol{v})^2] = \mathbb{E}[\sum_{i=1}^{d} u_i^2 \cdot \sum_{i=1}^{d} v_i^2]$. Since the components are independent, this simplifies to $d \cdot \mathbb{E}[u_i^2] \cdot \mathbb{E}[v_i^2]$, which equals $d$ as the variances are 1.) But still I am not sure how can it equate to RHS.

Any pointer is appreciated!

Math Attack
  • 1
Chris XU
  • 1
  • 1
    For part 1 - consider unit vectors $u,v$/divide through by the norms. Then it says $\langle u , v \rangle \approx \frac{1}{\sqrt{d}}$, so correlations decay by an inverse square root law in dimension. In the second part - use the precise statement of the CLT - you will pick up a $\sqrt{d}$ factor in the denominator like in the first because your variance is $d$ as you said – rubikscube09 Nov 10 '23 at 21:29
  • If you assume the probability distribution is invariant under rotations, then the density function is a function of the radial direction $r = |v|$. In that case, I believe that if you assume that $v$ and $w$ are independent, then the ratio of the expected values of both sides of the equation is independent of the radial function used. So you might as well assume the distribution is the standard Gaussian. I also believe that independence implies that you can fix $u = c\frac{1}{\sqrt{d}}(1, \dots, 1)$ and let $v$ be random. The rest of the computation should be straightforward. – Deane Nov 10 '23 at 21:41
  • Thanks @rubikscube09 @Deane! I think I understand now! – Chris XU Nov 10 '23 at 23:38

0 Answers0