18

I understand that each kernel implies a particular feature map. For instance for $x,z \in R^2$ the kernel $K(x,z)=(\textrm{dot}(x,z))^2$ implies a feature map $$\langle\phi(x_1), \phi(x_2)\rangle=\langle [x_1^2 , x_1 x_2 , x_1 x_2, x_2^2], [z_1^2 , z_1 z_2 , z_1 z_2, z_2^2] \rangle$$, as though my inputs are transformed from $R^2$ to $R^4$ in the process of computing $K(x,z)$.

The polynomial kernels are clear enough to see in this way. However the radial basis function kernel in particular, i.e. $K(x,z) = \exp(-\|x-z\|^2/c^2)$, supposedly implies an infinite dimensional feature map. Exactly why is this so?

TenaliRaman
  • 3,846

1 Answers1

19

Try expanding the exponential into an infinite series, and you'll see it. Abu-Mostafa has a great demonstration of this, in his classic style.

bright-star
  • 1,161