3

The "kernel" $k(x,y) = e^{-\|x-y\|} $ is used in the context of SVM. Here $x,y \in \mathbb R^n$ and $\|.\|$ is the Euclidean norm.

Is there a Hilbert space H and a map $\varphi:\mathbb R^n \rightarrow H$ such that $k(x,y) = <\varphi(x),\varphi(y)> $ for all $x,y \in \mathbb R^n$?

VictorZurkowski
  • 438
  • 2
  • 8

1 Answers1

2

Yes, this is guaranteed by the Moore–Aronszajn theorem.

$K(\mathbf{x},\mathbf{y}) = e^{-\|\mathbf{x} - \mathbf{y}\|}$ is a positive definite kernel. This means it is a symmetric function satisfying $$\sum_{i,j=1}^{n} c_i c_j K(\mathbf{x_i},\mathbf{x_j}) \ge 0$$ for all $n \in \mathbb{N}$, all $\mathbf{x_1},\dotsb ,\mathbf{x_n} \in \mathbb{R}^n$, and all $c_1, \dotsb , c_n \in \mathbb{R}$.

The Moore–Aronszajn theorem says that for each such function there exists a unique reproducing kernel Hilbert space with reproducing kernel $K(\mathbf{x},\mathbf{y})$.

That is, there must be a unique Hilbert space $H$ of functions $f: \mathbb{R}^n \to \mathbb{R}$ with a mapping $\phi: \mathbb{R}^n \to H$ such that $K(\mathbf{x},\mathbf{y}) = \langle \phi(\mathbf{x}), \phi(\mathbf{y}) \rangle$ for all $\mathbf{x}, \mathbf{y} \in \mathbb{R}^n$, and such that $K(\mathbf{x},\mathbf{y})$ satisfies the reproducing property: $$\langle f, K(\cdot, \mathbf{x}) \rangle = f(\mathbf{x}) \qquad \forall \mathbf{x} \in \mathbb{R}^n, \enspace \forall f \in H$$

While the reproducing kernel Hilbert space is unique for a given positive definite kernel $K(\mathbf{x}, \mathbf{y})$, there can be other mappings $\phi$ to other Hilbert spaces which also satisfy $K(\mathbf{x},\mathbf{y}) = \langle \phi(\mathbf{x}), \phi(\mathbf{y}) \rangle$ but without the reproducing property.

Tim Goodman
  • 3,100
  • 1
  • 17
  • 14
  • Tim, do you know a reference where the positive definite condition for $k$ is verified in $\mathbb R^n$ (with $n>1$)? – VictorZurkowski Oct 17 '16 at 02:13
  • 1
    Yes, this can be shown from a theorem by Schoenberg - see this answer and the references it links to: http://math.stackexchange.com/questions/248976/how-to-prove-that-exponential-kernel-is-positive-definite – Tim Goodman Oct 17 '16 at 09:05