1

I'm taking an introductory course in Machine Learning and we are looking at all the requirements a Vector Space must fufill to be a Hilbert Space. I'm not a very math versed person, i had to recently learn the inner product, equivalence relations, and what it even meant to use functions as vectors. I think those concepts are fine for me now but the Cauchy Criterion is where I feel completly lost. I have watched some Youtube videos on the matter, they usued a number line from 0 to 1 and points in between from which the distance in between to consecuitive points had to be less then a value epsilon. While the Cauchy Criterion made sense in such context I fail to see how it applies to a vector space of functions,discrete functions or random valued functions.

I though that the vectors that I would encounter in a Hilbert space would be functions like sinusoids or some signal's that decay and are convergent. Are all vectors on a Hilbert Space then part of a Cauchy sequence? Can, for example, sin(x) not be part of a Hilbert space because it's not part of a Cauchy sequence? Can there be elements of a cauchy sequence and other elements that are not part of a cauchy sequence? What if there are no other elements of a cauchy sequence in my vector space and just normal linear algebra vectors, if I use a Hilbert Space inner product will it still fufill the requirements for it to be a Hilbert Space?

hectorrmzla
  • 23
  • Your last paragraph makes little sense to me. – J. De Ro Jan 12 '21 at 13:48
  • Perhaps this question can give you insight in the Cauchy condition: https://math.stackexchange.com/questions/112168/c0-1-is-not-hilbert-space?rq=1 – J. De Ro Jan 12 '21 at 13:49
  • you need to take into account three definitions. 1) A sequence in set $X$ is a assignment $\mathbb N\to X$. 2) If $X$ has a metric function $d$ then a sequence $x_n$ in $X$ is dubbed Cauchy sequence if $\forall\varepsilon>0$ there exists a natural number $N$ such that $d(x_k,x_l)<\varepsilon$, for each $k,l\ge N$. 3) A metric space $(X,d)$ is called complete if each Cauchy sequence in $X$ its limit in $X$ itself. The Hilbert spaces are complete vector spaces. – janmarqz Mar 28 '22 at 16:09

2 Answers2

1

A Hilbert space is a set with elements and some operations. In your intended application these elements could be functions. All we care about is that we can add and substract them, we have a $0$ and scalar multiplication (they form a vector space) and we have an inner product $\langle x,y\rangle$ for any $x,y$ in the set, fulfilling some axioms (inspired by those on $\Bbb R^n$, the "mother of all Hilbert spaces"). This inner product allows us to define a norm (a notion of length) by defining $\|x\|=\sqrt{\langle x,x\rangle}$ and a norm allows us to define a distance (metric) on the set by setting $d(x,y)=\|x-y\|$ for this norm from the inner product. And whenever we have a set with a distance we can define a Cauchy sequence. Also when the elements are functions and the inner product is defined by some series or integral, e.g., those are just "implemntation details", as long as the defining axioms are fulfilled (for vector space and inner product) we can talk about Cauchy sequences of those elements. Constant sequences are always Cauchy (regardless of the space), and so always is any convergent sequence (the completeness condition for a Hilbert space says that the reverse must also be true: any Cauchy sequence is convergent).

The elements themselves don't matter, the distance function on them is the crucial element.

Henno Brandsma
  • 242,131
  • What do you mean by constant sequences here? If I have lets say v1=sin(x) v2=cos(x) and take their lenght I dont really get much and thats a sequence of vectors right? v1 v2 – hectorrmzla Jan 12 '21 at 17:00
  • @hectorrmzla $x_n = \sin x$ for all $n$ is a possible constant sequence. A sequence is a map that assigns for each $n$ a “vector”. What the distance between your $v_1$ and $v_2$ is depends on how you defined the inner product. – Henno Brandsma Jan 12 '21 at 17:05
  • I really appreciate your response. But I'm still confused as to when most definitons I have found go into every sequence in the space. Im still not sure what that even means. If we had defined the inner product as the integral in between the first element and the complex conjugate of the second the length in between sin(x) and cos(x) would not converge anywhere? What gives then? – hectorrmzla Jan 12 '21 at 17:46
1

I think it is easier to explain a space that wouldn't be Cauchy. We can estimate pi with the below equation. So this equation is a cauchy sequence in the rational numbers space. However pi is not part of the rational numbers space. So rational numbers are not a hilbert space. enter image description here

Why do we want this?

It makes sense, just like when we scale numbers we want the value also to be in the space. This is just an extra restriction.

You don't want to be doing limit operations within our space and accidentally end up in undefined. If we have this restriction method we can use limits when solving equations.

It is also not saying that every sequance is Cauchy. It is also not saying that sin(X) is cauchy. you can have sin(x) in your function space. That's fine, it doesn't need to approach a limit or anything.

Funzo
  • 111