What is convolution, how does it relate to inner product?

Question

Let's consider the formula of the inner product between two continuous functions: $$<f,g> = \int_{-\infty}^{+\infty} f(t) \cdot g(t) \ dt$$

The inner product is intuitively defined to give some real number that measures the degree of similarity between two functions. I understand that, and I arrive to find a meaning for some mathematical formulas involving integrals.

Let's now consider the formula of the convolution between two methods: $$(f \star g)(x) = \int_{-\infty}^{+\infty} f(t) \cdot g(x-t) \ dt$$ Based on the previous intuitive interpretation of the inner product of two functions, can we arrive at another intuitive interpretation for the convolution formula?

From the inner product point of view, I can interpret a special case of convolution and relate it to the inner product:

If we consider that $x=0$, we will have: $(f \star g)(0)= \int_{-\infty}^{+\infty} f(t) \cdot g(-t) \ dt$.

At this point, we can say the convolution with $x = 0$ is the inner product and thus the degree of similarity between a function and the mirror image of the other.

If we consider again that $f$ or $g$ is an even function(take $g$ in this example):

$$g(t)=g(-t) \implies (f \star g) = \int_{-\infty}^{+\infty} f(t) \cdot g(t) \ dt = <f, g>$$

So, in the case where one of the functions is even and by imposing $x=0$, we found that the convolution is a generalization of the inner product, what about the other functions?

If we had a formula like this: $\int_{-\infty}^{+\infty} f(t) \cdot g(t-x) \ dt$, I can persuade my self that it measures the similarity with some shifting parameter $x$ introduced(some kind of causality). However, in the original convolution formula, the sign of $t$ is inversed(what does this sign inversing mean?).

My final question is: what is the intuition behind convolution? what is its relation with the inner product?

I would appreciate it if you include the examples I gave above and correct me if I am wrong.

Answer 1

Convolution is a way of collecting like terms in a discrete or continuous sum. For example, \begin{align} \sum_{n=0}^{N}a_n x^n\sum_{n=0}^{N}b_n x^n &= \sum_{n=0}^{N}(\sum_{j+k=n}a_j b_k)x^n \\ &= \sum_{n=0}^{N}(\sum_{j=0}^{n} a_j b_{n-j})x^n \end{align} The convolution of $\{x_k\}_{k=0}^{n}$ and $\{y_k\}_{k=0}^{n}$ is the sequence $\{\sum_{j=0}^{k}a_j b_{k-j}\}_{k=0}^{n}$, which is a natural way of collecting like terms. The convolution of sequences $\{x_k\}$, $\{y_k\}$ is the sequence $\sum_{j=0}^{k}a_j b_{k-j}$.

A continuous version of convolution occurs with the Laplace transform: $$ \int_0^\infty e^{-st}f(t)dt\int_0^\infty e^{-st}g(t)dt \\= \int_0^\infty e^{-st}\left(\int_0^tf(u)g(t-u)du\right) dt $$ The convolution of coefficient functions $f,g$ is \begin{align} (f * g)(t) &= \int_0^t f(u)g(t-u)du \\ &= \int_0^t f(t-u)g(u)du \end{align} Convolution allows you to multiply two Laplace transforms by collecting the like powers of the exponential. As with ordinary polynomial multiplication, it doesn't matter which way you gather the like terms: $$ \int_0^t f(t-u)g(u)du = \int_0^t f(u)g(t-u)du. $$ The Fourier transform also has an associated convolution over all positive and negative powers: $$ \int_{-\infty}^{\infty}f(u)e^{-isu} du \int_{-\infty}^{\infty}g(u)e^{-isu}du $$ $$ =\int_{-\infty}^{\infty} \left(\int_{-\infty}^{\infty} f(v)g(u-v)dv\right)e^{-isu}du $$ The convolution is $$ (f *g)(u)=\int_{-\infty}f(v)g(u-v)dv $$ Note: The convolution may incorporate or omit the Fourier constant $1/\sqrt{2\pi}$.