1

Single variable functions $f: \mathbb{R} \to \mathbb{R}$ form a vector space, and each such function can be represented as a column vector, specifically an $N \times 1$ vector when the function is sampled at $N$ points. Operators acting on such functions, such as the derivative, can be represented as $N \times N$ matrices.

What changes for a function $g: \mathbb{R}^2 \to \mathbb{R} $ if the sample is on an $N \times N$ grid? Do the vectors now take the form of $N \times N$ matrices and the operators the form of an $N \times N$ matrix of $N \times N$ matrices?

Paul Frost
  • 76,394
  • 12
  • 43
  • 125
Tomek Dobrzynski
  • 155
  • 1
    The key here is the word “represented”. It’s not something uniquely intrinsic to the function and it’s domain itself. For instance we could just as easily represent the outputs of $g$ as a $n^2$ vector. And no, when we say that continuous functions form a vector space, we definitely don’t mean that they can be represented as column vectors. – V.S.e.H. Oct 04 '22 at 22:04
  • Give me an example of a single variable function that can't be represented with a column vector. – Tomek Dobrzynski Oct 04 '22 at 22:10
  • 1
    Well, any function that can’t be reconstructed from $N$ samples. But that’s besides the point I was trying to make: when we say that the set of continuous functions forms a vector space, we mean that said functions satisfy a set of axioms, not whether they can or can’t be represented by $n$-vectors. – V.S.e.H. Oct 04 '22 at 22:17
  • Seems to be related to my question https://math.stackexchange.com/q/4378716. – Kritiker der Elche Oct 04 '22 at 23:19
  • @V.S.e.H. You are completely missing the point of the question. This question is very obviously asking about implementation of numerical schemes. – K.defaoite Oct 05 '22 at 09:06
  • @K.defaoite Not really. I answered quite clearly with the first part of my first comment: the question is about how to represent $n$-dimensional samples of the output of a multi function on a finite grid, i.e. how to represent the "data" with a suitable data-structure, and my answer is that it is entirely subjective and not related to the function itself. In that sense, the OP can use their proposed scheme and notation, but it is entirely up to them. There isn't a standard convention. The rest is just notation (syntactic sugar), and it doesn't affect how computation is done. – V.S.e.H. Oct 05 '22 at 09:36

1 Answers1

1

When it comes to the specific values of the components of these arrays, that is something that will take care and time. But as to your question, about their shape, that is not so hard to answer.

We want these operators to preserve the shape of the original function. So, as you said, in one variable, if we have our function $f$ represented by a sampling at points $x_1,\dots,x_n$, it can be represented as a vector, $$\begin{bmatrix}f(x_1) \\ \vdots \\ f(x_n)\end{bmatrix}\equiv\begin{bmatrix}f^1 \\ \vdots \\ f^n\end{bmatrix}\equiv \boldsymbol f$$ The derivative of a function $\Bbb R\to\Bbb R$ is still a function from $\Bbb R\to \Bbb R$, so whatever array operation we use to represent the derivative, it must be of the right shape so that it maps column vectors to column vectors. Clearly, a matrix is the right choice. In particular, if $\mathbf D$ is the symbol for the matrix representation of the derivative, then $$(Df)^i=D^i{}_jf^j$$

You can see that by contracting a matrix ($(1,1)$ tensor) with a vector ($(1,0)$ tensor), we get a $(1,0)$ tensor. In particular,

$$\text{Contracting a}~ (a,b)~\text{tensor with a}~(c,d)~\text{tensor}~n~\text{times will give a }\\ (a+c-n,b+d-n)~\text{tensor}.$$

But what if we have a function $F:\Bbb R^2\to \Bbb R$ ? Now of course we need a matrix representation: $$\begin{bmatrix}F(\boldsymbol x_{1,1})&\cdots&F(\boldsymbol x_{1,n}) \\ \vdots & \ddots & \vdots \\ F(\boldsymbol x_{n,1}) & \cdots & F(\boldsymbol x_{n,n})\end{bmatrix}\equiv\begin{bmatrix}F^1{}_1 & \cdots & F^1{}_n \\ \vdots & \ddots & \vdots \\ F^n{}_1 & \cdots & F^n{}_n\end{bmatrix}\equiv\mathbf F$$

Now, again, partial derivatives should preserve the shape of the function. Our function here is represented as a matrix or $(1,1)$ tensor, so we need an array operation that takes a $(1,1)$ tensor to a $(1,1)$ tensor. So now, instead of two indices, we need four. So we need something that looks like

$$(\partial F)^i{}_j=\partial^{il}{}_{jk}F^k{}_l$$

So for functions $\Bbb R^2\to \Bbb R$, the partial derivative operator is a $(2,2)$ tensor, or, in rather crude terms, a "matrix of matrices".

You can generalize this approach easily to any function from $\Bbb R^M$ to $\Bbb R^N$.

K.defaoite
  • 12,536