I understand that it is important in the field of matrix-vector multiplication when vectors can be interpreted as a matrix of one row/column vector.
But I do not really understand when I should use column vectors and when to use row vectors generally - when does it make a difference? Is it more a question on asthetics/style?
Let $v \in \mathbb R^d$, is it a column vector or a row vector? (I think, it depends on interpretation, doesn’t it?) When does one care about it?
That mostly depends on the context.
If one wants to write a linear transformation on the vector space $\mathbb{R}^n$, like matrix multiplication, then one writes the element in $\mathbb{R}^n$ as a $1\times n$ or $n\times 1$ matrix and call it a row or column vector respectively.
Using column vectors, the linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$ is written as $Ax$ where $A$ is an $m\times n$ matrix.
Using row vectors, the linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$ is written as $xA$ where $A$ is an $n\times m$ matrix.
If in general, people only talk about linear transformations on the vector space $V=\mathbb{R}^n$ without thinking about matrix multiplications, they simply refer to an element of $V$ as a vector, without saying whether it is a "row vector" or "column vector".
Generally speaking, if $x \in \mathbb{R}^d$, then $x$ is conventionally chosen to be a column vector, and perhaps to avoid using extra space, it is written in text as $x = (x_1 , \cdots, x_d)^T$. Since $x$ is a column vector, the order of matrix multiplication with a $d \times d$ matrix $A$ will be $Ax$.
You can consider $x$ to be row vector if you like, then you just need to be careful with matrix multiplication.
