I am having trouble understanding the definition of p-norm unit ball. What I know is that for infinity (maximum norm), then it will shape as a square. I need a "click" to understand this, can someone be so kind to explain this to me in simple words?
If the norm is the distance of the vector, then where does this vector located? From where to where is the length of the vector? Is it from the (0,0) point?
If the center of the unit-ball is in the origin $(0,0)$, then each point on the unit-ball will have the same p-norm (i.e. 1). The unitball therefore describes all points that have "distance" 1 from the origin, where "distance" is measured by the p-norm.
The easiest unit balls to understand intuitively are the ones for the 2-norm and the 1-norm.
Example 1: The 2-norm is simply the length of the vector ($\sqrt{x_1^2 + x_2^2}$ for the 2-dimensional case). Therefore it makes sense that all points of the same length form a circle around the origin.
Example 2: The 1-norm ($|x_1| + |x_2|$) is another case that can be easily interpreted. Just imagine the special cases
$$ x_{horizontal} = (1,0), x_{vertical}=(0,1) $$
Their 1-norm is
$$ |x_{horizontal}| = | x_{vertical}| = 1 $$
Every point on the line between these two points will also have a 1-norm of 1 since you linearly decrease the $x_1$-component while you increase the $x_2$-component.
Example 3: The infinity norm is defined as $\|x\|_\infty=\max\{ |x_1|, \dots, |x_n| \}$. Therefore, $\|x\|_\infty=1$ for all $(x_1,x_2$) where either $|x_1|=1$ and$|x_2| \leq 1$ or $|x_1|\leq1$ and $|x_2| = 1$ - this is how the square is found!
