Thoroughly understand the concepts and formulas in Linear Algebra

Question

In linear algebra(I'm the beginner so far), I feel the concepts and ideas are based on on another, like layer by layer, and there are many relations among the properties.

For example, Matrix Multiplication, AB is essentially the composition of Linear Transformation（$A(Bx)=(AB)x$ to manually define AB) , and Linear Transformation is also from Matrix Equation($Ax=b, T(x)=b=Ax$)，and Matrix Equation is also from Vector Equation($x_1a_1+...+x_na_n=b$ equivalent to $Ax=b$)，and finally Vector Equation is the basic System of Linear Equations.

so Vector Equation could be intuitively understood as Linear System，Matrix Equation is just another formation。 Linear Transformation is fine too，but when it comes to Matrix Algebra， like Inverse Matrix and its properties, it seems very hard to be intuitively understood through the basic System of Linear Equations. It'll be only fine when I admit the manually definition the matrix multiplication from composition of linear transformation. Because as the process moves on, it's more and more like everything is constructed layer by layer, and more and more complicated as it's very hard to try to understand it from the very basic layer (e.g. linear system) intuitively.

I'm not sure whether it's the normal situation, because there are also intersected properties among many 'layers'. Therefore the whole 'knowledge structure' is not very clear in my brain.

And, when I think of the the basic Quadratic Formula in elementary algebra, ， its meaning is just to solve the quadratic equation with one variable, and then it is proved from completing square of $ax^2+bx+c=0$ to find general formation of $x$. But it is still impossible to 'understand thoroughly' from the original equation like how the relation among each variable and coefficient. So when I need to use it, I just refer to the book or prove it myself (assume if I forget)

So will it the same thing as learning the Linear Algebra, like the determinant of matrix, $\det A = \sum_{j=1}^{n}(-1)^{1+j}a_{1j}\det A_{1j}$. Is it okay that what I need to do is only to know two things,

1. What it really is.
2. How to prove it.

(When I use it, just refer to book or prove it, however, prove the determinant takes a little long time). Because this formula ($\det A$) is already impossible to be thoroughly understood from the very basic 'layer' like system of linear equations or even vector equation. I only know, Firstly its meaning is to determine whether a matrix is invertible (because invertible matrix must has $n\times n$ size and has pivot position in each row and no zeros in diagonal entries.), Secondly it's proved from $n\times n$ matrix row reduced to echelon form, then the right-bottom corner must be nonzero, the above formula is just the brief formation of the item in right-bottom corner.

I know this two things,could I say I'm already understand it ? Because even though I know 1.what is really is and 2.how to prove it, but I'm still not able to remember it when use it.

Answer 1

I think you're on the right track when you say you want to get a better intuitive understanding of what's going on.

Let's start with a geometric interpretation for the determinant of a matrix: Given an $n$-by-$n$ square matrix $A$ consisting of column vectors $v_1,v_2,\dots,v_n$, the volume of the parallelotope spanned the $n$ vectors is the absolute value of the determinant, $|\det(A)|$.

Now think of the matrix $A$ as a linear map from the $n$-dimenesional real vector space $\mathbb R^n$ to $\mathbb R^n$ given by $v \mapsto Av$ for $v \in \mathbb R^n$. Consider the unit cube $[0,1]^n=[0,1]\times\dots\times[0,1]$ in $\mathbb R^n$. The image of the unit cube under that linear map $v \mapsto Av$ will just be the parallelotope spanned by the $v_1,v_2,\dots,v_n$ which were the column vectors of $A$. Note that if $\det(A)=0$, then the volume will be zero.

This can tell us something intuitive about the invertibility of the linear map given by $A$. Think about what the image of the entire space $\mathbb R^n$ is under the map given by $A$ (what's also called the "range" of the map). It will either be all of $\mathbb R^n$, or some proper subspace of $\mathbb R^n$. Observe that the map given by $A$ is invertible if and only if the range of the map is all of $\mathbb R^n$. Indeed, if the range were not all of $\mathbb R^n$, then the map has "squashed" $\mathbb R^n$ into some subspace of smaller dimension, so you'll have a many-to-1 map rather than 1-1, so it won't be invertible.

What does this have to do with the determinant? If $\det(A)=0$, then the volume of the image of the unit cube is zero, so the unit cube must have been "squashed" by the map into a subspace of smaller dimension. You can then extrapolate this reasoning to see that if $\det(A)=0$, then the image of the whole space $\mathbb R^n$ will also be "squashed" into a proper subspace, so the map won't be invertible. Conversely, if $\det(A)\neq0$, then the volume of the image of the unit cube is greater than zero, and some thought might convince you that then the image of the whole space $\mathbb R^n$ in be all of $\mathbb R^n$, so that map will be invertible.

Thus, to summarize our intuitive argument:

matrix $A$ is invertible $\Leftrightarrow$ map on given $\mathbb R^n$ by $A$ is invertible $\Leftrightarrow$ image of the map is all of $\mathbb R^n$ $\Leftrightarrow$ the volume of the image of the unit cube is $>0$ $\Leftrightarrow$ $\det(A) \neq 0$.

The next step to understanding the determinant is to think about what "signed volume" should mean, what kind of properties it should have, and then work backwards from those properties to get the usual formulas for the determinant. A great discussion of this in in Chapter 5 of Peter Lax's Linear Algebra and its Applications.

Answer 2

À la Sheldon Axler, determinants are a horrible way of teaching linear algebra. I agree with him as well but sometimes determinants are convenient. They way you should learn linear algebra and matrices (at least to me) is to imagine a matrix as being some kind of linear map between two vector spaces of finite dimension (If you do not know what a vector space is, or for that matter the dimension of a vector space you can look at wikipedia).

What you need to know is this: A matrix being square is a necessary but not sufficient condition for a matrix to be invertible. Why? Well firstly I should say that the dimensions of a matrix determine the dimensions of the vector spaces in question. For example, if $T : V \rightarrow W$, where $V$ and $W$ are of finite dimension $n$ and $m$ respectively then your matrix is going to be of size $m \times n$ (Think about it in terms of how you multiply matrices with column vectors).

Prove the following: Every linear map between two finite dimensional vector spaces can be represented by a matrix.

Now perhaps in your class they have done row reduction, reducing a matrix into row-reduced-echelon form. Then perhaps you would have been told of the formula:

Number of columns in a matrix = number of pivot columns + number of columns with free variables.

This can be restated as follows: If $T$ is a linear map from $V \rightarrow W$, then dim $V = $ dim null $T$ + dim im $T$.

With this formula alone, prove that if either the dimensions of the two spaces $V$ and $W$ in question are not equal, the map is either not injective or not surjective. If you don't know what these mean, you can look at the wikipedia link here. For example, it dim $V > $dim $W$, then by the formula I stated above,

$\begin{eqnarray*} \text{dim null} T &=& \text{dim} V - \text{dim im} T \\ &>& \text{dim} W - \text{dim im} T \\ &\geq& 0. \quad \quad (\text{Why?}) \end{eqnarray*}$

So there is not only the trivial solution to the equation $Ax = 0$, where is the matrix represented by the linear map $T$, which is equivalent to saying that the map is not injective. A similar result can be proven for the case when dim $W > $ dim $V$. This relates back to what I said on a matrix being square being a necessary but not sufficient condition for it to be invertible.

So prove that a matrix is invertible iff it is surjective and injective. In fact proving that the following 3 statements are equivalent would be of great help to your understanding: Let $V$ and $W$ be finite dimensional vector spaces.

$\textbf{1.}$ There exists an invertible linear map from $V$ to $W$, call this $T$.

$\textbf{2.}$ The dimension of $V$ is equal to the dimension of $W$.

$\textbf{3.}$ The map $T$ is surjective and injective.

So really, all the properties of whether a linear system has a unique solution or not reduces to checking the dimensions of the null space, image, etc.

To give you an idea how to see connections:

Consider the linear system $Ax = 0$. This is equivalent to finding the null space of the linear map represented by the matrix $A$, which is equivalent to asking if the columns of $A$ are linearly independent (What does it mean to multiply a matrix by a vector?)

In three dimensions at least, three vectors being linearly independent is the same as saying that the parallepiped formed by the three vectors is not squashed, which is the same as saying that the determinant of $A$ is non-zero.

After a while, some connections will be seen and your eyes will light up!

I hope that helped!

$\textbf{Edit:}$. To answer your question on why the right most element last row of a matrix in reduced-row echelon form must be non-zero to have at least one solution, think about it in terms of dimensions.