How does augmenting a square matrix (LHS) with an identity matrix (RHS) and then reducing the square matrix to an identity matrix and performing the same operations on the identity matrix using elementary row operations give the inverse matrix on the (RHS)?
What's the logic behind this? I don't understand the mathematical equations that show this is true.
I still dont understand the logic behind it the equations dont help me.
What does Ei mean?
Suppose that your initial matrix is $A$ (for simplicity, let's say $A$ is 2 by 3, though you can easily see that the dimension doesn't affect this proof).
Each elementary row operation can be represented by multiplying $A$ by some matrix $E_i.$ For example, if you want to replace the second row by the first row minus the second row. Then this is represented by $$E_1 = \begin{bmatrix} 1 & 0 \\ 1 & -1 \end{bmatrix} \rightarrow E_1 A = \begin{bmatrix} 1 & 0 \\ 1 & -1 \end{bmatrix} A .$$ You can see that each elementary row operation is represented by left-multiplying $A$ by some matrix $E_i.$ Note, also, that each elementary row operation is invertible, and thus the matrix $E_i$ that represents this operation is invertible.
So suppose that through some sequence of $n$ elementary row operations, you reduce $A$ to the identity matrix. Then you have the equation $$E_n E_{n-1} \dots E_2 E_1 A = I,$$ where multiplying by the sequence of $E_i$ represents reducing your matrix $A$ to the identity matrix. Thus, from here, you can see that $$A^{-1} = E_n E_{n-1} \dots E_2 E_1 I.$$ Thus, the inverse of $A$ is exactly just by doing these elementary row operations on the identity matrix!
One carefully illustrated explanation can be found in Barnett et. al's College Algebra. The following example from the book describes the process of finding the inverse of matrix $A$ by performing elementary row operations on the augmented (coefficient) matrix $[A|I]$.
Source: Barnett et. al. College Algebra, McGraw Hill, 9th Edition, p.475.
The resulted matrix $B$ is indeed $A$'s inverse. This can be seen by noticing that, for every augmented matrix except the first one, the left component matrix (one is on the left side of the vertical bar) is equal to the right component matrix (of the same augmented matrix) multiplied by $A$. For example, consider the augmented matrix in the 4th row in the image above, we have:
$$ \begin{bmatrix} 1 & 0 & \frac{1}{2}\\ 0 & 1 & \frac{-1}{2}\\ 0 & 0 & \frac{1}{2}\\ \end{bmatrix}= \begin{bmatrix} 1 & \frac{1}{2} & 0\\ 0 & \frac{1}{2} & 0\\ -2 & -5 & 2\\ \end{bmatrix} \begin{bmatrix} 1 & -1 & 1\\ 0 & 2 & -1\\ 2 & 3 & 0 \end{bmatrix} $$
Thus the following theorem follows naturally:
If $[A | I]$ is transformed by row operations into $[I | B]$, then the resulting matrix $B$ is $A^{-1}$. If, however, we obtain all 0's in one or more rows to the left of the vertical line, then $A^{-1}$ does not exist.
Finding the inverse or even checking the inverse of a $n \times n$ square matrix is basically solving the following systems of equations (so there are $n$ different systems of equations): $$Ax=e_1, \quad Ax=e_2, \quad \ldots \quad Ax=e_k, \quad \ldots \quad Ax=e_n,$$ where $e_k$ is the $k-$th standard basis vector.
Instead of solving these one at a time, we can combine the right hand sides into one augmented matrix $[A | I]$ and row reduce (both sides) to see if we can get $[I | M]$. If we can, then we say $M$ is the inverse of $A$, otherwise the inverse does not exist. So even if one of the systems $Ax=e_j$ is inconsistent, inverse of $A$ will not exist.
Let us try with an example: Suppose $A=\begin{bmatrix}1&2\\3&4\end{bmatrix}$ and we want to find it's inverse (if it exists). Then we want a matrix $M$ such that $AM=\begin{bmatrix}1&0\\0&1\end{bmatrix}=\begin{bmatrix}\uparrow & \uparrow\\e_1&e_2\\\downarrow & \downarrow\end{bmatrix}$. We can write this as: $$AM=I \implies A\begin{bmatrix}\uparrow & \uparrow\\m_1&m_2\\\downarrow & \downarrow\end{bmatrix}=\begin{bmatrix}\uparrow & \uparrow\\e_1&e_2\\\downarrow & \downarrow\end{bmatrix}.$$ Using matrix multiplication etc., we can write this as $$A\begin{bmatrix}\uparrow\\m_1\\\downarrow\end{bmatrix}=\begin{bmatrix}\uparrow\\e_1\\\downarrow\end{bmatrix} \quad \text{and} \quad A\begin{bmatrix}\uparrow\\m_2\\\downarrow\end{bmatrix}=\begin{bmatrix}\uparrow\\e_2\\\downarrow\end{bmatrix}.$$ This means to find vectors $m_1$ and $m_2$, we should solve two systems of equations: $$Ax=\begin{bmatrix}\uparrow\\e_1\\\downarrow\end{bmatrix} \quad \text{and} \quad Ax=\begin{bmatrix}\uparrow\\e_2\\\downarrow\end{bmatrix}.$$ For each of the systems, the augmented matrix is as follows: $$\left[\begin{array}{cc|c}1&2&1\\3&4&0\end{array}\right] \quad \left[\begin{array}{cc|c}1&2&0\\3&4&1\end{array}\right]$$ So, instead of solving them separately, we can combine the two augmented matrices into one and solve: $$\left[\begin{array}{cc|cc}1&2&1&0\\3&4&0&1\end{array}\right] $$ Hope this helps.
