I'm currently taking an intro to linear algebra course. We have reached the section on dot products and I fail to understand the meaning of it. What does it tell me, besides spitting out a scalar?
I initially read that the answer is similar to asking what the point of multiplication is, but I understand why multiplication is useful.
The dot product is a specific case of something called an "inner product". This may seem surprising at first, but inner products (and in this case the dot product) is what gives you a notion of angle in the vector space.
For example, $(0,1)\cdot(1,0) = 0\times1+1\times0 = 0$, while $(1,0)\cdot(1,0) = 1\times1+0\times0 = 1$. In your first case, you know that the two vectors are "orthogonal" or "perpendicular" just by looking at them - one of them is in the $x$ direction, and the other one is in the $y$ direction. But what if you're in a more abstract setting? What if you can't "visualise" the vectors like lines any more? This is where the dot product comes in: If the dot product of two vectors is $0$, the two vectors are orthogonal.
In the second case, the "length" of each vector is $1$, and so is their dot product. (By length I mean the Euclidean norm: i.e. the length of $(1, 0)$ is $\Vert(1, 0)\Vert = \sqrt{1^2+0^2} = 1$.) In the opposite extreme from orthogonal vectors, you have $u\cdot v = \Vert u\Vert \Vert v\Vert$. This is when the two vectors are parallel.
How about everything in between? If we rearrange the previous results slightly, you'll see that if two vectors $u$ and $v$ are orthogonal, $\dfrac{u\cdot v}{\Vert u\Vert \Vert v\Vert} = 0$. On the other hand, if $u$ and $v$ are parallel, $\dfrac{u\cdot v}{\Vert u\Vert \Vert v\Vert} = 1$ (It can also be $-1$ if your two vectors are parallel in opposite directions). Everything else between these two extreme cases takes some number between $1$ and $0$ or $0$ and $-1$.
You might have noticed this is getting strangely trigonometric: for example, $\cos(\theta)$ takes values from $-1$ to $1$. And yes, $\dfrac{u\cdot v}{\Vert u\Vert \Vert v\Vert} = \cos(\theta)$ gives you the angle between two vectors.
To see why this is true, remember that cosine is "adjacent over hypotenuse". By thinking of the dot product as "the product of a vector and the component of the other vector in the same direction as the first vector", you can convince yourself this is true with some drawing!
The dot product gives you information about lengths and angles.
The dot product of two vectors tells you the angle between them, via the equation $$u \cdot v = ||u|| ||v|| \cos \theta.$$ A consequence of this is that the dot product of a vector with itself tells you the length of the vector, via the equation $$v \cdot v = ||v||^2.$$
If you wanted to, you could actually define length and angle by means of the dot product, using the two equations above.
Let me try to motivate the dot product from a slightly different perspective.
I assume you already buy that measuring lengths, $\|v\|,$ is important and useful.
Now you know that in Euclidean geometry, the Pythagorean Theorem holds; that for two perpendicular vectors $u$ and $v$, $\|u+v\|^2 = \|u\|^2+\|v\|^2$. This theorem tells you how to measure lengths of any vector, so that $\|(a,b)\|^2 = a^2 + b^2.$ You can encode this information in a matrix $g$, with $$\|u\|^2 = u^Tgu = u^T\left[\begin{array}{cc}1 & 0\\0 & 1\end{array}\right]u.$$
This matrix $g$, which is just the identity matrix, tells you how to measure lengths: you hit it on both sides with any vector $u$, and you get out as your answer the squared length of $u$.
Now once you have this matrix $g$ (called the inner product of Euclidean space) nothing stops you from hitting in on both sides with two different vectors $u,v$: $u^Tgv$. Notice that this vector-matrix-vector product simplifies to what you think of as the dot product, $u\cdot v$, but which I will write using slightly different notation $\langle u, v\rangle.$
What is the meaning of $u^Tgv$, though? We can get some insight by noticing the following identity: $$\|u+v\|^2 = (u+v)^Tg(u+v) = u^Tgu + 2u^Tgv + v^Tgv = \|u\|^2+\|v\|^2 + 2\langle u, v\rangle,$$ where I have used here the fact that $g$ is symmetric. Now you see that the inner product of $u,v$ measures the failure of these vectors to obey the Pythagorean theorem: $\langle u,v\rangle$ is zero when the vectors are perpendicular, positive when $u$ and $v$ point in the same direction, and negative when the vectors point in opposite directions. The inner product thus encodes the angle between the vectors $u$ and $v$, and in fact the angle can be defined in this way, as elaborated in the other answers. It is also clear from the fact that the inner product is equal to $u^Tgv$ that it is bilinear and symmetric in its arguments.
Sooner or later in your studies, you will encounter norms other than the Euclidean norm. For example, draw some vectors on a rectangular sheet of rubber, and stretch the rubber so that it is 2x as long in the horizontal direction. Vectors on this new sheet of rubber look stretched compared to their "real" shape on the undeformed piece of rubber. You can measure vectors on the stretched sheet, and determine their true lengths, by "undoing" the stretching: $$\|(a,b)\| = \sqrt{a^2/4 + b^2}.$$ You can encode this norm in a matrix, just like before, but this time $$g = \left[\begin{array}{cc}\frac{1}{4}&0\\0 & 1\end{array}\right]$$ and like before, this matrix defines an inner product $$\langle (a,b), (c,d)\rangle = \frac{ac}{4}+bd.$$ This inner product measures angles of vectors on the deformed sheet of rubber, as if they were being measured on the original sheet. So for example, $(-2,1)$ and $(2,1)$ are perpendicular, since their inner product is zero, and indeed, if you unstretch these back to their original shape, they are the vectors $(-1,1)$ and $(1,1)$ which are perpendicular in the usual Euclidean sense.
You can also get $g$ that are non-diagonal. This happens when you shear the sheet of rubber instead of, or in addition to, stretching it.
This ability of measuring lengths and angles in one space, as if they are distorted images of "true" vectors in a different space, is incredibly useful throughout geometry and physics.
A final note: it is not always possible to define an inner product from a norm. In order to be able to do so, the norm must satisfy the parallelogram law: for a parallelogram made of two vectors in your vector space, you need that the sum of the squared lengths of the four sides of the parallelogram equals the sum of the squared lengths of the diagonals: $$2\|u\|^2 + 2\|v\|^2 = \|u+v\|^2 + \|u-v\|^2.$$ Notice that the Pythagorean Theorem is a special case of this law. Euclidean space (any number of dimensions), and other spaces that you get by stretching or shearing Euclidean space, all satisfy the law. Whenever the law is satisfied, you can prove that there exists a matrix $g$ that measures square lengths (this is called the polarization identity) and therefore also measures inner products (and angles).
