40

So we vector spaces, linear transformations, inner products etc all have their own axioms that they have to satisfy in order to be considered to be what they are.

But how did we come to decide to include those axioms and not include others?

For example, why does this rule hold in inner product spaces $c\langle u,v\rangle=\langle cu,v\rangle$, when my intuition says that it should be $\langle cu,cv\rangle$?

And how did we decide that it was scalar multiplication and additive were sufficient criteria for something to be a linear map?

  • 6
    Flip of a coin, roll of a dice. – Asaf Karagila Nov 21 '15 at 15:51
  • 5
    How does your intuition says that $c\langle u,v \rangle=\langle cu,cv\rangle $? Take the vectors ${(a,b), (c,d)}$, then: $e (a,b) =(ea, eb) $ and $\langle (ea, eb), (c, d )\rangle = ea\cdot c +eb\cdot d = e (ac + bd) $. – Red Banana Nov 21 '15 at 16:53
  • @Voyska, I don't think you understand OP's question. – user1717828 Nov 22 '15 at 03:44
  • 2
    The examples you give are not axioms, but definitions. Your question is relevant anyway, but in mathematics language matters. – Kolaru Nov 22 '15 at 13:40
  • 1
    @Kolaru, for the difference between an axiom and a definition, please see http://math.stackexchange.com/questions/1428228/what-is-exactly-the-difference-between-a-definition-and-an-axiom – wythagoras Nov 22 '15 at 18:51
  • don't listen do the answers below, linear algebra+QM+TP=<3... – DVD Dec 28 '15 at 22:13

4 Answers4

77

Linear algebra is one of the first "abstractions" that you encounter in mathematics that is not very well motivated by experience. (Well... there are numerals, which are a pretty tricky abstraction as well, but most of us don't recall learning those.)

It helps to have the backstory.

Mathematicians studied geometry and simple transformations of the plane like "rotation" and "translation" (moving everything to the right by 3 inches, or up by 7 inches, or northeast by 3.2 inches, etc.) as far back as Euclid, and at some point, they noticed that you could do things like "do one translation after another", and the result was the same as if you'd done some different translation. And even if you did the first two translations in a different order, the resulting translation was still the same. So pretty soon they said "Hey, these translations are behaving a little like numbers do when we add them together: the order we add them in doesn't matter, and there's even something that behaves the way zero does: the "don't move at all" transformation, when composed with any other translation, gives that other translation."

So you have two different sets of things: ordinary numbers, and "translations of the plane", and for both, there's a way of combining ("+" for numbers, "composition of transformations" for translations), and each of these combining rules has an identity element ("0" for addition, "don't move at all" for translation), and for both operations ("+" and "compose"), the order of operations doesn't matter, and you start to realize something: if I proved something about numbers using only the notion of addition, and the fact that there's an identity, and that addition is commutative, I could just replace a bunch of words and I'd have a proof about the set of all translations of the plane!

And the next thing you know, you're starting to realize that other things have these kinds of shared properties as well, so you say "I'm going to give a name to sets of things like that: I'll call them 'groups'." (Later, you realize that the commutativity of addition is kind of special, and you really want to talk about other operations as well, so you enlarge your notion of "group" and instead call these things "Abelian groups," after Abel, the guy who did a lot of the early work on them.)

The same thing happened with linear algebra. There are some sets of things that have certain properties, and someone noticed that they ALL had the same properties, and said "let's name that kind of collection". It wasn't a pretty development -- the early history of vectors was complicated by people wanting to have a way to multiply vectors in analogy with multiplying real numbers or complex numbers, and it took a long time for folks to realize that having a "multiplication" was nice, but not essential, and that even for collections that didn't have multiplication, there were still a ton of important results.

In a way, though, the most interesting thing was not the sets themselves -- the "vector spaces", but rather, the class of transformations that preserve the properties of a vector space. These are called "linear transformations", and they are a generalization of the transformations you learn about in Euclid.

Why are these so important? One reason for their historical importance is that for a function from $n$-space to $k$-space, the derivative, evaluated at some point of $n$-space, is a linear transformation. In short: something we cared a lot about -- derivatives -- turns out to be very closely tied to linear transformations.

For a function $f: R \to R$, the derivative $f'(a)$ is usually regarded as "just a number". But consider for a moment $$ f(x) = \sqrt{x}\\ f'(x) = \frac{1}{2 \sqrt{x}}\\ f(100) = 10 \\ f'(100) = \frac{1}{20} $$ Suppose you wanted to compute the square root of a number that's a little way from 100, say, 102. We could say "we moved 2 units in the domain; how far do we have to move away from 10 (i.e., in the codomain)?" The answer is that the square root of $102$ is (very close to) the square root of 100, displaced by $2 \cdot \frac{1}{20}$, i.e., to $10.1$. (In fact, $10.1^2 = 102.01$, which is pretty accurate!)

So we can regard "multiplication by $1/20$" as the derivative of square-root at $a = 100$, and this gives a linear transformation from "displacements near 100, in the domain" to "displacements near 10, in the codomain."

The importance of derivatives made it really worthwhile to understand the properties of such transformations, and therefore to also understand their domains...and pretty soon, other situations that arose in other parts of math turned out to "look like those". For instance, the set of all polynomials of degree no more than $n$ turns out to be a vector space: you can add polynomials, you can multiply them by a constant, etc. And the space of all convergent sequences of real numbers turns out to be a vector space. And the set of all periodic functions of period 1 turns out to be a vector space. And pretty soon, so many things seemed to be sharing the same properties that someone gave 'sets that had those particular properties" a name: vector spaces.

Nowadays, seeing each new thing that's introduced through the lens of linear algebra can be a great aid...so we introduce the general notion first, and many students are baffled. My own preference, in teaching linear algebra, is to look at three or four examples, like "period-1 periodic functions" and "convergent sequences" and "polynomials of degree no more than $n$", and have the students notice that there are some similarities, and only then define "vector space". But that's a matter of taste.

John Hughes
  • 93,729
  • 7
    Great explanation, thank you. – jeremy radcliff Nov 21 '15 at 15:35
  • 2
    This is a really nice answer. Do you happen to know of any introductory books that adopt your approach to teaching ? – alexgiorev Nov 22 '15 at 10:23
  • 1
    Thanks. Alas, I don't know of any books that do this, although Massey's Algebraic Topology: An Introduction does something like this in the discussion of presentations of groups, and I think that book, which I read back when I was first learning topology, was something of an inspiration for me in my teaching. – John Hughes Nov 22 '15 at 16:58
  • "(...) the early history of vectors was complicated by people wanting to have a way to multiply vectors in analogy with multiplying real numbers or complex numbers (...)" This is very reasonable, but is there a written example? – Ian Mateus Dec 08 '15 at 03:12
  • 1
    I'm pretty sure I read this in a biography of Hamilton at some point. But Chapter 2 of Michael J, Crowe's "A History of Vector Analysis" seems to paint the picture pretty clearly: Hamilton was seeking a generalization of complex numbers to "triplets" in which a nice addition and mutliplication operation were defined, in analogy with those of the reals and complexes. – John Hughes Dec 08 '15 at 16:55
  • Awesome explanation, thank you. Just to clarify, you're saying that this idea of commutativity holds only for specifically translations of the plane, right? Since matrix multiplication is not commutative. Also can the composition of transformations be thought of as addition? It seems like your answer hints to yes, although as a linear algebra student I've only thought of combining linear transformations in the context of matrix multiplication, rather than any form of addition. – rb612 May 06 '18 at 06:48
  • Great question. I've only claimed commutativity for translations of the plane, yes. I also mentioned that "not commutative" operations can be interesting, and matrix multiplication is one of these. The set of all invertible $n \times n$ matrices is called $GL_n(\Bbb R$)$, the "general linear group over $\Bbb $R$", and has been extensively studied. On the other hand, the set of all $n \times k$ matrices, with the operation of matrix addition, is also a group (the all-zero matrix is the identity element). So these algebraic constructions DO apply to collections of matrices/transformations, yes – John Hughes May 06 '18 at 11:49
8

For the particular case of inner products being bilinear:

Inner products are intended to generalize the usual dot product from plane and space vector algebra. Therefore it would not make sense to require it to satisfy a property that the dot product doesn't have.

For example, in the plane we have $2\bigl[ (1,1)\cdot (1,2) \bigr] = 6$ whereas $$[2(1,1)]\cdot[2(1,2)]=(2,2)\cdot(2,4) = 12$$ so your proposed rule doesn't hold for the dot product.

5

Although he is talking about topological spaces, the paragraph in Munkres's Topology may still in some sense answer your question and hence is applicable here:

"They wanted, of course, a definition that was as broad as possible, so that it would include as special cases all the various examples that were useful in mathematics - euclidean space, infinite-dimensional euclidean space, and function spaces among them - but they also wanted the definition to be narrow enough that the standard theorems about these familiar spaces would hold for topological spaces in general."

Yes
  • 20,719
  • Linear algebra is giving me a headache. I thought I was good at math but I have realized that up until now the "proofs" i have been doing were just derivations. The computation side of linear algebra but I am totally lost when it comes to proofs. I have tried Axler's book and it just sounds like gibberish to me. Mind you I am not weak in math, in high school I taught myself everything straight from the textbook, no problem, and I wouldn't just rote learn, I would understand why it was true. I am really doubting whether I am good at maths now. – user289419 Nov 21 '15 at 13:33
  • To give an example, when teaching myself trig in high school, I didn't just rote learn it, but I immediately saw the connection between similar triangles and their ratios. I also realized the trig table was just a data collection of t. To most of my classmates sin,cos, tan sounded like gibberish, and they had no idea how we actually obtain the trig table values but for me it was something very concrete. I know that computation is not really mathematics though, and that a monkey can churn out results with enough practice. Maybe I am just not a mathematician. – user289419 Nov 21 '15 at 13:37
  • 3
    @user289419 Maybe I am just not a real mathematician: don't be so sure. Real proofs are just hard, and linear algebra is much harder to visualize than the name "vector-space" might lead you to believe. What you should keep in mind is that it's extremely difficult to remember what it's like not to know something. At some point, trig seemed difficult, but with practice you figured it out. Calculus seems strange when you first encounter it, but it becomes second nature. Likewise, you're seeing proofs for the first time. Give it time, and I promise it gets better. – Ben Grossmann Nov 21 '15 at 14:02
  • To be honest trig was never hard. It was actually very logical to me, if you can understand the concept of similarity in triangles then trig is an indispensable tool. Now, linear algebra on the other hand, is a totally different beast. When Axler starts talking about inner product spaces from the get go, with no context, I just feel like I am watching a foreign movie. – user289419 Nov 21 '15 at 14:19
  • 3
    @user289419 that's how it's supposed to feel. Math at the higher level is a foreign language that you learn the words to bit by bit. Coming out of your first or second proof-intensive class, you know enough to get by in conversation but you're not fluent. Eventually, however, you'll look back at what you're doing now and see how obvious it seems in retrospect. – Ben Grossmann Nov 21 '15 at 14:30
  • 1
    @user289319 notwithstanding, Axler's text is both praised and notorious for trying to get you to think of things "the right way" at the expense of making them easy to digest. It is good preparation, however, for the math that follows, which often consists of definitions and theorems for which you have no intuition. – Ben Grossmann Nov 21 '15 at 14:34
  • 1
    @user289419 also, good as you may have been at trig, I'm sure that if somebody taught you the basics of trig for a month and then asked you to prove the law of cosines, you would have a difficult time. This is tantamount to what's being asked of you now. – Ben Grossmann Nov 21 '15 at 14:36
  • 1
    @user289419 As I read the comments here, I thought this: it would be a good idea to pick up a number theory book to get a feeling of a math proof; I would recommend Landau's "Elementary number theory" or Apostol's "introduction to analytic number theory". Integers are familiar objects, so one can focus more on proofs themselves. :) – Yes Nov 21 '15 at 15:04
  • 2
    @user289419 if I may... there are unsolved problems in math. If you managed to go through your entire life without ever finding anything in math hard, I would immediately employ you to solve all the Millenium problems and then some. Yes, linear algebra is hard. For you, it sounds like your first real resistance coming from the challenge of math. That's what it feels like. That feeling is part of being a good mathematician, not a sign you are a bad one. – djechlin Nov 22 '15 at 05:10
  • 1
    @user289419, in his very nice answer above John Hughes says he prefers to motivate abstractions with a few examples, before introducing the abstraction. That is very much to my taste also. Some books are written in this way, and some in the opposite way. I struggled for decades with the latter type of book / teaching style. Eventually I discovered the other type of book and teaching style, and a million things started to make a lot more sense. Depending on how your brain works, I would recommend seeking "nice" books and teachers. – Simon Nov 22 '15 at 10:48
0

The motivation to the inner product is perhaps most serious point because linearity axioms are just abstract properties of the space. Without additional and scalar construction on it the very space is completely useless; even for mathematics. Hence we need (mathematics, physics and calculus require) to introduce a way of getting the numeric and invariant (base independent) quantities from vectors; like angles and lengths. This entails that we actually want to have an invariant numeric function of elements of the vector space. The first property of the function is to keep the linearity of the space itself. Hence it is called the linear functional. It is primary, not the inner product. The latter object is just a numeric representation of the functional above (Riesz's theorem). This also lead to the notion of a dual space and its elements. More precisely its coordinate represetations provide the computation tool of both the functional and inner product.

Sir168
  • 224
  • The claim that a space "without addition and scalar multiplication is completely useless" seems a little broad. The unit sphere, for instance, has proven to be a pretty interesting object over the years. I expect you can think of other examples. – John Hughes Jul 27 '20 at 12:39