73

I'm majoring in mathematics and currently enrolled in Linear Algebra. It's very different, but I like it (I think). My question is this: What doors does this course open? (I saw a post about Linear Algebra being the foundation for Applied Mathematics -- but I like doing math for the sake of math, not so much the applications.) Is this a stand-alone class, or will the new things I'm learning come into play later on?

Mallory
  • 1,187
  • 43
    Well sure. Functional analysis, differential geometry, representation theory... even graph theory makes extensive use of linear algebra, and especially of matrices and eigenvalues. In fact, apart from set theory, I can't really think of a branch of math that doesn't build on linear algebra! – goblin GONE Mar 14 '16 at 06:19
  • 52
    Literally Everything... – Sidharth Ghoshal Mar 14 '16 at 06:23
  • 9
    @goblin: I suggest posting your answer as a proper one instead of a comment. – Martin Argerami Mar 14 '16 at 06:26
  • 8
    The basic idea of the infinitesimal calculus is that, if you look at anything close enough, it becomes linear. Of course this is only mostly true, but it means that theoretically and practically, linear algebra shows up in everything requiring derivatives. Free abelian groups are just an application of linear algebra. Graph theorey relies on linear algebra. Yes, it shows up everywhere. Even fractals, which don't get linear as you look closer, need linear algebra to describe themselves. – Steven Alexis Gregory Mar 14 '16 at 06:27
  • @StevenGregory "Free abelian groups are just an application of linear algebra" - could you clarify that point please? – silvascientist Mar 14 '16 at 06:29
  • @MartinArgerami, thanks, but I don't feel comfortable doing so. My comment just isn't detailed enough, and therefore probably skirts altogether too close to simply dismissing the question altogether (which shouldn't be done, because questions about motivation are fundamentally good questions.) I think a good answer would explain the relevance of linear algebra to the subjects that it lists, and would not simply list them. – goblin GONE Mar 14 '16 at 06:31
  • 1
    @silvascientist, its linear algebra over the integers; a $\mathbb{Z}$-module is the same thing as an abelian group. – goblin GONE Mar 14 '16 at 06:32
  • @silvascientist I am referring to the importance of Smith normal forms. – Steven Alexis Gregory Mar 14 '16 at 06:34
  • @goblin I always thought linear algebra referred strictly to vector spaces. TIL – silvascientist Mar 14 '16 at 06:38
  • 7
    It's also also most students' first exposure to what mathematicians call algebra, and you'll see instances of quite a few good and broadly useful ideas: bases as generating sets, change-of-basis as conjugation, kernels, subspaces and quotients, and the list goes on! – pjs36 Mar 14 '16 at 06:40
  • Without linear algebra you just wouldn't be able to approach any advanced math or physics or statistics topic – Marco Disce Mar 14 '16 at 09:13
  • 2
    Surprised nobody's brought up computer graphics yet - 3D rendering is almost entirely about understanding matrices, vectors, etc. – Darrel Hoffman Mar 14 '16 at 13:40
  • 3
    Another Math student once explained the importance (and difficulty) of Linear Algebra like this: "There is no Non-linear Algebra course". – Lan Mar 14 '16 at 14:09
  • 4
    Isn't linear algebra important, large and complicated, and interesting enough in its own right as a field of study? Linear algebra is laying the foundation for more linear algebra! – Todd Wilcox Mar 14 '16 at 15:33
  • 1
    Wow! Thank you to everyone! I am so appreciative of all the responses. My heart races with excitement for future math courses! Thanks a million! – Mallory Mar 15 '16 at 14:00
  • Different from what? – Carsten S Mar 15 '16 at 15:20
  • @CarstenS -- Could you clarify, please? – Mallory Mar 15 '16 at 17:29
  • This is mentioned in an answer, but it's worth repeating: All of quantum mechanics rests of the shoulders of linear algebra. – zahbaz Mar 15 '16 at 17:36
  • You write that Linear Algebra is very different, but you do not say from what it differs so much. Here in Germany, Linear Algebra is a first semester course, so it may already be different from what students have encountered before, because all of that was not really mathematics. – Carsten S Mar 15 '16 at 18:53
  • 3
    @CarstenS Oh, I see! To me, Linear Algebra is different compared to calculus and calculus-based courses. It feels like a completely new type of algebra compared to what's been taught up to this point. Calculus II is a prerequisite for the course, which to me seems a little misleading. (So far we have used no calculus at all!) I wasn't really sure what I was getting into, and I'm not so sure what's to come after this. Hence the initial question! (: – Mallory Mar 15 '16 at 20:10
  • Is LA essential for studying Mathematical Logic? –  Feb 25 '20 at 06:15

8 Answers8

81

Linear Algebra is indeed one of the foundations of modern mathematics. There are a lot of things which use the language and tools developed in linear algebra:

  • Multidimensional Calculus, i.e. Analysis for functions of many variables, i.e. vectors (for example, the first derivative becomes a matrix)

  • Differential Geometry, which investigates structures which look locally like a vector space and functions on this.

  • Functional Analysis, which is essentially linear algebra on infinite-dimensional vector spaces which is the foundation of quantum mechanics.

  • Multivariate Statistics, which investigates vectors whose entries are random. For instance, to describe the relation between two components of such random vector, one can calculate the correlation matrix. Furthermore, one can apply a technique called singular value decomposition (which is close to calculating the eigenvalues of a matrix) to find which components are having a main influence on the data.

  • Tagging on to the Multivariate Statistics and multidimensional calculus, there are a number of Machine Learning techniques which require you to find a (local) minimum of a nonlinear function (the likelihood function), for example for neural nets. Generally speaking, on can try to find the parameters which maximize the likelihood, e.g. by applying the gradient descent method, which uses vectors arithmetic. (Thanks, frogeyedpeas!)

  • Control Theory and Dynamical Systems theory is mainly concerned with differential equations where matrices are factors in front of the functions. It helps tremendously to know the eigenvalues of the matrices involved to predict how the system will behave and also how to change the matrices in front to make sure the system behaves like you want it to - in Control Theory, this is related to the poles and zeros of the transfer function, but in essence it's all about placing eigenvalues at the right place. This is not only relevant for mechanical systems, but also for electric engineering. (Thanks, Look behind you!)

  • Optimization in general and Linear Programming in particular is closely related to multidimensional calculus, namely about finding minima (or maxima) of functions, but you can use the structure of vector spaces to simplify your problems. (Thanks, Look behind you!)

On top of that, there are a lot of applications in engineering and physics which use tools of linear algebra and the fields listed above to solve real-world problems (often calculating eigenvalues and solving differential equations).

In essence, a lot of things in the mathematical toolbox in one variable can be lifted up to the multivariable case with the help of Linear Algebra.

Edit: This list is by no means complete, these were just the topics which came to my mind at first thought. Not mentioning a particular field doesn't mean that this field irrelevant, but just that I don't feel qualified to write a paragraph about it.

Roland
  • 6,064
  • 11
    Adding to the stats point, Machine Learning has been nothing but Lin Alg for me :) – Sidharth Ghoshal Mar 14 '16 at 06:30
  • 7
    Control theory, signal processing, optimization, stochastic processes... – Fraïssé Mar 14 '16 at 06:44
  • 6
    dynamical systems theory – N. Virgo Mar 14 '16 at 11:39
  • 6
    Numerical analysis often boils down to sparse matrix calculations. – lisyarus Mar 14 '16 at 13:42
  • 2
    Examples from algebra: (almost) every course on field theory / Galois theory will begin by observing that if $K/k$ is an extension of fields, $K$ is a vector space over $k$ and use linear algebra to study this further. Representation theory is the study of groups through linear algebra (more precisely, through their action on a vector space). – Myself Mar 14 '16 at 16:34
  • 1
    @lisyarus, probably not "often", unless you deal with a lot of PDEs or networks. There's still much demand for dense methods, especially those that exploit structure. Nevertheless, linear algebra is one of the very handy tools in the box of a numericist. – J. M. ain't a mathematician Mar 14 '16 at 18:17
  • 1
    @lisyarus Not just sparse but dense matrix calculations too. Imagine using finite element on a volume of space and finding out that the resulting matrix is sparse but absolutely ginormous. Then you use the boundary element method to reduce the problem by a dimension so now you have a smaller matrix but it is dense. – Fixed Point Mar 14 '16 at 21:02
  • 5
    Cryptography!!! – vojta Mar 15 '16 at 08:07
  • I was very lucky to fall in love for my two-semester course on linear algebra (because solely of its mathematical beauty) while undergraduate student in Statistics. As it turned out, I was the only one in my last-year class that actually understood the depth, beauty, breadth and power of multivariate analysis techniques. I could see that every single one of my classmates were understanding zits and resigned to just memorize the basic facts and how to make statistical packages run those techniques. – Marcelo Ventura Apr 09 '16 at 18:33
  • After that, on my masters, I was also able to study by myself some other techniques that were new to me. The Spectral Theorem, in particular, is one of the seemingly arcane techniques that has a vast applicability. – Marcelo Ventura Apr 09 '16 at 18:34
27

By now, we can roughly say that all we fully understand in Mathematics is linear.

So I guess Linear Algebra is a good topic to master.

Both in Mathematics and in Physics one usually brings the problem to a Linear problem and then solve it with Linear Algebras techniques. This happens in Algebra with linear representations, in Functional Analysis with the study of Hilbert Spaces, in Differential Geometry with the tangent spaces, and so on almost in every field of Mathematics. Indeed I think there should be at least 3 courses of Linear Algebra (undergraduate, graduate, advanced), everytime with different insights on the subject.

Dac0
  • 9,154
  • 9
    Put another way: we approximate a nonlinear reality with a more easily imagined linear simulacrum. – J. M. ain't a mathematician Mar 14 '16 at 18:18
  • 11
    In mathematics, we divide problems into linear problems and non-linear problems. This is like dividing the universe into bananas and not-bananas. – Yakk Mar 14 '16 at 18:27
  • 4
    +1 for the line "all we fully understand in mathematics is linear." – Placidia Mar 14 '16 at 19:09
  • What do you mean by that? In which way do we not fully understand, say, number theory? – leftaroundabout Mar 14 '16 at 21:10
  • 6
    well, If you fully understand Number Theory be my guest... I would have few million-dollars-questions to ask you... :D :D :D – Dac0 Mar 14 '16 at 21:12
  • @Dac0: of course. There are always unanswered questions, in any nontrivial topic. Is Linear Algebra different in this regard? Is this some kind of Gödel argument? – leftaroundabout Mar 16 '16 at 11:05
  • 2
    @leftaroundabout Indeed linear algebra is different. If you study mathematics for a while you know that if you have a problem (in almost every context) is a common first practice to find some kind of way to reconduct you problem to a linear problem or your structure to a linear structure. Actually there's a reason for that, I'll let you guess what is... – Dac0 Mar 16 '16 at 12:57
  • I think you're evading my point a bit. Sure, the widespread use of linearisation techniques gives an empirical argument that LA is quite useful, and that there exist easy problems in LA. But you can't conclude from this that a problem which can't be linearised at any level is necessarily hard (example: tic tac toe), nor that any problem which can be reduced to linear algebra is necessarily easy / fully understood (example: generalised Ising ground state). Or am I missing something here? – leftaroundabout Mar 16 '16 at 15:50
  • 1
    I think you are missing a lot here, but you're entitled to think whatever you prefer... – Dac0 Mar 16 '16 at 16:23
  • For sure I am missing why you respond this way, but whatever... I made a new question of the subject. – leftaroundabout Mar 16 '16 at 21:04
14

Excellent answers above. I would just like to add:

  • Computer graphics. As soon as you scale, rotate and/or translate an object on a computer screen, that is Linear Algebra in action. Computer graphics is an academic field that depend heavily on Linear Algebra.
MichaelK
  • 267
  • This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - From Review – choco_addicted Mar 15 '16 at 07:55
  • 9
    Considering I have had classes in Computer Graphics (where we used Linear Algebra), that CG is an academic field, and that I have used LA professionally in that particular field, I strongly disagree with you. People above already mentioned many important fields that build on and/or use LA, and I saw no reason to repeat their excellent posts, but no-one mentioned CG. It would be folly of me to comment on each post above just to say "Do not forget CG". I made the judgement call it was best to mention CG in this way instead. – MichaelK Mar 15 '16 at 08:00
  • 5
    +1 Whenever a curious teenage student asks me "what is linear algebra, and why would I care?", I often root it in the real world by citing its use in graphics. – zahbaz Mar 15 '16 at 17:40
6

Mathematics are all about building theorems and properties from a set of ground hypothesis.

Linear algebra is the theory around the set of properties that define vector spaces and linear mappings, and these properties are verified by a huge number of other mathematical constructions, but can also verified as first order approximations of most physical systems.

So to answer your question, if you like math for themselves, you will love Linear Algebra for its own beauty. You really start from a simple set of properties and build very interesting objects. But it is also a mandatory step for most engineering or mathematics majors, as its properties are verified by a huge number of systems.

Remi D
  • 179
5

Well, let's try to imagine how to avoid linear algebra.

Anything that involves functions of $2$ or more variables, that can be differentiated, will very soon use linear operators or matrices. No geometry!

Also, no algebra or number theory or topology. Almost anything in algebra or its client subjects that involves associative (not necessarily commutative) multiplication, such as composition of functions, requires the simpler case of linear algebra as a preliminary.

That leaves calculus of one real or complex variable. There, it will still be hard to avoid linear algebra, unless you also avoid recursion relations, differential and difference equations, operators on the functions, measures, Fourier and Laplace transforms, special function and orthogonal polynomials, discrete or finite-element methods, calculus of variations. Sooner or later one meets linear-algebra concepts like eigenvalue, linear operator and quadratic form.

One could try to hide in parts of analysis that seem less algebraic. The Riemann zeta function does not look like it needs any linear algebra. But understanding it requires some algebraic number theory, and then there are all the conjectures about its relations to linear operators, and theorems that the zeros are spaced according to patterns typical of random... matrices. Fractal geometry doesn't look very algebraic, until you ask why the Mandelbrot set looks self-similar at all scales, or use iterated function systems, which are directly based on linear algebra. Probability theory is full of Markov chains, which are complicated linear algebra.

OK then. As long as one sticks to subjects that are outside of algebra, analysis and geometry, and are so qualitative that no serious algebra/analysis/geometry is used for calculating anything, then there might not be a need to ever know linear algebra! This excludes concrete pursuits like combinatorics, computer science and all of applied mathematics.

For the very few subjects that are left, it is still questionable whether anyone could really make use of them without knowing linear algebra. There are too many basic thought patterns that are learned in linear algebra, such as reasoning about higher-dimensional spaces, that have permeated the language and ideas in other parts of mathematics.

zyx
  • 35,436
4

Linear algebra is often (as seems to be the case here by your instructor) taught in a disjoint way from calculus and then eventually the threads are gathered together and merged, but usually hastily with what seems to be tangential asides, leaving most students to miss the point.

The truth of the matter is it turns out that to really understand the functions you study in calculus, you need to understand spaces of functions. You already get a hint of that when you talk about differentiation and integration as being "inverse" operators. The natural structure to put on these spaces is that of a vector space (it's not coincidence that high-schoolers learn about the "algebra of functions").

Linear algebra can be considered the study of vector spaces and natural operations upon them. Using the tools of linear algebra, such as eigenvalues, you can gain an understanding of linear operators on these function spaces. This will actually help you understand differential equations, which themselves are the chief motivation for studying calculus in the first place (historically Newton et al studied calculus to solve differential equations in physics).

Chan-Ho Suh
  • 549
3

Very good answers so far but I would like to add...

  • the matrix representation theory of groups (and fields) : That more advanced algebras can be calculated and systematically investigated using linear algebra of less advanced "numbers" (groups). So linear algebra for simple fields lays the foundation and provides the tools for both constructing and systematically investigating more advanced types of bananas, sorry numbers.
mathreadler
  • 25,824
3

Mathematical Logic, Type, Proof, and Category Theory, and Abstract Algebra do not depend on Linear Algebra that much, but rather underlay it.
Moreover say Number Theory and Homology rely on modules not vector spaces (vectors over rings, not fields).

So while Linear Algebra is indeed very widely used in practice, it is not yet any appropriate candidate for being foundation of mathematics.

Andrew
  • 449