What all maths do I need to know to become good at machine learning.

Question

I am a computer science engineer and I took a couple of maths classes in my first year they were on Fourier series(not transform) partial differential equations, vector calculus, infinite series matrices.

I find that none of those classes are helping me today and I am just starting with machine learning this semester and I find that my maths skills are horrible. I just want to know all the maths topics that I would be needing for this course so I could may be get a maths book and practice them.

PS: This is my syllabus

Answer 1

In my opinion the essentials are:

• Linear algebra. Comes up explicitly e.g. in neural nets but it is used implicitly all over the place e.g. in solving regression.

• Calculus, but focus on differentiation rather than integration. Be sure to include partial differentiation and finding minima and maxima of functions of multiple variables because this is used in...

• Optimization. Understand gradient descent at the very least, and what can go wrong when you try to optimize functions that aren't concave / convex.

• Probability and statistics. You definitely need to understand the basics such as what are random variables, what are means and expectations, measures of spread including standard deviation. It would also be useful to understand linear regression, and the different perspective provided by Bayesian approaches. The danger of going too far into statistics is that a significant part of statistics is concerned with hypothesis testing, which isn't very helpful for machine learning.

All in all, try to get the basics of these areas sorted, but don't go too far because:

1. It's perfectly possible to be a successful practitioner of machine learning without an in-depth understanding of the maths.

2. Once you have enough basics it's going to be better to take a "just in time" approach to learning because the underlying maths involved is so wide and deep that you couldn't possibly be expert in all of it even if you wanted to.

Finally, if you haven't already, Take Andrew Ng's machine learning course on coursera, which is a great overview of the area, and covers just enough maths to avoid glossing over important issues, but not so much that is inaccessible if you're willing to put the effort in.

Answer 2

It depends a lot on the kind of models you intend to work on, but just to add a few keywords and links:

• Statistics: useful to explore the data, build models potentially and assess models' performances.
• Probabilities: since most data is noisy, we need them.
• Linear algebra, especially eigenvectors, eigenvalues and SVD/PCA/ICA. MOOC: edX: Linear Algebra - Foundations to Frontiers. Also useful to vectorize your formulas and implementations (GPUs can be pretty useful).
• Calculus, mostly being able to compute gradients (ubiquitous in machine learning). To a lesser extent, integration, especially when handling continuous probabilities.
• Graph theory: knowing the basic terms is useful when thinking about graphical models.
• Logic: mostly GOFAI but still good to know.
• Computational complexity theory, since one key issue in machine learning is the computational time.

In addition to Ng's excellent course, if you want to focus on a more theoretical approach, I advise to look at the Caltech MOOC on machine learning which is pretty awesome too.

Answer 3

You will need multivariable calculus, linear algebra, and probability. I am creating an online course that goes through the math for machine learning, for beginners. You can find out more by checking out:

Math for Machine Learning Course