Preparing Math for Machine Learning at TU Kaiserslautern

Machine learning uses basic tools from mathematics to formulate its concepts. In order to understand machine learning, you need to first understand the math behind it. In the following, we have collected mathematical material that is, to the best of our knowledge, ideally taylored to machine learning. In particular, we are not suggesting you to study areas of math that are irrelevant for machine learning. The following material is ideally taylored to the TUK courses on machine learning: ML1, ML2, and ML3.

Option 1 All-In-One (workload: high)

Read the book Mathematics for Machine Learning, from Marc Peter, Deisenroth, A. Aldo Faisal, und Cheng Soon Ong. ISBN:978-1-108-47004-9

  • Sections 2-5 for ML1
  • Additionally section 6 for ML2
  • Additionally section 7 for ML3

Option 2 (workload: medium)

1.1 Lecture notes on "Mathematics for Machine Learning'

A good summary of the most basic prerequisites is contained in the following reference:

  • Maths for Intelligent Systems, the following sections/subsections can be ommitted: 3.5,3.6.1
  • For ML1 study sections 1,2, and 3.
  • For ML 2, section 5 should also be read.
  • For ML3 study Section 4 and Section 5.1

In order to master the material, it is crucial that you have a hands-on experience. You need attempt some (or better: all) of the exercises!

1.2 Video Lecture

If you are not confident in your mathematics skills, or if you prefer learning by watching video lectures, you can watch the following video lectures. Make sure, however, that, subsequently, you study the above material.

Option 3 (workload: high)

If you want to invest more time into your math, consider studying the following material:

  • Linear algebra (basics) (Except chapters 4 and 5). For exercises, see the example sheets from the same website. We recommended the following exercises (starred exercises are harder). Sheet 1: 1,2,4,5,6,12*. Sheet2: 1,7,8,12*. Sheet 3: 1,2,12*. Sheet4: none.
  • Linear algebra (review): basic diagonalisation exercises with solutions.
  • Probability (basics) (except the following chapters: 13,14,19,20). Exercises: see example sheets on the same webpage. We recommended the following exercises. Sheet 1: 1,2,4,5,6,14*. Sheet 2:1,2,3,7,12**. Sheet 3: 2,3,6,7. Sheet 4 : 1,2,4,7,8. Motivated students are  encouraged to attempt more excercises.

After studying the above, you can check your learning success by attempting to solve the following three questions:

  • Question 1:
    • (a) Prove that eigenvectors corresponding to different eigenvalues of a real symmetric matrix are orthogonal.
    • (b) Prove that the eigenvalues of a real symmetric matrix are real.
    • (c) Prove that every real symmetric matrix is diagonalisable.
  • Question 2: See exercise 5 in the following reference.
  • Question 3 (harder, only for highly motivated students): How many n\times n matrices $A$ are there whose entries are either 0 or 1, and such that $A^2=0$?

Option 4 (workload: very high)

Option 3 is for highly motivated students only. The following reference is a highly complete resource for probability: Probability and Random Processes

You may also want to read the following books for linear algebra: