Introductory description

N/A

Module web page

Module aims

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

Fundamentals of statistical learning theory

• Regression and classification
• Empirical risk minimization and regularization
• Concentration inequalitites
• VC theory
• Rademacher complexity and covering numbers

Optimization

• Convexity
• Optimality conditions, Lagrange duality, KKT conditions
• Quadratic optimization and support vector machines
• Gradient descent - Acceleration
• Proximal gradient methods
• Stochastic gradient descent

Deep Learning

• Neural networks
• Convolutional neural networks
• Universal approximation
• Adversarial perturbations
• Generative adversarial networks (GAN)
• Accuracy and stability
• Recurrent neural networks
• Applications

Learning outcomes

By the end of the module, students should be able to:

• Describe the problem of supervised learning from the point of view of function approximation, optimization, and statistics.
• Identify the most suitable optimization and modelling approach for a given machine learning problem.
• Analyse the performance of various optimization algirthms from the point of view of computational complexity (both space and time) and statistical accuracy.
• Implement a simple neural network architecture and apply it to a pattern recognition task.

Indicative reading list

1. Beck, Amir. First-Order Methods in Optimization. Vol. 25. SIAM, 2017.
2. Vapnik, Vladimir. The nature of statistical learning theory. Springer, 2013.
3. Cucker, Felipe, and Ding Xuan Zhou. Learning theory: an approximation theory viewpoint. Vol. 24. Cambridge University Press, 2007.
4. Higham, Catherine F., and Desmond J. Higham. "Deep Learning: An Introduction for Applied Mathematicians." arXiv preprint arXiv:1801.05894 (2018).
   5.Yurii Nesterov, Lectures on Convex Optimization, Springer, 2018
   6.Aaron Courville, Ian Goodfellow, and Yoshua Bengio, Deep Learning, MIT Press, 2016
   7.Gábor Lugosi, Pascal Massart, and Stéphane Boucheron, Concentration Inequalities, OUP, 2013

Subject specific skills

At the end of this course, the students will be able to apply state-of-the-art optimization methods to machine learning tasks, and evaluate their performance.

Transferable skills

Students will acquire key reasoning and problem solving skills which will empower them to address new problems with confidence.