ST420-15 Statistical Learning and Big Data

Introductory description

This module runs in Term 2 and is available for students on a course where it is a listed option (subject to restrictions*) and as an Unusual Option to students who have completed the prerequisite modules.

Pre-requisites:
Statistics Undergraduate students: ST218 Mathematical Statistics A, ST219 Mathematical Statistics B and ST221 Linear Statistical Modelling.*
MSc in Statistics students: ST903 Statistical Methods and ST952 Introduction to Statistical Practice.*
Master’s in Financial Mathematics students: MA907 Simulation and Machine Learning.
External Undergraduate students: ST220 Introduction to Mathematical Statistics and ST221 Linear Statistical Modelling.*

Module web page

Module aims

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

Statistical Learning – an introduction to statistical learning theory, using simple ML methods to illustrate the various ideas:
From over-fitting to apparently complex methods which can work well, such as VC dimension and shattering sets.
PAC bounds. Loss functions. Risk (in the learning theoretic sense) and posterior expected risk. Generalisation error.
Supervised, unsupervised and semi-supervised learning.
The use of distinct training, test and validation sets, particularly in the context of prediction problems.
The Bootstrap revisited. Bags of Little Bootstraps. Bootstrap aggregation. Boosting.
Big Data and Big Model – issues and (partial) solutions:
The “curse of dimensionality”. Multiple testing; voodoo correlations, false-discovery rate and family-wise error rate. Corrections: Bonferroni, Benjamini-Hochberg.
Sparsity and Regularisation. Variable selection; regression. Spike and slab priors. Ridge Regression. The Lasso. The Dantzig Selector.
Concentration of measure and related inferential issues.
MCMC in high dimensions – preconditioned Crank Nicholson; MALA, HMC. Preconditioning. Rates of convergence.

Learning outcomes

By the end of the module, students should be able to:

• Explain, critically discuss and apply fundamental concepts and analytic tools in Statistical Learning;
• Analyse and discuss issues and fundamental tools in the analysis of Big Data and Big Models;
• Implement and assess methods for prediction based on partitioning data;
• Apply fundamental tools based on sparsity, regularisation and the control of error rates to analyse large data sets.

Indicative reading list

View reading list on Talis Aspire

Subject specific skills

TBC

Transferable skills

TBC