Introductory description

N/A.

Module web page

Module aims

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

1. Probability theory (4 lectures - optional for students with strong Maths background)
   a. Basic concepts such as probability space, events, expectation, moments, densities, generating functions, conditioning, marginalization, independence
   b. Joint/conditional probability densities, (conditional) expectations
   c. Random variables/vectors, covariance, correlation, random processes, mean and covariance functions, cross-covariance, classification of processes, ergodicity
   d. Markov chains, Poisson processes, continuous time Markov chains, Brownian motion, martingales
   e. Classical Monte Carlo (rejection, importance sampling), convergence property, Markov chain Monte Carlo (Gibbs, Metropolis-Hastings) Monte Carlo, Random number generation, univariate and multivariate distributions
   f. Bayes formula, illustration with binary variables (events), Bayes formula for continuous variables: likelihood, conjugate priors
2. Machine learning essentials (4 lectures)
   a. Statistical learning introduction and background
   i. Decision theory; Bayes risk
   ii. Probabilistic models
   iii. Complexity, regularization, bias vs. variance
   iv. Resampling, cross-validation
   b. Unsupervised techniques
   i. Linear dimensionality reduction: PCA and SVD, MDS
   ii. Nonlinear dimensionality reduction: LLE, Isomap, kPCA, diffusion maps
   iii. Clustering methods, K-means; hierarchical algorithms; probabilistic model-based clustering; graph-based/spectral clustering
   iv. Density estimation
   v. Gaussian mixture models
   vi. Expectation-maximization
   c. Supervised techniques for regression and classification
   i. Linear methods: linear, logistic, Bayesian regression and generalized linear models, naive Bayes, LDA, SVM
   ii. Nonlinear methods: kernel methods, nearest neighbor, decision trees, neural networks, Gaussian process regression
   d. Semi-supervised techniques
   e. Ensemble methods (bagging, boosting, random forests)
3. Uncertainty propagation through surrogate-model construction (4 lectures)
   a. Statistical emulators
   b. Deterministic vs Bayesian training and cross-validation
   c. Gaussian processes and limitations,
   d. Multivariate RVM
   e. Mixtures/products of models (mixtures/products of experts)
   f. Spectral Stochastic Methods (generalized polynomial Chaos, gPC)
   i. Intrusive vs non-intrusive, collocation, sparse-grid, tensor products
   ii. Sparse Polynomial Chaos
   g. Illustration for ODE with various input dimensions
4. Predictive materials modelling (4 lectures)
   a. Statistical thermodynamics and Monte Carlo methods (stochastic exploration of potential energy surfaces, ab-initio thermodynamics, structure prediction).
   b. Random microstructures (effective properties, property variability, sampling of microstructures, microstructure and materials failure). 4 lectures.
   c. Model errors (constitutive model errors, limits of density functional theory, transferability of exchange-correlation functionals and pseudopotentials, uncertainty quantification of effective potentials, sampling of thermodynamic quantities).
   d. High dimensionality in materials modelling (challenges in simulation and uncertainty quantification, dimensionality reduction, coarse graining and microscopic model reconstruction, model selection).
   e. Machine learning and information (statistical learning approaches, materials genome, materials informatics)

Learning outcomes

By the end of the module, students should be able to:

• Demonstrate knowledge of statistical and mathematical methods for predictive modelling.
• Perform detailed, advanced analyses of complex data sets, extracting information and developing relationships using linear and nonlinear regression and classification techniques.
• Systematically develop models for predictive purposes using advanced techniques of model selection and evaluation.
• Understand and apply cutting-edge methods of machine learning.
• Demonstrate an understanding of complex modelling transferability issues arising from, e.g. choices of exchange-correlation functionals and pseudo-potentials in electronic structure, or the choice of force fields in atomistic and molecular models.
• Demonstrate a detailed knowledge of, and be able to apply models, for quantifying uncertainties arising in material structure and properties, constitutive models, from limited data scenarios and through coarse graining.

Indicative reading list

Probability Random Variables and Stochastic Processes, by A. Papoulis
Pattern Recognition and Machine Learning, by Christopher Bishop
Bayesian Data Analysis, 3nd Edit., by A. Gelman et al.
The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition, by Trevor Hastie, Robert Tibshirani, Jerome Friedman
Monte Carlo Strategies in Scientific Computing, by J.S. Liu
Information Theory, Inference, and Learning Algorithms, by D. MacKay

Subject specific skills

Demonstrate knowledge of statistical and mathematical methods for predictive modelling
Perform detailed, advanced analyses of complex data sets, extracting information and developing relationships using linear and nonlinear regression and classification techniques
Systematically develop models for predictive purposes using advanced techniques of model selection and evaluation
Understand and apply cutting-edge methods of machine learning
Demonstrate an understanding of complex modelling transferability issues arising from, e.g. choices of exchange-correlation functionals and pseudo-potentials in electronic structure, or the choice of force fields in atomistic and molecular models.
Demonstrate a detailed knowledge of, and be able to apply models, for quantifying uncertainties arising in material structure and properties, constitutive models, from limited data scenarios and through coarse graining.

Transferable skills

Mathematical analysis, statistics, coding, writing