Introductory description

This module introduces stochastic processes in discrete time and space. It is core for students with their home department in Statistics. It is available as an option or unusual option for other students.

Pre-requisites:
Statistics Students: ST119 Probability 2 AND (MA140 Mathematical Analysis 1 OR MA142 Calculus 1).
Non-Statistics Students: ST120 Introduction to Probability AND (MA141 Analysis 1 or equivalent).

Leads to:
ST333 Applied Stochastic Processes
ST406 Applied Stochastic Processes with Advanced Topics.

Module web page

Module aims

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

This module introduces the study of random processes in time. The module covers

1. Brief review of fundamental probability notions.
2. Introduction to Markov processes (definitions, Chapman-Kolmogorov equations, notions of recurrence, transience, reducible, periodic, ergodic, transition probability matrices).
3. Long-run behaviour of Markov Chains (hitting times and absorption, equilibrium distributions, convergence to equilibrium). Time reversal, detailed balance, and the ergodic theorem.
4. Applications of Markov chains (e.g. branching processes in epidemiology).

Learning outcomes

By the end of the module, students should be able to:

• Describe the notion of a Markov chain, and know how simple ideas of conditional probability and matrices can be used to give a thorough and effective account of discrete-time Markov chains.
• Analyse notions of long-time behaviour including transience, recurrence, and equilibrium.
• Know how to apply these ideas to answer basic questions in applied situations and to recognise where Markov chains are and are not appropriate models.
• Communicate solutions to problems accurately with structured and coherent arguments.

Indicative reading list

• S.M. Ross, Introduction to Probability Models. Academic Press.
• G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes. Oxford University Press.
• P.W. Jones and P. Smith, Stochastic Processes. Chapman & Hall.
• J.R. Norris, Markov Chains. Cambridge University Press.

View reading list on Talis Aspire

Interdisciplinary

This module requires students to develop balanced facility of rigorous mathematical argument together with appreciation of the over-riding relevance of statistical considerations.

Subject specific skills

Demonstrate facility with advanced mathematical and probabilistic methods

Demonstrate knowledge of key mathematical and statistical concepts, both explicitly and by applying them to the solution of mathematical problems.

Reason critically, carefully, and logically and derive (prove) mathematical results.

Create structured and coherent arguments communicating them in written form

Analyse problems, abstracting their essential information formulating them using appropriate mathematical language to facilitate their solution.

Transferable skills

Problem solving skills: The module requires students to solve problems presenting their conclusions as logical and coherent arguments.

Written communication skills: Students complete written assessments that require precise and unambiguous communication in the manner and style expected in mathematical sciences.

Verbal communication skills: Students are encouraged to discuss and debate formative assessment and lecture material within small-group tutorials sessions. Students can continually discuss specific aspects of the module with the module leader. This is facilitated by statistics staff office hours.

Team working and working effectively with others: Students are encouraged to discuss and debate formative assessment and lecture material within small-group tutorials sessions.

Professionalism: Students work autonomously by developing and sustain effective approaches to learning, including time-management, organisation, flexibility, creativity, collaboratively and intellectual integrity.