ST227-10 Stochastic Processes
Introductory description
This module introduces stochastic processes in discrete time and space. It is core for students with their home department in Statistics. It is available as an option or unusual option for other students.
Pre-requisites:
Statistics Students: ST119 Probability 2 AND (MA140 Mathematical Analysis 1 OR MA142 Calculus 1).
Non-Statistics Students: ST120 Introduction to Probability AND (MA141 Analysis 1 or equivalent).
Leads to:
ST333 Applied Stochastic Processes
ST406 Applied Stochastic Processes with Advanced Topics.
Module aims
To introduce the idea of a stochastic process, and to show how simple probability and matrix theory can be used to build this notion into a beautiful and useful piece of applied mathematics.
Loosely speaking, a stochastic or random process is any measurable phenomenon which develops randomly in time. Only the simplest models will be considered in this course, namely those where the process moves by a sequence of jumps in discrete time steps. We will focus on Markov chains, which use the idea of conditional probability to provide a flexible and widely applicable family of random processes; and random walks, which serve as fundamental building blocks for constructing other processes as well as being important in their own right. Such processes are common tools in economics, biology, psychology, and operations research, so they are very useful as well as attractive and interesting theories.
Outline syllabus
This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.
This module introduces the study of random processes in time. The module covers
- Brief review of fundamental probability notions.
- Introduction to Markov processes (definitions, Chapman-Kolmogorov equations, notions of recurrence, transience, reducible, periodic, ergodic, transition probability matrices).
- Long-run behaviour of Markov Chains (hitting times and absorption, equilibrium distributions, convergence to equilibrium). Time reversal, detailed balance, and the ergodic theorem.
- Applications of Markov chains (e.g. branching processes in epidemiology).
Learning outcomes
By the end of the module, students should be able to:
- Describe the notion of a Markov chain, and know how simple ideas of conditional probability and matrices can be used to give a thorough and effective account of discrete-time Markov chains.
- Analyse notions of long-time behaviour including transience, recurrence, and equilibrium.
- Know how to apply these ideas to answer basic questions in applied situations and to recognise where Markov chains are and are not appropriate models.
- Communicate solutions to problems accurately with structured and coherent arguments.
Indicative reading list
- S.M. Ross, Introduction to Probability Models. Academic Press.
- G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes. Oxford University Press.
- P.W. Jones and P. Smith, Stochastic Processes. Chapman & Hall.
- J.R. Norris, Markov Chains. Cambridge University Press.
View reading list on Talis Aspire
Interdisciplinary
This module requires students to develop balanced facility of rigorous mathematical argument together with appreciation of the over-riding relevance of statistical considerations.
Subject specific skills
Demonstrate facility with advanced mathematical and probabilistic methods
Demonstrate knowledge of key mathematical and statistical concepts, both explicitly and by applying them to the solution of mathematical problems.
Reason critically, carefully, and logically and derive (prove) mathematical results.
Create structured and coherent arguments communicating them in written form
Analyse problems, abstracting their essential information formulating them using appropriate mathematical language to facilitate their solution.
Transferable skills
Problem solving skills: The module requires students to solve problems presenting their conclusions as logical and coherent arguments.
Written communication skills: Students complete written assessments that require precise and unambiguous communication in the manner and style expected in mathematical sciences.
Verbal communication skills: Students are encouraged to discuss and debate formative assessment and lecture material within small-group tutorials sessions. Students can continually discuss specific aspects of the module with the module leader. This is facilitated by statistics staff office hours.
Team working and working effectively with others: Students are encouraged to discuss and debate formative assessment and lecture material within small-group tutorials sessions.
Professionalism: Students work autonomously by developing and sustain effective approaches to learning, including time-management, organisation, flexibility, creativity, collaboratively and intellectual integrity.
Study time
Type | Required | Optional |
---|---|---|
Lectures | 20 sessions of 1 hour (20%) | 2 sessions of 1 hour |
Seminars | 4 sessions of 1 hour (4%) | |
Private study | 66 hours (66%) | |
Assessment | 10 hours (10%) | |
Total | 100 hours |
Private study description
Weekly revision of lecture notes and materials, wider reading and practice exercises, working on problem sets and preparing for examination.
Costs
No further costs have been identified for this module.
You do not need to pass all assessment components to pass the module.
Assessment group D
Weighting | Study time | |
---|---|---|
Problem set 1 | 2% | 2 hours |
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. |
||
Problem set 2 | 3% | 2 hours |
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. |
||
Problem set 3 | 2% | 2 hours |
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. |
||
Problem set 4 | 3% | 2 hours |
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. |
||
Stochastic Processes Examination | 90% | 2 hours |
You will be required to answer all questions on this examination paper.
|
Assessment group R
Weighting | Study time | |
---|---|---|
In-person Examination - Resit | 100% | |
You will be required to answer all questions on this examination paper.
|
Feedback on assessment
Individual feedback will be provided on problem sheets by class tutors.
Solutions and cohort level feedback will be provided for the examination
Students are actively encouraged to make use of office hours to build up their understanding, and to view all their interactions with lecturers and class tutors as feedback.
Courses
This module is Core for:
- Year 2 of USTA-G302 Undergraduate Data Science
- Year 2 of USTA-G304 Undergraduate Data Science (MSci)
-
USTA-G300 Undergraduate Master of Mathematics,Operational Research,Statistics and Economics
- Year 2 of G30A Master of Maths, Op.Res, Stats & Economics (Actuarial and Financial Mathematics Stream)
- Year 2 of G30B Master of Maths, Op.Res, Stats & Economics (Econometrics and Mathematical Economics Stream)
- Year 2 of G30C Master of Maths, Op.Res, Stats & Economics (Operational Research and Statistics Stream)
- Year 2 of G30D Master of Maths, Op.Res, Stats & Economics (Statistics with Mathematics Stream)
- Year 2 of G300 Mathematics, Operational Research, Statistics and Economics
- Year 2 of USTA-G1G3 Undergraduate Mathematics and Statistics (BSc MMathStat)
- Year 2 of USTA-GG14 Undergraduate Mathematics and Statistics (BSc)
- Year 2 of USTA-Y602 Undergraduate Mathematics,Operational Research,Statistics and Economics