ST22710 Stochastic Processes
Introductory description
This module introduces stochastic processes in discrete time and space. It is core for students with their home department in Statistics. It is available as an option or unusual option for other students.
Prerequisites:
Statistics Students: ST119 Probability 2 AND (MA140 Mathematical Analysis 1 OR MA142 Calculus 1).
NonStatistics Students: ST120 Introduction to Probability AND (MA141 Analysis 1 or equivalent).
Leads to:
ST333 Applied Stochastic Processes
ST406 Applied Stochastic Processes with Advanced Topics.
Module aims
To introduce the idea of a stochastic process, and to show how simple probability and matrix theory can be used to build this notion into a beautiful and useful piece of applied mathematics.
Loosely speaking, a stochastic or random process is any measurable phenomenon which develops randomly in time. Only the simplest models will be considered in this course, namely those where the process moves by a sequence of jumps in discrete time steps. We will focus on Markov chains, which use the idea of conditional probability to provide a flexible and widely applicable family of random processes; and random walks, which serve as fundamental building blocks for constructing other processes as well as being important in their own right. Such processes are common tools in economics, biology, psychology, and operations research, so they are very useful as well as attractive and interesting theories.
Outline syllabus
This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.
This module introduces the study of random processes in time. The module covers
 Brief review of fundamental probability notions.
 Introduction to Markov processes (definitions, ChapmanKolmogorov equations, notions of recurrence, transience, reducible, periodic, ergodic, transition probability matrices).
 Longrun behaviour of Markov Chains (hitting times and absorption, equilibrium distributions, convergence to equilibrium). Time reversal, detailed balance, and the ergodic theorem.
 Applications of Markov chains (e.g. branching processes in epidemiology).
Learning outcomes
By the end of the module, students should be able to:
 Describe the notion of a Markov chain, and know how simple ideas of conditional probability and matrices can be used to give a thorough and effective account of discretetime Markov chains.
 Analyse notions of longtime behaviour including transience, recurrence, and equilibrium.
 Know how to apply these ideas to answer basic questions in applied situations and to recognise where Markov chains are and are not appropriate models.
 Communicate solutions to problems accurately with structured and coherent arguments.
Indicative reading list
 S.M. Ross, Introduction to Probability Models. Academic Press.
 G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes. Oxford University Press.
 P.W. Jones and P. Smith, Stochastic Processes. Chapman & Hall.
 J.R. Norris, Markov Chains. Cambridge University Press.
View reading list on Talis Aspire
Interdisciplinary
This module requires students to develop balanced facility of rigorous mathematical argument together with appreciation of the overriding relevance of statistical considerations.
Subject specific skills
Demonstrate facility with advanced mathematical and probabilistic methods
Demonstrate knowledge of key mathematical and statistical concepts, both explicitly and by applying them to the solution of mathematical problems.
Reason critically, carefully, and logically and derive (prove) mathematical results.
Create structured and coherent arguments communicating them in written form
Analyse problems, abstracting their essential information formulating them using appropriate mathematical language to facilitate their solution.
Transferable skills
Problem solving skills: The module requires students to solve problems presenting their conclusions as logical and coherent arguments.
Written communication skills: Students complete written assessments that require precise and unambiguous communication in the manner and style expected in mathematical sciences.
Verbal communication skills: Students are encouraged to discuss and debate formative assessment and lecture material within smallgroup tutorials sessions. Students can continually discuss specific aspects of the module with the module leader. This is facilitated by statistics staff office hours.
Team working and working effectively with others: Students are encouraged to discuss and debate formative assessment and lecture material within smallgroup tutorials sessions.
Professionalism: Students work autonomously by developing and sustain effective approaches to learning, including timemanagement, organisation, flexibility, creativity, collaboratively and intellectual integrity.
Study time
Type  Required  Optional 

Lectures  20 sessions of 1 hour (20%)  2 sessions of 1 hour 
Seminars  4 sessions of 1 hour (4%)  
Private study  66 hours (66%)  
Assessment  10 hours (10%)  
Total  100 hours 
Private study description
Weekly revision of lecture notes and materials, wider reading and practice exercises, working on problem sets and preparing for examination.
Costs
No further costs have been identified for this module.
You do not need to pass all assessment components to pass the module.
Assessment group D
Weighting  Study time  

Problem set 1  2%  2 hours 
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a wellprepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. 

Problem set 2  3%  2 hours 
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a wellprepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. 

Problem set 3  2%  2 hours 
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a wellprepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. 

Problem set 4  3%  2 hours 
A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a wellprepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment. 

Stochastic Processes Examination  90%  2 hours 
You will be required to answer all questions on this examination paper.

Assessment group R
Weighting  Study time  

Inperson Examination  Resit  100%  
You will be required to answer all questions on this examination paper.

Feedback on assessment
Individual feedback will be provided on problem sheets by class tutors.
Solutions and cohort level feedback will be provided for the examination
Students are actively encouraged to make use of office hours to build up their understanding, and to view all their interactions with lecturers and class tutors as feedback.
Courses
This module is Core for:

USTAG302 Undergraduate Data Science
 Year 2 of G302 Data Science
 Year 2 of G302 Data Science

USTAGG14 Undergraduate Mathematics and Statistics (BSc)
 Year 2 of GG14 Mathematics and Statistics
 Year 2 of GG14 Mathematics and Statistics

USTAY602 Undergraduate Mathematics,Operational Research,Statistics and Economics
 Year 2 of Y602 Mathematics,Operational Research,Stats,Economics
 Year 2 of Y602 Mathematics,Operational Research,Stats,Economics
This module is Option list A for:

UCSAG4G1 Undergraduate Discrete Mathematics
 Year 2 of G4G1 Discrete Mathematics
 Year 2 of G4G1 Discrete Mathematics
 Year 2 of UCSAG4G3 Undergraduate Discrete Mathematics
 Year 2 of UMAAG105 Undergraduate Master of Mathematics (with Intercalated Year)

UMAAG100 Undergraduate Mathematics (BSc)
 Year 2 of G100 Mathematics
 Year 2 of G100 Mathematics
 Year 2 of G100 Mathematics

UMAAG103 Undergraduate Mathematics (MMath)
 Year 2 of G100 Mathematics
 Year 2 of G103 Mathematics (MMath)
 Year 2 of G103 Mathematics (MMath)
 Year 2 of UMAAG1NC Undergraduate Mathematics and Business Studies
 Year 2 of UMAAG1N2 Undergraduate Mathematics and Business Studies (with Intercalated Year)
 Year 2 of UMAAGL11 Undergraduate Mathematics and Economics
 Year 2 of UECAGL12 Undergraduate Mathematics and Economics (with Intercalated Year)
 Year 2 of UMAAG101 Undergraduate Mathematics with Intercalated Year