# ST227-10 Stochastic Processes

Department
Statistics
Level
Zorana Lazic
Credit value
10
Module duration
10 weeks
Assessment
Multiple
Study location
University of Warwick main campus, Coventry

##### Introductory description

This module introduces stochastic processes in discrete time and space. It is core for students with their home department in Statistics. It is available as an option or unusual option for other students.

Pre-requisites:
Statistics Students: ST119 Probability 2 AND (MA140 Mathematical Analysis 1 OR MA142 Calculus 1).
Non-Statistics Students: ST120 Introduction to Probability AND (MA141 Analysis 1 or equivalent).

ST333 Applied Stochastic Processes
ST406 Applied Stochastic Processes with Advanced Topics.

##### Module aims

To introduce the idea of a stochastic process, and to show how simple probability and matrix theory can be used to build this notion into a beautiful and useful piece of applied mathematics.

Loosely speaking, a stochastic or random process is any measurable phenomenon which develops randomly in time. Only the simplest models will be considered in this course, namely those where the process moves by a sequence of jumps in discrete time steps. We will focus on Markov chains, which use the idea of conditional probability to provide a flexible and widely applicable family of random processes; and random walks, which serve as fundamental building blocks for constructing other processes as well as being important in their own right. Such processes are common tools in economics, biology, psychology, and operations research, so they are very useful as well as attractive and interesting theories.

##### Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

This module introduces the study of random processes in time. The module covers

1. Brief review of fundamental probability notions.
2. Introduction to Markov processes (definitions, Chapman-Kolmogorov equations, notions of recurrence, transience, reducible, periodic, ergodic, transition probability matrices).
3. Long-run behaviour of Markov Chains (hitting times and absorption, equilibrium distributions, convergence to equilibrium). Time reversal, detailed balance, and the ergodic theorem.
4. Applications of Markov chains (e.g. branching processes in epidemiology).
##### Learning outcomes

By the end of the module, students should be able to:

• Describe the notion of a Markov chain, and know how simple ideas of conditional probability and matrices can be used to give a thorough and effective account of discrete-time Markov chains.
• Analyse notions of long-time behaviour including transience, recurrence, and equilibrium.
• Know how to apply these ideas to answer basic questions in applied situations and to recognise where Markov chains are and are not appropriate models.
• Communicate solutions to problems accurately with structured and coherent arguments.
• S.M. Ross, Introduction to Probability Models. Academic Press.
• G.R. Grimmett and D.R. Stirzaker, Probability and Random Processes. Oxford University Press.
• P.W. Jones and P. Smith, Stochastic Processes. Chapman & Hall.
• J.R. Norris, Markov Chains. Cambridge University Press.
##### Interdisciplinary

This module requires students to develop balanced facility of rigorous mathematical argument together with appreciation of the over-riding relevance of statistical considerations.

##### Subject specific skills

Demonstrate facility with advanced mathematical and probabilistic methods

Demonstrate knowledge of key mathematical and statistical concepts, both explicitly and by applying them to the solution of mathematical problems.

Reason critically, carefully, and logically and derive (prove) mathematical results.

Create structured and coherent arguments communicating them in written form

Analyse problems, abstracting their essential information formulating them using appropriate mathematical language to facilitate their solution.

##### Transferable skills

Problem solving skills: The module requires students to solve problems presenting their conclusions as logical and coherent arguments.

Written communication skills: Students complete written assessments that require precise and unambiguous communication in the manner and style expected in mathematical sciences.

Verbal communication skills: Students are encouraged to discuss and debate formative assessment and lecture material within small-group tutorials sessions. Students can continually discuss specific aspects of the module with the module leader. This is facilitated by statistics staff office hours.

Team working and working effectively with others: Students are encouraged to discuss and debate formative assessment and lecture material within small-group tutorials sessions.

Professionalism: Students work autonomously by developing and sustain effective approaches to learning, including time-management, organisation, flexibility, creativity, collaboratively and intellectual integrity.

## Study time

Type Required Optional
Lectures 20 sessions of 1 hour (20%) 2 sessions of 1 hour
Seminars 4 sessions of 1 hour (4%)
Private study 66 hours (66%)
Assessment 10 hours (10%)
Total 100 hours
##### Private study description

Weekly revision of lecture notes and materials, wider reading and practice exercises, working on problem sets and preparing for examination.

## Costs

No further costs have been identified for this module.

You do not need to pass all assessment components to pass the module.

##### Assessment group D
Weighting Study time
Problem set 1 2% 2 hours

A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment.

Problem set 2 3% 2 hours

A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment.

Problem set 3 2% 2 hours

A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment.

Problem set 4 3% 2 hours

A problem sheet that include problem solving and calculations. Problem sheets will be set at fortnightly intervals. The problem sheets will contain a number of questions for which solutions and / or written responses will be required. The preparation and completion time noted below refers to the amount of time in hours that a well-prepared student who has attended lectures and carried out an appropriate amount of independent study on the material could expect to spend on this assignment.

Stochastic Processes Examination 90% 2 hours

You will be required to answer all questions on this examination paper.

• Students may use a calculator
##### Assessment group R
Weighting Study time
In-person Examination - Resit 100%

You will be required to answer all questions on this examination paper.

• Students may use a calculator
• Graph paper
##### Feedback on assessment

Individual feedback will be provided on problem sheets by class tutors.

Solutions and cohort level feedback will be provided for the examination

Students are actively encouraged to make use of office hours to build up their understanding, and to view all their interactions with lecturers and class tutors as feedback.

## Courses

This module is Core for:

• Year 2 of G302 Data Science
• Year 2 of G302 Data Science
• USTA-GG14 Undergraduate Mathematics and Statistics (BSc)
• Year 2 of GG14 Mathematics and Statistics
• Year 2 of GG14 Mathematics and Statistics
• USTA-Y602 Undergraduate Mathematics,Operational Research,Statistics and Economics
• Year 2 of Y602 Mathematics,Operational Research,Stats,Economics
• Year 2 of Y602 Mathematics,Operational Research,Stats,Economics

This module is Option list A for:

• Year 2 of G4G1 Discrete Mathematics
• Year 2 of G4G1 Discrete Mathematics
• Year 2 of UCSA-G4G3 Undergraduate Discrete Mathematics
• Year 2 of UMAA-G105 Undergraduate Master of Mathematics (with Intercalated Year)