Introductory description

This module is targeted toward students who seek to understand frameworks for the design and evaluation of policies, big or small. It aims to provide students with the skills and knowledge required to make informed decisions concerning the design of policy interventions, and the evaluation of their impact in a range of settings.

Module web page

Module aims

Outline syllabus

This is an indicative module outline only to give an indication of the sort of topics that may be covered. Actual sessions held may differ.

In order to facilitate the acquisition of knowledge and competency, the course is structured and taught via a number of core problems in the development and evaluation of policy which must be understood and overcome. Where possible, problems will be addressed through a case-study approach which focusses on the design and impact evaluation of real-world policy — including the study, replication, and extension of analysis carried out in reports and/or academic papers.

1. How and why do we consider evaluation when designing policy?

• What frameworks can we use to design effective policy?
• Why do we evaluate?
• What are the links between good policy design and effective impact evaluation?
• The power of data: Descriptive Statistics, Indicators, Indices

1. How can we collect data, what can it tell us, and what can it not?

• Data collection, sampling methods, distributions, sample v population
  Revision of correlation, statistical significance, spurious relationships, causality,
  formation of hypotheses

1. Are we observing correlation or causation?
   Revision of multivariate analysis: Principles, assumptions, methods, interpretation
   Difference in difference approaches and the importance of a control group

• Regression discontinuity

1. How do we overcome the issue of problematic data?

• Overcoming imperfect control groups and non-compliance
• The frequent issue of endogeneity, instrumental variables and the art of finding a good instrument

1. How can we maximise the impacts of our findings?

Learning outcomes

By the end of the module, students should be able to:

• Demonstrate an understanding of frameworks for policy design, be able to apply such frameworks to interpret existing policy, and use established frameworks as a tool for designing new policy interventions
• Demonstrate an understanding of the application of policy design frameworks to both localised interventions, and also broader applications
• Demonstrate an understanding of the importance of appropriate data collection and choice of intervention subjects, random sampling, and how data collection impacts on the ability to use sample data to make inferences about the broader population
• Interpret, produce, appropriately present, and explain (verbally and in writing) a range of descriptive statistics relevant to policy interventions
• Demonstrate (verbally and in writing) an understanding of the need for rigorous policy or intervention evaluation and be able to provide examples of good practice in this subject area
• Identify and implement appropriate statistical methods for the evaluation of policy intervention based upon the characteristics of the intervention and the available data, and to be able to use these approaches to overcome challenges of identification of causality

Subject specific skills

-Demonstrate an understanding of frameworks for policy design, be able to apply such frameworks to interpret existing policy, and use established frameworks as a tool for designing new policy interventions
-Demonstrate an understanding of the application of policy design frameworks to both localised interventions, and also broader applications

Transferable skills

-Demonstrate an understanding of the importance of appropriate data collection and choice of intervention subjects, random sampling, and how data collection impacts on the ability to use sample data to make inferences about the broader population
-Interpret, produce, appropriately present, and explain (verbally and in writing) a range of descriptive statistics relevant to policy interventions
-Demonstrate (verbally and in writing) an understanding of the need for rigorous policy evaluation and be able to provide examples of good practice in this subject area