37510 - Policy Assessment and Analysis

Learning outcomes

At the end of the course the student is expected to know the approaches and tools used to evaluate economic and social programs or decisions, and the main statistical techniques used in impact evaluation. In particular the student should be able: - to design the evaluation of a program; - to choose the appropriate design for impact evaluation basing on the available data; - to understand the results of impact evaluation.

Course contents

Background

Student are assumed to have some background on statistical inference and linear model or be willing to learn the necessary tecniques. Basic concepts can be found in the book "Introduction to econometrics" reported in the recommended reading list, chapters 1 to 5.

Topics

1) Introduction to Program Evaluation: concepts and goals, different purposes for evaluation

2) Program Evaluation design: implementation evaluation, impact evaluation, monitoring

3) Evaluation of program implementation

4) Impact evaluation: the counterfactual approach

5) The model of potential outcomes; impact definition and measures; selection and omitted variables bias

6) Experimental design

7) Quasi experimental designs and the method of difference in difference 

8) Use of regression analysis for impact estimation 

9) Statistical methods for impact assessment in non experimental designs:

- Longitudinal data: DID and panel data models 

- Regression Discontinuity Design

- Statistical matching and Propensity Score Matching

- Instrumental Variables

- Mixed methods

Readings/Bibliography

Reference texts:

A. Martini, M. Sisti (2009), Valutare il successo delle politiche pubbliche, Il Mulino

J. D. Angrist, J. S. Pischke (2009), Mostly Harmless Econometrics: An Empiricistic’s Companion, Princeton University Press

Recommended reading:

P. H. Rossi, M. W. Lipsey, H. E. Freeman (2003), Evaluation: A Systematic Approach, Sage, chap. 1, 2, 3, 5, 7

J. H. Stock, M. W. Watson, Introduzione all'econometria, Edizione italiana a cura di F. Peracchi (2005), Pearson, Prentice-Hall

Further reading (optional):

R. Blundell, M. Costa Dias (2002), Alternative approaches to evaluation in empirical microeconomics, Cemmap working paper CWP10/02, The Institute for fiscal studies, Department of economics, Ucl, http://www.cemmap.ac.uk/wps/cwp1002.pdf

Heckman J.J., Smith J.A. (1999), The pre-program earnings dip and the determinants of participation in a social programme. Implications for simple program evaluation strategies, The Economic Journal, 109, 313-348.

James J. Heckman; V. Joseph Hotz (1989), Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training, Journal of the American Statistical Association, Vol. 84, No. 408, pp. 862-874.

Angrist J.D. and A.B. Krueger (1999), Empirical strategies in labor economics, in O. Ashenfelter and D. Card (eds.), Handbook of Labor Economics, Vol. 3A, Amsterdam, North-Holland, pp.1277-1366.

Becker S.O., Ichino A. (2002), Estimation of average treatment effects based on propensity scores, The Stata Journal, 2(4), 358-377.

Dehejia R.H., Wahba S. (1999), Causal effects in nonexperimental studies: reevaluating the evaluation of training programs, Journal of the America Statistical Association, 94, 1053-1062

Dehejia R.H., Wahba S. (2002), Propensity score matching methods for nonexperimental casual studies, The Review of Economics and Statistics, 84(1), 151-161.

Card and Krueger (1994), Minumum wages and employment: a case study of the fast food industry in New Jersey and Pennsylvania, American Economic Review, 84, 4

D. Bondonio (2000), Statistical methods to evaluate geographically-targeted economic development programs, Statistica applicata, vol. 12, n. 2, pp. 177-204)

T. Lemieux, K. Milligan (2008), Incentive effects of social assistance: A regression discontinuity approach, Journal of Econometrics, 142, pp. 807–828

Teaching methods

- Teacher lectures

- Case studies

- Computer sessions on application of statistical techniques

Assessment methods

Student learning will be assesed by a written test, to be held at the end of the course, in order to check:
- the knowledge of the main issues in designing the evaluation of a program; 
- the knowledge and the uptake of the approaches and the main techniques for impact evaluation;
- the ability to choose between alternative methods for impact assessment depending on available data.
Questions may have closed multiple answers or an open-ended format, and may regard theory and methodology,  interpretation of the output of the statistical software used in computer sessions, simple exercises on impact estimation. 

Teaching tools

- Slides

The slides are available on the web at http://campus.unibo.it/ . Slides can be dowloded by students who subscribe to the mailing list of the course. In order to subscribe to the mailing list, students need a password that will be notified by the teacher at the beginning of the course. Students who do not attend classes can ask the teacher for password by e-mail.

- Statistical software for the application of estimation techniques in the computer lab.

Office hours

See the website of Pinuccia Pasqualina Calia