18079 - Assessment of Public Policies

Learning outcomes

At the end of the course the student is expected to know the approaches and tools used to evaluate economic and social programs, and the main statistical techniques used in impact evaluation. In particular the student should be able:

- to design the evaluation of a program

- to carry out monitoring and implementation evaluation of a program

- to choose the appropriate design for impact evaluation basing on the available data

- to understand the results of impact evaluation

Course contents

Background

 Student are assumed to have some background on statistics/econometrics (statistical inference and linear model) or be willing to learn the necessary tecniques. Basic concepts can be found in the book "Introduction to econometrics" reported in the recommended reading list, chapters 1 to 5.

Topics

1) Introduction to Program Evaluation: concepts and goals, different purposes for evaluation 

2) Program evaluation design: theory of change, monitoring, implementation evaluation, impact evaluation

3) Evaluation of program implementation, monitoring and indicators

4) Impact evaluation: the counterfactual approach and the model of potential outcomes

5) Estimation of the counterfactual: selection and omitted variables bias

6) Experimental and non experimental designs

8) Statistical methods for estimation of program impact with non experimental designs:

- Difference in Differences estimator

- Random Growth Rate Model

- Regression Discontinuity Design

- Propensity Score Matching

- Instrumental Variables

Readings/Bibliography

Reference text:

A. Martini, M. Sisti (2009), Valutare il successo delle politiche pubbliche, Il Mulino 

Recommended reading:

P. H. Rossi, M. W. Lipsey, H. E. Freeman (2003), Evaluation: A Systematic Approach, Sage, cap. 1, 2, 3, 5, 7

J. H. Stock, M. W. Watson, Introduzione all'econometria, Edizione italiana a cura di F. Peracchi (2005), Pearson, Prentice-Hall

Further reading (optional):

R. Blundell, M. Costa Dias (2002), Alternative approaches to evaluation in empirical microeconomics, Cemmap working paper CWP10/02, The Institute for fiscal studies, Department of economics, Ucl

Heckman J.J., Smith J.A. (1999), The pre-program earnings dip and the determinants of participation in a social programme. Implications for simple program evaluation strategies, The Economic Journal, 109, 313-348.

James J. Heckman; V. Joseph Hotz (1989), Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: The Case of Manpower Training, Journal of the American Statistical Association, Vol. 84, No. 408, pp. 862-874.

Angrist J.D. and A.B. Krueger (1999), Empirical strategies in labor economics, in O.  Ashenfelter and D. Card (eds.), Handbook of Labor Economics, Vol. 3A, Amsterdam, North-Holland,  pp.1277-1366.

Rettore E., U. Trivellato e A. Martini (2003), La valutazione delle politiche del lavoro in presenza di selezione: migliorare la teoria, i metodi o i dati?, Politica Economica, n.3, 2003, 301-342.

Becker S.O., Ichino A. (2002), Estimation of average treatment effects based on propensity scores, The Stata Journal, 2(4), 358-377.

Dehejia R.H., Wahba S. (1999), Causal effects in nonexperimental studies: reevaluating the evaluation of training programs, Journal of the America Statistical Association, 94, 1053-1062

Dehejia R.H., Wahba S. (2002), Propensity score matching methods for nonexperimental casual studies, The Review of Economics and Statistics, 84(1), 151-161.

Card and Krueger (1994), Minumum wages and employment: a case study of the fast food industry in New Jersey and Pennsylvania, American Economic Review, 84, 4

D. Bondonio (2000), Statistical methods to evaluate geographically-targeted economic development programs, Statistica applicata, vol. 12, n. 2, pp. 177-204)

T. Lemieux, K. Milligan (2008), Incentive effects of social assistance: A regression discontinuity approach, Journal of Econometrics, 142, pp. 807–828

Teaching methods

- Teacher lectures

- Readings on case studies which will be discussed at the class sessions

- Computer sessions on application of statistical techniques 

Assessment methods

Student learning will be assesed by an oral examination, to be held at the end of the course,  in order to check:
- the knowledge of the main issues in designing the evaluation of a policy (program);
- the knowledge of  the approaches in evaluating  the implementation of a policy and the main issues to focus;
- the knowledge and the uptake of the approaches and the main techniques for impact evaluation;
- the ability to choose between alternative methods for impact assessment  depending on program's characteristics and available data.
Oral examination will assess also the student's familiarity with the formal language of the counterfactual approach and the statistical modelling.

Teaching tools

- Slides 

The slides are available on the web at http://campus.unibo.it/ . Slides can be dowloded by students who subscribe to the mailing list of the course. In order to subscribe to the mailing list, students need a password that will be notified by the teacher at the beginning of the course. Students who do not attend classes can ask the teacher for password  by e-mail.

- Statistical software for the application of estimation techniques in the computer lab.

Office hours

See the website of Pinuccia Pasqualina Calia