99651 - VALUTAZIONE DI IMPATTO DELLE POLITICHE

Learning outcomes

The course aims to provide in-depth knowledge about the main strategies aimed at evaluating the impact of public policies, deepening both theoretical and methodological aspects. At the end of the course, the student is able: a) to understand the logic of the main impact evaluation designs; b) to use evaluation results to inform policy making; c) to identify the most suitable evaluation designs for identifying causal effects of policies in different practical situations; d) to critically evaluate existing studies and discuss the validity of the results, the limitations, and their transferability to other contexts.

Course contents

The course will have a specific focus on experiments in the social sciences.

The lectures are organised as follow:

Lecture 1: Course introduction and description of the objectives of impact evaluation.

Lectures 2-5: Systematic approach; process evaluation, introduction to the causal inference and to the potential outcomes language.

Lectures 6-9: Experimental research in social sciences: RCT, field experiment, survey experiment and vignette studies.

Lectures 10-12: Identification strategies in observational studies: propensity score matching, regression discontinuity design, difference-in-differences.

***********************************************

The lecturer reserves the right to modify the readings list before the beginning of the course.

Lecture 13: The evaluation of educational policies (part 1): compulsory schooling

Barbetta, G. P., Canino, P., & Cima, S. (2021). Let’s tweet again? Social networks and literature achievement in high school students. Education Finance and Policy, 1-64.

De Poli, S., Vergolini, L., & Zanini, N. (2014). L'impatto dei programmi di studio all'estero. Evidenze da un disegno sperimentale. Scuola democratica, (3), 549-576.

Lecuture 14: The evaluation of educational policies (part 2): tertiary education

Martini, A., Azzolini, D., Romano, B., & Vergolini, L. (2021). Increasing College Going by Incentivizing Savings: Evidence from a Randomized Controlled Trial in Italy. Journal of Policy Analysis and Management, 40(3), 814-840.

Bettinger, E. P., & Evans, B. J. (2019). College guidance for all: A randomized experiment in pre‐college advising. Journal of Policy Analysis and Management, 38(3), 579-599.

Lecture 15:The evaluation of anti-poverty policies

Kangas, O., Jauhiainen, S., Simanainen, M., & Ylikännö, M. (2019). The basic income experiment 2017–2018 in Finland: Preliminary results. Reports and Memorandums of the Ministry of Social Affairs and Health.

Haushofer, J., & Shapiro, J. (2016). The short-term impact of unconditional cash transfers to the poor: experimental evidence from Kenya. The Quarterly Journal of Economics, 131(4), 1973-2042.

Lecture 16: Education and labour market via vignette studies

Di Stasio, V. (2014). Education as a signal of trainability: Results from a vignette study with Italian employers. European Sociological Review, 30(6), 796-809.

Di Stasio, V., & Van de Werfhorst, H. G. (2016). Why does education matter to employers in different institutional contexts? A vignette study in England and the Netherlands. Social Forces, 95(1), 77-106.

Lecture 17: Gender inequalities analysed via survey experiment

Auspurg, K., Hinz, T., & Sauer, C. (2017). Why should women get less? Evidence on the gender pay gap from multifactorial survey experiments. American Sociological Review, 82(1), 179-210.

Carlsson, M., Finseraas, H., Midtbøen, A. H., & Rafnsdóttir, G. L. (2021). Gender bias in academic recruitment? Evidence from a survey experiment in the Nordic region. European Sociological Review, 37(3), 399-410.

Lecture 18: The evaluation of policies concerning ethnic inequalities

Carlana, M., La Ferrara, E., & Pinotti, P. (2022). Goals and gaps: Educational careers of immigrant children. Econometrica, 90(1), 1-29.

Abascal, M., Huang, T. J., & Tran, V. C. (2021). Intervening in anti-immigrant sentiments: The causal effects of factual information on attitudes toward immigration. The ANNALS of the American Academy of Political and Social Science, 697(1), 174-191.

***********************************************

Lecture 18-20: Class work aimed at designing an experiment in the social sciences.

Readings/Bibliography

Attending students: 

Martini, A., & Sisti, M. (2009). Valutare il successo delle politiche pubbliche. Bologna: Il Mulino.

Mutz, D. C. (2011). Population-based survey experiments. Princeton: Princeton University Press.

Papers listed in the section "Course contents" available on virtual.

Optional readings:

Angrist, J. D., & Pischke, J. S. (2014). Mastering'metrics: The path from cause to effect. Princeton: Princeton university press.

De Blasio, G., Nicita, A., & Pammolli F. (a cura di) (2021). Evidence-based Policy! Ovvero perché politiche pubbliche basate sull'evidenza empirica rendono migliore l’Italia. Bologna: Il Mulino.

***

From the fifth lecture, the roll call will be taken.

***

Non-attending students:

Textbook:

Martini, A., & Sisti, M. (2009). Valutare il successo delle politiche pubbliche. Bologna: Il Mulino.

Mutz, D. C. (2011). Population-based survey experiments. Princeton: Princeton University Press.

Additional readings:

De Blasio, G., Nicita, A., & Pammolli F. (a cura di) (2021). Evidence-based Policy! Ovvero perché politiche pubbliche basate sull'evidenza empirica rendono migliore l’Italia. Bologna: Il Mulino.

Papers on the effectiveness of financial aid in Higher Education:

Martini, A., Azzolini, D., Romano, B., & Vergolini, L. (2021). Increasing College Going by Incentivizing Savings: Evidence from a Randomized Controlled Trial in Italy. Journal of Policy Analysis and Management, 40(3), 814-840.

Modena, F., Rettore, E., & Tanzi, G. M. (2020). The effect of grants on university dropout rates: Evidence from the Italian case. Journal of Human Capital, 14(3), 343-370.

Facchini, M., Triventi, M., & Vergolini, L. (2021). Do grants improve the outcomes of university students in a challenging context? Evidence from a matching approach. Higher Education, 81(5), 917-934.

Vergolini, L., & Zanini, N. (2015). Away, but not too far from home. The effects of financial aid on university enrolment decisions. Economics of Education Review, 49, 91-109.

Teaching methods

With at least 20 "attending students", the course will be organised in lectures and seminars, according to the following programme. The first section consists of 8 lectures (16 hours) and introduce students to the acquisition of basic conceptual and theoretical tools. The second section is organised in seminar mode (12 hours) with the aim of favouring occasions for in-depth discussions of class materials and exercises.

Students will be divided into two groups for the seminar section. The classroom commitment will therefore be 28 hours. Students are required to read the assigned material and - in the case of the seminars - will be required to actively participate through presentations of research papers and case studies.

Should the number of attendees be less than 20, a seminar-style approach will be maintained in order to discuss in class certain evaluation studies deemed particularly relevant.

Assessment methods

The final grade for the attending students will be calculated as follow:

• Presentation and class participation: 50%.
• Design and presentation of the experimental project: 50%.
• Discretionary oral examination on the topics covered during the course: +/- points.

In order to obtatin the status of attending students you must attend at least the 80% of the classes in both parts. This means 9 lessons in the first part and 6 in the second one.

For non-attending students, the exam will be based on an oral exams on the reading listed in the bibliography.

Please note that exam modes may be subject to change according to the health emergency.

Candidates who pass the exam can refuse the final grade (thus requesting to re-take the exam) only once, in accordance with the university’s teaching regulations (art. 16, comma 5).

After having rejected a passing mark, any other subsequent passing mark will be recorded definitively in candidates’ transcripts.

Teaching tools

Teaching material uploaded on virtual.

Office hours

See the website of Loris Vergolini