Explore by…

More

YSI

​Replication and Transparency in Economic Research

Jan 5–6, 2016 Download .ics

San Francisco | Mozilla, 2 Harrison St

Join Richard Ball, Johannes Pfeifer, Edward Miguel, Jan H. Höffler, and Thomas Herndon for a workshop after the AEA meetings in San Francisco.

The workshop will take place right after the January 3-5, 2016 Annual Meeting of the Allied Social Science Associations (ASSA), which includes the Annual Meeting of the American Economic Association (AEA) and academic job market candidate interviews. Experiencing this conference and job market can be a valuable experience for those who are considering applying for an academic job after their studies. During the conference there will be a special session on Replication in Economics, also organized by organizers of this post-conference workshop described here.

The workshop will consist of mini-courses covering research transparency, a topic that is often neglected in the conventional economics curriculum, with an emphasis on empirical research and macro models. For young scholars it can be very useful to orient themselves by looking at how established researchers do their studies. Though economics has recently seen an increase in replication materials made publicly available, it is often frustrating to actually replicate analyses since work is poorly documented or poorly organized, or necessary data and code are missing. This workshop intends to help young scholars find out how to replicate others’ studies and how to archive their own research for future use.

The workshop also will feature student presentation sessions, which will give Ph.D. candidates the opportunity to present and discuss their research in a collaborative environment. Moreover, during the joined lunch and dinner there will be ample time for social interaction with students and teachers

Course descriptions:

Integrity in Empirical Research

Richard Ball, Haverford College, Project Teaching Integrity in Empirical Research (TIER)

This course will be useful to participants in both the pedagogical and research aspects of their careers. For instructors of courses on econometrics and related topics, as well as supervisors of student research, the TIER Protocol provides a well-structured way of incorporating principles and methods of transparency and replicability in the training of graduate and undergraduate students. For professional researchers, the Protocol serves as a model of conventions for constructing replication documentation to accompany empirical work.

Participants will be encouraged to bring with them data and code they are using in their current teaching and research so that they can begin applying the TIER Protocol in their own work during the course.

Readings:

Transparency in Macro models

Johannes Pfeifer, University of Mannheim

The replication of macroeconomic models poses its own unique challenges for researchers due to the analytical intractability of even simple DSGE models. The course will discuss how the transparency issues arising in macroeconomic research differ from the ones typically encountered in regression-type empirical analysis and what this means for researchers. Just getting the model solution and simulating the model often requires considerable programming effort. At the same time, the adaptation of standardized software packages by researchers has been slow, partially due to the fear of using “black boxes”. As the complexity of DSGE models in use has massively increased in recent years, the potential for introducing programming errors has also increased. The course will discuss some recent important papers where such bugs have been detected and how the academic community has dealt with the fallout. We will then talk about the lessons different stakeholders like researchers, students, referees, and journal editors can take away from this. Johannes Pfeifer will also talk about his own experience with detecting replication issues in research and publishing about it.

The second part of the course will describe attempts to increase the transparency and replicability of DSGE modelling. A particular focus will be on the Macroeconomic Model Data Base ( http://www.macromodelbase.com/), a collection of important replicated DSGE models that focuses on model comparison, as well as Dynare (www.dynare.org), an open-source software package based on MATLAB/Octave, which the Macroeconomic Model Data Base relies on for solving and simulating the models. A particular focus will be on the lessons that can be learned from these projects, including software engineering aspects.

Readings:

Further presentations:

The Use of Pre-Analysis Plans in Economics Edward Miguel, Berkeley Institute for Transparency in the Social Sciences (BITSS)

Readings:

An Inroduction to ReplicationWiki, a Database of Published Replications and Replication Materials, Jan H. Höffler, University of Göttingen

Readings:

Lessons on the Process of Replication from the 2013 Public Debt and Growth Debate, Thomas Herndon, University of Massachusetts, Amherst

Readings:

Young Scholars Presentations

Select participants will be given the opportunity to present their replication research, be it a completed dissertation chapter or work in progress, on any topic. Presenting is voluntary. Applicants shall enter the title and abstract in the registration form and submit the complete version to the Institute no later than December 1st, 2015. Presenters will be assigned as respondents to other students’ papers.

Travel and Accommodation

The participation fee is $20. Drinks and snacks will be provided throughout the workshops. Participants are responsible for arranging their own travel and accommodation.

The AEA provides information about hotels in San Francisco and the opportunity to book a room on the conference website. The conference fee for students is only $25 but the accommodation starts at $115 per room and night and one has to book quickly after the booking option is opened. Also for cheaper non-conference accommodation it is advisable to book way in advance because for the job market many candidates will come and demand will be high - not so much for the workshop time when the conference itself is already over.

Workshop participants seeking roommates should indicate this on the registration form and will be matched with fellow participants.

Registration

Participants should be enrolled in a PhD program in economics or a related field. Postdocs, assistant professors, and qualified masters students are also welcome.

Complete the web form to register. The number of participants is capped at 30 per course. It is recommended that you register sooner rather than later.

Questions? Email Jan Hoeffler at Jan-H.Hoeffler@wiwi.uni-goettingen.de (YSI Event Coordinator)

Workshop Partners and Sponsors

BITSS ( http://www.bitss.org/)

Mozilla Science ( https://www.mozillascience.org/)

Project TIER ( https://www.haverford.edu/tier)

Program (preliminary)

Wed, January 6th

9:00 – 10:30

Mini Course
Integrity in Empirical Research
Richard Ball, Haverford College

10:30 – 11:00

Coffee Break

11:00 – 12:30

Mini Course
Replication of Macro Models
Johannes Pfeifer, University of Mannheim

12:30 – 14:00

Lunch

14:00 - 15:00

Pre-analysis Plans and their Relationship with Replication
Edward Miguel, Berkeley Institute for Transparency in the Social Sciences (BITSS)

15:00 – 16:00

Young Scholars Presentations (1):

“Top-coded Earnings”, Zhiqui Zhao

“Nitrogen Management under Uncertainty: An investigation of Subjective Decision Processes”, Sandip Agarwal

16:30 – 17:30

ReplicationWiki
Jan H. Höffler, University of Göttingen

17:30 - 18:30

Young Scholars Presentations (2):

“Transparency issues with the FA model and learning dynamics of Boz and Mendoza (2014)”, Eddie Gerba

“Analyzing the Anomalies in the Choice of Crop Insurance by US Farmers”, Sulagna Sarkar

Thu, January 7th

9:00 – 10:30

Mini Courses (continued)

10:30 – 11:00

Coffee Break

11:00 – 12:30

Mini Courses (continued)

12:30 – 14:00

Lunch

14:00 - 15:00

Lessons on the Process of Replication from the 2013 Public Debt and Growth Debate

Thomas Herndon, University of Massachusetts, Amherst

15:00 – 16:00

Young Scholars Presentations (3):

“Gulf of Mexico Bidding Database Codebook”, Ross Manley

“Economics of Radical Uncertainty and Cybersecurity”, Dr. Suchitra Abel

16:30 – 17:30

Young Scholars Presentations (4) and Open Discussion:

“Do Social Security Recipients Smooth Consumption Between Checks? Evidence Using New Data With Variation in Pay Dates”, Nathanael Vellekoop

Open Discussion

18:30

Cocktail Reception

Register here

Partners and sponsors

Event Partners