Yiqing Xu - How Much Should We Trust Estimates from Instrumental Variable Designs in Political Science? Practical Advice Based on Sixty-three Replications

Mon, Mar 8 2021, 11:30am - 12:30pm

Instrumental variable (IV) strategies are commonly used in political science to establish causal relationships, yet the identifying assumptions required by an IV design are demanding, and it remains challenging for researchers to correctly apply the method. We replicate 63 papers published in three top journals in political science during the past decade and find several troubling patterns: (1) researchers often miscalculate the first-stage F statistics, thus overestimating the strength of their IVs; (2) most researchers rely on classical asymptotic standard errors, which often severely underestimate the uncertainties around the two-stage-least-squared (2SLS) estimates; (3) in the majority of the replicated studies, the 2SLS estimates are much bigger than the ordinary-least-squared estimates, and their ratio is negatively correlated with the strength of the IVs when the IVs are non-experimental, suggesting potential violation of the exclusion restriction; such a relationship does not exist when the IVs are produced by experiments. To improve practice, we provide a checklist for researchers to avoid these pitfalls and recommend a local-to-zero test to guard against failure of the identifying assumptions.



Dr. Xu's primary research covers political methodology, Chinese politics, and their intersection. He received a PhD in Political Science from Massachusetts Institute of Technology (2016), an MA in Economics from China Center for Economic Research at Peking University (2010) and a BA in Economics (2007) from Fudan University.

His work has appeared in American Political Science Review, American Journal of Political Science, Journal of Politics, Political Analysis, Political Science Research and Methods, among other peer-reviewed journals. He has won several professional awards, including the best article award from American Journal of Political Science in 2016 and the Miller Prize for the best work appearing in Political Analysis in 2017