Laura Stoker - Response Propensity Modeling: Applications for Experimental and Non-experimental Survey Research
Response Propensity (RP) models are used by some commercial survey research firms to generate weights for non-probability samples. RP models work with data from two samples, one unlikely to be representative of the population of interest (e.g., MTurk workers, opt-in Web panelists), and one of high quality on that dimension (e.g., CPS, ANES). This talk will first describe the RP approach and distinguish it from other methods of devising sample weights, emphasizing its simplicity, low cost, and adaptability for individual researchers. I will then illustrate the utility of RP modeling in two contexts, one experimental and one non-experimental: (1) RP can be useful when survey researchers face questions about the generalization of experimental results found using a convenience sample, or about why findings fail to replicate across studies. Here I work with data from a series of experiments replicated using MTurk and population-based samples, analyzed by Mullinix et al. in “The Generalizability of Survey Experiments” (JEPS 2015). (2) RP can also be used to devise weights when none exist for one’s sample or to improve upon the “one size fits all” weights that studies like the ANES and GSS provide. I illustrate its potential with analyses using the ANES 2012 and 2016 Internet and Face-to-Face samples.
Laura Stoker is Professor of the Graduate School in the Department of Political Science at the University of California, Berkeley. Her research focuses on the development and change of political attitudes and behavior with a focus on family influences and generational change. She also writes on topics at the intersection of research design and statistics, including the optimal design of multi-level studies, problems of aggregation, and the estimation of cohort effects. She has regularly taught undergraduate and graduate courses on political psychology and research methods. Her publications have appeared in many venues including the American Political Science Review, American Journal of Political Science, British Journal of Political Science, Electoral Studies, and Journal of Politics. Stoker is the recipient of fellowships from the Center for Advanced Study in the Behavioral Sciences (CASBS), Oxford University, and the University of Manchester. Stoker has served on the Board of the American National Election Studies (2000-2002, 2018-present; Chair 2000-2002), the British National Election Studies (2014-2016), and the CASBS Causal Inference for Social Impact Lab (2019-present).