Methodological Experiments to Inform Adaptive Survey Designs in Longitudinal Research

This presentation reports on recent experimental work aimed at improving survey performance with respect to the joint optimization of response rates, nonresponse bias, and fieldwork costs in a long-running large-scale household panel. Drawing on the main study of the Socio-Economic Panel (SOEP), including the IAB-SOEP Migration Samples and the IAB-BAMF-SOEP Refugee Survey, I discuss four field experiments designed to identify leverage points for more efficient and less biased data collection.
First, I present two incentive experiments: (1) a study testing the effectiveness of prepaid incentives in combination with post-paid incentives in a panel study, and (2) an experiment evaluating unconditional pre-paid and “early-bird” incentives in a mature panel context. Second, I summarize preliminary findings from two multimode experiments. One assigns respondents at random to one of three interviewing modes: video interviewing (CALVI), interviewer-administered face-to-face (CAPI), or web self-completion (CAWI). The other experiment randomly transitions 50% of households from CAPI to a CAWI-with-CAPI-follow-up design, while the remaining 50% continue in their familiar CAPI mode.

Using evidence from these studies, I outline the derivation of an adaptive mode-assignment strategy for household panels - an approach that tailors data-collection modes to household characteristics and response propensities with the goal of stabilizing data quality while reducing costs. I conclude by highlighting methodological pitfalls encountered in the experiments and by outlining next steps toward fully operational adaptive survey designs.

This presentation draws partly on joint work with colleagues from the SOEP team, researchers at the Federal Institute for Population Research, and GESIS.

Date

Venue

Registration