Assessing and Improving Data Quality in Linked Panel Surveys - With a Focus on Panel and Linkage Consent (Bias)

Abstract

"The analytical potential of linked panel surveys is great. Nevertheless, rising costs, declining response and linkage consent rates threaten their data quality. They frequently rely on respondent´s informed consent for further panel participation and linkage which is only provided by a (selective) subgroup. Failure to obtain panel and linkage consent decreases their analytical potential and affects their data quality. This dissertation examines the data quality of linked panel surveys, focusing on panel consent (bias) and linkage consent (bias) (RQ1) and strategies for improvement (RQ2). It introduces a new conceptual framework for assessing and improving the data quality of linked panel surveys, focusing on panel and linkage consent, compatible with the Total Survey Error paradigm. The framework models crucial data-generating processes that affect data quality in the first panel wave, namely nonresponse bias, panel consent bias, linkage consent bias, and validity. The framework is flexible as it covers the option of improving data quality through repeated requests for panel and linkage consent, and can therefore be applied to unlinked panels or linked cross-sectional surveys. This cumulative dissertation consists of four research articles and is located within survey methodology. The first article examines mode effects on linkage consent rates and linkage consent bias. The analyses are based on the „Legitimation of Inequality over the Life-Span“ (LINOS) panel that has an experimental design that randomly allocated respondents to a face-to-face sample or a web/mail sample. The self-administered modes yielded a much lower linkage consent rate, as the mode effect of 40 percentage points indicates. To assess linkage consent bias, the consent indicator was linked to an administrative database. Reassuringly, the results showed that, on average, linkage consent biases were relatively small for several administrative variables, regardless of mode. This supports the growing practice of replacing expensive interviewer-administered modes with less expensive self-administered modes. The second article consequently continues the investigation of mode effects, but with respect to panel consent rates and bias, and examines how repeated panel consent requests can improve these outcomes. The analyses are also based on the LINOS panel. The findings revealed that repeated consent requests in the form of follow-up phone calls and postcards can significantly improve panel consent rates (by around 26 percentage points) in the self-administered modes, thus reducing the gap in panel consent rates compared to the interviewer-administered mode by almost half. The panel consent bias analysis showed that, although the biases in the self-administered sample (compared to the face-to-face sample) still tended to be larger, the repeated panel consent requests helped to reduce these biases to a more tolerable level. A contribution to the TSE literature is the finding that panel consent bias was a smaller source of bias than nonresponse bias, and that sometimes these biases offset each other. The third article reviews literature on linkage and panel consent, differentiating between the survey design aspects of requests´ wording (arguments and framings) and placement (within and between waves) as well as repeated requests (within and between waves). It introduces Goal-Framing Theory (GFT) as theoretical framework for deriving hypotheses about the effectiveness of linkage arguments to survey methodology. Analyses are based on the recruitment wave of the panel survey „Quality of Life and Social Participation“ that has an innovative survey design optimized to maximize panel and linkage consent. It is the first panel covering repeated requests for both consents. Repeated requests reduced the number of non-consenters by 57 % for panel consent and by 54 % for linkage consent. The linkage wording is randomly varied in an experiment to test hypotheses from GFT that are largely supported. E.g. linkage consent rates are higher at the first request when applying the argument of time savings (hedonic goal frame) rather than referring to an improved meaningfulness of study results (normative goal frame). Linkage consent rates were (29 percentage points) higher when using different arguments between repeated requests during the interview, as opposed to repeating the same argument as before. The results on the determinants of (changing) linkage and panel consent due to repeated requests show a negative effect of privacy concerns and a positive effect of intrinsic motivation for both types of consent and a positive effect of extrinsic motivation for linkage consent. The article closes with recommendations of how to improve linkage and panel consent rates. The fourth article shifts the focus to assessing and improving the validity of linked panel surveys, particularly the data quality for substantive constructs. Using order-related justice attitudes (ORJA) as an example, it addresses the shortcomings of previous measures. Using three different datasets, the article introduces and validates the Basic Social Justice Orientations (BSJO) scale, which provides a more comprehensive measurement instrument of ORJA. This scale is compatible with psychological, sociological justice, and welfare state research and captures individual preferences for the four basic justice principles: equality, need, equity, and entitlement. In conclusion, this dissertation has developed a conceptual framework for maximizing panel and linkage consent rates, focusing on the effectiveness of repeated consent requests within a panel's recruitment wave. The results show that these repeated requests can reduce the number of non-consenters by half, making them a valuable survey design feature, particularly to counteract the consequences of both decreasing response and linkage consent rates, and in the context of spreading web surveys. Repeated requests therefore represent a notable advance in the survey methodology toolkit in terms of improving linkage and panel consent rates, potentially minimizing respective biases, improving data quality and enhancing the research potential of a linked panels." (Author's abstract, IAB-Doku) ((en))

Cite article

Hülle, S. (2024): Assessing and Improving Data Quality in Linked Panel Surveys - With a Focus on Panel and Linkage Consent (Bias). Bielefeld, 197 p. DOI:10.4119/unibi/2992206