Implementierung eines Web-basierten Datenerhebungsmodus in einem bestehenden Panel der Allgemeinbevölkerung: Eine Analyse der Selektions- & Messeffekte

Abstract

"This report investigates selection and measurement effects of the first websurvey of the Panel Study Labour Market and Social Security (PASS). The weighting of the websurvey is based selection effect models presented here. This report can therefore be viewed as a detailed documentation of the weighting process. The investigation of measurement effects revaels for which question comparability between the interviewer-administered modes of PASS and the PASS-websurvey is limited. Web surveys are becoming increasingly popular. However, in addition to the advantages such as faster availability of data and lower costs, there are also disadvantages. For example, web surveys can often reach fewer people than modes with interviewers (lower response rates) and certain groups (older, low educated) are underrepresented (nonresponse bias). Respondents must have a certain level of Internet affinity and cannot be motivated to participate through interviewers. Therefore, the web mode is often used not as a substitute but as a supplement to existing modes. The aim here is to ensure that the selection effects of the modes used balance each other out. At the same time, there is a risk that the different modes lead to measurement differences. In this paper, the selection effects of introducing a web mode into an existing panel, the Labor Market and Social Security Panel (PASS), are investigated. In addition, the possibility of compensating for existing nonresponse bias by means of weighting is assessed. It is examined whether the comparatively new method of random forest outperforms the classically used method of logistic regression. Finally, the measurement errors between the web mode and the existing modes in PASS (CAPI and CATI) are compared. Specifically, item nonresponse, acquiescence, recency, and socially desirable responding are addressed. We find that the web mode systematically excludes marginalized groups. In particular, economically disadvantaged people, people with a migration background as well as people with a low level of education are underrepresented in the web survey. Overall, compensating for this bias by weighting works better with the logistic regression method than with the random forest method. In some cases, large differences are found within the measurement errors of the different modes, although these do not always go in the expected direction." (Author's abstract, IAB-Doku) ((en))

Cite article

Mühlbacher, V. & Trappmann, M. (2024): Implementierung eines Web-basierten Datenerhebungsmodus in einem bestehenden Panel der Allgemeinbevölkerung: Eine Analyse der Selektions- & Messeffekte. (FDZ-Methodenreport 01/2024 (de)), Nürnberg, 74 p. DOI:10.5164/IAB.FDZM.2401.de.v1