Averaging Non-Probability Online Surveys to Avoid Maximal Estimation Error (im Erscheinen)

Abstract

"Data from online non-probability samples are often analyzed as if they were based on a simple random sample drawn from the general population. As the exact sampling frame for these non-probability samples are usually unknown, there is no general method to construct unbiased estimators. This raises the question of whether estimates based on online non-probability samples are consistent across sample vendors and concerning estimates based on probability samples. To address this question, we analyze data collected from 8 different online non-probability sample vendors and one online probability-based sample. We find that estimates from the different non-probability samples can be very inconsistent. We suggest averaging estimates across multiple vendor samples to avoid the risk of a maximum estimation error. We evaluate several averaging approaches, including a LASSO regression procedure which identifies a subset of vendors that, when averaged, produce estimates that are more consistent with the reference probability-based estimates, compared to any single vendor. Our results show that estimates based on different vendors’ samples display different selection biases, but there is also some commonality among some vendor-specific estimates, thus there could be strong gains in estimation precision by averaging across a selection of multiple non-probability sample vendors." (Author's abstract, IAB-Doku) ((en))

Cite article

Zins, S., Sakshaug, J., Murray-Watters, A. & Cornesse, C. (2025): Averaging Non-Probability Online Surveys to Avoid Maximal Estimation Error (im Erscheinen). In: Journal of Official Statistics, accepted on December 20, 2024.