Confronting 2016 and 2020 polling limitations

As context, the Center conducts surveys using its online panel, the American Trends Panel (ATP). The ATP is recruited offline via random national sampling of residential addresses. Each survey is statistically adjusted to match national estimates for political party identification and registered voter status in addition to demographics and other benchmarks.2 The analysis in this report probes whether the ATP is in any way underrepresenting Republicans, either by recruiting too few into the panel or by losing Republicans at a higher rate. Among the key findings: Adults joining the ATP in recent years are less Republican than those joining in earlier years. The raw, unweighted share of new ATP recruits identifying as Republican or leaning Republican was 45% in 2015, 40% in the 2018 and 38% in the 2020. This trend could reflect real-world change in participation (i.e., Republicans are increasingly resistant to polling) or real-world change in party affiliation (i.e., that there is a decline in the share of the public identifying as Republican), but it might also reflect methodological changes over time in how the ATP is recruited. Switching from telephone-based recruitment to address-based recruitment in 2018 may have been a factor. Regardless of the cause(s), more weighting correction was needed in 2020 than 2014 (when the panel was created) to make sure that Republicans and Democrats were represented proportional to their estimated share of the population.3 Donald Trump voters were somewhat more likely than others to leave the panel (stop taking surveys) since 2016, though this is explained by their demographics. The overall retention rate of panelists on the ATP is quite high, as 78% of respondents in 2016 were still taking surveys in 2020. But a higher share of 2016 Trump voters (22%) than Hillary Clinton or third-party voters (19%) stopped participating in the ATP during the subsequent four years. The demographic make-up of 2016 Trump voters basically explains this difference. In analysis controlling for voters’ age, race and education level, presidential vote preference does not help predict whether later they decided to leave the panel.