In Reviewing Survey Data, Details Matter

In recent days, we’ve posted a fair number of updates about Supreme Court nominee Brett Kavanaugh. We’ve noted that among those following news of his confirmation process, roughly 50% have a favorable opinion of him and 50% an unfavorable view.

Today, we released a poll measuring the favorability of all Supreme Court Justices plus Kavanaugh. In this survey, just 35% have a favorable view of the Judge and 33% an unfavorable view. The rest don’t know enough to form an opinion.

For those skeptical about polls, the apparent discrepancy between 50% favorable in one poll and 35% raises obvious questions.

The answers are to be found in the details of the survey. In this case, there were two major differences. One was the question wording. In the poll with the lower numbers, we just asked about “Brett Kavanaugh.” We didn’t mention any identifying information. In the survey with higher numbers, we identified him as “Supreme Court nominee Brett Kavanaugh.” That tied him more directly into the news cycle for people who weren’t quite sure who he was.

Secondly, in the poll with higher numbers, we only asked that question of people who were paying at least a little bit of attention to news of the confirmation process. This eliminated about 20% of voters who weren’t following the story at all. It is natural that those who follow the story are more likely to have an opinion about Kavanaugh.

Why did we do it that way? We began asking about the Kavanaugh confirmation process in August when few were paying attention (only 13% were paying close attention at the time). So, we wanted to gauge the process among those who might have some knowledge of it. We have continued with that practice to provide a consistent base of measurement.

It’s also important to note that while the specific numbers are different between the two approaches are different, the message is the same. Voters are pretty equally divided on Brett Kavanaugh.


Posted in Deeper Currents

Scott's Newsletter
Sign up for Scott's newsletter and get his political insight delivered right to your inbox!