4.6 - Evaluating Public Opinion Data
Analyzing Data and Polling Reliability
Being able to read polling data in charts, graphs, and infographics matters, along with judging whether a given poll is actually trustworthy.
Public Opinion as Political Influence
Public opinion is a source of political power, and polls are how candidates, officials, and the media measure it. That's why poll numbers can change what happens in an election or a policy debate.
- Elections: campaigns use polls to decide where to spend time and money and which issues to stress. Strong poll numbers help candidates raise money and get media coverage, and the parties have used poll numbers to decide who qualifies for primary debates. Weak numbers can push candidates out of the race early.
- Bandwagon effect: some voters drift toward the candidate who looks like the winner in the polls. Media coverage focused on who's ahead (horse race journalism, see 5.12) can make this stronger.
- Policy debates: officials are more likely to act when a large, steady majority supports something, and less likely to push a policy most voters oppose. A president with high approval ratings has more political capital to push an agenda through Congress, while low ratings make it harder. Both sides in a debate also cite polls to claim the public is on their side.
- How much it matters depends on the situation: public opinion has the most influence on high-profile issues that voters care a lot about, especially close to an election. On low-profile or technical issues, interest groups and experts often matter more than polls.
Changes in Polling Methods
Polling has shifted a lot, especially since 2016.
- Decline of live phone polling: fewer people answer calls from unknown numbers, and many households are cellphone-only.
- New methods: online opt-in panels, probability-based paneling, and texting have all grown, and most pollsters now combine multiple methods.
Reliability and Veracity
- Reliability: results are consistent and repeatable, meaning redoing the same poll right away should get similar results.
- Veracity: results are actually accurate, not just consistent. A poll's veracity shows up if it correctly predicts an election outcome.
- What makes a poll less credible: a non-random or opt-in sample, a small sample (bigger margin of error), loaded or confusing wording, a sponsor with a stake in the result (like a campaign or interest group), old data, or polling the wrong group (all adults instead of likely voters when predicting an election).
- Nonresponse bias: if certain kinds of people are less likely to answer polls, the sample misses them, even when the method looks random. Pollsters try to fix this by weighting the results to match the population.
- Example, 2016: national polls were fairly close (final averages had Clinton ahead by about 3 points, and she won the popular vote by about 2). But many state polls in Wisconsin, Michigan, and Pennsylvania underestimated Trump, partly because many didn't weight by education (voters without college degrees broke heavily for Trump) and because late-deciding voters went his way. Trump won all three states and the Electoral College. The state polls agreed with each other (reliable) but were off together (a veracity problem).
Big picture
Judging a poll means checking both reliability (consistent, repeatable results) and veracity (actual accuracy). As traditional phone polling has declined, pollsters have leaned more on texting and online panels to keep their methods current and their results trustworthy.