President Obama's chances of winning the Electoral College were 64.8 percent as of Tuesday's FiveThirtyEight forecast, down slightly from 66.0 percent on Monday.
The forecast will not yet reflect any effects from Tuesday night's debate in New York, any impact of which will require a few days to become apparent in the forecast.
Tuesday featured an interesting set of surveys, however. While Mr. Obama's numbers were middling on the whole, one set of them implied that the polls may be inclined to overstate the effect of events like the party conventions and the debates.
Mr. Obama's poorest set of numbers came from a pair of national polls. A weekly survey conducted by Public Policy Polling for the blog Daily Kos had Mitt Romney ahead by four points among likely voters. So did the Gallup national tracking poll, which had Mr. Romney hitting 50 percent of the vote for the first time.
It would be terrific news for M r. Romney if he were consistently at 50 percent in the polls. That threshold would ensure that he could win (or at least tie) the popular vote, even without picking up any additional support from undecided voters.
But other national polls published on Tuesday were not in agreement with the Gallup and Public Policy Polling numbers. Rather, three of the six national polls published on Tuesday had Mr. Obama leading the race. The same three polls also had Mr. Obama improving his numbers from the previous edition of the same survey, while the other three had him declining.
This is a curious distribution of polls. While on average the polls showed almost an exact tie in the race, none of the individual polls did so, each lining up on the side of Mr. Obama or Mr. Romney.
What to make of this pattern? Not much, I don't think. On Monday, the national polls did seem to form a nice, tight distribution, clustered around the mean. On Tuesday, they didn't. If the differences persist, they will be worthy of our attention, but the strong suspect here is simply statistical noise. People spend far too much time trying to determine which polls are right, when the correct attitude is to endeavor to infer which polls are less wrong.
Every poll has an irreducible degree of error introduced by statistical sampling methods (this is what is measured by the margin of error reported along with the poll). Most polls also introduce some additional error on top of that because of methodological shortcuts or ambiguities.
Ideally, we would focus on those surveys that have only sampling error, and not any error introduced by methodological flaws. To so me extent, by weighting the polls differently based on their past performance and their methodological standards, the FiveThirtyEight model seeks to do that.
But even the best polls will give an inexact estimate of the condition of the race. Many polls are better than one, but six polls aren't all that many, and will inevitably produce odd patterns from time to time.
There was another set of polls released on Tuesday, however, that diverged from the consensus in a significant way - and in a way that almost certainly can't be explained by statistical variance alone.
It was a set of 25 state polls conducted by the online firm YouGov. The polls showed Mr. Obama with a lead in most battleground states.
Perhaps more important, they showed little decline for Mr. Obama from the previous set of YouGov surveys in September, which were conducted just after the party conventions. On average, Mr. Romney gained only about half a percentage point in the surveys, in contrast to other polls that have given him an average of about a four-point bounce from the Denver debate.
The YouGov polls were also different in another respect. Instead of contacting a new group of voters, they resurveyed the same ones that they did in their September set of polls. They were able to reach about 80 percent of these voters, and found that very few of them had changed their minds.
This recontacting procedure is uncommon in survey research, but not unprecedented (The New York Times and CBS News have used the technique occasionally in their polls). It has advantages and disadvantages.
Let's start with the potential downsides. First, to the extent that the original sample was off the mark because of random variance, the new one will be also. If the 800 voters you reached in Ohio or Virginia originally happened to be more Republican- or Democratic-leaning than the broader group of voters in those states, you'll reproduce the flaw when you sur vey them again.
Second, there is the possibility that the voters could be affected by the act of taking the survey multiple times. Either they might be reluctant to change their answers out of concern about appearing inconsistent, or the act of taking the poll could make them more attentive to following political events.
There are also two potential advantages, however. First, the technique potentially reduces statistical error for measuring the trend in voter opinion. Perhaps the original sample was off the mark, but at least you're measuring changes in voter sentiment against a consistent baseline rather than a moving one: Is there evidence that individual voters are in fact changing their minds? Particularly for methods like the FiveThirtyEight forecast model, which relies heavily on tracking the changes over time within individual groups of surveys, this is potentially helpful.
Next, the technique potentially reduces what is known as nonresponse bias. E ven the best surveys these days only manage to get about 10 percent of people on the phone, while the shoddy ones might struggle to get 3 or 5 percent of voters to return their calls. These percentages have fallen precipitously over the past two decades.
Polling firms are hoping that the 10 percent of people that they do reach are representative of the 90 percent that they don't, but who will nevertheless vote. But there are no guarantees of this, and it is really something of a leap of faith. The willingness to respond to surveys may depend in part on the enthusiasm that voters have about the election on any given day.
Aren't adults who are more enthusiastic about politics also more likely to vote? Sure, but the polls have other ways of handling that problem. The platonic ideal is to reach a random sample of all American adults, and then to apply a likely voter method to eliminate those who are unregistered or unlikely to vote. If you have a sample that is biase d toward enthusiastic respondents to begin with, and then apply a likely voter screen, it risks double-counting the enthusiasm factor, especially in cases like presidential general elections when overall turnout is quite high.
The YouGov polls, which contacted the same group of respondents, are potentially suggestive that some of Mr. Romney's gains since the Denver debate reflect the increased willingness of those who might be inclined to vote for him to participate in surveys, rather than very many Americans having changed their minds.
This does not necessarily reflect any characteristic of Mr. Romney, Republican-leaning voters or the Denver debate in particular, however. It is potentially a problem whenever there is a news event that could affect voter enthusiasm and their willingness to respond to surveys.
One reason that Mr. Romney may have received such a large bounce following the Denver debate, for example, is because it came right on the heels of a strong run of news for Mr. Obama - particularly the Democratic convention, and the release of the “47 percent†tape.
Perhaps if YouGov had also contacted this same group of voters in August in advance of the party conventions, they would also have shown a relatively small bounce for Mr. Obama in September.
Stated more simply: Perhaps Mr. Obama's numbers went from being artificially inflated to artificially deflated, exaggerating the degree of change in the race on both ends.
It continues to be noteworthy, in my view, that in slow news cycles - as in most of the spring and summer months - the polls have generally converged to show an overall advantage for Mr. Obama of about two percentage points. After his best news cycles, like after the Democratic convention, Mr. Obama pulled ahead in the polls by four to five points, while Mr. Romney remained about tied with Mr. Obama after his best series of events. But some of these effects could be artificial, as a result of nonresponse bias.
Perhaps the New York debate - viewed as a modest win for Mr. Obama by instant reaction polls - will reset the news cycle to a neutral enough point that the potential effects of nonresponse bias will be reduced, giving us a test of YouGov's hypothesis.
On the other hand, it's possible that Mr. Obama's “win†in the debate will seem more definitive in the coming days, and that he'll get a bigger bounce in the polls. If so, there will be some reason to be suspicious of it.
No comments:
Post a Comment