In this update, I'm going to focus on the polling we've seen in one particular state, Michigan, where there is an exceptionally wide spread between firms showing relatively strong numbers for Barack Obama, and those which have good numbers for Mitt Romney instead.
But first let me mention a poll that is a bit easier to interpret. The Washington Post and ABC News released their latest national poll on Monday, and it had favorable numbers for Mr. Romney, showing him one point ahead of Barack Obama among registered voters.
Past Washington Post/ABC News polls have often shown a pretty tight race, so this poll doesn't necessarily speak to a dramatic change in the state of play. But sometimes a more basic point gets lost when we get as detail-oriented about the polls as we sometimes do at FiveThirtyEight.
The Post/ABC poll is a very good survey, with strong methodology and a strong track record. The point is simply that it's good for a candidate when a strong polling firm shows him doing well. In any election, there are going to be polling firms that publish relatively stronger and relatively weaker numbers for a candidate; this is one of those surveys that you'd very much like to have on your side. And so far this cycle, Mr. Romney has tended to get the better results from the Post/ABC poll.
The Mysterious Polling in Michigan
There are also surveys that you don't necessarily want on your side, however, and some states seem to have a concentration of them. One of them is Michigan, where since mid-July we've seen polls showing everything from a 14-point lead for Mr. Obama in Michigan to a 4-point lead for Mr. Romney. That is a remarkably wide spread - too wide to be caused by statistical variance alone. Someone is getting this race wrong.
Our forecast model tends to side with the polls showing Mr. Obama in a reasonably good position in Michigan. It's extremely doubtful that he's ahead by double-digits there - and almost certainly not by 14 points - but the forecast projects a six- or seven-point victory for Mr. Obama in Michigan on Nov. 6. Let me explain how it comes to that conclusion.
In the table below, I've broken down the Michigan polls into two categories: those conducted by firms that frequently survey states outside of Michigan, and those that poll Michigan only, or Michigan and one other state.
The national polling firms, like Rasmussen Reports, Public Policy Polling, and Marist College, show Mr. Obama ahead by an average of seven points in Michigan. By contrast, the Michigan-only pollsters show him ahead by only two points.
The advantage of the national pollsters is that we have a good sense for how to interpret their results. Some are Democratic-leaning and some are Republican-leaning - but whatever “house effect†they tend to show, it is usually fairly consistent from state to state.
By contrast, when a polling firm takes surveys in one state only, and its results diverge substantially from the consensus, it's harder to tell what is going on. It could be that there is something about Michigan in particular they are seeing. Or it could be something more intrinsic to the methodology: that if the firm shows good numbers for Mr. Romney in Michigan, for example, it would also do so in Colorado, or Virginia, or Ohio, if only they surveyed those states.
The model is designed to mostly prefer the latter interpretation. If a local polling firm diverges substantially from the consensus, it infers that the results show a strong “house effect,†rather than providing any particular insight about the state. It doesn't entirely dismiss the poll, but it treats it skeptically.
But isn't local knowledge a good thing? It can be, and there are some outstanding state polls conducted by local newspapers or local universities, which are rewarded accordingly in our pollster ratings . However, there are also some very questionable ones.
In Michigan, there are two local polling firms in particular that have shown poor numbers for Mr. Obama. One is Mitchell Research, which just published a poll showing a tie there. The other is Foster McCollum White Baydoun, which had Mr. Romney ahead by four points in its last survey.
Foster McCollum White Baydoun actually does survey one state apart from Michigan: it released a poll of Florida last week. In that survey, it showed Mr. Romney ahead by almost 15 points - perhaps the biggest outlier we've seen in any individual poll so far this year.
As I mentioned when that Florida poll came out, this firm anticipates a big decline in participation among groups like younger voters that are ordinarily inclined to support Democratic candidates. The initial news release of the Florida poll appeared to show that the firm anticipated that just 11 percent of Florida voters would be under age 50 - in contrast to about half the Florida electorate in 2008, according to exit polls. A principal of the firm, Eric Foster, later wrote me to say that the news release had been incomplete, and voters under the age of 50 had actually been weighted to make up 27 percent of the poll instead.
That's still an implausibly large drop-off, in my view, from 2008. It may be because the Foster McCollum White Baydoun likely-voter model looks for voters who, as they described it, participated in “odd year municipal and county elections†- where turnout is tiny as compared with what it will be on Nov. 6. In any event, this method seems likely to produce extremely Republican-leaning results - whether applied to Michigan, to Florida or to any other state.
At least Foster McCollum White Baydoun have a methodology of some kind, even if it isn't one that I would recommend. What Mitchell Research is doing in Michigan is a little more troubling.
The head of Mitchell Research, Steve Mitchell , wrote a long memo accompanying his poll release on Monday. In that poll, he weighted the survey to assume that African-Americans would make up only 8 percent of Michigan's turnout. By contrast, black voters represented 12 percent of the turnout in Michigan in 2008 according to exit polls, and 14 percent according to another source, the Current Population Survey. Blacks also made up 13 percent of Michigan's vote in 2004 and 11 percent in 2000, according to exit polls. African-American participation is sometimes lower in midterm election years, but blacks were 12 percent of Michigan's electorate in 2006, the exit poll reported that year. (There was no exit polling in Michigan in 2010.)
So why only 8 percent this year? Mr. Mitchell wrote that he simply doesn't believe the exit polls:
“African-American participation in this poll is 8%, not 12%, which is the percentage of the population but not likely voters. I do not believe blacks represented 12% of the vote in 2008 and I don't believe they will in 2012. Having polled this state for 26 years, blacks have represented about 7%-8% of all votes cast in every statewide race. At best, it went to 9% four years ago. It will not reach that level this year.â€
What is the evidence for Mr. Mitchell's claim? He didn't present any of it in the memo. The exit polls certainly aren't perfect, but they've been consistent from year to year and also seem to agree with Census Bureau data. (Another fact that Mr. Mitchell cites - that African-Americans represent 12 percent of the overall population in Michigan - is also slightly incorrect; they made up 14 percent of Michigan's population as of 2011, according to the Census Bureau.)
Perhaps Mr. Mitchell means that only about 8 percent of respondents are black when he takes his surveys. But response rates to political surveys have been much lower among African-Americans for many years. The problems may be worse in a su rvey, like Mr. Mitchell's, that does not call cellphone voters, since African-Americans are more likely to rely on cellphones in lieu of landline phones.
It is precisely because of this problem that essentially all polling firms do weight by race, usually calibrating their numbers based on Census Bureau or exit poll data. Mr. Mitchell, however, seems to have his own ideas about what Michigan's electorate will look like. There is a certain amount of art in political polling, but I've never heard of a pollster treating the demographic makeup of a state as essentially a matter of opinion.
Mr. Mitchell's demographic weights account for much of the difference between his polls and those of the other firms that are active in Michigan; a decrease in the African-American vote from 12 percent to 8 percent would harm Mr. Obama's numbers by a net of about four percentage points there. If you have some reason to discount the Mitchell Research and Foster McCollum White Baydou n polls, it is much harder to make the case that the presidential and Senate races in Michigan are a “tossup,†as some other Web sites have characterized it.
To be clear, our forecast model doesn't “know†all of this detail about the Mitchell Research and Foster McCollum White Baydoun polls. It discounts the results from these firms simply because that's what it's programmed to do whenever it encounters a local polling firm whose results diverge substantially from the consensus; this method works to Mr. Obama's benefit in some states, and Mr. Romney's in others.
But my personal view is that the model is making the right inference in the case of Michigan, and that the methodological choices that these polls have made are hard to defend. Perhaps 90 or 95 percent of the time, taking a simple average of the polls will work just about as well as the more complicated FiveThirtyEight method. But this is rare instance where taking all the polls at face value may be a mistake, and the additional checks-and-balances the FiveThirtyEight method applies are worth the trouble.
No comments:
Post a Comment