Just a reminder about Rasmussen Polls. They suck. Here’s a snippet from Nate Silver’s post-2010 midterms analysis:
The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research, missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points, including one in the Hawaii Senate race that missed the final margin between the candidates by 40 points, the largest error ever recorded in a general election in FiveThirtyEight’s database, which includes all polls conducted since 1998.
Moreover, Rasmussen’s polls were quite biased, overestimating the standing of the Republican candidate by almost 4 points on average. In just 12 cases, Rasmussen’s polls overestimated the margin for the Democrat by 3 or more points. But it did so for the Republican candidate in 55 cases — that is, in more than half of the polls that it issued.
If one focused solely on the final poll issued by Rasmussen Reports or Pulse Opinion Research in each state — rather than including all polls within the three-week interval — it would not have made much difference. Their average error would be 5.7 points rather than 5.8, and their average bias 3.8 points rather than 3.9.
Nor did it make much difference whether the polls were branded as Rasmussen Reports surveys, or instead, were commissioned for Fox News by its subsidiary Pulse Opinion Research. (Both sets of surveys used an essentially identical methodology.) Polls branded as Rasmussen Reports missed by an average of 5.9 points and had a 3.9 point bias. The polls it commissioned on behalf of Fox News had a 5.1 point error, and a 3.6 point bias.
I mention this because not a few right-wingers are hanging their hats on the fact that Rasmussen shows better results for Mitt Romney and other Republican office-seekers than the consensus of other polls. That appears to be the point of Rasmussen Polls, especially since most outfits are now using aggregates of all the polls that are released. Nate and I actually had a liquor-soaked conversation about Rasmussen’s shitty-polling during a party at the Netroots Nation conference in Pittsburgh (in 2009). Yet, he recently admitted that his current model allows Rasmussen to have quite a bit of influence.
…the Gallup and Rasmussen Reports polls can have a lot of influence on the forecast at times when there is a potential turning point in the race. The trendline adjustment that the model calculates compares changes in the results produced by the same polling firms over time. Since the Gallup and Rasmussen Reports national tracking polls have been published almost every day since the spring, they represent highly important data series in this process.
Nate does have a way to correct for consistent bias over time, but merely by being a Republican-supporting data point that plays an important role in his model, and in the other aggregators which mostly do not correct for such bias, Rasmussen’s overall effect is to skew the picture of the political landscape in a rightward direction.
In other words, if you’re reading analysis that takes Rasmussen seriously at all, you’re a dummy. Nate hopes that a consistently flawed polling methodology can nonetheless provide trend lines as long as the methodology remains consistent. In other words, even if Rasmussen is consistently showing results that are four points too optimistic for the Republican, if you account for that you can see when the contests are growing closer or farther apart.
I’d prefer it if their results were simply ignored and their outfit ostracized as bad-faith operators. It’s not that I don’t believe that Nate can utilize their data usefully, but he’s the only one who is making the correction for bias. I do not believe you have any such correction at Real Clear Politics or Talking Points Memo, although I will update and correct this post if I am wrong.
It seems to me that Rasmussen’s polls are designed to skew the perception of the state of play in the Republicans’ direction, which can help fundraising, press coverage, and morale among activists. It’s dishonest and dishonorable, and it is also effective.
People, including, Nate, should know better than to treat them as an honest data point.
He’s not treating them as an honest data point, he’s treating them as a biased one.
In polling, no data point is honest. But if you have a reliably biased one with a consistent history of bias, that seems to me to be a better data point than an ‘unbiased’ poll without a history.
He’s really treating them no differently than any other polling outfit. He measures their distance from the mean of other polls and adds a correction. He does that for PPP and Gallup and every other pollster.
But he assumes a consistency of methodological error will prevail, which isn’t a safe assumption when working with a bad-faith actor.
Yes, but even good-faith actors (if they exist) are subject to some kind of bias. Polling is not an exact science – it attempts to describe and predict trends in voting, which sure as hell isn’t an exact science due to all of the factors involved. I would argue that attempting to account for consistent errors inherent in polling is really the only way to take polling seriously.
I’m not complaining about the methodology Silver uses to account for bias. I’m saying that I don’t think Rasmussen’s systemic bias is a bug. It is, rather, a feature. And my treating them like all other pollers, he’s legitimizing them so that they can do their damage all across our political spectrum. I also don’t trust Rasmussen to use a consistent methodology.
It doesn’t matter whether it’s a bug or a feature, if it’s systematic instead of truly random it still reflects some underlying data. And that data is likely not fabricated from whole cloth (just because the cost of doing it and the difficulty of hiding it). And the bias can be corrected the same as other polls with systematic bias (but not a recognizeably explanatory cause).
Once a pollster sets out to influence the debate rather than report their best estimate, their methodology cannot be trusted to be consistent. They’ve already violated the code.
Fact:
Rasmussen’s polls were the most accurate of all polls in both the 2004 and 2008 Presidential elections, predicting all four candidates’ final vote percentages within .5 percent.
Nate Silver’s “model” showed a close race for control of the House in 2010…how did that work out?
No, he didn’t:
His predictions for the House were off by 10 (he had Dems at 203, and GOP at 232, end result was 193, 242).
At the very end of the campaign he did.
Have you looked at his September predictions in 2010? He gave the GOP only a 50/50 chance of taking the House.
It’s much easier to predict elections in November as the polls tend to converge.
Ya know, Nick, you’d be a better troll if you weren’t so rabidly insistent that you are always right.
Even the most cursory of examination of Nate’s analysis of Rasmussen would show that the house effect diminishes as the election approaches.
In words of less than 3 parts:
Raz uses bad samples. On purpose. A long time before the day of voting. So they cannot be checked for how right or wrong they are. As you get closer to the day of voting, the samples get better.
By the week before the voting is being done the samples used are about the same as all the other polls.
The term used by people who do this work to describe these actions is LYING.
Note: bold faced terms are words/phrases that replaced 3+ syllable words:
parts:Syllable
purpose:Deliberately
day of voting:Election
better: More Accurate
week before:penultimate week
being done:actually being performed
people who do this work:Statisticians
I hope this helps you to understand the depths of your deliberate ignorance.
Thank you and have a good day
Exactly
Who was more accurate in predicting the 2010 House vote? Rasmussen projected 64 seat GOP gain, while your precious Nate Silver projected a 45 seat GOP gain.
Also…what about Gallup? Are they dishonest also? Why dies Gallup mirror Rasmussen?
Zombie Lie:
Not to mention, that if you game to change media coverage and popular perceptions, your final poll should be your most accurate, because it’s too late for media reactions to matter and because you’ll actually be measured against a result (the real polls).
I already cited Nate on their dismal 2010 results. And their bias.
You have to go back to 2004 to find them nailing an election.
It’s true that Rasmussen’s national poll did well in 2004, but CBS/NY Times, CNN/USA Today/Gallup and Pew Research were all more accurate than Rasmussen’s final poll showing Bush ahead 50.2-48.5
http://poq.oxfordjournals.org/content/69/5/642/T1.expansion.html
In 2000, Rasmussen’s national and state polling was way off, rife with Republican bias, and in 2008, for the 31 states Rasmussen polled close to the election, his error rate was 3.68 and his Republican bias was 1.25. The bias doesn’t seem too bad until we find that it was dramatically reduced by a lot of strange polls in deep red states Obama couldn’t win, showing him performing much better than he actually did.
In the 20 states decided within 10 points of Obama’s popular vote margin, Rasmussen had an error rate of 3.73 and a Republican bias of 3.55, favoring McCain by 1+ in 17 of 20 states and Obama by 1+ in only 1 state. Rasmussen was off in favor of McCain by:
8.5 in Nevada
6.9 in Wisconsin
6.4 in Michigan
6.2 in Washington
5.1 in New Mexico
4.6 in Ohio
4.4 in Oregon
4.3 in Pennsylvania
4.3 in Maine
4.1 in Indiana
4 in Colorado
2.8 in Florida
2.6 in New Hampshire
2.3 in Virginia
1.8 in Montana
1.5 in Iowa
1.3 in North Carolina
Rasmussen off in favor of Obama by 1.7 in Minnesota.
Public Policy Polling, which Republicans love to bash, and which RCP didn’t even include in their 2008 averages, had an error rate of 2.21 and Republican bias of 0.24 for the 16 states it polled close to the election. Like most pollsters, they missed badly in Nevada (8.5 in favor of McCain), but they called 7 of 16 states to an accuracy of 1 point or less: http://postimage.org/image/a6c3du7ir/
And? That’s your best case?
PPP’s likely voter screen is focused on PAST elections!!
Not relevant!
In any estimate of turnout, the last similar election is the most important consideration.
So, 2010 turnout models will do a decent job of predicting 2014 turnout models, but a terrible job of predicting this year’s turnout. The best predictor for this year’s turnout is 2008, followed by 2004, followed by 2000.
That doesn’t mean that you just limit yourself to the last presidential election cycle, though. You have to consider measures of voter enthusiasm, differences in regional strength and/or ideological appeal, levels of new and changed party identification.
It was a reasonable assumption that black turnout would be higher in 2008 than in 2004, and it’s a safe bet that Mormon turnout will be higher in 2012 than in 2008. But you have to measure for that. You can’t make up some number.
If the Democrats have registered 80,000 more voters in Nevada than the Republicans, that should impact your turnout model.
Or you just do what Rasmussen does, which is create an unrealistically old and land-lined sample which assures a right-wing bias in their polls.
Okay, Progs, let’s say the RCP average reflects the current state of the race, and Obama is up 48 to 45. Let’s assume one percent goes to third parties…that leaves six percent undecided…
The majority of these six percent disapprove of Obama, and feel the country is on the wrong track…
What the hell makes you think they’ll pull the lever for Obama?
They just need to feel Romney is acceptable…the debates will do that, assuming Romney doesn’t blow it (crossing fingers!)
I think you should go with that and donate the max to Romney, since it will be money well spent (and, you know, he’s having a bit of a cash flow problem).
Keep in mind that it doesn’t really matter from an electoral perspective if Romney is huge in Alabama, or wherever, even if that skews the national numbers.
I’m sorry, Nick. I don’t think I can explain it in terms of one syllable or less. But maybe I can:
Here it is:
Thank you and have a nice day.
Nice condescension…but you did not address the point…
In any poll in which Obama receives less than 50 percent, why do you believe the “undecideds” will vote for him?
Why?
Worst “recovery” since Great Depression…
Median income down 11 percent…
Record levels of folks on Food Stamps…
Lowest labor force pArticipation in 30 years…
This anti business PResident is running this country into the ground, and the Independents know it…
They want the Marxist out of the White House!!!
Nate Silver has a ton of information about the swing states and who is winning there. They are, of course, the ones that matter. You won’t like what you see.
Nate Silver’s numbers give Obama nearly an 80% chance of victory. His “now-cast” shows over a 95% chance of Obama winning.
Do you really believe all three of Ohio, Florida, and Virginia are going Romney? Really??
Because if Obama wins just one of those states, it’s virtually impossible for Romney to win the election.
Rasmussen is one thing, but I really wish he’d address that Sam Wang (the Princeton guy) is basically calling him out every other post these days. If for no other reason than I want to see a massive meta-fight between pollsters across the internets.
No, Nate’s right on this point. If they are consistently off, they are a perfectly honest gauge of the election. Scientists and engineers apply instrument corrections all the time. (If a thermometer always shows that water freezes at -3C and boils at 97C, then it’s a fair bet that you can add 3C to any measurement you get from it).
The real concern is using polls that show Obama up 14 points one week and 2 points down the next. There’s simply no reason to believe that opinion will shift that much in such a short amount of time, and the poll is probably very imprecise. Even when averaged with other polls, it will skew the results in ways that can’t be predicted.