Four years can be a long time in politics, and one midterm cycle may not resemble the one immediately preceding it. For a pollster, political scientist, or basement-dwelling blogger, there are tools and practices that can be put to use to assist the gut in making predictions about electoral outcomes. The ones we are most familiar with are efforts to discern patterns in the electorate’s behavior by looking at the broadest relevant data set that can be obtained. Yet, from a scientific point of view, we hold so few elections and our elections (e.g. general vs. midterm vs. local) are so distinct from each other that we never really have a very robust data set. Even with the data we have, there are events that change the nature of the country and that make old election data suspect when used in conjunction with relatively recent election data. Big events like the Vietnam War, Watergate, the Reagan and Gingrich revolutions, the end of the Cold War, 9/11, the second Iraq War, Hurricane Katrina, and the Great Recession combine with slow but persistent demographic change and internal migration, as well as changing election law to change both the composition of the electorate and how it views the world.
We know for example that in the four years since the 2010 midterms some states have undergone some significant changes or experienced some unusual events. California has been suffering from an historic drought. North Dakota has been experiencing a massive oil and gas boom. North Carolina has been governed by a radical Republican Party for the first time since the 19th Century. Pennsylvania has had close to the slowest job growth of any state in the country. Colorado and Washington created a market for the legal sale of marijuana. Georgia has seen a big reverse-migration of African Americans. And many states now have gay marriage which did not in 2010.
Most of these types of developments are not typically considered or captured in the models we make to predict elections, and are usually relegated to the gut if they are considered by analysts at all.
How, after all, could one scientifically predict the political impact of marijuana legalization or gay marriage?
What tends to happen is that we poll people and use the results as our baseline. Then we try to explain those polls. But that’s a problem. It’s a problem right now because the rule is that midterm Senate election polls are biased in one direction or another, usually by at least two points. If you take a look at the Real Clear Politics aggregation of polls, four of the top ten Senate races show a lead of less than two percent and the same is true of eight of the top ten gubernatorial races. If history is a reliable guide, these polls are off by a little bit in one direction or another, meaning that there is a skew in favor of either the Democrats or the Republicans. This is irrespective of the unique or particular factors at play in each state and race.
And those factors can be decisive. It’s going to be harder for Bruce Braley to exceed expectations in Iowa because the incumbent Republican governor at the top of the ticket is going to win by a large margin. It’s going to be harder for Thom Tillis to win in North Carolina because he was the Speaker of the House in a very unpopular state legislature. The hope is that these features of campaigns are somehow captured in the polls, but they may not be, or they may be captured only partially.
Over this weekend, I am going to look at a few Senate races and I am going to try to go a little beyond the polls to make predictions about who will win and why.
Any election poll that does not specify the margin of error and does not take that figure seriously is quackery and likely an effort to propagandize a self-fulfilling prophecy for those folks who want to vote the winner. Sample size and the number of polls taken for a particular race cause the aggregate margin of error to be narrower (but it can be wider if the central tendency jumps around between samples), which is what pollsters are hoping as their aggregations approach election day. Variation among samples is an important statistic in itself if there are systematic and not random arguments for why the variance is occurring.
Correlation of one race with others on the ballot, correlation of candidate support with nearness to the candidate’s home, and other correlations that can be measured in the population statistics after the election become elements of the secret sauce of opinion poll analysts. The data does not exist prior to election to determine some of those effects and might not be statistically significant before the election to permit estimating the effect.
There are hidden assumptions in political opinion polling. One assumes that each sample is an independent trial, despite campaign media attempts to make them dependent by feeding poll results back to the public. One assumes that an intention to vote translates uniformly over the sample into the actual action of voting and that the vote is as stated in the interview for the poll. You can see from this where differential effectiveness in turning out the votes through organization can mark a departure from the opinion polls. One assumes that a personal willingness to answer for one’s voting intentions is independent of what they are, that enough representative Republicans and Democrats can be found to make a valid sample for the poll.
The fact that pollster know that those assumptions are not correct is why they create devices like voting screens to estimate how incorrect and devise strategies for making the poll more accurate in predicting the election outcome (if that is their intention for creating the screen, that is).
So an analysis of the polling should, but rarely does, analysis of the possible systematic contributors that might cause the race to come outside the margin of error. However, those might also be hidden variation captured statistically within the margin of error. So some idea how much a governor’s race can pull a US Senate race with a coattail effect is a necessary understanding, which might not have been studied for exactly the situation of the race being analyzed. Or the extent to which the poor actions of the legislative body pull down the support for individual legislators. And the extent to which those are dismissed bipartisanly as all politicians are corrupt.
The opinion polls that precede and election are not the reality. The election itself should be and is the “big reveal”. Unless there has been systematic tampering with the results after voters have voted. Statistics then are used to bring suspicion and sometimes charges of fraud, but they are not determinative themselves of the existence of fraud. That determination requires documentation of the actions and behavior in the elections system that show clear evidence of non-standard and auditable missteps.
This is going to be very important this year as Jodi Ernst’s campaign seems to be planning on winning a US Senate seat with Norm Coleman-style post-election doggedness.
MOE is a simple calculation and all of the published polls include it and most at the 95% confidence level. It’s a statistical fudge factor for not having gotten a sample that is correctly representative of the whole population. Larger sample sizes and more replications mathematically reduces the odds that the sample isn’t representative of the whole. However, that says nothing about how the poll was constructed and conducted which can remain biased and that information is less likely to be disclosed. Pew polls generally provide such information. The early September Loras poll on the Iowa Senate race did as well (revealing that it was a high quality poll).
A problem is that readers don’t seem to understand what it means. For example (and I see this very often), if the poll is 47 to 45 with an MOE of 3 and the writers support the candidate at 45, they’ll say that the race is tied within the MOE because 45 plus 3 is 48 and that beats 47. When the poll only means that there’s a 95% chance (another fudge factor) that one candidate is somewhere between 44% and 50% and the other is somewhere between 42% and 48%.
I made so much money at Intrade just betting on what Sam Wang and Drew Linzer put at the 95% confidence level.
Well, that and betting that the planet will get warmer. That’s a no-brainer though.
Wonder what the payoff would have been for me if I’d placed a wager 1/08 on Obama to win in the GE and 1/11 to do it again. Also thought GOP wins in 2002 and 2004 and DEM wins in 2006 and 2008 were no-brainers.
Exactly. And error works upwards and downwards, which is also overlooked in hasty analysis.
But that looks only at the randomness that results from sample size based on deductively worked through mathematical logic.
Errors from improper screens, failure to get representative samples as a result of differential telephone access and political saliency of polling calls, and other things add to the potential errors one way or the other. But one of the operating assumptions is that some of these are randomly canceled out by errors in the direction of the other outcome.
In your example, the election could come down the other candidate winning 48% to 44% of the vote based on opinion. The other 8% randomly marked their ballots, either through indecision, last minute misgivings about the candidate they were going to vote for, or just an impulsive mood, or other random event. Get out the vote campaigns work to reduce those sort of random events by strengthening clarity and commitment. GOTV also increases name awareness of who the challenging candidate or new candidate is and what they stand for. That works to increase sentiment voting and can show up in polling, but late-stage GOTV does not show up in polling. Which is why the last weekend push of GOTV even if a candidate is polling strongly makes a lot of sense. (The GOP activated this in their 2004 campaigns and likely denied John Kerry the election in other states than Ohio as a result. The goings on by Kenneth Blackwell never were adequately examined to prove that there were no shenanigans.)
Errors from improper screens, …
There will always be errors in how the poll is constructed and administered — but even crappy pollsters are a bit more sophisticated than you seem to assume.
“Undecideds” don’t break randomly — there are predictable patterns within a certain range. In depth polling interviews can detect which way it is more likely to fall, but those polls are expensive to conduct. So, we just get more cheap polling relying on external validation and regression to the mean.
It’s interesting to note that as polling models have become more sophisticated, getting representative sample populations has become more difficult. The more stable period was after 1948 up until 2000. When almost everybody had a landline and answered their phones and USians moved less frequently, but that hasn’t been overlooked by pollsters that have a vested interest in getting it correct.
Watch the trendlines — that’s where MO for Nunn and Carter can be seen.
I would think that undecideds break in different directions in different races and for different but locally explainable causes. I am sure the each pollster has its historical database and rules of thumb for deciding how undecideds are going break and put that into the written analysis instead of the marquee numbers.
Another thing that makes getting representative samples more difficult is cynicism about (1) unsolicited phone calls in general and (2) political polling in an age of surveillance and hyperpartisanship in particular.
Trend lines are the best thing about longitudinal polling of hot races. But most Congressional Districts do good to have a poll at all. A lot of House models are predicting from state and national sentiment and precinct PVIs. And it shows.
My position is that this election has been made so close so many places in the polls that overall it is difficult to predict. But for GOTV, that’s an ideal canvassing situation in which the most effective work is rewarded. And rewarding GOTV is what builds volunteers the next time around.
Don’t need an increased level of cynicism as an explanation of possible less representative samples. Fewer landlines and fewer people answering their phones is a more concrete, robust, explanation.
In some elections and in some locations, a strong ground GOTV operation will make a difference if the opposition doesn’t match it, the race is authentically close, and it increases the number of LV. (Don’t both parties always cite their better ground game when they win?) Whether needed or not, Begich has built a strong ground game in the more remote parts of AK. Weiland started late on this, but may be catching up. CO Democrats are also claiming that this is their strength.
A candidate that excites voters enough that they want to vote for him/her is a more powerful inducement. I’m thinking of 2000 when black voters showed up in droves in FL and PA and stood in line for hours (because the precincts weren’t prepared for that level of participation) to cast their vote for Gore. Pathetic that this country didn’t demand that all those FL ballots be counted.
Oh how I miss Intrade…
The folks who accurately predicted trends based on people’s ability to squander money on betting on the election? How was that a random sample?
It works well enough when money sways elections. I don’t think it will do as well when there is been more systematic get-out-the-vote activities.
The psychology of the market is the people act on their preferences no matter how illogical. And that the preferences of people with money to waste on Intrade skew in a certain direction.
I miss it because I made money at it. Fact is, there’s plenty of research that show that prediction markets are exceptionally good at predicting elections.
Exceptionally good is not good enough. I doubt they could predict an election that was close to totally GOTV unless everyone in the market was involved in the GOTV activity. Presidential elections are much easier for prediction markets. Congressional composition based on multiple subsidiary elections less so. A market like that depends on crowdsourcing a variety of aggregative analyst opinions. Given the sophistication of aggregation techniques like Nate Silver’s, national Congressional elections can probably be predicted in a market. State legislature composition or aggregate party power in states less so.
Glad you made money on Intrade. To what extent was that random or based on good information?
It’s all about getting the good info. While the right was unskewing, I was looking at the Demographics, and seeing that a lot of the polling was missing cell phones and minorities who happen to use cell phones more. But even in 2010 and 2006 Intrade did very well. The incentives to get it right make smart people get the right price by jumping on deals from those who bet with their hearts (confirmation bias, long-shot bias, etc)…
So, for example, anyone following the GOTV efforts in, say, NC in 2008, or CO in 2012, or even, for a long shot, FL in 2012, would have made good money. I was buying shares of Obama to win FL for about 3 bucks (pays 10). That was risky, but the price was right and the GOTV efforts down there really paid off.
But Intrade was pretty accurate, basically like Sam Wang and Drew Linzer (except that Linzer and I called FL 2012 right, while Intrade missed it).
The great thing about it was that smart people, who know how this stuff works, are taking money from idiots. That’s why I love it so much. Redistributing some of their ill-gotten cash and Bush tax cuts.
After the 2012 election, Intrade announced that ONE account had lost $7 million on Willard Romney. It was raining money in there, and I even sent in another check so I could get some of it.
I like to think it was Trump.
I think people are talking about Marujuana out of state more than they are in Colorado, though I can’t say how they are discussing it in right wing circles. I haven’t heard any ads about it that I can remember. I wonder how many voters are even thinking about pot.