When it comes to midterm elections prognostication, it’s helpful to look at demographic trends and polling data, but, at least when it comes to the House of Representatives, you need to focus on the make-up of the districts. Growth of the minority population, higher minority turnout, and less support from minorities for Republican candidates would be unwelcome developments for the GOP, but those factors would also be unlikely to change the outcome in more than a small handful of House elections. That’s because most Republican-held districts have relatively few minorities and the whites are reliably Republican. If the Republicans are really in jeopardy of losing control of the House, it’s because they’re losing the support of people who they used to be able to depend upon for support.

That’s where their handling of the budget, economy, retirement programs, health care, farming, and even transportation, could become a problem. Historically, no organization has been more closely aligned with the Republican Party than the national Chamber of Commerce, but that has to be about to change considering that the modern GOP simply refuses to do its bidding anymore. The Chamber wants immigration reform, but it also wants more transportation spending and opposes a government shutdown or imposition of the debt limit.

The agricultural industry is another source of strength for the GOP, especially considering that the Republicans dominate in rural areas. But failing to produce a Farm Bill is enraging farmers.

I’ve already discussed recent Democracy Corps polling that showed severe erosion of the Republican advantage with elderly voters. This is probably primarily a result of lasting damage done by the voucher plan for Medicare that Paul Ryan pushed over the last several years. But the GOP’s general attitude about entitlements (that we’re not entitled to the benefits we paid for during our working lives) is always going to be a drag on their performance with the elderly. If it’s getting worse, some safe districts are going to become competitive.

There are other factors that could be turning off long-time Republicans. One is the way Republican governors have behaved. Most of them are very unpopular. Pushing voter suppression schemes and blocking a decent fix for college loans turns off young voters. Extreme anti-abortion legislation is alienating many right-leaning women. Attacks on school budgets are worrying countless parents. And there are plenty of people who mildly disagree with the president on most things but acknowledge that he was reelected and think he deserves an opposition that will compromise.

Beyond that, as the culture changes to be more tolerant about things like gay marriage and a pathway to citizenship, and climate change become more of an undeniable reality, the culture wars become a less effective wedge.

Finally, one of the Republicans’ traditional areas of strength has been their message discipline: lower taxes, less regulation, a strong military, etc. But the GOP is no longer united on national defense, taxes and budgetary issues, or even on immigration reform and gay rights. As a result, they will not be able to convey a clear concise message effectively, as they have in the past.

All of these things are factors that could combine to do enough damage in districts that Romney carried easily that a Democrat might be able to sneak out a win in November 2014.

We saw something similar happen in 2006. President Bush was such an epic fuck-up, that a lot of Republicans just left the party, many for good. It could happen again.