The first time I had a private conversation with a politician (Republican state rep), he told me that the voting public is fickle. We don't live in a direct democracy and for good reasons. The core idea behind a Republic, in my opinion, is for people who know law and law-making to advance their constituents' interests at the assembly or any political office. The popularity of a decision among the electorate does not necessarily reflect the public's interests. The elected representative will make that determination. Sometimes the public will hate that decision and will vote the representative out of office only to change their mind a few years later. Any public representative who bases his decisions on the popularity of issues is not doing his job right. This has little to do with ideology. Otherwise, we would just let public opinion polls run the country. Indeed, there were many Democrats who cast their votes on the healthcare bill knowing that it would lead to their demise in their particular districts.
While there is a point to be made about the knee-jerk nature of the electorate, I think in this particular case they have actually been quite consistent. Most Americans just don’t want socialized healthcare. They rejected it in the 90s, and they did it again this year. Both private and universal systems have their flaws, and I think the Democrats just need to come to terms with the reality that their vision for healthcare is not shared by the rest of the country. The most practical agenda for any administration would be to push for common sense reforms like children’s coverage and acceptance of pre-existing conditions, and leave it at that.
As for the public not really knowing what’s good for them, this is something that each party only seems to believe when their own policies are involved. They rarely give this benefit of the doubt to the other side. Don’t forget that advocates of the Bush wars frequently invoke long term benefits that “the public doesn’t see yet.” That didn’t stop the left from opposing them from day one.
I also think we should not let hindsight bias entirely affect our judgment of a decision made for its time. The voters were right when they gave the already Democratic US Congress a mandate in 2008 and the Democrats delivered the goods, however half-assedly. Perhaps voters didn't like too much change at once and broke for GOP this time. And perhaps, when the majority of the changes actually go into effect the voters will like come to appreciate them.
We also should not ignore the urgent political reality of keeping your base happy. There is nothing worse than a disaffected base. In other words, if Democrats sat on their asses and did nothing, they could have lost a 100 seats even in the bluest of blue states. Healthcare may not have been a priority for the "fickle" independent voter who swung for GOP this time, but it was for the base of the Democratic party and it may have been enough not to lose the Senate. Remember, Democrats gained the majority not in 2008 but in 2006 and they had very little to show for the previous term in which they were in majority except an unpopular TARP program. The thinking at the time was, we have had our corporate welfare, what do people get? They got a healthcare bill that ended up mostly being an insurance giveaway (with a few measly bones thrown towards Democrats such as removal of pre-existing conditions) but in politics you celebrate anything as an achievement. I think your discussion with Mehdi actually proves this point in particular regarding Obama: Healthcare was an issue that he inevitably had to tackle and could not afford to let go. Healthcare was not an urgent matter for Obama personally. In fact, that is true if one looks at his campaign. Obama did not have a concrete healthcare plan even after he was elected. It changed every day. But he was forced into adopting the issue because of base pressures and he took the issue on very unenthusiastically.
Sitting and doing nothing would have certainly hurt the dems, but pushing the wrong agenda at the wrong time turned out to be equally damaging. The mandate given to them in 2008 was centered on the economy and wasn't a carte blanc to push just anything on their agenda, just as the mandate given to republicans now has little to do with their positions on stem cell or gay marriage, and they would be stupid to spend any considerable time on those.
Moving to the center at the expense of the base doesn’t necessarily translate to defeat. Clinton looked like a one-termer until he did exactly that. There is a reason why candidates pander to the base in the primaries but then move to the center to win the election. Because the core bases of these parties are too small to win on their own. They’re nowhere close to a majority. Also, for every disaffected ideologue that chooses to stay home on election day, there are plenty of partisans on both sides who will still vote the party line regardless of the circumstances. Obama may have had to address his party’s fixation on healthcare at some point, but this was absolutely the wrong time. Even if he’d pushed head on with it, the energized left would have still been outnumbered by the enraged, reawakened right (conservatives do have bigger numbers). And then he would have independents to worry about. Those two groups together would have neutralized any partisan goodwill.
I also think that keeping the base happy, while important, shouldn’t come at the expense of alienating or exploiting the rest of the electorate.
As an independent, I certainly wouldn’t want a scenario where my vote is gained with false promises planned to be abandoned after election day to please a small but loud minority. Not to say that this doesn’t happen already, but I wouldn’t want it to become an acceptable part of the process.