A recent Twitter exchange vividly brought to mind a critical flaw in Iowa caucus polling.
Peter Daou recalled a national poll showing then-Sen. John Kerry (D-Mass.) so far behind in the 2004 race, he was trailing Al Sharpton. Addisu Demissie responded with this kind recollection: “I remember this WELL as well as the @MarkMellman poll briefing soon after that explained why we were going to win” Iowa and the nomination. At that point, we showed Kerry leading in the Iowa caucuses despite the fact that every public poll was putting him well behind.
The reason for our greater accuracy remains relevant in assessing Iowa caucus polls on both sides of the aisle today.
We were surveying the right people — those who would actually show up at the caucuses — while public polls were interviewing those who merely claimed they would participate.
Most polls that purport to provide Iowa caucus results simply ask each interviewee whether they are likely to attend a caucus. The tendency to say yes yields responses so highly inflated as to be worthless.
For example, the recent YouGov poll conducted for CBS implicitly assumes a Democratic caucus turnout of 750,000 and Republican turnout of about 800,000. In fact, the 2008 caucuses brought a record 240,000 to the Democratic event and 121,000 to that of the Republicans. In 2004, it was half that many for the Democrats and 119,000 for the Republicans. Thus, the YouGov poll overestimates caucus turnout by a whopping 300-700 percent.
However, YouGov is hardly alone. Almost all the public polls exhibit this same flaw.
Honestly, this would not be a problem if caucus attendees were pretty much like other voters. But they aren’t.
Unlike public pollsters, campaigns pay substantial sums to purchase lists of previous attendees and expend vast efforts assessing who the new participants might be. They then lavish millions on that segment — phone calls, mail, door knocks, invitations to meet the candidate, etc. — rendering these individuals an entirely different species from the ordinary Iowa registrant.
That’s one reason to take these polls with many grains of salt. Another related concern is their tremendous volatility.
At the equivalent of this point four years ago, it was Mitt Romney in first for the GOP nod, Rick Perry in second and Michele Bachmann holding down third place. Then, just weeks before those 2012 GOP caucuses, Newt Gingrich surged into the lead, garnering over 25 percent of the vote on average, followed at a distance by Ron Paul and Romney who were essentially tied.
On caucus night, it was Rick Santorum who came in first, followed very closely by Romney and then Paul. Former front-runner Gingrich could only muster 13 percent, while Perry barely broke into double digits and Bachmann didn’t.
Such volatility is not confined to the GOP. Leading up to 2004, nine polls through October showed Dick Gephardt winning the Democratic caucuses.
His widest margin was in a SurveyUSA poll that put him 13 points ahead of Howard Dean. Days later, a Des Moines Register poll had Dean 2 points ahead of Gephardt.
The Register poll had eventual winner John Kerry with just half as many votes as Dean, while ultimate second-place finisher John Edwards was tied for fourth with Joe Lieberman at 5 percent.
An American Research Group survey in 2008 gave Hillary Clinton a 9-point lead — she lost the Iowa caucuses to President Obama by 8 points.
Indeed, only half the leaders in early September polling went on to win the caucuses.
On average, Iowa caucus winners picked up 15 points between September and caucus night, with half adding more than 20 points to their totals.
With every gyration in the Iowa polls producing speculation about tectonic shifts in both parties’ contests, it’s worth noting that Iowa caucus polls generally — and certainly those in September, October and November — often offer no useful guidance in ferreting out the eventual result.
Mellman is president of The Mellman Group and has worked for Democratic candidates and causes since 1982. Current clients include the minority leader of the Senate and the Democratic whip in the House.