Supporters raised their hands New Year's Day when Barrack Obama asked if they would take part in the Iowa Democratic caucuses. (AP photo)

By Barry Sussman
bsussman@niemanwatchdog.org

The Des Moines Register’s poll of likely Democratic caucus-goers published Monday — widely regarded as the single best pre-caucus measure — exhibits many of the worst characteristics of election-year polling.

It could serve news organizations that do polls as a “how not to” guide for the rest of the 2008 campaign.

The poll focused only on the horse race and on vague bumper-sticker voter motivations, like desire for change or unity. That’s a disservice.

In a 33-paragraph story and in sidebar charts, the number-one issue for many Democrats, Iraq, was never mentioned. Not once. This seems standard procedure for the Register; earlier on its editor banned discussion of Iraq in televised Democratic and Republican candidate debates she moderated.

Did the poll questionnaire even try to determine what bearing Iraq or any other issues might have on people’s choices? Not as far as I can tell — but I can’t rule that out, either, as there was next to no transparency in the paper’s report.

The poll had Barack Obama with 32 percent of likely caucus-goers, Hillary Clinton with 25 percent and John Edwards with 24 percent.

Eight hundred people were surveyed by telephone Dec. 27 to 30. A striking assertion was that 40 percent of the Democratic caucus-goers will be Independents, another 5 percent Republicans and that 60 percent of the 800 have never attended a caucus before.

Perhaps these unlikely-sounding predictions will be correct; there’s a lot of interest in the Democrats this year. But the Register should have explained its turnout conclusions. How did it determine who’s likely to show up and who isn’t? In years past, the Gallup poll had a nine-question filter to determine likely voters. What’s the Register’s filter? The story doesn’t say or even give a hint.

Another problem is the difficulty of polling during the Christmas-New Year’s holiday period; fewer people than usual are available for interviews, and there’s no easy way of knowing if those who are available have somewhat different candidate preferences.

In addition, the poll report goes too far in analyzing candidate preferences among subgroups. The overall margin of sampling error for a survey of 800 people is about 3.5 percentage points, as the Register story states. But for smaller groups, such as women under the age of 35, or between 35 and 54, or 55 and older, sampling error is substantially higher. But that didn’t stop the Register from stating, without noting the higher error margin, the preferences of each group.

Polls can tell us a lot about the electorate — but not when they are so fixed on the horse race to the exclusion of everything else.

Bandied about over the years is the question of whether pre-election polls change voter behavior. If they do, it seems certain that the change would be only slight — but slight differences could result in different outcomes. For that reason alone, news organizations should be extremely cautious in their polling.

Two of the first items to appear on this Web site in 2004 were primers on election polls, one by Barry Sussman, the other by the late Leo Bogart.

Barry Sussman is the editor of the Nieman Watchdog Project. E-mail: bsussman@niemanwatchdog.org

Comments

Submit a comment - Jump to bottom of list

How not to critique a poll Posted by Perry Washburn - ex-editor 01/04/2008, 12:33 AM Mr. Sussman: I was suprised to see the errors and lack of understanding exhibited in your commentary on the Des Moines Register's most recent Iowa Poll. You say that "Iraq, was never mentioned. Not once." Just plain wrong. It's there, both online and in print. You say there was "next to no transparency" about the poll. Given that you couldn't find the two "influential issues" charts, perhaps you missed the 21-paragraph piece explaining the poll. Again, it's both in print and online. You need to understand - a little research easily uncovers this - that this Iowa Poll is just the latest in a series. Issues have been prominent in others (for a meaty one, check November's, it's online). Read those fascinating poll results, and you will see the reason Register editor Carolyn Washburn (full disclosure: my wife) did not ask questions about Iraq or immigration. From Andrew Romano in this week's Newsweek: "Carolyn Washburn, editor of The Des Moines Register, launched her newspaper's Republican debate by saying, 'We're going to focus on issues Iowans say they still want to know more about.' When the roster didn't include Iraq or immigration, the pundits pounced. 'The worst debate in Western history,' said Charles Krauthammer. And yet Washburn delivered: taxes, education, global warming and trade. There were no snowmen, no slapfests. The 1,440/7 media emerged empty-handed. Iowans emerged informed. Who did a better job? We report. You decide." I find your style of drive-by criticism, full of "flip" and containing goofy errors, a "disservice." Your errors leave me wondering which of your remaining points have merit. My J-professors might have said back in the day: "Do a little reporting next time." Perry Washburn ex-editor Des Moines

Sussman responds Posted by Barry Sussman - Editor, Watchdog Project 01/04/2008, 01:22 PM Mr. Washburn is correct in several of his important criticisms. Although I looked for it online before I wrote, I didn’t see the sidebar describing the methodology and questions – but it was there. >http://www.desmoinesregister.com/apps/pbcs.dll/article?AID=/20071231/NEWS09/71231041/-1/iowapoll07< Neither did I see the “influential issues” chart. It’s fair for him to say I should do a little reporting next time. He also could have mentioned that the poll was highly accurate in key points: Obama won going away and while the survey got the second and third place order wrong, it was off by only a miniscule amount. It was correct in anticipating a “dramatic influx of first-time caucusgoers,” a main factor. I feel strongly that omitting Iraq as an issue in the Register’s poll story and in candidate debates were disservices. Iraq is the elephant in the room. The chart I missed showed Iraq as the No. 1 issue. A simple crosstab would have helped explain whether it had any bearing on how those interviewed were choosing candidates. Sadly, the Register isn’t alone in ignoring Iraq – almost all the coverage I saw focused on how, for Iowans, change trumped experience. Change from what? It’s during election campaigns that Americans are most attentive to national issues and problems. The press needs to deal with important issues, not put them aside or bury them. As for poll reviews, I’ll try to do better next time. --Barry Sussman Submit a comment - Jump to top of list