Virtually veryone has seen the historic 1948 photograph of Harry S. Truman grasping the front page of the Chicago Tribune that erroneously declared in large type “Dewey Beats Truman.” The paper wanted to be the first to publish and made the mistake of believing polls.After that polling embarrassment, there were recriminations and self-examination. Despite the fact that Truman won by two million votes, the story in the Electoral College was much closer. The switch of 30,000 votes, distributed the right way would have given an Electoral College victory to Dewey. Nonetheless, there was no excuse on the part of pollsters and the newspapers in the rush to judgment. Dewey, a liberal Republican, was popular in intellectual circles and perhaps the papers were letting their wishes get ahead of their practical assessments.
The mispredictions by Roper and Gallup were not the same as the problems with the Literary Digest. After polling its subscribers, by no means a representative cross section of voters, the publishers predicted that Alfred Landon would defeat Franklin Roosevelt in 1936. Landon lost all but two states and the Literary Digest went out of business. It is heartening that foolishness at least used to have consequences.
It turns out that the key mistake of the pollsters in 1948 was to stop polling two weeks before the election. No one considered the possibility of a last minute surge towards Truman. Some polls used data compiled as late as August 1948. In addition, pollsters were too cavalier in their treatment of undecided voters. They just allotted the undecided to the candidates in the same proportion as the decided vote. [1]
All modern pollsters know these lessons. However, the costs of conducting well thought out polls with statistically significant numbers of respondents tempt people into taking shortcuts. The Newsweek Poll is a modern example of how not to conduct a poll and as a consequence its results are volatile and not credible. That is why Newsweek can claim one week that Gore has a 14-point lead and a week later that Gore’s lead has plummeted to 2 percentage points. Even given the volatility of a complacent polity, this is too much of a change in too short a period of time. It is the sort of change one expects after a political convention, not a week where the big news is a minor release of oil from the strategic petroleum reserve.
A polling organization can pay for lists of registered voters. This cost money so Newsweek obtains less expensive, more general lists, and asks voters if they are registered. One does not always get truthful answers. Even more importantly reputable pollsters must take the time to cull from respondents, likely voters. Various organizations have different ways to do this. Basically they ask respondents if they voted in the last election. There is a strong correlation between voting in the past and future voting. However, if pollsters want to save money they will stick with registered voters or spend less time assessing whether a registered voter is likely to vote.
Pollsters must also consider at what time they query respondents. Newsweek stops its polling at 8:30 PM Eastern Time to save money. However, as a consequence, they only sample people who are at home during the day in Western Time Zone states. By contrast, the Battleground Poll, only polls Monday through Thursday because weekend activities make weekend polls notoriously unreliable.
One way to estimate future polling performance is past performance. In 1996, Bill Clinton defeated Bob Dole by 8 percentage points. This was a significant popular vote win for Clinton, but far less than the 18-point margin predicted by CBS News and the 12-point difference found by the Harris, NBC News/Wall Street Journal, and ABC Polls. Two polls did conspicuously well, the Reuters/Zogby Poll and the Battleground Poll. They were within a percentage point of the final outcome.
The Zogby Poll is the brainchild of John Zogby who realized that many Conservatives do not like to respond to polls. His polling techniques attempt to compensate for this. The Battleground poll is a cooperative venture between Republican pollster Ed Goeas and Democratic pollster Celinda Lake. This week Zogby has Gore up by two points, while the Battleground Poll has Bush up by five points. Interestingly, the Battleground Poll still finds 21% of likely voters undecided.
It is clear that this election is still close. The Reuters/Zogby and the Battleground Polls appear to be the ones to watch.
1. Irwin Ross,
The Loneliest Campaign, 1968. 245-252.
This entry was posted on Sunday, October 1st, 2000 at 8:52 am and is filed under Social Commentary. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
Polls to Follow
Virtually veryone has seen the historic 1948 photograph of Harry S. Truman grasping the front page of the Chicago Tribune that erroneously declared in large type “Dewey Beats Truman.” The paper wanted to be the first to publish and made the mistake of believing polls.After that polling embarrassment, there were recriminations and self-examination. Despite the fact that Truman won by two million votes, the story in the Electoral College was much closer. The switch of 30,000 votes, distributed the right way would have given an Electoral College victory to Dewey. Nonetheless, there was no excuse on the part of pollsters and the newspapers in the rush to judgment. Dewey, a liberal Republican, was popular in intellectual circles and perhaps the papers were letting their wishes get ahead of their practical assessments.
The mispredictions by Roper and Gallup were not the same as the problems with the Literary Digest. After polling its subscribers, by no means a representative cross section of voters, the publishers predicted that Alfred Landon would defeat Franklin Roosevelt in 1936. Landon lost all but two states and the Literary Digest went out of business. It is heartening that foolishness at least used to have consequences.
It turns out that the key mistake of the pollsters in 1948 was to stop polling two weeks before the election. No one considered the possibility of a last minute surge towards Truman. Some polls used data compiled as late as August 1948. In addition, pollsters were too cavalier in their treatment of undecided voters. They just allotted the undecided to the candidates in the same proportion as the decided vote. [1]
All modern pollsters know these lessons. However, the costs of conducting well thought out polls with statistically significant numbers of respondents tempt people into taking shortcuts. The Newsweek Poll is a modern example of how not to conduct a poll and as a consequence its results are volatile and not credible. That is why Newsweek can claim one week that Gore has a 14-point lead and a week later that Gore’s lead has plummeted to 2 percentage points. Even given the volatility of a complacent polity, this is too much of a change in too short a period of time. It is the sort of change one expects after a political convention, not a week where the big news is a minor release of oil from the strategic petroleum reserve.
A polling organization can pay for lists of registered voters. This cost money so Newsweek obtains less expensive, more general lists, and asks voters if they are registered. One does not always get truthful answers. Even more importantly reputable pollsters must take the time to cull from respondents, likely voters. Various organizations have different ways to do this. Basically they ask respondents if they voted in the last election. There is a strong correlation between voting in the past and future voting. However, if pollsters want to save money they will stick with registered voters or spend less time assessing whether a registered voter is likely to vote.
Pollsters must also consider at what time they query respondents. Newsweek stops its polling at 8:30 PM Eastern Time to save money. However, as a consequence, they only sample people who are at home during the day in Western Time Zone states. By contrast, the Battleground Poll, only polls Monday through Thursday because weekend activities make weekend polls notoriously unreliable.
One way to estimate future polling performance is past performance. In 1996, Bill Clinton defeated Bob Dole by 8 percentage points. This was a significant popular vote win for Clinton, but far less than the 18-point margin predicted by CBS News and the 12-point difference found by the Harris, NBC News/Wall Street Journal, and ABC Polls. Two polls did conspicuously well, the Reuters/Zogby Poll and the Battleground Poll. They were within a percentage point of the final outcome.
The Zogby Poll is the brainchild of John Zogby who realized that many Conservatives do not like to respond to polls. His polling techniques attempt to compensate for this. The Battleground poll is a cooperative venture between Republican pollster Ed Goeas and Democratic pollster Celinda Lake. This week Zogby has Gore up by two points, while the Battleground Poll has Bush up by five points. Interestingly, the Battleground Poll still finds 21% of likely voters undecided.
It is clear that this election is still close. The Reuters/Zogby and the Battleground Polls appear to be the ones to watch.
1. Irwin Ross, The Loneliest Campaign, 1968. 245-252.
This entry was posted on Sunday, October 1st, 2000 at 8:52 am and is filed under Social Commentary. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.