Political pollsters quite accurately boast of the reliability of their science, but polling’s biggest ever goof remains alive 50 years later in a headline seared into America’s collective memory: “Dewey Defeats Truman.”
Fifty years ago this month, all major polls predicted victory for New York Gov. Thomas Dewey over Harry Truman, the incumbent president. Reasons for that mistake are still a cause of debate.
“I don’t think the polls were wrong in terms of measuring national sentiment,” said Burns W. “Bud” Roper, retired chairman of Roper Starch Worldwide and son of pioneering pollster Elmo Roper. “Clearly they were wrong in determining the election. I think the 1948 polls were more accurate than the 1948 election.”
Far from killing the fledgling industry, which had become popular in the 1930s, the pollsters’ embarrassing mistake laid the foundation for modern polling techniques. It also offered a valuable reminder that “it ain’t over ‘til it’s over.”
“We stopped polling a few weeks too soon,” said George Gallup Jr., co-chairman of the Gallup organization and son and namesake of another of polling’s giants. “We had been lulled into thinking that nothing much changes in the last few weeks of the campaign.”
The problem was that major pollsters of the day, Elmo Roper, George Gallup and Archibald M. Crossley, cut their teeth on elections involving Franklin D. Roosevelt.
“Roosevelt was the issue. People were either for him or against him. The whole thing was built around Roosevelt,” said Burns Roper, explaining the approach to polling in presidential elections of 1936, 1940 and 1944.
In the 1948 presidential election, there was no Roosevelt, but a field of the two major candidates, as well as Dixiecrat Strom Thurmond and Progressive Party candidate Henry Wallace.
The polls predicted a Dewey victory of between 5 to 15 percentage points, but Truman won by 4.4 percentage points. The labor vote was energized as Democrats worried about Dewey’s strength in preelection polls, and Republicans felt their candidate would win “so they played golf that day,” Roper said.
The offspring of the famous pollsters from 1948 remember the days around the election as stressful.
Helen Crossley recalls “a very tense household” as her father worried aloud that Truman, who drew increasingly enthusiastic crowds at his preelection speeches, was gaining momentum. George Gallup Jr. says his father had to visit many newspaper clients after the election to lure them back after 30 canceled their poll service. Burns Roper said the election came just weeks after the suicide of his brother, and he recalled that he and his father voted for Truman.
“It sort of looked like the end of the world,” Roper said. “It was the definition of mixed emotions. We saw our man winning, but our company going down the tubes.”
Roper company officials huddled and came up with an approach for Elmo Roper’s next newspaper column: “We were wrong. We couldn’t have been more wrong. We’re going to find out why.”
The polling pioneers admitted their mistakes, reexamined their methods and plunged back to work. They moved gradually away from quota sampling, which questioned a set number of people from different ethnic and age groups, and moved toward random sampling. They extended polling deadlines up until election day and developed their ability to predict those likely to come out and vote.
“Political polling was non-probability, and for a number of years they got away with it,” says New York pollster Warren Mitofsky, a pioneer of random-digit dialing and the use of extensive telephone sampling 20 years ago. “In 1948, they got burned.”
There’s never been a comparable election disaster since 1948, when all the major players picked the wrong winner, said Tom W. Smith, director of the general social survey at the National Opinion Research Center in Chicago. The scale of the disaster was such that a panel of scientists analyzed the industry for the Social Science Research Council.
Since then, preelection polls have become far more accurate, although some years are more precise than others. The margin of Ronald Reagan’s 1980 victory was underestimated by some pollsters, for example.
While the polling industry has made progress in methods of sampling and poll timing, it still has plenty to learn about probability methods and the wording and order of questions, Mitofsky said.
“Identifying likely voters is still a mystery to most polling organizations,” he said.
And a big lesson for pollsters from 1948 still holds true today.
“There’s a lot of room for humility in polling,” Mitofsky said. “Every time you get cocky, you lose.”