Com 450: Politics and Media

The Polls and Polling

Dr. Janet McMullen

Last Updated:  11/13/2002

The roll of political polling and research in campaigns and elections is one which is growing substantially with each succeeding election.  In this lecture, we're going to look at some of the key issues:

Reading:  Powell and Cowert, Ch. 10; Plissner Chapter 3 and 4 (On reserve at the library)   I really should have kept the Plissner book, The Control Room: How Television Calls the Shots in Presidential Elections on as a required text.  While it was written before the 2000 election, it really is an excellent book.  So what I have done is this: We will not be using the Graber book because it was received too late.  It is also way more expensive than I thought it would be.  We have ordered Plissner into the University Book Store and you will like the price MUCH better.

Additional Resources:  Graber: Section 3;  Jamieson, Ch 27. 

Pre-Election / Campaign Polls

Pre-Election Polls are the most familiar and the ones we hear about every day prior to the election. They may be sponsored in several ways:

USES of Campaign Polls:

Impact of Campaign Polls:

A little background:

While some public opinion polling was done in the first have of the 20th century, public opinion polling for elections really became important in the last fifty years.  In the process, "calling" a winner has become a key mark of prestige for television networks and news organizations, one for which they were willing to expend hundreds of thousands, if not millions, of dollars.  Getting it wrong, was a humiliating experience personally and institutionally, and stakes kept getting higher and higher.

In the 1940s, Gallup, Crossley and Roper were the primary pollsters for campaigns.  Their methodologies varied, but included selecting groups demographically parallel to the voting public, people who owned cars or had home telephones and  mail-backs from popular magazines.  Amazingly, they sometimes got it right.  They didn't get it particularly right in 1948 when Truman won after public opinion polls had predicted a landslide for Dewey.   Interestingly enough, some pollsters thought Dewey was far enough ahead that they ceased doing the research six weeks before the election.  Gallup conducted its final survey in mid-October, declaring that Dewey would win and the whole world would see "how good we are."  Oooops.   Gallup decided after that, that it would poll through the Saturday prior to the election and has done so ever since.

Computers became part of network election coverage technology in the early 1950s and as early as 1956, the networks began calling the races before all the polls closed.  By 1960, the race was on to see who would "call" the race first and most accurately.  That proved to be embarrassing because ABC and CBS both predicted Nixon would win: ABC at 6:45 and CBS at 7:16.   He didn't.

Lou Harris had been Kennedy's polling director and had used model precincts in his evaluations. These "cells" included the sub-groups for a state, and if representative, the way the precinct went would predict the way the state would go.  The "recipes" included all kinds of demographic information including urban/suburban/rural, economic, ethnic and educational data.   He called the method Voter Profile Analysis. It was extensive, expensive, and in 1962, CBS bought it, hoping to get the "egg off the face."   They did.  CBS with Walter Cronkite presiding and Don Hewitt producing provided "estimates" that a state would go a certain way and all of their call stood up.  Even so, more people watched NBC.

At NBC, the methodology was a little different.  Here polling director Richard Scammon discovered precincts with a history of going Democratic in one election and Republican in another and separated them from precincts with a stable voting history.  He did the tabulations on an adding machine.  In 1964, NBC had to use one RCA 's new line of computers.  Unfortunately, the thing kept breaking down leaving Scammon with only his adding machine and typewriter.

The interesting thing about 1964 was that when the networks "called" a state, they didn't identify it as a projection, but as a "done deal."  This raised the penalties for being wrong......and by 1967 out went the profile precincts and in came statistical analysis.  They had these high powered computers, and by golly, they were going to use them!   Harris was out at CBS and a statistician, Warren Mitofsky, was in.  While some key precinct research continued, random sampling was the new way.

Exit polling was introduced in 1980 as a way to "call".  Lou Harris had first done them in 1964 using the old VPA precincts and it turned out to be pretty dependable.  Exit polls were used initially to explain WHY voters had voted as they had.  By 1980 all the networks were using them, and they showed very early on that Ronald Reagan was winning big.   NBC tracked them and called states with them, while CBS hedged, waiting for final tallies of actual votes.  That put CBS two hours behind NBC in calling any given race. CBS knew, based on exit polling projections, by 4:30 p.m. that Reagan had won.  They did not announce he had won until after 10:30 p.m., 30 minutes AFTER Carter had conceded.

Some thought CBS had done the right thing.  When NBC made the call at 8:15 E.S.T., the polls had not closed in California.  It was reported that after the NBC call, election volunteers did not show up, phones went silent and "would-be voters became non-voters."  After a Congressional hearing on the issue, the network news chiefs agreed that they would NOT call a race until after the polling places for that race had all closed.  They would ask their reporters NOT to tell what they knew.  That hasn't worked very well, because the leaks usually happen.

By 1988, the networks had other things to worry about.  Viewership had dropped for network news, especially election coverage.  Expenses were up.  Revenues were down.  New ownership was eyeballing the bottom line in a big way.  Millions of dollars spent to beat the other guy in "calling" a race just didn't seem cost effective any more.  The statistical analysis was honed to the point that they usually all came to the same conclusions about the same time, but had to sit on the information because of the agreement with Congress.  So what was the point?  Better yet, what was the solution?

The solution: A POOL!   NBC had been trying to convince the others a pool was a good idea since the 1970s, but by 1988, CBS and ABC were ready to consider it.  When NBC announced it would fold its election unit and join with another network to save costs on election coverage, CNN jumped at the opportunity.  ABC decided for a number of reasons (including the immanent retirement of its election unit chief) to join with Mitofsky at CBS.  So the four networks formed Voter Research and Surveys (VRS) which began operating in the off-year 1990 elections.  VRS was up an running with the wrinkles ironed out by 1992, but the presence of Ross Perot clouded the results during the early primaries.

Cost cutting was still an issue with network management, so the company that counted votes for networks, Election News Service, was merged with VRS and the combined organization was called Voter News Service (VNS).   All the networks would get the same information at the same time, so the "war" was off.  Apparently Roone Arledge at ABC didn't read the memo, because he hired a political scientist to look at the numbers as they came in and analyze them. That allowed ABC to "call" a few states before the others did and the WAR was back on.

CBS went back to Warren Mitofsky who was by then a successful political consultant.  He showed them how to beat VNS and make the competitive calls -- if they wanted to spend a million dollars.  CBS did so because they didn't want to relinquish their image as a news leader.   That was fine until the WRONG call was made in an Arizona primary.  It turns out the exit poll was wrong and all the networks had it wrong.  Oooops.

On election night, 1992, most of the networks stayed true to their promise of not announcing exit poll results until the polls in a state had closed.  NBC gave some pretty big hints, however, with such statements as " ...it's going to be a long night for President Bush." (Plissner, p. 97)

By 1996, the thinking at the news division levels had changed.  As long as they had other information, aside from the exit polls on which to base the information, news people could speculate about who was going to win in a state before the polls closed.  By 7:00 p.m., exit polling showed President Clinton already had 300 electoral votes, enough to win.  While he couldn't call the race, he could say that no president had won without Florida in years, and Clinton had won Florida.  At NBC, Tim Russert was playing the key state game as well.....VNS made the call around 9:00 when a large number of state polls closed.  Despite the competition and the consultants, no network came out ahead on the outcome of the final state which would put him over the 270 mark before 9:00 p.m..  It had been too close to call.  

VNS had done pretty well.  Two errors -- one in a primary and one state race -- compared to over 400 accurate calls in four elections  was deemed to a success.

Polling was affected in another way by concerns to save money.  By the mid-seventies, computers and new statistical methods were changing the way election campaigns were reported.  Where in the 1950s and 1960s the "gut" instincts of the reporters were primary predictive tool, now frequent statistical research was not only possible but necessary.  Still, there was only so much money.  As a result, television networks joined major print institutions to cooperate on the gathering of campaign polling data.

These polls had some problems early on, especially in 1980 when nobody predicted Reagan's win of more than 10 points.  While no explanation was agreed upon for those problems, it may have been than the momentum was shifting, and the polling stopped on Saturday before the election.

Tracking polls emerged in 1984 and the horse race was on.

By 1992 CNN had  joined the polling effort, offering polls every two weeks up to October and then every day after that until the election.  CNN hooked up with various networks and publications over the years. In 2000, it's USATODAY  and Gallup.

 

TYPES OF Pre-Election Polls:

1.  Benchmark Survey:

The survey also asks questions in three KEY AREAS:

The benchmark survey is done early in the campaign and may in fact be done before the potential voters know much if anything about a challenger for an office.  Frequently results of these surveys are never made public.  

2.   Trial Heat Survey

These are the polls we're seeing in the news every day.  Trial heats may not actually be surveys, but rather specific questions or series of questions within a larger survey.  Trial heat questions read something like, "If the election were held today, would you vote for candidate X or candidate Y?"

These are the "horse race" coverage polls, and it is important to remember some key warnings....

3.  Tracking Polls:

These are also used in "horse race" reporting as well as to measure voter responses to campaign advertising and strategy.  They may be done by campaign managers or by media outlet and are usually done daily as the end of the campaign season approaches.

Tracking polls are done with a larger sample.  A ROLLING SAMPLE is used in which a sample of several hundred or more will be drawn on consecutive days.  The results may indicate an average of the MTWTH sample on Friday and Saturday's results will reflect data drawn on TWTHF, etc.  The over all sample is larger  than any single given day and that reduces sampling error. 

Some caution should be used... Any one day could be unusual for any number of reasons.  That could throw off the results for every day in which the aberrant data is included. 

Tracking polls are very valuable in allowing candidates to monitor trends in voter perceptions and attitudes and to modify a campaign accordingly.  They were used extensively by major media in 1996 and again in 2000. 

4.  Cross-Sectional Surveys:

These are surveys done with large sample sizes drawn to represent the various demographic characteristics of the electorate.  While they are done at various periods during the campaign, each time a different sample is drawn to meet the required demographics. 

These surveys can provide information about voter perception and attitude changes, but they don't provide much information about why.

To answer those questions the same respondents should be used at different times in the campaign.

5.  Panel Design Survey:

In this design, the same people are questioned at different times during the campaign.  While they are drawn to reflect demographic characteristics of the electorate, the fact that the same people are queried at different points in the campaign allows data to be collected which answers WHY changes occur.

There are some problems. 

6.  Focus Groups:

Focus groups are in-depth interviews with a group of people, usually not more than 10 or 12, who are selected because of their demographic or psychographic characteristics.  Focus groups may be asked to watch a candidate's speech, commercial, or performance and be asked to give reactions.  Because this is a qualitative method, focus group leaders can ask follow-up questions and pursue additional lines of questioning.  As one author puts it, "Unlike traditional quantitative research, focus groups are centrally concerned with understanding attitudes rather than measuring them. (Luntz, 1994)

Focus groups have been responsible for radical changes in campaigns.  The most famous incident occurred in 1988 when the Bush campaign team asked a group about their impressions of Democratic opponent, Michael Dukakis.  The group was made up of working class Democrats who favored Dukakis, but the focus group leader kept asking questions.  He uncovered the fact that these individuals did not like his opposition to the death penalty, his opposition to a law requiring school children to say the Pledge of Allegiance each school day, and they didn't like his support for program which let criminals out of prison for weekend passes.  Out of this focus group research came a re-targeted issue campaign against Dukakis, one that included the famous "revolving door" commercial about Willy Horton.  

Focus groups are also used to pre-screen campaign commercials and other campaign communication.  They can be used to test specific words, phrases, images and messages in order understand whether people like them and why.

They can also find concepts and issues which may have been missed by the campaign team.  Johnson(2002) provides an example of a Democratic incumbent who thought he had a great issue in the fact that his Republican opponent was against increasing minimum wage.  The focus groups revealed that his constituents didn't care about that.  They worked at jobs that paid substantially more, and their kids were making more than minimum wage at McDonalds. It was a non-issue. 

In order to be effective, Focus groups must have:

Focus groups have limitations:

7.  Deliberative Opinion Polls

This is a relatively new process in which a standard public opinion poll is collected and then a subset of that sample is asked to participate in focus group-type meetings.  While this method offers advantages of both survey and focus groups, there is still a question about the generalizability of the focus group data.  It is also expensive and complex to administer.

8.   Exit Polls

One of the most controversial polling methods employed in an election or campaign is the exit poll.  These polls are collected by asking voters how they voted as they leave the polling places.  The television networks have employed these polls for several years with some significant controversy.

Advantages of Exit Polls:

Disadvantages of Exit Polls:

So what are the responsibilities of media decision-makers regarding exit polls?

The National Council on Public Polls has a great website with lots of resources for students of the campaign and electoral process.  One of them deals with the ethical concerns journalists have in dealing with polls and evaluating the properly.  Consider "20 Questions for Journalists" to be required reading. You will find it at

http://www.rci/rutgers.edu/~ncppolls/20qs.htm

Criticisms of exit polls have been vocal and frequent:

One columnist urged voters to lie to exit pollsters.  The U.S. House of Representatives appointed a task force to hold hearings on the issue in 1985.  The state of Washington made it illegal to conduct exit polls in that state within 300 feet of a polling place, but the law was thrown out by a federal court.

For an excellent recent article on the issue of exit polls, see "The Poll Watchers", a regular column in The Washington Post.  In September 27, 2000 column, Richard Morin and Claudia Deane express concern about the roll of the Internet in the upcoming election.  Their concern is that while early exit poll results were known to media professionals long before the polls closed in 1996, these people had paid for access to the data and they were also gatekeepers.  While they may have leaked results to their peers within news organizations, they kept the information "in house" for the most part.

Morin and Deane predicted a "guerrilla uprising" on November 7th when they predict Internet sites all over the country will post exit poll results long before they are announced by major media outlets who have paid for them.  This may result in the demise of VNS itself.  Organized to collect and control such data, if the control part is gone.....  Exit polling data is released at mid-day, mid-afternoon and when the polls close.  So when it's noon in California, it's mid-afternoon on the east coast.  News media could very easily know who will win the election by early afternoon.  

If the information is leaked on the internet, it is doubtful that the networks will refrain from passing the information as well.  If VNS refused to release the numbers until the polls closed in California, but it would be too late to make much use of the data. 

As you all, know, the VNS data did indeed present problems....Florida was called first one way, then the other and the election of the 2000 had no outcome for weeks.  

Some excellent sources on the 2000 election..... (There are many more, but I wouldn't want to "overload you all".....)

Columbia Journalism Review : January, 2001  "A Wild Ride into History"  This entire issue deals with various aspects of the election and it has some outstanding articles http://cjr.org/year/01/1/index.asp   Be sure to read "Low Marks for Polls, Media" at http://cjr.org/year/01/1/kohut.asp and "The Big Mistake" by Neil Hickey at http://cjr.org/year/01/1/hickey.asp  and if you have time, browse the issue. It has some great stuff!

Another great source is American Journalism Review  which you can find at http://www.ajr.org (you should bookmark both of these).  Be sure to read "How they Blew It" by Alicia Sheppard at http://www.ajr.org/Article.asp?id=519 and "Polled Enough for Ya?" by Lori Robinson at http://www.ajr.org/Article.asp?id=564   

Of course, the Exit Polls weren't up and running for the 2002 elections, and that made for a much later night than most of us were used to on election night.  VNS is being revamped, but the task is not yet completed..... 

9.  Auditorium Testing or Dial Testing. 

While this isn't a poll in the usual sense of the word, it is a more and more widely used research technique to test commercials, ads and key campaign messages.   This is the kind of thing done at ASI in which respondents sit in an auditorium with a dial.  As they watch the commercial (or whatever) they turn the dial one way if they like what they see and the other way if they don't.  It's a minute by minute, almost second-by-second assessment of audience response.  As a result it can identify words, characters, logos, etc. that work well or don't work at all.  The problem is that it doesn't identify why an audience responds as it does.

 

What are the ethical implications for journalists?

Statistical Assessment service offers a checklist for journalists reporter polls:

To see some examples of companies that specialize in campaign polling, see the links below:

KEY QUESTIONS ABOUT POLLING:

Several sources offer answers to key questions voters ask about polls.  One of the best is at PollingReport.com.  You can find it at http://www.pollingreport.com/ncpp.htm  (I strongly encourage you to check it out....)  I will paraphrase some of those here.

What about Sampling Error?

Harry O'Neill, of Roper was quoted as saying, "God forbid the public should ever find out that all sampling error and its 95 percent confidence interval really say is that if you conduct the same biased survey among the same unrepresentative sample 100 times you will get the same meaningless results 95 times within plus or minus some percentage points of the results of the particular survey being reported." (Statistical Assessment Service, 2000)

Humphrey Taylor writes in an article for The Polling Report that "all surveys, all opinion polls (and, indeed all censuses) are estimates, which may be wrong."  Sampling error is potentially infinite.  The problems could result any number of things:

It's also hard to predict voter turn out which could impact the accuracy of the poll.  The people who answer the poll may not actually vote.

Then there's the "late swing" factor.  People who change their minds after the last poll was taken.

The standard acceptable sampling error is +/- 5% or .05.  As sample sizes increase, probability or error decreases, but similarly, if sample size DEcreases, the probability of error increases.  That's why a sample of 400 is much better than a sample of 100.  After about 1200 to 1500 respondents, the probabilities of error don't decrease that much, so most sample sizes average between 400-1200.

For an excellent discussion of why numbers are volatile and what sorts of things can sway a poll, see "What's behind Gallup's Volatile Poll Numbers?"  at http://www.washingtonpost.com/wp-dyn/articles/A51737-2000Oct11.html 

Why don't people I know get asked polling questions?

The average poll only uses a sample of about 1000 people.  That means there would have to be 200,000 polls to get around to everybody.

How accurate are polls?

When the NCPP checked the accuracy of polls conducted over the last 50 years by national media, it found that accuracy has improved in recent years. Polling error for each candidate averaged only 1.9 percentage points between 1956 and 1996.

Can wording of questions affect the results of polls?

Definitely!  Questions should be neutral and balanced and language must be clear and unambiguous.  Order of questions must also be considered. Professional pollsters review multiple drafts of questionnaires before they are finally approved.

Plissner cites an example from the 1992:

"Before asking respondents their choice for President, for example, CBS News/New York Times asks respondents to give an opinion of each candidate or say they don't know enough about him.  Perot at this point was the least know of the candidates.  Those who said they didn't know enough about him to have an opinion may have been inhibited from picking him--even though they might otherwise have done so."

"CNN/Time regularly reports the least support for President Bush. A likely reason: before asking about preference, the survey asks: "How well do you thinks are going in the country these days?" This may deter voters who say things are not going well from backing the man in office." (Plissner, p. 113)

 

To see an actual political poll questionnaire, so the the Victory Store at http://www.victorystore.com/polling/actualpoll.htm  (Consider this required reading)

How are polls conducted?

For an excellent resource on how the polls are conducted in this election, see the Gallup Organization faqs "How the Polls are Conducted"    at http://www.gallup.com/help/FAQs/poll1.asp   

Another good look at political polling can be found at the Roper site: 

Another good site is the Campaigns and Elections magazine at http://www.ropercenter.uconn.edu/pubper/pdf/pp13_3d.pdf 

Still another interesting site is Polling Report.com at http://www.pollingreport.com/ 

THE PROBLEMS WITH POLLING:

Harry Truman: "How far would Moses have gone if he had taken a poll in Egypt? What would Jesus Christ have preached if he had taken a poll in the land of Isreal?  Wgat wiykd gave happened to the reformation if Martin Luther had taken a poll?  It isn't polls or public opinion of the moment that count.  It's right and wrong in leadership." (Dionne , 1991, p. 311 quoted in Cowell and Cowart, 2002 p. 188)

Further, methodological errors can occur in a number of ways:

 

Call-in Polls or Pseudo Polls: 

These are non-scientific polls (and therefore, not polls at all) conducted by television stations to gauge viewer opinion about an issue.  More often, they are designed not to gather any meaningful data, but rather to provide viewers with some means of interactivity and an illusion of participation in the station or issue.  They are misleading and I believe unethical because most viewers do not understand they are meaningless.  Further, when one considers the prestige conferral associated with anything on television, because the "poll" is on TV, it is assumed to be valid in the mind of the average viewer.  The data collected has no generalizability, no external validity and is a waste of time and electricity.  However, the problem is that these polls can influence perception of public opinion and give viewers as false impression of where public opinion really is on an issue.  This is dangerous and the antithesis of what news organizations are supposed to do.  

Dennis Johnson reports in his excellent book, No Place for Amatures, that USA TODAY conducted a poll in 1990 which asked if readers thought Donald Trump symbolized what was good for America?  More than 6,000 people called an 800 number and 82% of them said Trump was "great".  The problem was, 72% of those calls came from two phone numbers! (p. 100)

Internet Polling:

With the growth of the Internet, polls taken there are becoming more prevalent.  There are lots of challenges which must be addressed with Internet polls, but some researchers are working on weighted techniques they believe will be as reliable and valid as telephone polls.  Much like the call-in polls, these polls have serious problems with the sample collected and are fairly meaningless at the present time. Their influence on perception of public opinion may be very real however, and therein lies the danger. Time will tell.  

Push Polls: 

These aren't really polls at all, but rather instruments designed to persuade voters under the guise of a poll.  They are usually short (though don't have to be) and they are distributed to many more people than usually receive a standard, legitimate polls. They often contain skewed or outright misinformation. 

While these didn't get a lot of attention until 1996, they had been on conducted for years in state and local elections.  Push polling has been condemned by professional associations associated with public opinion research, including The National Council on Public Polls, the American Association of Public Opinion Research and The American Association of Political Consultants.

What exactly is a push poll?  It's a phony poll in which massive numbers of voters are called and fed false or "misleading" information about a candidate while pretending to be determining how voters feel about this information.   They are essentially a telemarketing scheme designed to manipulate or "push" public opinion rather than discern what it already is.   Push polls have been used with "success" in several campaigns. [ For more information about them, see Asher, p. 123]

Push polls should not be confused with legitimate polling questions designed to discern public opinion about a legitimate negative campaign ad or strategy.  Sometimes a campaign may have a good reason to test a commercial which has some negative information about an opponent. 

So how do you tell the difference between legitimate survey research and push polling?  Herbert Asher and the American Association of Political Consultants offer some guidelines:

Johnson provides a number of examples:  In 1994, a "Democratic Party state phone bank called probably African-American voters and linked (Oliver) North and the Virginia Republican Party with Louisiana's David Duke and the Ku Klux Klan." (p. 164)

Remember......

Polling and related research offers a temptation to politicians and consultants.  Remember, even a bad poll will yield numbers of some kind.  You always get results.  The problem is, the public may have no clue about whether those numbers actually mean anything or not.  Sometimes even the experts don't have a clue.   In 2002 it was common knowledge that polling data showed Bob Riley anywhere between 8 and 11 points ahead of Don Siegleman.  At this writing, the election has still not been called..........

Resources and valuable Links:

Copyright, 2002

Dr. Janet McMullen

University of North Alabama