March 31, 2006

Beware Of Online Polls

There are a bunch of new poll numbers circulating in a bunch of states, thanks to the release of the latest online polls Zogby Int'l conducts for the Wall Street Journal's web site. We don't publish or acknowledge the existence of these numbers in any of our outlets because we are just not comfortable that online panels are reliable indicators.

It's a very new technology and we applaud Zogby for trying because some mix of phone calls, door-to-door and online will be used to create reliable polls in the future. Zogby is uniquely situated for the future, in fact, as he regularly conducts door-to-door surveys via his int'l arm, so he's gaining crucial experience when he moves toward using all three technologies for the same survey. (Note: Zogby does do a few calls in each state he polls online, but, frankly, it's not enough calls.)

But, to date, his online poll results are not just quirky, in some cases they don't make any sense. BTW, we have the same policy on Rasmussen (who uses automated callers) as well as his numbers sometimes show movement where none should have occurred. Until the track records of these surveys are proven over a long period of time, we'll continue to ignore these poll results when conducting our own analysis.

Finally, and here's the clincher for why we look at both Zogby online and Rasmussen with such a jaundiced eye: if either method of polling (online panels or automated callers) were consistently reliable, wouldn't professional campaign pollsters be using it? It's certainly cheaper [CHUCK TODD]

Posted at 01:02 PM

Comments

This is great!!

I just wanted to tell you The Parker Group is doing it properly ... with a blended system-

1. Search for and identify respondents with IVR
2. Screen on the phone with live callers and get e-mail address and check quotas.
3. Send link to respondent while respondent is still on the phone.
4. Guarantee that the link is only used once.
5. Guarantee the correct person did the poll with an IVR validation upon completion.
6. Guarantee that they're just not clicking through questions ? the system will not let them.
7. Optionally call back live for longer version after 'net/IVR questions completed ... allows for in-depth analysis of hard-to-find / interesting groups.
8. Two versions test and a voice version which doesn't require the respondent to read anything.

Tony Parker | 04.19.06 03:32 PM

I agree with your Zogby decision, but automated polls are the future. The reason consultants haven't used automated polls is because they're, as a class, super-conservative and because (yes, this is cynical) they get paid a commission and automateds are relatively cheap.

In Illinois, automated polls are being used by two major candidates that I know of (GOP governor and the Dem county board president candidate).

Automated polling was used in Illinois statewide races by the Sun-Times in 2002 and CBS2 in 2004. There were few, if any problems with those polls (except I thought the Sun-Times sample sizes were too small). Rasmusssen and SurveyUSA have both done automated polling in Illinois and the results have been pretty reliable and comparable to "live human" polling.

Instead of listening to those insanely overpaid DC consultants who specialize in extracting as much money from clients as possible, you should check automated poll results with "live human" results. There's little if any difference, except the size of consultant commissions.

rich miller | 08.22.06 04:48 PM

Post a comment