C&E: How about the pollster as pundit? The ones who seem to spend more time on cable TV than on campaigns?
Lundry: I actually thought you were going somewhere different with the question, which is this new crop of pollsters—especially the IVR (interactive voice response) ones—who are clearly polling just for media hits. I think Public Policy Polling is probably the primary example of that. They do a good job from time to time, but some of the questions they ask are just clearly generated as link bait. They’re asking Barack Obama versus Charlie Sheen in their polls, which is just ridiculous. I think that’s more of an issue than the pollster as pundit.
Hickman: At least PPP has people who pay them for polls. Some of these people don’t even have paying clients. They just do it for publicity.
Lundry: And to their credit it has worked. They’ve got their name out there and built their brand very aggressively. But every once in a while PPP asks a question that really makes me cringe and wonder whether that’s making the role of pollster in the public domain better or worse.
C&E: How can you make use of social media for survey research? Have you been testing ways to incorporate it?
Lundry: Absolutely. We’re doing just that sort of thing with sentiment analysis of data that’s out there on the Internet. I think that we’re never going to get to a place where analysis of social network data or blogs or online news is going to replace polling. But it’s a useful indicator of when you should go in the field and what you should go in the field about. It’s a signaling mechanism.
Hickman: Is it leading or is it anomalous?
Lundry: It’s leading. In fact, all of the studies that we’ve done and others that have come out publicly by academics indicate that there is predictive power in these things. There was one simple study that took tweets with the word Obama and coded them for sentiment—either positive or negative—and used it to predict the Gallup daily track three days ahead of time. There was a correlation of about .75 between the two. Twitter has also been used to predict the stock market; it’s been used to accurately predict opening weekend box office returns for movies. It’s not perfect and it’s not a replacement, but I think it’s a pretty incredible complement that increases the efficiency when you go in the field.
Bolger: We’re still trying to figure out how to maximize analysis of blogs and Twitter and everything else. But people aren’t using social networking for politics as much as politicians think that they are or might want them to. These are more like early indicators or early warning systems as opposed to something that gives you a true sense of the state of the race.
Lundry: I also think the conversation on Twitter is somewhat limited. There’s a lot of great ability to look at the aggregate effect of communication—what news articles are out there, what’s on radio, what’s on TV. That’s where we find real value in it. It’s a leading indicator of what information people are being exposed to. And opinions don’t change without exposure to new information.
Hickman: You do see some things that are really fascinating about the number of mentions in social media. I’m sure that the number of mentions of Herman Cain has gone from some relatively small number to x plus a really big number.
Bolger: But he’s the exception and not the rule. I can’t tell you after 2008 how many candidates would say, “I want an Internet presence like Obama.” Well, ok, because that’s really easy.
Hickman: Sure, and I’d like to play in the NBA. (laughter)
Hogan: This is separate from social networking, but given how difficult it is to get young people on the phone I think we’ll see more use of online polling. It’s not representative, so you don’t want to use it for your basic polls. But for message testing I think there’s a lot of stuff you can do online that you can’t do over the phone.
Lundry: I actually think we’ve been having the wrong debate in the community about online research. We’ve been wondering how we can ask the same questions online and get the same answers we get on the phone. That’s not the right way to think about it. We need to figure out how to ask different questions that take advantage of the medium and come up with more interesting and maybe more insightful answers.
Hickman: And this is the hardest thing for me, but you do have to sort of suspend this commitment to having a perfectly random sample. You have to just accept that it’s a cross-section of some sort.
C&E: Same question when it comes to smartphones and tablets—how will those apply to survey research?
Lundry: We’ve never done anything with it, but I have seen some very interesting demos of ethnographic research—the idea of taking pictures of what you’re seeing in the shopping aisle. There are some companies that are experimenting with push surveys that are based on where you are at a given moment. So an application could know when you were in a supermarket and then push a text asking you to take a picture of what you’re looking at on the shelf.
Bolger: The challenge with all of this is that ultimately the bread and butter of what we do isn’t changing. It’s still about the status of the race, what messages are working and what we don’t want to put our money into. From that standpoint, all of this technology just isn’t there yet. We can do great things in terms of ad testing, but in a campaign we still need to know what groups are moving and what groups we’re lagging with. As an industry, we’re a long way from figuring out the new technology in terms of being able to reach these folks.
Hickman: I think the bigger use for most of us will be in the distribution of our findings to our clients—especially the tablets. That’s something clients will increasingly want from us so they can have their data in a form that they can manipulate on a tablet.
Lundry: I’m waiting for Facebook to open up its list for survey research. If they’re smart, I think they’ll realize it’s a way to monetize users very quickly and I’m sure that they’re having internal discussions about it. It would be the biggest survey panel in the world and it would be very interesting to see what sort of accuracy you can get out of it.