Online Surveys, No Longer Much Fun


-- Download Online Surveys, No Longer Much Fun as PDF --


Once upon a time, it was fun to take online polls. For lots of reasons. But one of my favourites was to watch how poorly polls could be constructed.

https://i2.wp.com/dgivista.org/uploaded_images/Canada20:20-713097.jpg?resize=377%2C236Once, six years ago, Innovative Research Group put a racist poll into the field. It included questions about whether I had favourable or unfavourable feelings about various races and religious groups, sometimes lumping in folks who come from nearby places. It was disgusting. What DID I think about Blacks, South Asians [as opposed to, say, Asians], Muslims, recent immigrants…you get the picture. They didn’t ask about South Americans or Jews or Hindus. Or Christians. I wonder why.

At any rate, I took a bunch of screenshots of the offensive survey, so that I could write about it and show everyone what kind of data they were trying to mine. Maybe they were just trying to figure out who were the racists in their polling pool.

angus.bannerBut as you can see below, Angus Reid has developed a new tool that it is using in at least some of its online polls:

No fun allowed.
No fun allowed.

Aside from the understandable agreement to not share client’s proprietary information, it requires participants to not take screen shots of the poll itself. To continue with the survey we are required to agree that we will not “photograph, record, publish on the Internet, copy, or in any way reproduce any of the confidential information included in this study.”

Well, that’s just sad.

If they end up asking some really notable questions, we can’t share that. Or, if they trot out a racist survey like IRG did six years ago, I wouldn’t be able to share the love.

I wonder what kind of consequence there would be if I were to share, not client confidential information in the survey, but really crappy survey methodology that doesn’t violate a third party’s privacy. Is the worst that can happen that I would be barred from future surveys?

All I know for sure is that pollsters have really screwed up lately, completely blowing the Alberta and BC provincial elections, and being out in left field on the US presidential election. It comes from having unrepresentative samples because people don’t want to answer calls, or certain types of people not having landlines, and people just lying about how much they intend to actually vote. Then polling firms try to adjust for underrepresented populations. Often badly.

What if pollsters, going forward, realizing their methodologies are…suspect…now try some new kinds of engagement. And now, they don’t want us sharing that.

Their credibility is in the toilet. Adding this agreement stage to the whole process may poison their cherished online polling community. It’s called Werner Heisenberg’s Uncertainty Principle. And Angus Reid might have just stepped in it.

The following two tabs change content below.

Stephen Elliott-Buckley

Post-partisan eco-socialist. at Politics, Re-Spun
Stephen Elliott-Buckley is a husband, father, professor, speaker, consultant, former suburban Vancouver high school English and Social Studies teacher who changed careers because the BC Liberal Party has been working hard to ruin public education. He has various English and Political Science degrees and has been writing political, social and economic editorials since November 2002. Stephen is in Twitter, Miro and iTunes, and the email thing, and at his website, dgiVista.org.

Latest posts by Stephen Elliott-Buckley (see all)

4 thoughts on “Online Surveys, No Longer Much Fun”

  1. I no longer answer polls (other than marking a ballot at election times) because, inevitably, the answers to their questions bear little on what I really want to tell people. As well, it is often plain what the poll is trying to get you to say. It’s a little like the snippets of the Lang and O’Leary Exchange to which I’ve been subjected,a debate where there is no debate because the real issues are neatly skirted and what back-and-forth there is moves from the Right to the NutJob Right. I suppose that people who subscribe to polls might watch drivel like that. I’d rather be out in the yard in both cases. The issue of not reporting what pollsters are asking is another link in the chainmail armor of secrecy that shrouds so many of the dealings that, affecting the public, should rightly be openly public. Thanks for the post.

  2. I can actually see why pollsters might not people seeing all the poll questions before answering the poll. Often the questions are designed to be indirect to find out what people really think, not what they want you to think that they think or even not what they think that they think. For example asking someone if they have prejudices against a particular group might elicit a direct no, because people do not want to believe they are racist. But indirect questions can reveal what people really think. If you figure out the point of the questions ahead of time your answers may not be as honest.

  3. Pollsters invite participants to donate information that has commercial value so it can be sold to customers who find value in the data. Actually, they don’t ask for it to be donated; if you are a loyal and long term participant, they’ll provide a reward that retroactively values your time at about $1 an hour.

    It’s a fine deal for the pollsters but a fool’s game for people supplying the information.

Leave a Reply to Norm Farrell Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.