Tag Archives: Innovative Research Group

Online Surveys, No Longer Much Fun

Once upon a time, it was fun to take online polls. For lots of reasons. But one of my favourites was to watch how poorly polls could be constructed.

https://i2.wp.com/dgivista.org/uploaded_images/Canada20:20-713097.jpg?resize=377%2C236Once, six years ago, Innovative Research Group put a racist poll into the field. It included questions about whether I had favourable or unfavourable feelings about various races and religious groups, sometimes lumping in folks who come from nearby places. It was disgusting. What DID I think about Blacks, South Asians [as opposed to, say, Asians], Muslims, recent immigrants…you get the picture. They didn’t ask about South Americans or Jews or Hindus. Or Christians. I wonder why.

At any rate, I took a bunch of screenshots of the offensive survey, so that I could write about it and show everyone what kind of data they were trying to mine. Maybe they were just trying to figure out who were the racists in their polling pool.

angus.bannerBut as you can see below, Angus Reid has developed a new tool that it is using in at least some of its online polls:

No fun allowed.
No fun allowed.

Aside from the understandable agreement to not share client’s proprietary information, it requires participants to not take screen shots of the poll itself. To continue with the survey we are required to agree that we will not “photograph, record, publish on the Internet, copy, or in any way reproduce any of the confidential information included in this study.”

Well, that’s just sad.

If they end up asking some really notable questions, we can’t share that. Or, if they trot out a racist survey like IRG did six years ago, I wouldn’t be able to share the love.

I wonder what kind of consequence there would be if I were to share, not client confidential information in the survey, but really crappy survey methodology that doesn’t violate a third party’s privacy. Is the worst that can happen that I would be barred from future surveys?

All I know for sure is that pollsters have really screwed up lately, completely blowing the Alberta and BC provincial elections, and being out in left field on the US presidential election. It comes from having unrepresentative samples because people don’t want to answer calls, or certain types of people not having landlines, and people just lying about how much they intend to actually vote. Then polling firms try to adjust for underrepresented populations. Often badly.

What if pollsters, going forward, realizing their methodologies are…suspect…now try some new kinds of engagement. And now, they don’t want us sharing that.

Their credibility is in the toilet. Adding this agreement stage to the whole process may poison their cherished online polling community. It’s called Werner Heisenberg’s Uncertainty Principle. And Angus Reid might have just stepped in it.