Tag Archives: polling

Democracy

Ontario Election: Polling Fails Again!

Leave a comment

After spectacular failures during the Alberta and BC elections, polling, as an industry struggling for credibility, lost yesterday’s election in Ontario along with the neoliberal, anti-social Hudak.

308.com reviewed the polling as of the day before the election. Polls indicated that there would most likely be a Liberal minority government. Those polls missed the extra 10 seats the Liberals earned at the expense of the PC’s, and had the PC’s 4 points higher than what they really earned.

What’s wrong with polling?

  1. Random sampling is necessary to make inferences for all of the population. When people block calls, use call display to ignore polling, and self-select for online poll samples, the polling industry has to adjust/calibrate their results “somehow” to try to get an accurate prediction. BC, Alberta and Ontario demonstrate that they can’t do that. And people opting out of poll participation skew the results. And the polling industry doesn’t even know which demographics of people are opting out. So WE SHOULD NOT PAY ANY ATTENTION TO POLLS ANYMORE! Get it?
  2. People lie to pollsters. Those who opt-in to responding to a poll are more likely to have a political agenda. They’re also more likely to lie to pollsters, particularly about how strong their intention to vote is. They don’t want to look stupid by saying they’d vote in one way, but then admit they aren’t likely to bother to vote. And in Ontario’s case yesterday, it looked like PC supporters were more likely to express high voter intention, but they couldn’t get the vote out.

Polling is lazy.

If people or organizations want to get a sense of how the population is feeling, they can pay some cash, get some polling data, then set policy or do whatever they want with it. But if the polling model is broken, they’re going to have to go out, get talking to people on the ground and truly, authentically engage with people.

Shock! Talking to people? It’s something political parties have shown real inability to do in any meaningful way. The first one that gets off its ass and starts interacting with actual people, en masse, should see some real electoral gains. No?

British Columbia, Canada, Freedom of Information, Journalism, Media, Racism

Online Surveys, No Longer Much Fun

4 Comments

Once upon a time, it was fun to take online polls. For lots of reasons. But one of my favourites was to watch how poorly polls could be constructed.

http://dgivista.org/uploaded_images/Canada20:20-713097.jpgOnce, six years ago, Innovative Research Group put a racist poll into the field. It included questions about whether I had favourable or unfavourable feelings about various races and religious groups, sometimes lumping in folks who come from nearby places. It was disgusting. What DID I think about Blacks, South Asians [as opposed to, say, Asians], Muslims, recent immigrants…you get the picture. They didn’t ask about South Americans or Jews or Hindus. Or Christians. I wonder why.

At any rate, I took a bunch of screenshots of the offensive survey, so that I could write about it and show everyone what kind of data they were trying to mine. Maybe they were just trying to figure out who were the racists in their polling pool.

angus.bannerBut as you can see below, Angus Reid has developed a new tool that it is using in at least some of its online polls:

No fun allowed.
No fun allowed.

Aside from the understandable agreement to not share client’s proprietary information, it requires participants to not take screen shots of the poll itself. To continue with the survey we are required to agree that we will not “photograph, record, publish on the Internet, copy, or in any way reproduce any of the confidential information included in this study.”

Well, that’s just sad.

If they end up asking some really notable questions, we can’t share that. Or, if they trot out a racist survey like IRG did six years ago, I wouldn’t be able to share the love.

I wonder what kind of consequence there would be if I were to share, not client confidential information in the survey, but really crappy survey methodology that doesn’t violate a third party’s privacy. Is the worst that can happen that I would be barred from future surveys?

All I know for sure is that pollsters have really screwed up lately, completely blowing the Alberta and BC provincial elections, and being out in left field on the US presidential election. It comes from having unrepresentative samples because people don’t want to answer calls, or certain types of people not having landlines, and people just lying about how much they intend to actually vote. Then polling firms try to adjust for underrepresented populations. Often badly.

What if pollsters, going forward, realizing their methodologies are…suspect…now try some new kinds of engagement. And now, they don’t want us sharing that.

Their credibility is in the toilet. Adding this agreement stage to the whole process may poison their cherished online polling community. It’s called Werner Heisenberg’s Uncertainty Principle. And Angus Reid might have just stepped in it.

British Columbia

A brief reflection on BC election polling

Leave a comment

Once the disappointment over the BC election isn’t so painful, there’s something that needs to be talked about regarding polling.  This is a quick, stream-of-consciousness post, and I plan on writing more later.

While Mario Canseco of Angus Reid said, this morning, that it’s not a “methodology” problem, I think it is. Forum Research, while still off, was the closest to the results, with a survey sample of about 1,100 using telephone voting. Angus Reid, Ekos, Ipsos, etc., all used web-based voting with samples of about 800, and were wildly off.

The polling firms claim that web panels can work well, because you can weight respondents based on demographics. Sure, but you’re weighting a non-random sample, which can be really problematic scientifically. And it’s web based. Everyone knows that political parties send out e-blasts to freep polls – “go here and click this.” Because everyone thinks that polls lead opinion, rather than the other way around.

I know that a lot of people point at telephone polls and flail, because “cellphones aren’t polled.” Sure, they’re not. Normally. Some are, depending on where you put your phone number and such. But more importantly, the demographic most likely to be excluded from telephone polls because of over-reliance on landlines – the 18-34 demographic – is also the demographic least likely to vote. So, perhaps telephone polling acts a filter, removing the least likely voters from the poll sample to start with?

Even more importantly, 800 respondents is a dismally low sample, especially, in my mind, for a web-based vote. If we take a voting population of about 3.1 million, for a +/- 3%, 9 times out of 10 confidence level, you want to have more than 1,800 respondents.

So, for a confident poll of BC voters, you need 1,800 respondents. Forum was closest to that sample, and they were closest in their projections. Angus Reid and the rest, while they claim to weight their samples based on demographics and other fancy calculus… their sample is less than 1/3 of a number statistically required for confidence.

This doesn’t even get into the larger campaign issues.