Wednesday, September 28, 2011

Always Beware the Probe!

In the coming days, the Winnipeg Free Press will publish election survey results from Probe Research.

It's almost guaranteed that very general questions will have been asked in the survey, and it's almost guaranteed that Scott McKay, Chris Adams, and maybe even Curtis Brown will be quoted extensively on why they think the data is the way it is.

For an otherwise credible research company, it has always fascinated the Blackberry Addicts the extent to which researchers themselves provide their own personal opinions on why the public appears to think the way they think rather than actually researching why the public thinks the way they think.

Nobody pays a research company for the opinions of researchers, and it's always a hoot to read the Free Press extensively quote Scott McKay et al on why Scott McKay et al think the public thinks a certain way when they didn't even bother asking why.

Nevertheless, Mr. Dan Lett has devoted an entire blog post arguing Probe's upcoming survey results will be the definitive survey of the campaign. And readers should expect the Free Press to make a big deal out of the results.

It's been rare, in fact, for Probe to accurately predict recent election results.

In the 2007 election, the NDP won 48 percent of the popular vote. Probe's quarterly polls leading up to the 2007 general election were off between 8 and 12 percent.
  • June 2006 = 38%
  • September 2006 = 36%
  • December 2006 = 38%
  • March 2007 = 40%
And we all remember what happened in 2003 when Probe thought the NDP was at 57%, right? Probe was wrong.

So, take the upcoming Probe survey results with a gigantic grain of salt everyone.

And, most importantly, always beware the probe even if it's only presented as a bunch of questions!