For LeadNow, the organization behind the Vote Together initiative operating in swing ridings across the country, the decision of who to vote for is straightforward: if you want to stop Stephen Harper from being reelected, vote for the local candidate with the best chance of beating the Conservatives.
To that end, they’ve mobilized a small army of volunteers, and nearly ninety thousand people in key ridings have pledged to vote according to their recommendations on Election Day.
Now, regular readers of The Alfalfafield will know that I have my issues with “strategic” voting – its inherently centrist bias is well-known, and it’s a woefully inadequate solution to the systemic problems with our political system. But today, I’d like to put forth a purely procedural criticism of LeadNow’s effort, which is surely the most well-organized strategic voting initiative in Canada’s history.
It’s a simple question: why should we trust their recommendations?
In order to ascertain which ridings are the closest and which candidates have the greatest prospects for defeating the Conservative Party, LeadNow has commissioned a series of riding-level polls. On the basis of these polls, they’ve issued recommendations so far for sixteen key ridings, with more recommendations to come in the final week of the campaign. So, for example, in the neighbouring Vancouver Island ridings of Nanaimo-Ladysmith and Cowichan-Malahat-Langford, LeadNow recommends that strategic voters vote for the NDP, on the basis of local polling which shows that party’s candidates in the best position to win.
Except that there’s a small problem with LeadNow’s numbers – they aren’t accurate.
Their latest posted poll, from the period of October 1-5, was conducted by the Dogwood Initiative, and Leadnow lists the results as being:
May 8 – 11
Oct 1 – 5
Sept 18 – 21
Oct 1 – 5
This looks pretty conclusive – the NDP in both cases is in a commanding lead over their nearest non-Conservative opponent, the Green Party. It would seem to make them the obvious choice.
But when we click through to the linked full report, we see not only a different story, but an entirely different set of numbers.
That’s because the Dogwood Initiative’s reading of the poll was not “NDP in commanding lead”, but instead, “Riding-level polls show many mid-island voters still undecided.” Undecided likely voters made up a slightly larger share of the total than did voters committed to any single party.
The poll’s sample sizes were also relatively small, such that the margin of error is +/- 4.9% for both ridings. Factoring in the uncertainty about how solid the numbers are and the huge amount of voters who had yet to decide, it’s actually borderline impossible to predict what’s actually going to happen in these ridings, or which party has the best shot of defeating the Conservatives.
But LeadNow essentially edited out all the undecided voters from the picture. When you erase the quarter of voters who have yet to decide and adjust the percentages accordingly, the NDP’s lead suddenly looks a whole lot bigger.
Which is the problem with polls. They’re presented with a certain air of authenticity and authority – the impression one gets is that here are some objective numbers showing exactly what the state of play is in a given area. But there are many ways to present a single set of data, and in this case I think most people would agree that Dogwood’s approach is much more honest.
In fact, the way that LeadNow is presenting their numbers on this issue has the potential to be extremely misleading. There’s nothing to indicate that the undecided voters in these ridings will distribute themselves proportionally to the decided voters. If anything, there’s a strong case to be made that they won’t – decided voters in an election this close are likely the most partisan voters, whereas the undecided are most likely to be lacking in strong party affiliation and therefore more open to persuasion.
In other words, by systemically excluding the presence of undecided voters from their projections, even in cases where undecideds make up a plurality of voters in a riding, LeadNow is potentially presenting a massively distorted picture of their polling data.
Whether or not that data is even accurate is another fair question. Margins of error are always inserted as a caveat with any posted poll, but in recent years in this country, there have been numerous egregious examples of pollsters making predictions which were just plain wrong. In the last federal election, the strength of the Conservative Party’s vote was underestimated by, on average, about 4% – pundits were virtually unanimous in predicting yet another Conservative minority, and few people saw the actual results coming. And when it came to predictions at the regional or provincial level, the pollsters did atrociously; in Ontario, the Conservatives beat their projected totals by a whopping 10%.
That debacle was followed closely by badly mispredicted provincial races: in 2012, the Wildrose Party of Alberta seemed to be a shoe-in to form government until Election Day, when the ruling Progressive Conservatives were returned with a huge majority, and then in 2013, a widely predicted NDP victory in B.C. absolutely failed to materialize. Along the way, we’ve also had wrong-but-not-that-wrong outings by pollsters in the Quebec election in 2012 and last year’s Ontario election; in both cases the polls accurately predicted which party would win but badly misjudged their eventual share of the vote.
Given the extremely poor performance of polling companies in recent years, why on earth should people be so confident about the results of a single riding-level poll that they’re willing to make their voting decision on that basis alone?
This isn’t to say that polling is hopeless or doomed to be wrong. Some companies, and some methods, are more effective than others, and in particular polling aggregators, if they weigh the numerous and complex factors correctly, can be powerful predictive tools. But I’m frankly completely unsatisfied that LeadNow’s method has the analytical rigour necessary to make extremely difficult calls in dozens of close ridings across Canada.
That, however, is exactly what they’re doing, and the decisions they make may have a massive impact on the final results.