Now that the official results of the first round are known, it is time to reflect on how accurate voting intention polls have been — both in an absolute sense and as compared with their performances in 2007. Did pollsters do better or worse than in 2007?
Five years ago, the horse race polls conducted in the final days of the campaign correctly predicted the exact order of the four main candidates (Sarkozy, Royal, Bayrou, Le Pen). Together with the absence of a ‘huge surprise’ –such as Le Pen in 2002 or Balladur in 1995–, this ‘superfecta’ forecast led many to consider that the 2007 polls had been reasonably good. Yet, this view must be nuanced.
First, the polls largely missed the margin between Sarkozy and Royal, mainly because the former’s score was underestimated by 3 per cent on average. Second, pollsters proved yet again unable to forecast accurately the FN vote: learning from the lessons of 2002, most of them adjusted Le Pen’s numbers upwards, which led to an overestimate of over 3 per cent.
Did pollsters pop open bottles of champagne last night? Like in 2007, their last week forecasts have predicted the final order of the five main contenders across the finish line. Taking the average of the seven polls published in the campaign’s final week gives a very accurate picture of the balance of forces between the two frontrunners with a 0.8-point variation in both Hollande and Sarkozy. This year the main source of error is found in the two peripheral ‘protest’ candidates: Mélenchon (-2.7) and Le Pen (+2.1).
Final week polls and official results
Overall, a ‘crude’ measurement of polling error can be taken from the sum of absolute errors for all candidates: in 2007, the total of errors was 12.2 across all twelve contenders. This year the same calculation gives an eight-point error for the ten candidates in the presidential race. In sum, the global performance by France’s pollsters has improved compared with 2007. Pollsters could –and will– legitimately claim that even the largest differences are within the margin of error.
The Marine Le Pen vote
Key to this year’s election was of course the ability by pollsters to give an accurate estimate for Marine Le Pen: after nearly 40 years of reign, her father finally stepped down from party leadership in early 2011. There have been speculations about the electoral impact of the so-called ‘de-demonization’ strategy pursued by his daughter. During the campaign, pollsters made no secret of the fact that their raw data had less of the ‘social desirability bias’ traditionally associated with the FN vote in France.
This is corroborated by the election returns: whilst Marine Le Pen has fallen short of replicating the 2002 political ‘coup’ of progressing to the run-off, she has achieved the best score ever for the extreme right in France, surpassing even the total number of votes won by her father in the second round of the 2002 election (6.4 million votes in 2012 compared with 5.5 million ten years ago).
With an average score of 15.8 per cent in the final week, polls have failed to measure the exact level of support for the FN leader, as have in fact most commentators. The symbolic significance of this is hardly debatable and was probably further amplified by some pollsters unwisely giving Le Pen up to 20 per cent of the vote in their early evening estimates. From a methodological perspective, however, the final week polls have managed to reduce the level of uncertainty in predicting the extreme right: in absolute terms, their average 2012 error (2.1) is smaller than both their 2002 and 2007 misforecasts (respectively 3.3 and 3.4).
CATI vs CAWI
In a previous post we provided some empirical evidence for a consistent bias in internet polls (CAWI) relative to telephone polls (CATI). An updated analysis of all 119 horse race polls published since June 2011, controlling for the time-period in which they took place, confirmed that Hollande tended to score around 2 per cent less in internet polls, and Marine Le Pen 1.7 points higher
One first observation is that internet polling seems to help break the ‘spiral of silence’ that prevents voters to disclose their preferences for protest or extreme political forces. In yet another plea for transparency, pollsters should agree to publish their raw data to allow for a more scientific assessment of such effects which are anything but trivial. In the light of the massive overestimation of Marine Le Pen’s vote by some of the first exit polls of the evening, most polling organisations are hardly in a position to argue that their reticence about releasing their adjustments serves a public good.
Differences have been waning in the most recent surveys –probably due to pollsters looking at each other’s forecasts to smooth their own numbers. Such differences have become negligible in the final week, with the exception of the numbers given for the two finalists: in anticipating a Hollande / Sarkozy tie, internet polls have failed to show the real margin between the frontrunners, mostly because of their underestimating the PS candidate (1.3).
Predicting the left / right balance of forces
Finally, contrary to 2002, the left versus right balance was almost perfectly estimated in 2007 (less than half a percentage point for the left bloc as a whole). This year, polls were anticipating a surge in electoral support for the left: in the final week, the forecast for the total score of the five candidates representing the left camp was up to an average 46 per cent.
The final results show that such wave of popular enthusiasm was overestimated (-2.2), mostly because of the amplification of the Mélenchon phenomenon in the polls.
Professor Jocelyn Evans and Gilles Ivaldi: guest bloggers from 500 signatures