The Polling Observatory Forecast #4: Conservative hopes recede slowly

As explained in our inaugural election forecast, up until May next year the Polling Observatory team will be producing a long term forecast for the 2015 General Election, using methods we first applied ahead of the 2010 election (and which are also well-established in the United States). Our method involves trying to make the best use of past polling evidence as a guide to forecast the likeliest support levels for each party in next May’s election, based on current polling, and then using these support levels to estimate the parties’ chances of winning each seat in the Parliament. We will later add a seat-based element to this forecast.

rob

This month’s Polling Observatory reported falls in support for both Labour and the Conservatives. Our forecast again finds the parties locked in a statistical dead heat, although Labour has edged up slightly, by 0.3 points, to 36.5%, and the Conservatives have slid back further, down 0.6 points to 34.9%. The continued stagnation in the polls is starting to harm the Conservatives in our forecast, with a slight widening of the gap between the parties.

Continue reading

Polling Observatory #37: No Westminster polling aftershock from European Parliament earthquake

This is the thirty-seventh in a series of posts that report on the state of the parties as measured by opinion polls. By pooling together all the available polling evidence we can reduce the impact of the random variation each individual survey inevitably produces. Most of the short term advances and setbacks in party polling fortunes are nothing more than random noise; the underlying trends – in which we are interested and which best assess the parties’ standings – are relatively stable and little influenced by day-to-day events. If there can ever be a definitive assessment of the parties’ standings, this is it. Further details of the method we use to build our estimates of public opinion can be found here.

pol

This month’s Polling Observatory comes in the aftermath of the European Parliament elections and the so-called UKIP earthquake for the electoral landscape. Despite much volatility in the polls ahead of those elections, with a few even putting the Conservatives ahead of Labour for the first time in over two years, underlying support for both main parties remained stable over the course of the month. Labour may have fallen early in the month in the run-up to the European elections, or the Conservative leads may have been the result of random variation. In any event, by the end of the month, we had Labour polling at 33.8%, just 0.2 points down on their support a month ago. The Conservatives are also broadly flat at 30.9%, 0.3 points below their standing a month ago. The Lib Dems have suffered slightly more of a post-election hangover, perhaps set back by infighting over the botched coup by Lord Oakeshott and the widespread ridicule over the Clegg/Cable beer-pulling photo op, on 9.3%, down 0.4 points. UKIP support remained stable at record high levels, as they enjoyed a moment in the limelight around the European Parliament elections. We have them rising 0.2 points on last month to 14.9%, their highest support level to date. Note that all these figures are based on our adjusted methodology, which is explained in detail below.

It is noticeable that while Labour’s support has been in decline for the last six to nine months (having plateaued for a period before that) underlying Conservative support has remained incredibly stable around the 31% level. In fact, setting aside the slight slump around the time of the last UKIP surge at the 2013 local elections, their standing with the electorate has been flat since its crash of April 2012 around the time of the ‘omnishambles’ budget. The narrowing in Labour’s lead over the past year is entirely the result of Labour losing support, not of the Conservatives gaining it. We have written at length previously about how the fate of the Liberal Democrats was sealed in late 2010, and as such it is remarkable that in this parliament there has been so little movement in the polls for the parties in government. The prevalent anti-politics mood out in the country and continued pessimism about personal/household finances has meant that neither of the Coalition partners have yet been able to convert the economic recovery into a political recovery. Instead, both are gaining ground relatively as the main opposition party also leak support, perhaps also succumbing to the anti-Europe, anti-immigration, anti-Westminster politics of UKIP.

As explained in our methodological mission statement, our method estimates current electoral sentiment by pooling all the currently available polling data, while taking into account the estimated biases of the individual pollsters (“house effects”). Our method therefore treats the 2010 election result as a reference point for judging the accuracy of pollsters, and adjusts the poll figures to reflect the estimated biases in the pollsters figures based on this reference point. Election results are the best available test of the accuracy of pollsters, so when we started our Polling Observatory estimates, the most recent general election was the obvious choice to “anchor” our statistical model. However, the political environment has changed dramatically since the Polling Observatory began, and over time we have become steadily more concerned that the changes have rendered our method out of date. Yet changing the method of estimation is also costly, as it interrupts the continuity of our estimates, and makes it harder to compare our current estimate with the figures we reported in our past monthly updates.

There were three concerns about the general election anchoring method. Firstly, it was harsh on the Liberal Democrats, who were over-estimated by pollsters ahead of 2010 but have been scoring very low in the polls ever since they lost over half their general election support after joining the Coalition. The negative public views of the Liberal Democrats, and their very different political position as a party of government, make it less likely that the current polls are over-estimating their underlying support. Secondly, a general election anchor provides little guidance on UKIP, who scored only 3% in the general election but poll in the mid-teens now, but with large disagreements in estimated support between pollsters (see discussion of house effects below). Thirdly, the polling ecosystem itself has changed dramatically since 2010, with several new pollsters starting operations, and several other established pollsters making such significant changes to their methodology that they were equivalent to new pollsters as well.

We have decided that these concerns are sufficiently serious to warrant an adjustment to our methodology. Rather than basing our statistical adjustment on the last general election, we now make adjustments relative to the “average pollster”. This assumes that the polling industry as a whole will not be biased. This assumption could prove wrong, of course, as it did in 2010 (and, in a different way, 1992). However, it seems pretty likely that any systematic bias in 2015 will look very different to 2010, and as we have no way of knowing what the biases in the next election might be, we apply the “average pollster” method as the best interim guide to underlying public opinion.

This change in our methodology has a slight negative impact on our current estimates for both leading parties. Labour would be 34.5% if anchored against the 2010 election, rather than the new estimate of 33.8%, while the Conservatives would be on 31.5% rather than 30.9%. Yet as both parties fall back by the same amount, their relative position is unchanged.  UKIP gain slightly from the new methodology – our new estimate is now 14.9%, under the old method they would score 14.5%. However, the big gainers are the Lib Dems, who were punished under our old method for their strong polling in advance of the 2010 general election.  We now estimate their vote share is estimated at 9.3%, significantly above the anaemic 6.7% estimate produced under the previous method. This is in line with our expectations in earlier discussions of the method in previous posts. It is worth noting that none of these changes affect the overall trends in public opinion that we have been tracking over the last few years, as will be clear from the charts above.

The European Parliament elections prompted the usual inquest into who among the nation’s pollsters had the lowest average error of the final polls compared against the result (see here). We cannot simply extrapolate the accuracy of polling for the European elections to next year’s general election. For one thing, these sorts of ‘final poll league table’ are subject to sampling error, making it extremely difficult to separate the accuracy of the polls once this is taken into account (as we have shown here). Nevertheless, with debate likely to continue to rage over the extent of the inroads being made by UKIP as May 2015 approaches, some of the differences observed in the figures reported by the polling companies will come increasingly under the spotlight. These ‘house effects’ are interesting in themselves because they provide us with prior information about whether an apparent high or low poll rating for a party, reported by a particular pollster, is likely to reflect an actual change in electoral sentiment or is more likely be down the particular patterns of support associated with the pollster.

Our new method makes it possible to estimate the ‘house effect’ for each polling company for each party, relative to the vote intention figures we would expect from the average pollster. That is, it tells us simply whether the reported vote intention for a given pollster is above or below the industry average. This does not indicate ‘accuracy’, since there is no election to benchmark the accuracy of the polls against. It could be, in fact, that pollsters at one end of the extreme or the other are giving a more accurate picture of voters’ intentions – but an election is the only real test, and even that is imperfect.

In the table below, we report all current polling companies’ ‘bias’ for each of the parties. We also report details of whether the mode of polling is telephone or Internet-based, and adjustments used to calculate the final headline figures (such as weighting by likelihood to vote or voting behaviour at the 2010 election). From this, it is quickly apparent that the largest range of house effects come in the estimation of UKIP support, and seem to be associated with the method a pollster employs to field a survey. All the companies who poll by telephone (except Lord Ashcroft’s new weekly poll) tend to give low scores to UKIP. By contrast, three of the five companies which poll using internet panels give higher than average estimates for UKIP. ComRes provide a particularly interesting example of this “mode effect”, as they conduct polls with overlapping fieldwork periods by telephone and internet panel. The ComRes telephone-based polls give UKIP support levels well below average, while the web polls give support levels well above it. It is not clear what is driving this methodological difference – something seems to be making people more reluctant to report UKIP support over the telephone, more eager to report it over the internet, or both. The diversity of estimates most likely reflects the inherent difficulty of accurately estimating support for a new party whose overall popularity has risen rapidly, and where the pollsters have little previous information to use to calibrate their estimates.

House

Mode

Adjustment

Prompt

Con

Lab

Lib Dem

UKIP

ICM

Telephone

Past vote, likelihood to vote

UKIP prompted if ‘other’

1.3

-0.9

2.8

-2.4

Ipsos-MORI

Telephone

Likelihood (certain) to vote

Unprompted

0.5

0.4

0.5

-1.6

Lord Ashcroft

Telephone

Likelihood to vote, past vote (2010)

UKIP prompted if ‘other’

-0.7

-0.8

-1.2

0.9

ComRes (1)

Telephone

Past vote, squeeze, party identification

UKIP prompted if ‘other’

0.6

0.0

0.2

-2.5

ComRes (2)

Internet

Past vote, squeeze, party identification

UKIP prompted if ‘other’

0.3

-0.7

-1.0

1.8

YouGov

Internet

Newspaper readership, party identification (2010)

UKIP prompted if ‘other’

1.9

2.1

-1.3

-0.2

Opinium

Internet

Likelihood to vote

UKIP prompted if ‘other’

-0.8

-0.9

-2.3

3.0

Survation

Internet

Likelihood to vote, past vote (2010)

UKIP prompted

-1.8

-1.5

-0.2

4.4

Populus

Internet

Likelihood to vote, party identification (2010)

UKIP prompted if ‘other’

2.3

1.5

0.2

-2.2

 

Robert FordWill JenningsMark Pickup and Christopher Wlezien

Polling Observatory #36 (April 2014): Farage’s Spring Uprising

GOOD ONE

 

 

 

This is the thirty-sixth in a series of posts that report on the state of the parties as measured by opinion polls. By pooling together all the available polling evidence we can reduce the impact of the random variation each individual survey inevitably produces. Most of the short term advances and setbacks in party polling fortunes are nothing more than random noise; the underlying trends – in which we are interested and which best assess the parties’ standings – are relatively stable and little influenced by day-to-day events. If there can ever be a definitive assessment of the parties’ standings, this is it. Further details of the method we use to build our estimates of public opinion can be found here.

 

This month’s Polling Observatory suggests that, not for the first time, Nigel Farage’s UKIP uprising has thrown a spanner into the Westminster political machine. The slow but steady tightening in the gap between the big two parties comes to a halt this month, as both lose ground while UKIP surges. Our estimate for Labour this month is 35.3%, down 0.9% on last month, and continuing the past year’s pattern of slow but steady decline. Labour have now hit their lowest point in the polls since Ed Miliband was elected leader of the party in the autumn of 2010. However, the Conservatives have not been able to capitalise on Labour’s continued decline as their support has fallen even more sharply this month, down 1.4 points at 31.6%. The Liberal Democrats have also seen no electoral benefit from their leader’s high profile combat with Nigel Farage over the EU – we have them down 0.2% this month at 7.4%.

All the momentum, and the media focus, lies instead with Nigel Farage and UKIP, whose surge in European Parliament polling is, as we predicted in earlier posts, being echoed in Westminster polling: UKIP stand at 14.1% this month, close to their highest ever share of 14.4% achieved in the aftermath of their local election success last summer. With UKIP currently favourites to win the European Parliament election, and become the first new party in nearly a century to top the poll in a nationwide election, further advances in the next few months look likely. It is likely that the upcoming Newark by-election will take place with UKIP support in domestic polling at record levels.

The current UKIP surge, however, does owe something to the unusual political context with a European Parliament election looming. In each of the past two European elections, UKIP has advanced in the polls, as their defining issue dominates the political agenda, and they expand beyond their traditional base of disaffected, struggling working class voters angry about immigration by winning over better off, often Conservative-leaning Eurosceptic voters who find UKIP attractive in European elections but less so in Westminster ones. In past election cycles, UKIP has struggled to hold on to these “strategic defectors” – voters who view UKIP as a vehicle for sending a message about the EU, but not a natural political home.

It remains to be seen whether the same dynamic will assert itself in the rather different political climate of 2014 – UKIP’s much stronger poll standing, greater media coverage, and improved financial and organisational resources may make it easier for them to retain new recruits. However, the Conservatives will certainly be hoping that the pattern of 2009, and (to a lesser extent) 2013 repeats itself – with UKIP’s spring surge fading away as voters return to the domestic political agenda and start to drift back to the mainstream parties, boosting Tory vote shares in particular.

Labour will hope that, with UKIP now targeting both main parties for votes, the decline in their support this month also reflects the temporary inflation in UKIP support, and that they too will recover more ground if and when UKIP decline than was true before. This is certainly possible, but it might be unwise to pin their hopes on a Farage fallback. The fall in Labour share this month was the continuation of a now well established trend that has seen the party shed 7 percentage points of support since their peak in the summer of 2012. Those within the party pushing for a “35% strategy” might be a little concerned that Labour support has already fallen back to this level, with over a year to go, and clear evidence of a downward trend in support. It is quite plausible that the remaining Labour backers are more firmly behind the party than those who have leaked away over the past 18 months, but the continued loss of support, and the precarious lead remaining, must give Labour plenty to worry about.

Those within all the parties wondering what today’s polling means for next year’s election will have some additional information to chew on next week when, exactly one year ahead of the general election, we launch the Polling Observatory forecast model, which utilises historical polling trends to estimate the likeliest pattern of results next May based on where things stand today. Tune in on May 7th to find out our current estimate of where support for the parties will be come election day.

Robert FordWill Jennings and Mark Pickup

Polling Observatory #35 : Politics, Fast and Slow

 

This is the thirty-fifth in a series of posts that report on the state of the parties as measured by opinion polls. By pooling together all the available polling evidence we can reduce the impact of the random variation each individual survey inevitably produces. Most of the short term advances and setbacks in party polling fortunes are nothing more than random noise; the underlying trends – in which we are interested and which best assess the parties’ standings – are relatively stable and little influenced by day-to-day events. If there can ever be a definitive assessment of the parties’ standings, this is it. Further details of the method we use to build our estimates of public opinion can be found here.

 

sadf

 

This month’s Polling Observatory is the first since the March budget, widely reported to have driven a bounce in support for the Conservatives. There is certainly evidence of a narrowing in the gap between the top two parties: our estimate puts Labour on 36.2%, down 0.8 percentage points on last month, and their lowest share since the end of 2010.  The Conservatives are up on 33.0%, up 0.9 points on last month and their highest share since early 2012. This leaves the gap between the parties at 3.2%, the lowest we have recorded since the parties briefly drew level in the aftermath of David Cameron’s European treaty veto in late 2011. It seems our speculation about a possible “voteless recovery” last month may have been premature. Meanwhile, in the undercard fight, UKIP come in at 12.1% this month, down 0.6% on last month, while the Liberal Democrats post a figure of 7.6%, up 0.4% on last month’s score.

The Conservatives’ renewed strength in the polls may owe something to a well-received budget from George Osborne, but when we take a longer view it becomes apparent that this latest narrowing is in fact the a continuation of a very gradual trend that has been proceeding, in stop-start fashion, for more than a year – as our excellent colleague Anthony Wells noted in his year-end review. In the early months of last year, Labour leads over the Conservatives were in double digits. This fell to under six points in the summer, before rebounding slightly in the autumn. Since November 2013, though, the narrowing has continued, and the lead has fallen from just under seven points to just over three.

Regular readers of this blog will know that we are cautious about identifying trends in what is often stable opinion, and also wary of using figures on polling leads, which are subject to more volatility and random variation. The underlying pattern here is however clear – the gap between the top two parties is steadily narrowing. Our main chart suggests this is the product both of rising Conservative support and falling Labour support, and also suggests that this is happening despite no decline in support for UKIP, who many argue are the main cause of recent Conservative weakness

Labour leads over the Conservatives since January 2013

sdgdfh

 

This slow but steady change in public opinion is not, however, the product of short term shifts in the political weather – the gentle ebbs and flows in public opinion over the Parliament do not have much relationship to the torrent of scandals, gaffes, policy announcements and message battles which constitutes the day to day world of politics for those in the media. This disconnect between the narratives about political events that are popularised by politicians and the media, and the slowly shifting tides of public opinion highlights the two-speed nature of politics. In Westminster, politics is a frantic day-to-day activity, conducted in the merciless glare of a 24 hour news cycle. Out in the country, politics is a marginal interest, on the fringes of attention, and opinions change slowly.

Those who make a habit of following political stories on newspaper front pages – or via highly charged political Twitter feeds easily forget that most voters are paying little to no attention to the events which are filling their days. Even major political set-pieces like the budget, Prime Minister’s Questions, or the Clegg-Farage debates barely register with many voters. Indeed, most of the population are at work during PMQs, unable to tune in to a television or a politics live blog, while the vast majority had much more interest in the latest goings on in Albert Square or Coronation Street than the duelling rhetoric of Nigel and Nick. Even economic news, which is of more immediate interest to many voters, tends to trickle down as a gradual process of diffusion, either in a general sense among the public that things are getting better or worse for the country or more directly as people feel better about the pounds in their pocket, and more eager to spend them. In practice, the fallout from political events is usually slow and sluggish, as it takes a long time for voters to notice and respond to things which are a long way from their everyday concerns.

Shifts in public opinion therefore tend to take place slowly, over long periods of time (with rare, but important, exceptions such as the Cameron veto and the “omnishambles” budget). Political commentators, however, exist entirely in the “high frequency politics” world, and interpret every day’s and week’s events in terms set by this world, projecting them onto the wider public, whose indifference is so far from their own everyday experience that it is hard for them to keep it in mind. Every minor event is expected to produce a sharp public reaction and every momentary twitch in the pulse of public opinion is attributed to the stories attracting Westminster buzz. As we write, much ink is being spilled over the public opinion implications of Maria Miller’s expenses, yet there is no sign that this is having any impact on the polls, nor should we expect it to when most voters have only a sketchy understanding of who Ms Miller is or what she is supposed to have done. This may change as the story gradually diffuses through the electorate, or if it takes a new turn. More likely, however, is that it will simply come and go, like many earlier stories before it, without leaving any lasting mark, aside perhaps from intensifying a little the “pox on all your houses” sentiment which has helped to fuel the rise in UKIP support.

It is important to bear in mind that the important, and lasting, changes in public opinion take place gradually, often too slowly to perceive without the benefit of months of data, because as election fever takes hold in the Westminster village, the focus on “fast politics” will become ever more intense. There will be many moments and many stories that are presented to us as ‘game-changers’ or turning points. In most cases they will be nothing of the sort, while the real action takes place away from the noisy slapstick comedy playing out on the news channels. Millions of voters will begin settling on their choices, nudged this way and that by forces which are, to some extent, predictable. The regularity of the tidal forces which move the “slow politics” of voters’ shifting opinions allow us to make modestly informed forecasts about the direction public opinion will take next, based on the lessons of history. Such forecasts are inevitably limited, particularly this far out – every election is in some respects unique, and we only have polling data for about 15 previous election cycles. Nonetheless, they tell us something useful about the likely direction of public opinion, and do better than guessing the outcome based on where opinion is now. Next month, with exactly a year to go until the general election, we will unveil our first forecast of the most likely state of public opinion on polling day next year, and how this will translate into seats in the House of Commons. There is still a long way to go, but our initial analysis suggests that the narrowing over the past year is in accordance with past trends. Tune in next month to find out if we expect it to continue…

 

Robert FordWill Jennings and Mark Pickup

Polling Observatory 34 (Feb 2014): A voteless recovery so far but still time to turn the tide?

final

This is the thirty-third in a series of posts that report on the state of the parties as measured by opinion polls. By pooling together all the available polling evidence we can reduce the impact of the random variation each individual survey inevitably produces. Most of the short term advances and setbacks in party polling fortunes are nothing more than random noise; the underlying trends – in which we are interested and which best assess the parties’ standings – are relatively stable and little influenced by day-to-day events. If there can ever be a definitive assessment of the parties’ standings, this is it. Further details of the method we use to build our estimates of public opinion can be found here.

On Wednesday, George Osborne will get up to deliver what is certain to be the sunniest Budget statement of his Treasury career. All the statistical indicators now point to a robust recovery – steady growth in GDP, falls in unemployment, and rising business and consumer confidence. The cloud on the horizon for Osborne and his Conservative colleagues is that there is, as yet, little evidence that the improving economic climate is improving the government’s political fortunes. With each passing month, the lack of any meaningful movement in the polls increases anxiety about a voteless recovery in 2015, with British voters unwilling to give the government any electoral reward for the much discussed “tough decisions” of the past few years. .

The story from February’s Polling Observatory estimates is much the same as in previous months. Once again, we can report that these suggest no dramatic changes in the status quo that has been in place since last autumn. We estimate Labour support this month at 37.0%, down 0.8% on January. The Conservatives come in at 32.1%, up 0.2% on last month. The overall gap has thus narrowed to under 5%, but both parties’ shares are broadly in line with previous estimates. There is not yet any evidence of sustained movement away from Labour or towards the Conservatives. We estimate UKIP support at 12.7%, up 0.9% on last month, and our highest estimate for the party since their peak after last May’s unexpected local election triumphs. The approaching European Parliament elections should provide a favourable environment for UKIP, so they may see further increases in support in the coming months. The Liberal Democrats have endured another poor month, and our estimates place them at 7.2%, the same as last month. Time will tell whether Nick Clegg’s new strategy of direct engagement with UKIP will help to reverse his party’s long polling slump.

There are, however, some green shoots of recovery in other polling. In YouGov’s regular tracking poll of government performance on the economy, the share saying the government handling the economy well now stands at 41%, up 16 percentage points on its low point in the summer of 2012, and the highest level seen since December 2010. The share rating the state of the economy as “bad” stands at 45% with the same pollster, which sounds pretty awful but is in fact the lowest figure recorded in this Parliament. For most of 2011 and 2012, between 70 and 80% of voters took a dim view of economic conditions. Finally, YouGov’s “feel-good factor” index – the balance of households expecting things to improve minus those who think things will get worse – now stands at -20. Once again, this sounds bad but we must factor in a general Eeyorish tendency in the British public – the last time the figure was this high was in the first half of 2010, when Labour staged a sustained rally in the polls in the run-up to the 2010 election. Other pollsters tell a similar story – IPSOS-MORI’s economic optimism index has shown a sustained run of strongly positive figures, the most positive since the beginning of New Labour’s government in 1997. Osborne can take additional comfort from his sobering experience in 2012 – when his “omnishambles” budget was followed by a sharp decline in Conservative poll ratings – while this was not the public reaction he was hoping for it did at least demonstrate that the budget is one of the rare political events with some capacity to move public opinion. With the public finances improving at last, he may hope to be able to offer enough goodies to voters to move the dial in the other direction this week.

While public opinion remains in a steady state, there are thus reasons for hope for government politicians: voters have noticed the improved economic climate, though they have yet to give the government any credit for it. The government can take further hope from polling history:  our analysis of historical polling trends, suggest that Labour still have plenty to worry about over the next 14 months. Our Polling Observatory forecast model, which analyses  historical polling trends to project the current state of public opinion forward to election day, suggests the likeliest outcome currently is a modest recovery in the Conservatives’ poll share, sufficient to make the next election a statistical dead heat in vote shares. This outcome would likely still translate into a small Labour majority because of biases built into the electoral system, but this projection highlights the very real possibility of another hung parliament if  Labour cannot maintain its poll lead in the coming months. The evidence from past election campaigns suggests that there is currently still plenty of time for opinion to change, and the economic opinion data provides one strong reason to suggest that it may do so. Each month from here on in is crucial for the parties’ electoral fates: the link between current polling and subsequent election results becomes steadily closer with each passing month in the last year of a Parliament, making a 5 point lead harder and harder to overhaul. There is still time for the parties, and their leaders, to change their electoral fates, but it is starting to  run out.

Robert FordWill Jennings and Mark Pickup