When it comes to forecasting elections, opinion polls that are often proved wrong bombard the public. Meanwhile, the betting markets get it right every time, so when they say Hillary Clinton will score a comfortable win over Donald Trump, we can take it as read.
Liberal Americans are becoming alarmed at the prospect of a Donald Trump Presidency. In his now inevitable battle with Democrat Hillary Clinton, Trump has taken a lead of 0.2% based on an average of polls, causing commentators to say, “it’s now time to panic”. Meanwhile, Professor Helmut, from Stony Brook University, has published a study claiming Trump has a 97% chance of winning the US Presidency. Norpath says his model has only proved inaccurate in one election since 1912 – the Bush-Gore race of 2000.
However, according to the leading British expert on election forecasting, liberal Americans have no reason to panic. Professor Leighton Vaughan Williams, who is Director of the Political Forecasting Unit at Nottingham Business School, says Trump has very little chance of winning the US election.
Poll data, he argues, consistently gets its political predictions wrong because of its weak methodology. There is, though, a vastly more reliable resource available. The betting markets have proved almost infallible in predicting the results of elections in both the US and the UK for decades. In addition, right now, they make Clinton a two in three favourite.
“Betting markets are much more accurate historically than polls,” he said. “There is a lot of money at stake, so the bookmakers carefully select which polls are worth looking at and which ones have a political bias. The Rasmussen polls have a consistent 2% Republican bias, but you don’t need to discard them, just factor the bias in. Other polls are so feeble that it’s akin to wildly throwing darts and hoping one hit the bull’s eye.”
The bettors themselves also influence the odds. Experts who are more likely to make correct predictions tend to stake the largest sums. The bookmakers know to ‘follow the money’ and pay attention to these punters when they set the odds. Some betting websites, such as Betfair, owned by high-street bookmaker Paddy Power, work as peer-to-peer gambling exchanges so the punters bet with each other.
Betting exchanges update the odds immediately, whereas polls take days to publish new data. “With the betting markets, we’ve created, almost by accident, a ‘high-tech’ crystal ball that taps into the accumulated expertise of mankind and makes it available to all,” said Professor Vaughan Williams.
The pollsters’ methods are comparatively crude. Their snapshots of voter intentions rarely identify candidates, but simply ask people for which party they would vote. Neither do they take into account the likelihood of tactical voting and late switches. Another flaw is that they always overestimate voter turnouts. Finally, they ignore the tendency towards a late swing to the incumbent candidates, or to the status quo. “The bookmakers always adjust raw poll data for this late swing,” he said.
Professor Vaughan Williams has published research comparing the success of bookmakers in predicting the winners of elections in both the US and the UK, with the many failures of the pollsters. No one studying his data would ever trust the polls again. Since 1985, the bookmakers have called every British general election correctly, whereas the polls have failed almost as often.
In the most recent General Election of May 2015, the pollsters showed the race between Labour and Conservatives as neck and neck. On Election Day, they made a confident prediction that the Labour Party would win significantly more seats. However, they were wrong. The Conservatives won a lot more seats, which was just what the betting markets had been saying consistently for months.
"The betting market was unmoved even by last-minute polls indicating a late swing to Labour. The polls were indicating most seats for Labour while the betting markets had the Conservatives as short as 6 to 1 on favourites to win most seats - meaning £6 to win £1. That shows great confidence,” said Professor Vaughan Williams.
The pollsters were interviewed everywhere in the British media and made excuses for the failure. However, it wasn’t the first time they had erred spectacularly. In the 1992 UK General Election, they predicted a Labour victory, only for John Major’s Conservatives to win by seven percentage points. Of course, the betting markets had forecast a comfortable Tory win all along.
Armed with his knowledge of the accuracy of the betting market forecasts, Professor Vaughan Williams told The Economist magazine two weeks before the 2005 UK General Election that there would be a 60-seat majority for the Labour Party. In a BBC live debate, a leading pollster wanted to bet him a large sum that his own figure of a majority of 100 for Labour would be more accurate.
“I declined the bet and saved him some money. The Labour majority was 66 seats. The assumption here is that the collective wisdom of many people is greater than the conclusions of a few,” said Professor Vaughan Williams. Similarly, in 2010, the polls had the Tories 12 points ahead on Election Day, whereas the betting markets predicted a hung parliament. Needless to say, betting markets were spot on again.
In American politics, the pattern is identical. In one famous study, the betting markets predicted correctly almost every US presidential election between 1868 and 1940. In only one year, 1916, did the candidate favoured in the betting the month before the election end up losing. However, that was in a very tight race and the markets got it right on the day.
A more recent example was in the 2012 US Presidential Election when both Gallup and Rasmussen called it for Mitt Romney by 4%, and the national polling average had the candidates tied. Meanwhile, Barack Obama was very short odds-on to win with the bookmakers and triumphed easily by a 4% margin.
In 2004, one betting exchange hit the jackpot in the US when their market favourite won every state. Professor Vaughan Williams compared this result to a “tipster calling the winner of 50 football matches in a row simply by naming the favourite”. Meanwhile, Gallup failed to forecast the winner of the 2004 Presidential Election.
The betting markets are just as successful at forecasting the results of referendums. In 2014’s Scottish Referendum, the polls were tight and some called it for independence. However, the betting markets pointed unwaveringly to a solid ‘No’. Likewise, in the 1995 Quebec Separation Referendum, the final polling showed ‘Yes to separation’ with a 6% lead.
In the event, ‘No to separation’ won by 1%. Naturally, the betting markets had seen that all along. Finally, in the Irish Gay Marriage Referendum of 2015, the polls pointed to a 70%-30% win for ‘Yes’, whereas the betting markets anticipated a 60%-40% split. In the event, ‘Yes’ won by 62%-38%.
Professor Vaughan Williams also respects the work of the Sabato Crystal Ball Project at the University of Virginia’s Center for Politics. The project uses expert judgment on a state-by-state level and much of the same methodology as the bookmakers’ websites.
They predict a strong win for Clinton by 347 votes to 191 for Trump. By comparison, Obama won 332 in 2012 to Romney’s 206. The betting and prediction markets, Professor Vaughan Williams says, tell a broadly similar story. Another source of accurate forecasts is the PollyVote project, which combines evidence from polls, prediction markets, and econometric models. It has been spot on with its forecasts since 2004 and predicts a Clinton win by 52% to 48%, the same as Obama’s defeat of Romney.
Though arguments will rage for months, the November 8 Presidential Election is already predictable, Professor Vaughan Williams says. We can expect Trump to launch attacks on Clinton’s integrity and bring up the scandals that have stuck to her name. He will use some of Bernie Sander’s criticisms about her close ties to the big banks and the political ‘establishment’. He will call her ‘crooked Hillary’ and the fight will get dirty. However, the bookmakers have already factored all these things into their predictions.
“Both Clinton and Trump are known quantities. If Bernie Sanders were running, it might be a different matter. He has not been victim to the sorts of personal attacks levelled at Trump and Clinton and it’s possible Trump could dent his ratings by accusing him of being a Communist, spending his honeymoon in Russia and wanting to abolish Christmas, and so on.
Clinton hasn’t attacked him viciously because she will need his supporters to vote for her. But it’s hard to see what Trump can throw at Clinton that isn’t already in the open,” he said.
For an accusation to influence popularity ratings, it has to be unpredictable and change the broad perception of the candidate. In the 2010 UK general election Labour leader, Gordon Brown was heard on camera accusing a supporter of being a “bigoted woman”.
Despite widespread negative publicity, it had no effect on his ratings as he was already perceived as “a bit of a bully boy leader”, Professor Vaughan Williams says. The Bill Clinton scandal with Monica Lewinsky also barely affected Clinton’s ratings, although a similar event would prove disastrous for a politician such as Obama whose appeal is based on being a ‘decent family guy’.
In the case of both Clinton and Trump, their views are so well-known it would be hard to change them now. Mitt Romney managed to achieve this trick in 2012. He went from calling himself a “severe Conservative” in the Republican race, to claiming to be a “compassionate Conservative” in his early debates with Obama. Romney got away with his transformation because his views were less well-publicised.
Over in the UK, there is also trepidation about the outcome of the June referendum on European Union membership. According to the Economist Brexit poll tracker, 40% of voters back the remain campaign, 39% want to leave and the rest are undecided. Anyone following the polls would expect it to be a close-run thing. However, the bookmakers disagree. Most of them rate a ‘vote to remain’ a near certainty at 7-1 on. A punter would need to bet £70 to win just £10.
In addition, Professor Vaughan Williams says wins for Clinton and the Brexit Remain campaign are even more likely than the odds imply. The phenomenon of “favourite longshot bias,” means bettors tend to overvalue “long shots” and undervalue favourites, he says. “This means that if the odds suggest the favourite is going to win 80% of the time, it’s probably more like 90% of the time,” he said.
As for Professor Norpath’s claims that Trump is a near certainty to win the Presidential Election, Professor Vaughan Williams dismisses them out of hand. “If he really believes that, he will become a millionaire just by betting on Trump and he can retire after the election. However, I’d rather listen to the betting markets. It seems only sensible to make the most of them when they are so consistently accurate.”