World Cup: Knockouts are difficult to forecast

The approach we took was simple. Teams for each country were ranked according to the transfer value of their 23 man squads. Higher valued teams were assumed to beat lower valued teams.

Spain’s loss to Russia was not the only game since the start of the knockout phase where the more highly valued team lost. Portugal lost to Uruguay, but the Uruguay team was ranked only one position below Portugal on value. In the four knock out games so far, the value ranking approach worked with France defeating Argentina and Croatia beating Denmark.

A score of two out of four might suggest the approach gives a prediction that is no better than chance. However, there is a way to challenge this.

It’s important to realise forecasts and models work best when there is repeated rather than one-off or knockout events. Looking at results from the group rather than the knockout phase gives more useful information.

The group stage of the tournament consisted of 48 games. Every team played three games with the aim of accumulating enough points to get into the upper half of their group.

In this stage, the higher valued team won 26 games – a 54% success rate. But nine games were draws. The value only approach does not allow for draws. Excluding those, the success rate increases to 67%.

Still, that may not have predicted which two teams would top each group. The approach was correct for four of the eight groups – a 50% success rate. That's actually pretty good. My colleague Sebastian Franke notes that for every group of four teams, there are six possible combinations of two coming out on top. A 50% success rate against a one-in-six chance is more than acceptable.

A second approach looked at 'value for money'. This compared the difference between a team’s value and its FIFA ranking. A team with low value but high FIFA ranking was considered better value for money. This approach was less successful. It had a 44% correct response rate for all games and 54% excluding draws. Peru, the best value for money team, did not progress to the next stage of the competition.

Arguably the biggest surprise in the tournament so far has been Germany’s failure to move out of the group stage despite playing three games. Germany was one of the most highly valued teams in the competition, has the highest FIFA ranking and is the current title holder. Losses by Spain and Portugal in one-off, knock-out games are less surprising. Germany disappointed multiple times. Spain and Portugal had only one chance on Sunday. The Iberian countries were exposed to unpredictability.

Tournaments, such as the World Cup, are designed to allow for unpredictability. Although my colleagues in Germany, Spain and Portugal may disagree, it's a big part of their attraction.

On a more serious note, there are useful lessons to learn from the World Cup. That the German and Spanish teams have been eliminated does not mean they're not good – even great – football teams. Luck did not run their way. Uruguay and Russia are worthy winners. They have used their luck well.

Economist Robert Frank argues that luck plays an important role in life and the economy. Tournaments can be cruel because they give an even greater role to luck.

Recognising when you're competing in a market that is essentially a tournament can be important in understanding why your plans don't work out as you hoped. Many job interviews can be considered tournaments. If you're in a tournament situation, it may be worth persisting with something you want to do.

I suspect 2001 Nobel prize winning economist George Ackerlof recognised he was essentially in a tournament while trying to have his influential paper “The market for lemons” published. The paper challenged the idea of perfect knowledge in markets by using the example of sales of second-hand cars. The seller has more information than the buyer, who could end up with an unreliable vehicle – a “lemon” in common language. The seller would know this but the buyer wouldn’t. The information is asymmetric.

Ackerlof recounts in his Nobel acceptance speech that the paper was rejected by major journals as trivial and incorrect. Ackerlof persisted, submitting his paper to the Quarterly Journal of Economics where it was finally published.