Data driven decision making in football: judging the process, not the result

Data driven decision making in football is often described as a matter of having more numbers. The harder part is deciding what to do with a probability once you have one. A forecast that gives a side a 40% chance of winning is a statement that the side loses more often than it wins, and also that it wins in a meaningful share of cases. A board that hears only the outcome will judge the decision by the result and learn the wrong lesson half the time.

The core discipline is to grade the decision on what was known when it was made. Rotating a tired centre-back before a congested week, setting a points target, or choosing a pressing intensity against a stronger opponent can each be sound and still end badly, because the match has a large random component. A useful meeting habit is to write the decision, the probabilities you believed and the reasoning on one page before kickoff. Afterwards, review the page first and the scoreline second. If the reasoning was good and the result was poor, you record that and change nothing. If the reasoning was weak and the result was good, you flag it, because a lucky win teaches the same bad habit as an unlucky loss.

Boards need expectations stated as ranges. Instead of a single predicted league position, ask for the spread of plausible outcomes and what share of that spread counts as acceptable. This protects the head coach and the executive alike: a season that lands in the lower part of a range you published in August is a different conversation from one that lands outside it. Probabilities also make squad rotation arguments concrete. If a fixture is one where you are already a heavy favourite on the facts, the cost of resting a key player is lower than in a fixture priced as near even, and the staff can say so in numbers rather than by instinct.

How do you tell whether a forecast or a data product is any good? Start with the questions that are easy to ask and hard to fake. Is the record public, and does it keep the last call before kickoff rather than a flattering earlier one? Is it graded on a stated definition of result? Does it report the base rate, meaning what a naive rule would have achieved on the same matches? And does it say how often its confident calls fail? Calibration is the central test: of all the calls made at roughly 70%, about seven in ten should come true, and the other three are the system working, not breaking. A forecast that is never wrong at 70% is miscalibrated, not brilliant.

A worked example shows how to read such a record. Kickwise publishes a probability for home win, draw and away win, built from a language model's reading of the match facts blended with the market's consensus where bookmaker prices exist, and updated when the confirmed line-ups land. Its public record keeps the last call made before kickoff, graded on the result after 90 minutes plus any extra time. Over the last 90 days, calls made at 65% confidence or higher were right 150 times out of 184 (82%), while the home side won 62% of those matches, so always backing the home team would have scored 62%. The comparison with the naive rule is what gives the 82% meaning.

The same record also shows why a short window is not a verdict. Over the last 30 days, across every confidence level, 504 of 881 calls were right (57%), with a Brier score of 0.56 against 0.636 for a baseline that gives every match the sample's own home, draw and away rates. Over the last 7 days it was 159 of 295 (54%), Brier 0.572 against 0.642. The Brier score rewards probabilities that were both confident and correct, and lower is better, so beating the baseline in both windows matters more than the headline percentage. The wobble between 57% and 54% is what a week of football looks like, and the stated limits apply: the draw is the hardest outcome to call, friendlies and thin-data matches are close to coin flips, and a 70% call is still expected to miss about three times in ten.

Where does this stop being useful? Probabilities do not tell you what to do, only how uncertain you are, and they say little about a single match once the sample is small. They cannot replace the judgement of a coach who knows a player is carrying a knock that never reached a report. They are weakest exactly where data is thinnest. And a forecast built for match outcomes should not be stretched into a claim about a player's long-term value or a manager's quality.

For the staff meeting, three habits carry most of the weight. Record the probability before the match, not after. Review decisions in batches of many matches rather than one weekend, so luck has room to average out. And keep a standing item that asks which of last month's calls we would make again with the same information. That item, repeated, is how the importance of data in football shows up in practice: not in any single forecast, but in a room that has learned to tell a good decision from a good result.