The use of data analysis in football has exploded over the past ten years and like any start-up industry it has faced many challenges. Ian will share some lessons he has learned along the way. Football is a combination of skill and luck, and understanding the difference between the two is critical to sustainable success. Ian will discuss how they’ve attempted to disentangle the signal of skill from the noise of luck, and how this can give teams an edge over the competition.
Auto-generated transcript - may contain errors.
Tap a timestamp to jump the video.
When I used to go evangelizing at sports conferences, this was the talk title I used to use. My background's physics, mathematical and statistical modeling. And it was very mysterious to sports organizations why a club like Liverpool would hire someone like me. These days, sports organizations know very well why they should hire a person like me.
So instead, I'm gonna ask why the heck am I talking at a tech conference? Hopefully, I'll convince you that I should be talking at a tech conference. And the easiest way to convince you is by telling you that my story's already been told in a different sport.
There was a book and a film called Moneyball, in which a suave young baseball manager called Billy Beane, who's played by Brad Pitt. He believed that baseball was fundamentally flawed in the way it measured performance. And he thought you could buy more wins for less money.
The geek that he got to do the analysis to prove this was played by Jonah Hill. And just to be clear, I'm not the Brad Pitt character in this story. I'm the Jonah Hill character. So that's Moneyball. Why does it apply to a tech conference?
I want to talk to you about three things today. The first thing is data analysis in football is really a startup industry, just like a lot of tech businesses and startup industries. And I've learned a lot of hard lessons along the way. So I shall tell you the story of how the football data industry started.
And hopefully, you can learn something from it as well. The second thing is data analysis in sports at its core is about separating skill from lack or signal from noise. And that's exactly what businesses needs to do. Understand what really matters, I. E.
The skill element and don't focus on the things out of your control, lack. The third thing I wanna talk about is we spent a lot of time trying to understand where we could maximize our impact of data analysis in football. And I think that's true for every business.
You have to work out where am I going to spend the time that will make the biggest impact? Where is the leverage? So I'll talk about those three things from a football point of view. So this was our pitch fifteen years ago. So I used to work for a startup company called Decision Technology.
We had some data about football where we thought we could improve teams' performances through using that data. If you make any sort of financial investment, you see something like this. Past performance is not indicative of future results. Our pitch was just that when it comes to sport, past performance is indicative of future results.
You can look at players past performances and they will continue to perform or they should continue to perform in a similar way. So that was our belief. What was the evidence behind that belief? First, it's already happened in basketball and baseball. I suppose the same is true in the tech industry.
If you want to know what will happen in the UK in ten years time, you look at what happens in America today. It had already happened by the time we were making this pitch. Second, the data is now available. Two thousand and seven was the first time people could buy large quantities of data that was good enough to analyze performances of football players.
Third, I knew from personal experience that gambling firms were making large amounts of money using data analysis. In fact, in two thousand and nine, Brighton was bought by a professional gambler. In twenty twelve, Brentford was bought by a professional gambler. Those clubs were not Premier League teams at the time.
They are today. Next, football clubs have got a dreadful track record of player transfers. I could give many, many examples. Most transfers fail. That's still true today. Clubs basically burn money buying and selling players. And finally, what data analysis does is separate out underlying performance from short term luck.
Looking from the outside, and now I say from the inside, football clubs make decisions based on short term luck rather than on underlying performances when it comes to signing players, selling players, firing managers, any number of decisions. If that evidence isn't enough for you, then you can ask Billy Beane, the hero of money ball.
In two thousand and three, he said about Premier League football in England, there's so much emotion going into football. There must be a lot of inefficiency. That means there's a lot of opportunity. So it was an easy sell, right? So I'll give you the history of how it actually happened.
There's a quote from an old union leader, Nicholas Klein, that was used by Gandhi and then used by Donald Trump in twenty sixteen. But it really outlines how data analysis in football has gone. And I think it's probably true of a lot of tech startups.
First, they ignore you. Then they ridicule you. Then they attack you. And then they build monuments too. So this was definitely my experience of the startup industry of football later. So we take it stage by stage. Ignorance is the first stage. So this is one of my heroes.
It's a guy called Charles Reep, who was an accountant in the RAF. Nearly everyone ignored him. Nineteen fifty three, he hand collected five seventy eight games of data over a fourteen year period. Nineteen sixty eight, he published probably the first soccer analysis result, which was the average goal chance from a shot on goal is eleven percent.
One in nine shots become goals. Seems obvious now, but no one knew that at the time. He needed to collect the data to prove it. In nineteen ninety seven, again based on hand collected data, he published something which is now called expected goals.
The same paper also had the first version of something that's now called expected possession value. They were yeah, they're the foundations of soccer analysis today. And that's his nineteen ninety seven paper in The Statistician. Is it ignored by all football clubs? This is one of the figures from that nineteen ninety seven paper.
It's the first expected goals model. And all this shows is that if you're on the penalty spot in open play, you've got about a twenty percent chance of scoring if you take a shot. If you move outside the box anywhere on this line, you've got one percent chance of scoring if you take a shot.
That's what expected goals is. It weights each shot by the chance that it becomes a goal. We've had some success. He was employed by wolves in the 50s and 60s. And throughout his career, 70s and 80s was employed by some smaller clubs. But he was mostly ignored.
There were many reasons why he was ignored. But to my mind, the biggest one was the lack of raw material. So you need data in order to do data analysis. It took fifteen years to collect six hundred games worth of data. So he really couldn't scale what he was doing.
He couldn't tell you about how a player on another team was performing. He said, I've had to go to all of the games to record the data. The sort of data that's historically been available is this. You get a full time score. You know which teams are playing, full time score, a list of the players that were involved, who scored, who got a yellow card.
It's not enough to tell you anything about player performance. Around about two thousand, more data started becoming available. These are some match statistics for Liverpool players about the number of passes that they attempted in a game, the number of those passes that were successful, how many passes were long, and so on.
This is still not good enough data to tell you anything about player performance. Or it doesn't tell you much about player performance. Two thousand and seven was the first time the event data became available. That's a history, ball touch by ball touch, of who did what, where on the pitch, and what happens next.
So here, number eleven, De Maria recovers the ball, passes it to seven, who then runs about a bit. There's a few more passes, gets passed back to De Maria, who then scores a goal. That is enough data to say something about player performance.
In two thousand and seven, you could buy about two thousand games of this data. Today, you can probably buy about thirty thousand games per year of this data. REAP took years to collect five hundred games of this style of data. Twenty thirteen, the Premier League started paying for tracking data, so we get access to all of this this data for all Premier League games.
Here, you can see what every player is doing twenty five frames per second. Definitely enough to tell you about player performance. The next element is skeletal data that we're now getting for champions league games. So twenty one points, twenty five frames per second per player.
Interestingly, the data was being collected but it wasn't available. There were gatekeepers. There was a company called Prozone that collected event data and tracking data and had deals with lots of Premier League teams. But they didn't give you access to the raw data.
You have to access it through their terrible software, and you could never see the raw data. You could only see stupid aggregations of the data. So they were really gatekeepers. And that secret data did a lot to hold back analytics. In two thousand and seven, a company called Opta were a media company.
And because of that, they were just happy to sell the raw data to anyone. And that's how we really got started. So yeah, this secret data is huge barrier to progress. Breaking it down was needed before analytics could make a difference. For the general public, that opted it was incredibly expensive.
But for a Premier League club, it's a reasonably small amount of money. And it gives a big advantage. Okay, so that's ignorance. Next is ridicule and attack. So I'm not gonna go through them in detail, but we've got many, many scathing articles about us in the British press.
Liverpool are idiots for trying to use data analysis was the summary of of these comments. Now British football is very traditional reactionary. Anyone doing anything new at all has to be ridiculed. It felt very safe to ridicule us because our results weren't particularly good in the early days of trying to apply a data analysis.
Liverpool had something called a transfer committee. All this meant was that the manager, traditional scouting and data analysis all had a say in player transfer decisions. It sounds very obvious as an idea and something that can only add rather than take away. But the football press didn't like it.
We also had attack internally. Harry Redknapp on the left was Spurs manager when I worked for Spurs. My company wasn't allowed to be in the same building as Harry for many years. He didn't know of our existence. Brendan Rodgers was the first manager I worked with at Liverpool.
He had a modern facade, but he was equally as old school in his thinking as Harry. English traditional managers like this, they demand full control over the football club. They want the final say and the only say on any player decisions. That's the way it had always been and they didn't wanna compromise.
Finally, got some success. And really, it was a cultural reason for success. Managers who worked in European clubs were used to working in this collaborative way. They'd work with a director of football and the chief scouts and other staff members to come to a collective agreement about what we should do about which players to buy and sell.
So that's the history of my business. But I have not really told you any details about what we do. So really everything is about trying to distill football data into two categories, skill or things that are under control and repeatable, and lack or things that are out of your control or unrepeatable.
Here's a really simple example. Twenty eleven, twenty twelve, there was a Premier League goal glut. There are two ninety five goals in ninety nine games. The esteemed members of the football press, always eager to whip themselves into a frenzy, said, it was the death of defending in the Premier League. There were loads of new strikers in the Premier League.
What was the real story? I had a very simple model of team strengths that just estimated how many goals a team may score or concede just based on their past performances. And I simulated the first ninety nine games of the season. This was the result.
Most of the time, well, most likely would be two seventy five, two eighty goals. Two ninety five goals, yeah, it's above average, but it's a one in six chance. So all these articles were written about a one in six chance. The real story was nothing to see here.
And that season ended with three more goals than the previous season. It really was nothing to see. Here's a more important example. In September twenty fifteen, Liverpool wanted a new manager. This guy, Jurgen Klopp, was top of our preferred candidates list. That January, his team Dortmund were in the relegation zone and the German tabloids called them absolute garbage.
So we needed to understand what had gone wrong at Dortmund. Coincidentally, I'd been in Frankfurt that March giving a talk to German video analysts. Dortmund had recovered to tenth in the table at that point. This is just the German league table in March twenty fifteen.
They'd recovered to tenth, but the opinion of the analysts was Dortmund were done and Juergen was done complete disaster. I also showed our analysis of the team's performances based on expected goals that I showed you from Charles Rip before, a similar model to what Rip used.
And that showed that Dortmund might have been tenth, but they comfortably had the third best performance in the league based on expected goals. So our conclusion was, if you take away the luck or the unrepeatable things that happen, lucky goals are scored and lucky goals are conceded, Take that away and look at the underlying performance.
Dortmund season was still the third best in Germany. And in fact, if you look at a wider definition of team performance, it was still the second best team in Germany. These are our predictions for the table last season based on the performances that we saw.
Dortmund finished seventh. But we said, based on the performances in game, they had a forty five percent chance of finishing second that season. So Dortmund's Echter Schroth, the terrible performance of Jurgen and his team, we put it down to bad luck. Here's a more technical view of that story.
This looks at how many points you've overperformed one season versus how many points you've overperformed next season. Dortmund's terrible season this year, a huge sixteen point underperformance was followed by a moderate overperformance. If teams could consistently overperform, you'd see lots of teams up here in this top corner.
I overperformed this season and I overperformed next season. But you don't. It's a random cloud of points. And spare a thought for Hertha Berlin in two thousand and eight, 'nine. A mid table performance led to twenty points more than expected in Champions League.
Next season, the same performance, more or less, led to last place in the Bundesliga and relegation. So finally, I want to talk about maximizing impact. Understanding skill and luck is crucial to maximizing impact. So as a data analyst turning up at a football club, you need to ask yourself, where should I focus my efforts?
I could work with the medical and fitness department to keep our players on the pitch for longer, less injuries. I can work with the academy. Let's try and produce more players for the first team that we don't have to pay transfer fees for.
I could work on match analysis and come up with some new tactic that will give us an advantage against upcoming opponents. Or I can work on recruitment and try and find the best players to sign in the future. What's each of these things worth?
Well, medical and fitness, maybe I save a few injuries. That means over the whole squad, the equivalent of one player, extra players available. It's quite valuable. In the academy, I can save one transfer fee for every academy player that comes through. That's that's valuable as well. Right?
My super new tactic, I could get five more points per season. That's very generous, very generous to say that it'll gain me an extra five points. And as soon as the opposition have worked out my new tactic, it's back to the drawing board and I need to work out another new tactic.
Recruitment, there are lots of player transfers per year. We buy three or four players per year, sell three or four players, have to decide whether to extend contracts of three or four players. So there's many of those decisions. How much is each of those decisions worth?
This is a chart of the distribution of player quality in each team in the Premier League. So you can see on average, the better teams, Manchester City and Liverpool, have got higher quality players than the teams that ended up getting relegated, unsurprisingly. But there's quite a wide distribution of quality.
Some of the best players at Leeds United or Aston Villa, they would easily fit in at Manchester City or Liverpool. This is just the same thing on putting everyone on the same scale. So comparing them to their teammates rather than comparing them to the league.
And you can see the spread in quality of players is about the same as you go down the Premier League. Let's work out what happens if I can take one player who is below my average and replace him with one player who's above my average.
A thirtieth percentile player costs you zero point zero four goal difference. Seventieth percentile gains you zero point zero four goal difference. It's quite so changing them gains you zero point zero eight. It's quite easy to show that point zero zero eight goal difference per game is about two points per season.
So that's two points per season per decision that you're making. And remember, three or four purchases, three or four sales, three or four extension decisions. If you can replace a fifteenth percentile player with an eighty fifth percentile player, that's four points per season.
So recruitment really is where the action is. If you don't believe performance argument, you can look at the money arguments, right? The word money ball has got money in it. Money ball is not about improving performance. It's about how do you maximize your improvement for minimum cost?
Where the teams spend their revenue? So this is revenue of Premier League teams up to one hundred percent. Blue is what they spend on player wages. Pink is what they spend on player transfer fees. Ninety percent of club revenue is spent on the playing staff.
So if you're doing money ball, the playing staff is the only place where the money is. So it's the only place you can make a difference. This next slide is just showing that Premier League clubs tend to spend more than they earn on transfer fees.
The other argument is that as you spend more on wages, the points of your team increases. There's a question about causation and correlation there, but I don't have time to get into it. So hopefully, I've proved to you that recruitment is a place where the leverage is.
I'll finish off by talking about a couple of recruitment stories from Liverpool's past. This chap, Mo Salar, is a Liverpool player for the last five years. He's been top scorer in three of those seasons. Chelsea signed him in twenty thirteen. He failed at Chelsea.
We signed him in twenty seventeen for forty two million euros. To put that in context, the same summer, Arsenal signed Lacazette for fifty three million. Chelsea signed Morata for sixty six million euros. Manchester United signed Lukaku for eighty five million euros. None of those three players are still at their clubs.
This is the number of minutes he played and some basic data about him. So he'd been two and a half seasons in Italy, scoring and assisting at a really high rate. Our analysis of his underlying performance, which goes beyond these raw numbers, also agrees that he was an excellent player.
But he was available because it was seen that he failed in England. He'd failed to get on the pitch at Chelsea. But in that low number of minutes, he's actually scored and assisted at zero point five two per ninety minutes, which is fine for a twenty one year old.
The data analysis allowed us to understand that those Italian performances were based on underlying skill of the player. And the data analysis allowed us to just ignore the negativity surrounding Salah failed at Chelsea. We were convinced that he would succeed in the Premier League.
There's also some luck involved. So luck's not repeatable, but it's quite often necessary for success. So Salah turned up at Chelsea where he had to displace Eden Hazard in order to get minutes, which wasn't gonna be happen because Hazard was a superstar of the Premier League.
His manager, Mourinho, famously prefers to play older players. So it's another reason Sala didn't get a chance. If he did get a chance at Chelsea, there's no way we would have been able to sign him in twenty seventeen. We also had some luck in that his team in Italy, Roma, were under some financial stress.
And so they were motivated to sell. So even though I'd like to say our skill in really understanding Salaz performances led us to signing him. We also needed the luck to be in place for the signing to actually happen. I say luck, it's things out of our control.
Then my other favorite player is Coutinho. He again played for Liverpool for five years. Again, his raw numbers in terms of goals plus assists per ninety minutes were excellent for Liverpool, and his underlying performances were excellent as well. We were back in the Champions League for the first time in like three or four years.
And he was playing brilliantly, and we definitely didn't want to sell him when Barcelona offered us one hundred and thirty five million euros for him. Now, again, the data analysis suggested that was an overvaluation. So he's a Premier League superstar, but one hundred and thirty five million would be the second highest transfer fee ever at the time.
Our team was also unbalanced at the time. We had four brilliant attackers, Coutinho plus three forwards, one of whom was Salah. And our defense was not so great. So because Barcelona were offering to overpay for Coutinho, that allowed us to redistribute his qualities to places that needed much more quality in the team, defense and goalkeeper.
Again, luck was needed. Barcelona were under huge pressure to replace Neymar, which was why they were willing to overpay. We were under no pressure at all to sign Coutinho. So that's the only reason we were able to wait and say, no, you're not offering it.
Or well, no, we're not interested in selling. That drove the price up. Our preferred replacements, Van **** was playing at Southampton who were not a Champions League team. So he was much more attainable than if he was playing at a Champions League team.
And similarly, the goalkeeper that we wanted to sign was a smaller team that was again under pressure to sell. So again, there was some skill involved in understanding the factors that led to Coutinho's performance, the relative strengths and weaknesses of our team. And to understand that one hundred and thirty five million euros was an overvaluation, so we should sell.
But there was also luck involved because if Barcelona didn't have two hundred million euros in the bank from selling Neymar, they wouldn't have been offering us that huge amount of money to buy Coutinho. Coutinho's story at Barcelona is also quite interesting. Here is I've just called it rating, but it's our analysis of his underlying performances for the club.
And you can see they're kind of very correlated with his raw output in terms of goals and assists. So he'd been contributing about zero point two goal difference per ninety minutes to Liverpool over the past few seasons. He moved to Barcelona in January twenty seventeen.
And in that first half season at Barcelona, he was really successful. So his raw data, zero point nine goals and assists per ninety minutes was the same as it had been at Liverpool more or less. And the underlying performance was also good. Now in twenty eighteen, his first full season at Barcelona, his raw numbers dipped a lot.
Zero point three was lower than we'd ever seen him score an assist at Liverpool. But if you look at his underlying performances, they didn't change too much. Zero point two one, okay, it's a little bit lower than previous seasons, but well within the range of expected variation in player performance over two thousand minutes.
Looking from the outsides, I don't know what the truth is. But what happened was it was decided eighteen months after spending one hundred and thirty five million euros on this player that he shouldn't be at Barcelona anymore. So he was loaned out to Bayern Munich, which not a great use of money.
From the outside, his underlying performances were kind of the same as they'd ever been. So I don't know, maybe Barcelona paid more attention to the output, the luck side of the equation rather than the skill side of the equation. I still think he's a brilliant player by the way.
So just to conclude, hopefully, I've been able to tell you about three things. First is that the path from ignorance to acceptance through ridicule and conflict is a long and painful one. But it's worth it in the end. Second, separating luck from skill is essential to understanding your business, but you do need both for success.
So we were lucky in the two signings that I've mentioned. We were also skillful. I could tell you about any number of other signings where that same combination of skill to understand that, yeah, they would be great players for Liverpool, but also lack that they were available and that we were able to actually make the transfer happen.
Yeah, both things are important. For sports clubs especially, mistaking lack of skill happens all the time and it's just lethal to your long term health. And finally, focus on the things that make a difference. So in football, hopefully I've showed you that it's player recruitment and retention is the thing that makes the difference.
Everything else is like a second order effect compared to that. Find the same thing in your own industry and just just focus on that. So that's it. Thanks. Thanks for listening.
