The growing use of statistics in hockey—advanced, enhanced, fancy, whatever you want to call them—has come with its own controversy. Virtually every claim has been contested and every new statistic a point of conflict. That’s how it should be. The great benefit of using this kind of data to study a game like hockey is that it is objective and any conclusion can be considered from multiple angles.
While the whole stats field can at times seem contentious, conclusions about defence have been particularly controversial. After all, we’re used to measuring offence through statistics – goal and point totals are generally acknowledged to do a good job of capturing offensive ability. However, prior to the last decade or so there were no widely accepted defensive metrics in the mainstream; the closest was plus-minus, which is a deeply flawed statistic that has, rightly, become a punchline.
The lack of defensive statistics has made it difficult to evaluate defencemen. The new numbers, though, offer us some interesting tools and a new vantage point on a position that has been ignored for too long.
To demonstrate, we decided to look at how defencemen performed year over year in the past two seasons. We made a list of defencemen who had played at least 500 minutes in both 2013-14 and 2014-15. For added confidence, we only looked at defencemen who hadn’t changed teams, reasoning that players who stay on the same team would be used in basically the same roles in both seasons.
This is a stat we call relative goals, and it basically shows why plus-minus is garbage.
Relative goals shows how a team does in terms of goal differential with a player on the ice vs. how it does with that same player on the bench. It’s pretty simple really. If a team scores 50 per cent of all goals with one player out there and then 40 per cent when he’s on the bench, he gets a mark of +10.0; his team’s performance goes up 10 per cent when he’s on the ice.
What we find here is there’s almost no relationship in how a player does in this stat from one year to the next.
Vancouver’s Alex Edler is a great example of this. In 2013-14, Vancouver was out-scored at more than a 2:1 rate when he was on the ice; the Canucks were a whopping 23 per cent worse with him on the ice than they were when he was on the bench. The next year, though, Vancouver was actually six per cent better with Edler on the ice.
When we look at Fenwick instead of goals, we find something else entirely. Relative Fenwick works the same way as relative goals, except that instead of counting goals for and against we’re counting shots and missed shots. Fenwick, Corsi and scoring chances are all highly related; teams that are good or bad at one tend to be good or bad at all.
This isn’t a perfect stat by any measure, but what we see is a much higher degree of repeatability. If a team is good at out-shooting the opposition with Player X on the ice one year, they’re likely to be good at it the next year, too.
This is why stats people place so much emphasis on players like Montreal’s P.K. Subban. In 2013-14, Subban had lousy goal numbers, but really strong Fenwick totals; the next year his Fenwick numbers were still excellent and the goal totals had come around. Goal numbers fluctuate wildly, but players like Subban who help their teams out-shoot the opposition one season are generally able to do it the next year, too. (Note: Mark Giordano is the player on the far right.)
The reason shot metrics are more reliable is that shooting and save percentage tend to bounce around. A lot of times, we think about defencemen as being able to improve their teams’ save percentage by limiting top quality chances, but when we look at the data it’s hard to find evidence of that. Eric Tulsky (now working for the Carolina Hurricanes) compared three years of data and found that while defencemen had more impact than forwards on save percentage, it was so small that even with three years of data it was all but impossible to predict how a defenceman would do the next year.
In our own sample of 119 defencemen, there’s no relationship whatsoever between performance one year and performance the next. For those wondering, that diamond in the top right corner is Boston’s Kevan Miller, who may have been lucky enough to have an on-ice save percentage above 0.950 in consecutive years, but almost certainly isn’t the cause.
This is true over full seasons, and it’s really true over shorter spans. A good example this year is Edmonton’s Mark Fayne. When Fayne is on the ice, the Oilers do a relatively good job of out-shooting their opponents, but because of a lousy on-ice save percentage the team has been lit up for a lot of goals when he’s out there. We know from the charts above that defencemen have little control over save percentage, and so it would be a mistake to blame the player for his lousy on-ice goal numbers.
The really surprising thing is that defencemen seem to influence not just shots against, but also shots for. Our sample of 119 defencemen had a high correlation year-over-year in unblocked shots against (0.56) but almost an equally high correlation in terms of shots for (0.54). That’s surprising, but when we think about how much a defenceman’s ability to make a good outlet pass or create turnovers at the defensive blue line matters to offence it makes sense.
Shot metrics such as Fenwick are controversial at the individual level even for forwards, and especially so for defencemen. But looking at charts like these, it’s easy to see why analytics types are intrigued by these numbers.