It would be hard to make up a story with the narrative arc of the Toronto Maple Leafs and their relationship with analytics over the past 18 months. In 2012–13, the Leafs enjoyed a thrilling, successful season that those familiar with analytics suggested was unlikely to repeat. The Leafs (and their friends in the media) were publicly contemptuous of said predictions.
The Leafs got off to a great start in 2013–14, emboldening their media friends and the anti-analytics corners of their organization. Just when it seemed they were going to defy the math for a second season, they ran into a statistically improbable slump and missed the playoffs. The analytics people were vindicated (sort of) and the Leafs went into the off-season derided as a team from the Stone Age.
A funny thing happened in the off-season though. The Leafs found religion (or science, depending on your view of things). There was acknowledgment that the analytics guys may have had a point. Brendan Shanahan took it a step further in an interview with the Globe and Mail and said the Leafs analytics budget would be spent this year, a reference to Leafs GM Dave Nonis having said they chose not to spend their analytics budget in the past.
Of course, it’s one thing to have an analytics budget that you intend to spend. It’s quite another to spend it well. For reasons that I’ll discuss, this is harder than people realize. In today’s hiring of Kyle Dubas, the Maple Leafs have removed one of the biggest potential hurdles to becoming an organization that can use data well: They’ve hired a gatekeeper who can screen out a lot of the bad analytics.
I'm still in shock that the AGM of the Leafs is well-versed in ideas that were once talked about by six guys on the Internet.
— mc79hockey (@mc79hockey) July 22, 2014
Dubas, for those of you who are unfamiliar with him, worked as the GM of the Sault St. Marie Greyhounds from 2011–12 to 2013–14. The Greyhounds went from a 56-point season in 2010–11 to 95 points in 2013–14, their best season since 1984–85.
Sault St. Marie is, in some ways, an Oakland A’s or Tampa Bay Rays of the Ontario Hockey League. It doesn’t have the financial resources of some of the OHL’s larger clubs and it can’t offer players from southern Ontario a chance to play relatively close to home. In order to compete, the Greyhounds had to figure out a way to overcome those obstacles.
One of the tools that Dubas brought into the Soo was analytics. Dubas built on the publicly available work that had been done in the NHL and started gathering data at the OHL level. What’s more, when he was quoted talking about analytics, his comments made sense, which isn’t always the case with hockey executives and coaches.
He understands why shot-based metrics like Corsi are preferable to goal-based analysis in a lot of cases. He understands the randomness of hockey. He can talk about quality of competition. He’s well versed in the current state of online research on hockey.
Dubas is going to have to be pretty smooth to deal with this. If I was him, after today, I wouldn’t talk to the media for a year.
— mc79hockey (@mc79hockey) July 22, 2014
Did the Soo enjoy the success they enjoyed because of their use of analytics? That’s impossible for any outsider to say. As Yahoo!’s Cam Charron pointed out, Dubas brought more than just analytics to the Greyhounds. There were undoubtedly some non-analytics changes and things that happened in the Soo over the past three years that contributed to their success.
That said, it’s the analytics angle to all of this that’s the most interesting. Dubas is a very astute hire for the Maple Leafs as they begin to follow a trend in the NHL and incorporate analytics into what they’re doing to try to win hockey games.
One of the challenges that NHL teams face in incorporating analytics into what they’re doing is that they aren’t particularly well equipped to separate the good analytics work from the bad. Dubas’s experience in doing so will enable the Maple Leafs to avoid a lot of the potential pitfalls that exist.
There is a lot of bad analytics work out there, and there are certain common threads running through it. Most importantly, it tends to be done by people who want to keep their formulas secret, thinking that if their formulas were public, nobody would pay them for them. This renders any meaningful review of their work very difficult. The profit motive creates an incentive for them to prevent that meaningful review, and to churn out garbage that catches the eye of a decision maker within an NHL team.
Baseball’s history suggests that this isn’t the way the best work comes about. While teams have now surpassed the work that’s being done in the public sphere, it was built on what was, for lack of a better term, the open-source work of people like Bill James in the 1980s and 1990s. Baseball’s advances didn’t come from someone developing a black box and selling it to teams; it came from teams adopting and improving upon the ideas that were developed in the public sphere and survived being publicly examined in that sphere.
Dubas is likely aware of this history—he’s a big baseball fan. Listening to him speak over the years about the Soo Greyhounds, he’s seemed to be very familiar with the better ideas that have emerged from the debate on hockey analytics that’s taken place on the internet. Some of this may be a product of necessity—the Greyhounds don’t have the same financial resources that the Maple Leafs have—but it will likely serve him (and the Maple Leafs) well in terms of building an analytics department that is the class of the NHL.