Why quantifying scoring chances in hockey is a unique challenge

Vegas Golden Knights right wing Mark Stone (61) attempts to get a rebound shot past Edmonton Oilers goaltender Mikko Koskinen (19) during the second period of an NHL hockey game Saturday, Nov. 27, 2021, in Las Vegas. (Chase Stevens/AP)

Here’s what an intermission looked like for the Toronto Maple Leafs and the Toronto Marlies during my time as a video coach with the organization: the coaching staff would come back in the office and the first thing the head coach would do is sit at his computer and go through the folder labelled “scoring chances against” from the previous period. This is one of the things a video coach tags throughout a game, pressing the right keys to tell the program to clip and sort what just happened. (The coaches went through scoring chances for as well, but it’s the “against” stuff that’s far more within the control of a coach.)

It was a major video coach faux pas for the coach to come in, watch the clips, and be left asking “Where's the chance from [somewhere in the period]?” The issue, of course, is not that a video coach isn’t watching the game, but rather the definition of “scoring chance against” is subjective. You and I may watch the same play and disagree about whether a moment was truly a scoring chance or not. With that in mind, when I logged games for the staff I erred on the side of over-labelling. You don’t want to go too far there -- coaches don’t want to pick through wishy-washy shots in the scoring chances against category either -- but it was better to ensure things were there rather than not. It was easier to dispose of clips than have to re-find them and tag them.

I usually refined the clips sometime after the game, getting rid of the mis-labelled stuff, and so by the end of the year our team would be able to fairly approximate how many scoring chances we gave up per game, who was involved in more or less of them, and how the chances tended to unfold.

This extremely human process is fraught with bias though, as all human processes are. I had my own relationships and interests within our hockey team. I’m not saying this ever happened intentionally, but if I were to be labelling a rush for our team, and a guy I liked and rooted for -- let’s say Connor Brown -- made a shifty pass across on a 2-on-1 to a teammate who fanned on it entirely, I might be more likely to qualify that as a scoring chance than I would be if it were a player who I had negative feelings towards to ensure the coach saw the great play.

The above scenario is a fun one to discuss, because it was at the root of one of the few debates I was involved in as a member of the Marlies staff. Kyle Dubas wanted a formal definition of what a scoring chance was for our statistical purposes, but I thought any hard definition (for our team anyway) missed too much. The crux of my point was this: I’ve been in arenas watching NHL hockey since I was, quite literally, a newborn. From my Dad’s career, through to my own, to my time in the media (doing play breakdowns that led me to the Marlies), all I’ve known is watching hockey and doing so from the perspective of someone who wants goals. I have a darn good idea when a play is a legitimate chance to score, and if we had instituted a hard definition that would rely on shot attempts (or worse, shots on goal), we wouldn’t get the true story of chances created in the games.

Players fan on wide open chances, they lose the puck stickhandling on partial breakaways, and make imperfect passes on 3-on-1s that were an inch or two from not getting broken up, and instead becoming tap-in goals. Those are all still chances your defensive structure allowed. With no shot attempts being registered on any of those examples, quantify scoring chances with hard statistics misses too much. Is there one of those every other game, or even every game? I’m guessing here, but I think it’s reasonable to speculate somewhere in that range.

The statistical argument is that by using a hard definition (one of which you can find here in Natural Stat Trick’s glossary) the subjectivity of it gets stripped away, which is particularly great for the public who are trying to compare across teams without the luxury of one person watching 82 games for 32 teams to keep the playing field closer to level. A definition for scoring chances also allows for the sample size to get big enough that you could point to my examples of “unrecorded chances” and say those occurrences are a drop in the bucket with this much data, or that what we have gets us close enough to the answers we’re after (which are generally simple: which teams and players are involved in the most/fewest chances for/against in the league?).

There’s another problem with the hard definition, though. It’s not just the chances that don’t get recorded, there’s the idea that all unblocked shots from the “medium danger” ice are worth calling a scoring chance, which requires a level of inspection and refinement that still lacks. This is a bigger problem than the missed chances, in that a variety of factors influence just how dangerous an attempt may be.

If a tightly covered player at the hashmarks whacks at a lose puck that makes it to the net with a D-man between him and a squared-up goalie, it’s certainly not a true scoring chance, particularly in the way I believe coaches and fans want a “scoring chance” stat to help inform them. There was no breakdown, there was nothing created offensively, but there was a puck on net from somewhere you’d loosely describe as dangerous, but in this case isn’t. The bigger offender here may be deflections and redirections in high danger areas -- players mindlessly tip and redirect pucks wide with some regularity around there, and a sizeable fraction of them are never scoring chances at all, yet are recorded as such. Combining these “false positive” chances with the missed ones I described, we end up with a stat that’s not just “noisy,” but at times flat-out inaccurate by wide margins.

On Nov. 21 the Toronto Maple Leafs played the New York Islanders extremely tight, beating them 3-0 with young Joseph Woll in net, who was getting his second NHL start. The Islanders only registered 20 shots, which made the game feel like it could’ve been just about any of 200 goalies from around the world in the Leafs' net and it still would’ve ended with the same 3-0 score.

The thing about that perception was that the data told us the Islanders actually registered 18 “high danger” scoring chances based on the definition of where the shot attempts came from, 15 of which were at 5-on-5. In 23 games this season, that’s the second-highest total of chances allowed by Toronto, and the most in their past 20 games.

That’s two awfully different angles on evaluating both Woll’s play and the performance of the Leafs.

The marking on InStat isn’t perfect, but in watching back the chances Toronto allowed in that game, a lot of what would’ve qualified for those “high danger chances” we can plainly see aren’t all that “highly dangerous,” and in many cases aren’t much of a “chance” at all. Here’s a selection of them.

How many of those are actually high danger chances, considering a good number of them would be labelled as such? It just doesn’t square up, or square up close enough to feel good about using the data.

The point of all this isn’t to disparage the overall usefulness of publicly available scoring chance data (though it does seem to be a less effective predictor of future performance than the possession-based stats like Corsi), but to simply acknowledge this is an area of hockey where we still struggle to quantify what’s going on. Statisticians don’t want the subjectivity of the human eye on this sort of thing (with good reason), but the definition of “scoring chances” leaves us wanting better too.

The revolution in how the game is analyzed is upon us now, with companies like Sportlogiq and Stathletes and the NHL itself doing more player and puck tracking. This is an area that will get refined and improved, and almost certainly already is within individual NHL teams. I care about this because I mentioned scoring chance data isn’t the best predictor of future success, which to my eyes and ears and brain sounds counterintuitive.

It’s what coaches around the NHL care about in their game evaluation, so it’s fair for fans to have interest in the topic too.

When submitting content, please abide by our  submission guidelines, and avoid posting profanity, personal attacks or harassment. Should you violate our submissions guidelines, we reserve the right to remove your comments and block your account. Sportsnet reserves the right to close a story’s comment section at any time.
We use cookies to improve your experience. Learn More or change your cookie preferences. By continuing to use this site, you agree to the use of cookies.