''We've shown an ability to play the game a certain way that that gives us the best chance to win and it starts with an attitude,'' Penguins interim coach Mike Sullivan said. ''Then it's a commitment to play in the game the right way and attention to detail, knowing what your job is and trusting that your linemate or your teammate is going to do theirs.
''When we play that way, and we have that collective effort and we have that trust among one another that you know each guy is going to do their job, I think we're stingy as any team in the league.''
Gusta Nyquist and Boone Jenner scored for the Blue Jackets, who have lost four of their last five home games.
Casey DeSmith stopped 10 shots in the first period before being pulled and replaced in the second by Tristan Jarry, who stopped all 15 shots he faced.
Joonas Korpisalo saved 26 shots in the loss.
It took just 32 seconds for Pittsburgh to take the lead on Dumoulin's wrister from the left circle, his third of the season. Assists from Crosby and Kris Letang extended their point streaks to four and 10 games, respectively.
Nyquist pulled Columbus even with his NHL-leading fourth short-handed goal, which he slid behind DeSmith at 10:40 of the first.
The Blue Jackets took a 2-1 lead at 18:46 when Jenner pushed the puck through DeSmith's pads after a bounce off the end boards. Jakub Voracek's primary assist was his 22nd of the season, tying him for second in the NHL.
The lead was short-lived as Crosby scored for the third straight game, beating Korpisalo with a shot from between the circles that made it 2-2 with 28 seconds left in the first period.
''It's nice to get some rhythm and timing,'' Crosby said. ''We've got a lot of games coming up and hopefully we can work through and get better.''
Matheson recorded his fifth game-winner of the season at 5:35 of the third period, tying him with Guentzel for the team high, with a puck that Korpisalo thought he stopped until it dribbled behind him into the net.
Columbus thought Max Domi tied it at 10:39, but it was waved off for offsides - the sixth such call against Columbus this season.
''We played a heck of a game,'' Blue Jackets coach Brad Larsen said. ''I have no issue with our effort. ... We were one shot away. We were in it. Then we imploded.''
Crosby added an insurance power-play goal at 13:32 of the third and an empty-netter with 1:58 remaining. The hat trick was Crosby's first since Nov. 28, 2018.
LONG TIME NO SEE
Pittsburgh and Columbus faced off for the first time in more than two years, having last played Dec. 19, 2019, when the Penguins won 1-0 at home.
KINGS OF THE ROAD
The Penguins have been one of the league's best teams on the road this season, boasting a 14-5-3 record for 31 points, second most in the NHL. After playing seven of their past eight games on the road, the Penguins head back to Pittsburgh for a seven-game stretch that will allow them to sleep at home until Feb. 8.
Four of Nyquist's 10 goals this season have come down a man, while Pittsburgh has given up three goals with a man advantage.
Jake Guentzel's assist was his team-leading 40th point.
In four sets of back-to-back games, Columbus is 2-2-0 in the first game and 2-2-0 in the second. The Pens have had five back-to-back situations, with a record of 6-3-0 (4-1-0 on the first night; 2-2-0 the second night).
Penguins: Host Winnipeg on Sunday.
Blue Jackets: Host Ottawa on Sunday.