Gladbach 'keeper Moritz Nicolas, who conceded the second penalty for punching Luca Waldschmidt instead of the ball, saved Kainz's spot kick in the 75th, but was penalised for moving forward off his line.
That gave Kainz the chance to take it again, and he made no mistake, firing inside the left post at the second attempt as his side grabbed the points in Sunday's fixture.
Waldschmidt sealed the victory with Cologne's third goal which wernt in off the bottom of the right-hand post in injury time.
The late drama came after Gladbach midfielder Kouadio Kone had earlier been sent off after a VAR check for a dangerous tackle on Dejan Ljubicic.
It was a game to forget for Koné, who conceded the first penalty through VAR for hand ball.
Kainz opened the scoring from the spot in the ninth minute, but Cologne failed to convert more chances in the first half and Nico Elvedi equalised in the 63rd.
Kone's sending off in the 72nd gave Cologne all the encouragement they needed to claim three points for the first time this season.
Elsewhere, Augsburg's Danish coach Jess Thorup enjoyed a morale-boosting 5-2 win at promoted Heidenheim in his first game in charge.
The visitors got off to a bad start by conceding two goals in two minutes by the 18th.
But Phillip Tietz pulled one back in the 29th, then set up Mads Pedersen to equalise in the 41st and there was still time before the break for Ermedin Demirovic to score Augsburg's third.
Felix Uduokhai and Elvis Rexhbecaj completed the scoring in the second half.
Thorup replaced the fired Enrico Maassen after Augsburg won only one of their opening seven league games and were also knocked out in the first round of the German Cup.
Bayer Leverkusen lead the Bundesliga after eight rounds, one point ahead of Stuttgart and two points clear of defending champions Bayern Munich.
8°C