Hosts South Africa won the toss and elected to bowl first in Sunday's second match, but after a slow start the Jos Buttler-inspired visitors accelerated superbly to post a formidable 7-342 off their 50 overs.
Yet the hosts reached their target with five balls remaining for a record chase at the Mangaung Oval, smashing the previous best mark of 274 by South Africa against Australia in 2020.
They finished on 5-347, with only India having chased down more runs to beat England in ODI cricket when they reached 7-356 in Pune in 2016.
The result is a boost for South Africa, who need a clean sweep of the series to stay on course for guaranteed qualification for the World Cup in India this year.
"Playing England is always tough, your disciplines are always being tested," Bavuma said at the post-match presentation.
"Chasing 340 like that will give us a lot of confidence. The wicket in Bloemfontein is nice and suits the way we want to play. I was glad to remind myself how to count to 100!"
The visitors' total on Sunday was boosted by a magnificent unbeaten 94 in 82 balls from captain Buttler and an excellent 80 from 75 deliveries from Harry Brook, who made a duck on his ODI debut in the opening match of the series.
Moeen Ali chipped in with a fluent 51 from 45 balls as England smashed 181 from the last 20 overs of their innings.
But South Africa were always up with the rate in their reply, which was anchored by Bavuma, who scored a timely third ODI century having come in for recent criticism following poor white-ball form.
David Miller took the team home with a masterful 58 from 37 balls and Aiden Markram chipped in with a 49 at better than a run a ball.
"To post 342 was a fantastic effort from the guys, but it was a great chase," Buttler said. "Credit to South Africa, Temba led from the front.
"We have come to win games of cricket, but there is also a bigger picture as well in terms of building towards the World Cup."
The final match of the series is in Kimberley on Wednesday.
