The former Red Devils boss had never won a Premier League match as visiting manager in 17 previous attempts, and a moment of madness 13 minutes into Monday's meeting looked set to extend that record.
Gueye was shown a straight red after striking Keane during a heated row, but 10-man Everton dug deep and Dewsbury-Hall's curling effort proved the difference in a 1-0 victory against toothless United.
"I think it was just a moment of madness," goalscorer Dewsbury-Hall said of Gueye's actions.
"It was avoidable, but Idrissa has apologised to us all at full-time and said his piece. We move on from it. The reaction from the lads after that is unbelievable. We could easily have crumbled and lost the game comfortably, and if anything it made us grow as a team."
Gueye apologised to Keane and to "my teammates, the staff, the fans and the club", adding he takes "full responsibility for my reaction".
"What happened does not reflect who I am or the values I stand for," Gueye wrote on his Instagram account. "Emotions can run high, but nothing justifies such behavior. I'll make sure it never happens again."
It was the Toffees' first win at Old Trafford since December 2013, when Moyes was in the home dugout, and saw Ruben Amorim's side return to earth with a bump, having gone five games unbeaten.
"If you'd have said to me when we went to 10 men we were going to get a result, still at 0-0, I'd have said that would be really tough," Moyes said. "That was a brilliant, brilliant performance from the players.
"Resilience, toughness, commitments, all the words you want as a manager, and a great goal as well."
It was a miserable night for United, who started the evening by honouring club great George Best on the eve of the 20th anniversary of his death.
Everton began brightest when play got under way, but things unravelled for them in specular fashion within 13 minutes of kick-off.
First skipper Seamus Coleman limped off to be replaced by Jake O'Brien, then Gueye lost his head.
The midfielder's poor pass saw Bruno Fernandes going close and led to an argument with Keane, who he slapped during the heated exchange.
Referee Tony Harrington brandished a red card and Jordan Pickford had to wrestle Gueye away from a bemused, and understandably angry, Keane.
That sending-off should have put United in control, but they offered little by way of threat, and meek defending cost them.
Dewsbury-Hall cut between Fernandes and Leny Yoro, whose weak attempt at a challenge only prodded on for the midfielder to hit a curling effort from the edge of the box that Senne Lammens was unable to stop.
Everton's away support went wild and United slowly built a response to the 29th-minute strike.
Patrick Dorgu lashed wide and Amad Diallo went close after some smart footwork in the box, with Pickford required to pull off a wonderful save when Fernandes took aim from 25 yards just before the break.
The halftime whistle was met by jeers and Mason Mount replaced Noussair Mazraoui when the teams re-emerged, but United continued to misplace numerous passes and create little more than half-chances.
"They were the better team, with 11 players or 10," Amorim said.
"I felt from the first minute that we were not there with the same intensity, so they deserve the win."
- with AP
