United had struggled to create clear chances on Sunday against Patrick Vieira's well-drilled side but the oft-criticised Fred beat Vincente Guaita with a wonderful strike from the edge of the box after being set up by substitute Mason Greenwood.
Palace should have gone ahead minutes earlier but Jordan Ayew missed from point-blank range at the back post after James Tomkins had headed down a corner.
German Rangnick was leading United for the first time after taking over from caretaker manager Michael Carrrick on Friday.
His side have moved up to sixth place in the table on 24 points, a point behind Tottenham, who went fifth after carving out a third successive win, defeating struggling Norwich 3-0 at home.
An early goal from Lucas Moura and two after halftime from Davinson Sanchez and Son Heung-min powered Spurs' further improvement under new boss Antonio Conte.
Elsewhere, Leeds striker Patrick Bamford marked his return from injury with a goal in added time as they snatched a 2-2 draw against Brentford in a high-intensity match at Elland Road.
Back after an absence with an ankle problem, substitute Bamford fired the ball into the net off the crossbar following a corner after Brentford's Sergi Canos and Shandon Baptiste had scored to give the visitors a 2-1 lead.
Earlier, Leeds striker Tyler Roberts had netted his first of the campaign, meeting Raphinha's whipped cross with a sliding finish.
