Gates, 70, denied knowing about the disgraced financier's ongoing criminal conduct, but acknowledged being aware of Epstein's prior legal issues during a closed-door testimony to the US House Oversight Committee.
The tech mogul expressed surprise that Epstein appeared to know about his two affairs involving young Russian women.
Gates identified the women as Mila Antonova, a bridge player previously identified by The Wall Street Journal, and Karima Nigmatulina, a nuclear scientist. He also identified Alice Jacobs Nesselrodt, a doctor he met at a conference before 2010, as another woman with whom he had an affair.
Gates acknowledged Epstein had at one point requested several thousand dollars in reimbursements related to one of the women.
"These affairs had nothing to do with my interactions with Epstein, but they were painful for my family," Gates said. In February, the Microsoft co-founder apologised to staff at the Gates Foundation for his ties to Epstein.
Gates said he wasn't blackmailed, but told the committee that emails suggest Epstein may have been attempting to move in that direction.
Gates told Congress he never discussed Epstein with the disgraced former prince Andrew, but says the pedophile financier "weirdly" kept a picture of the pair together at an official function.
Gates acknowledged that meeting Epstein, who was sentenced to prison in 2008 for soliciting a child for prostitution, "was a grave error in judgment" but said the relationship was limited to philanthropy.
He denied any wrongdoing or knowledge of the sex predator's abuse of girls.
Asked at the committee hearing whether he had ever discussed the financier with the former UK prince, Gates said "nothing connected to Epstein".
According to the transcript, he said of Andrew: "I encountered him two times in official visits in the UK where he was serving as something like the trade representative."
Pressed on this, he said the interaction was "never about Jeffrey Epstein".
He added: "Weirdly in the set of photos that I guess Epstein kept, there is a photo of me and prince Andrew at one of those official state meetings.
"But nothing in connection with Epstein, even though Epstein seemed to have that picture and it shows up in the Epstein files."
Asked whether Epstein had been at the function, Gates said: "No, no, I never saw him in the UK, and those were both meetings in the UK."
Andrew stepped down as UK trade envoy in 2011, where he earned the nickname "Air Miles Andy" for his lavish globetrotting, amid controversy over his continued friendship with Epstein, who cultivated ties to wealthy and powerful figures.
Epstein was found dead in his cell at a federal jail in Manhattan, New York, in August 2019 while he awaited trial on sex trafficking charges. His death was ruled as a suicide.
His accomplice, the British socialite Ghislaine Maxwell, is serving a 20-year prison sentence in the US for luring young girls for Epstein to sexually abuse.
Virginia Giuffre, who died by suicide last year, had accused Andrew of sexually assaulting her when she was a teenager after being trafficked by Epstein.
He has strongly denied this but in 2022 reached an out-of-court settlement with her, despite claiming never to have met her.
Britain's King Charles last year stripped his brother of his royal titles after fresh revelations about his Epstein ties and Andrew was forced to leave his Windsor mansion and move to the monarch's private Sandringham estate in Norfolk.
1800 RESPECT (1800 737 732)
National Sexual Abuse and Redress Support Service 1800 211 028
7°C