Independent candidate Monique Ryan will now be the next MP for the Melbourne-based electorate.
In a statement, Mr Frydenberg said he had called Dr Ryan to concede the election.
"Today I rang Dr Monique Ryan to congratulate her on the election result in Kooyong and wished her well for the term ahead," he said.
"It's been an incredible privilege to have served as the local member for the last 12 years."
The former treasurer was one of many high-profile Liberals to lose their safe, blue-ribbon seats to the so-called teal independent candidates.
Mr Frydenberg said he had given his job as an MP his all.
"Inspired by the wonderful local people I have met, among them those who volunteer in our charities and sporting clubs and work in our small businesses and schools," he said.
"We have achieved so much together. It is their contribution that makes our community great."
Mr Frydenberg said he was looking forward to spending more time with his family.
20°C