The racism intensified following his involvement in the public broadcaster's coverage of the King's coronation.
The veteran correspondent was invited to participate in an ABC TV discussion about the legacy of the monarchy on coronation day, during which he pointed out that the crown represents the invasion and theft of Aboriginal land.
Barely a week goes by in which he and his family are not targeted by racial abuse, Grant said in a column on the ABC News website on Friday, and he needs space from "the stench of the media".
Those in the media have distorted his words and accused him of maligning Australia, the column reads.
Not a single ABC executive has publicly refuted the lies that have been written and spoken about him following the coronation broadcast, he said.
"I don't hold any individual responsible; this is an institutional failure."
While the ABC's news director Justin Stevens has been a "support and a comfort", he said, the national broadcaster has its own legacy of racism.
It is not clear how long he will step away from the Q&A hosting role for.
Grant said he would not give racists the satisfaction of blaming racism for his decision to take time out.
Rather, he is leaving because the hard truths of Australia's Indigenous people are too big and fragile for an intensely politicised media to handle.
Grant's latest book, The Queen is Dead, argues for a republic, a just settlement with Indigenous people, and an end to the monarchy in Australia.
8°C