Mr Abbott has been a staunch critic of the voice proposal, but said he was in favour of Indigenous constitutional recognition.
"I'm all in favour of proper respect and honour for the first Australians, but I don't think anyone, however much we might admire them and appreciate them, should be given a special say," he told 2GB on Monday.
"I don't want to see our country divided by ancestry or by race ... and I don't want to see the business of government gummed up even further."
The government has rejected claims the voice would stall legislation or the decision-making process with High Court challenges.
While some polls have put support for the voice among young people almost as high as 75 per cent, recent polling has shown overall support waning, with four states on track to vote against the proposal.
A Sydney Morning Herald poll shows NSW, Queensland, South Australia and Western Australia voting 'no' at the referendum, with support in Victoria and Tasmania only slightly above 50 per cent.
Four of the six states and more than 50 per cent of all Australians need to vote in favour of a referendum for it to succeed with a double majority.
Mr Abbott said while polls showing reduced support were "encouraging", the 'no' side could not afford to be complacent.
"There is still an avalanche of money being provided by woke public companies, woke foundations and billionaires," he said.
"If it is possible to buy a referendum outcome, I think that the 'yes' campaign will do their best to buy it."
More than 30 young Indigenous Australians will lead conversations about why a constitutionally enshrined voice is integral to closing the gap.
First Nations leaders from NSW, Queensland, South Australia, Western Australia and the Torres Strait gathered for the the Uluru Youth Dialogue ambassador program over the weekend.
Co-Chair Bridget Cama said young people would play an important part in the referendum achieving success.
"Our young people are committed to this pragmatic reform that will achieve a better future for First Peoples and for this country," said Ms Cama.
Renowned Indigenous activist Noel Pearson also issued a call to arms for the 'yes' campaign, saying success wouldn't just "fall into our laps".
Mr Pearson said the 'yes' campaign had its work cut out but could draw upon momentum.
"We need to be at the railway stations, we need to be at the town halls, we need to be meeting people in the malls and we need to be appealing to the better angels of the Australian nature."
He said a 'no' vote would leave Australia "in the darkness".
7°C