All 44 of Melbourne's high-rise public housing estates will be demolished and redeveloped by 2051 with five towers in Flemington, North Melbourne and Carlton to be replaced by 2031.
The urban renewal project is expected to lift the number of residents across the sites from roughly 10,000 to 30,000 and its social homes by 10 per cent.
Merri-bek City Council, a Greens-led local government in Melbourne's inner north, said the 10 per cent commitment fell far short of combating growing public housing demand.
"Fundamentally, this fails to address crippling shortages throughout the state," council mayor Angelica Panopoulos said.
Federal Greens leader Adam Bandt and state party leader Samantha Ratnam have suggested the plan could signal the "end of public housing in Victoria".
Selling his government's housing statement atop a Box Hill high-rise apartment building on Thursday, Mr Andrews accused the Greens of spreading misinformation and scaring people.
"This is a really big project and a critically important one," he said.
"If your answer is just the status quo and to scare people, that's not leadership."
Asked why all the extra homes couldn't be earmarked for public tenants, Mr Andrews pointed to the "enormous" cost of redeveloping the towers.
"If you don't have a partnership with the private sector, where will that money come from?" the Victorian premier said.
"Lots of people want to live on those sites and we can get a mix of housing, a mix of different families living side by side."
Construction timelines and costs have not been outlined for the other 39 towers, nor how residents will be housed during the redevelopments.
Mr Andrews insisted all displaced residents would be found alternative accommodation and invited to return once construction was finished.
"Those who want to come back will be on the same terms they left, that'd be our aim," he said.
Opposition Leader John Pesutto isn't confident displaced public tenants will be relocated, pointing to the government's decision to lock down nine public housing towers without notice in July 2020 during the pandemic.
"We saw during COVID how they treated people in our social and affordable housing units," he said.
"They don't have a great track record at respecting the interests and the needs of people in those residences."
The housing statement was headlined by a commitment to build 800,000 homes across Victoria over the next decade and an Australian-first statewide levy on all short-term accommodation bookings with online platforms.
The consumer-facing 7.5 per cent levy on short-stay providers like Airbnb and Stayz is slated to come into effect from 2025 and raise $70 million a year to build and maintain social housing.
Mr Andrews dismissed a warning from Stayz that owners could avoid the levy by taking their properties off platforms and listing them elsewhere.
"There are many different ways in which people try and avoid paying tax and the State Revenue Office are expert in finding those people and making sure that people do pay their fair share," he said.
"If you're eligible to pay the tax then you will be paying that tax."
Other highlight reforms include closing a rental bidding loophole, slashing planning red tape and bolstering apartment design standards.
