Specialist heavy machinery is on site and will begin knocking down what is left of the seven-storey, 1910s-era brick building in inner city Surry Hills on Tuesday morning.
Police haven't ruled out finding bodies in the rubble of the heritage building but the site needs to be made safe before it can be thoroughly inspected and residents in neighbouring buildings can return home.
Fire and Rescue NSW Superintendent Adam Dewberry says a 60-tonne long reach excavator arrived before dawn on Tuesday for the tricky operation to ensure the job is done safely.
"It can reach in about 23 metres so that's the distance it can stay away from the building," he told Sydney radio 2GB.
"We want to make sure everyone remains safe but we also don't want to cause any damage to adjoining structures and utilities that are in and around and under the road," he said.
"We've got gas mains underneath, we've got electricity and water so we just need to make sure we don't cause any more problems."
The site will be handed over to police, allowing arson squad investigators and cadaver-detection dogs to examine the scene as rubble is removed.
"While police have not received any reports of missing people, investigators are unable to definitively say there is no one inside," police said in a statement on Monday.
The brick and timber building, and a neighbouring structure formerly home to karaoke bar Ding Dong Dang, was known for regularly housing 15 rough sleepers. Thirteen of them were contacted by police on Friday.
There are calls for authorities to search other buildings in the CBD that are standing empty.
Business Sydney Executive Director Paul Nicolao has written to Premier Chris Minns and Sydney Lord Mayor Clover Moore asking for an audit.
"It is unsatisfactory and potentially unsafe for buildings to be left empty and insecure with the risk of them being used by squatters and others who shouldn't be in them," he said.
Since Thursday's blaze lit up the sky, closed streets and diverted public transport, more than 100 people living in the area have registered as being displaced.
Welfare services will continue to update displaced residents and keep them informed.
The brick hat factory was built in 1912 and operated until 1954 when proprietor RC Henderson went into liquidation.
The property was later leased to individual tenants in the 1960s and '70s, and plans were afoot to use the brick structure in a $40 million redevelopment of the site into a 123-room hotel.
16°C