The four-storey Loafers Lodge hostel in Newtown, in Wellington's south, caught alight overnight, prompting an evacuation.
Around 90 emergency personnel responded to the blaze, which included around five people winched off the roof by helicopter.
Fire and Emergency New Zealand said "a number of people" were unaccounted for but is yet to confirm deaths.
Earlier, officials were privately bracing for dozens of fatalities, but inspector Dean Silvester said the initial assessment of police is "fewer than 10".
"We are not in a position to be any more specific until we can access the building," he said.
FENZ district manager Nick Pyatt said "it doesn't get worse than this" for his crews.
"This is a once-in-a-decade fire for Wellington. It's the worst nightmare for us," he said.
"This is a tragic event for all involved. My heartfelt condolences go to the loved ones of those who have lost their lives," Mr Pyatt said.
Around 50 people were taken out of the building after being notified of the blaze at 12.25am.
"It's a really concerning development and obviously our thoughts will be with all the firefighters involved and all the victims of the terrible tragedy that's unfolding," NZ Prime Minister Chris Hipkins told local television.
There were no vacancies at the hostel, which is also used for transient housing, except for a number of rooms being treated for bedbugs, Radio NZ reported.
Mayor Tory Whanau, who described it as a "truly tragic situation", said the building housed a mix of short and long term residents.
"Possibly a lot of vulnerable folks which makes this even more devastating," she told AAP.
"We'll find out more in the coming hours and days.
"Wellington is feeling deep sadness and sorrow at what has happened."
12°C