Junee Correctional Centre, in the state's Riverina region, has been run by the US-owned GEO Group since it opened in 1993, but the company's contract will not be renewed when it expires in 2025.
Corrective Services NSW will take over responsibility for the operation of the prison, a mixed-security facility that houses more than 1000 male inmates.
No decision has been made on unwinding agreements for the state's two remaining privately operated prisons, Clarence Correctional Centre near Grafton and Parklea Correctional Centre in Sydney.
The contract for Parklea's operation also expires in 2025, but no decision has been made on its future.
Corrections Minister Anoulack Chanthivong said each contract would be assessed as its end date neared.
"No two correctional facilities are the same, they are very different in their risk profiles, in their facilities, in their operations, so we will make those decisions when the contracts fall due," he told Sydney radio 2GB.
The operators of the Junee prison have been criticised at multiple coronial inquests into inmate deaths.
In July, a coroner found Indigenous man Reuben Button received poor health care at the facility due to "significant resourcing issues" and other systemic issues before his death from heart disease.
GEO Group, which also operates two prisons in Victoria, said the decision not to undertake a fresh tender process for Junee had been met with "much disappointment and surprise" by management after 30 years of private operation.
"Even though staff are disappointed by this decision, and given the uncertainties and challenges brought about by transitioning to a new employer, GEO and its staff will work with the NSW government to ensure a smooth handover," the company said in a statement.
But Public Service Association of NSW general secretary Stewart Little said privatising prisons was a "stupid idea" and the union welcomed the state government's recognition it had been a policy failure.
"You can't squeeze a profit from a prison unless you cut corners on safety, on wages and on rehabilitation," he said.
Labor won the March election with an anti-privatisation platform with Premier Chris Minns and other ministers arguing asset sales have led to higher costs for taxpayers.
9°C