The Albanese government announced the federal prohibition earlier this year, which could see people jailed for a year if they publicly display the insignia.
The ban will include flags, armbands and T-shirts, and will extend to posting the symbols online.
Selling Nazi memorabilia will be made an offence, and will be banned in retail and online stores.
Australian military collectors told a parliamentary committee the legislation would harm the trade of items.
Jason Gehrke, from the Shooters Union arms collector's branch, said the bill implied people owning military items were Nazis or advocates of the ideology, which was "deeply hurtful and offensive".
Mr Gehrke said it was estimated the bill would adversely impact 1.2 million Australians who own up to five million objects worth $760 million.
"The families and heirs of collectors ... may inherit valuable objects ... but will be denied the opportunity to liquidate those assets because the value of their inheritance will be stolen by this bill," he told the hearing on Friday.
"Unless the government is proposing to offer compensation on just terms, this bill represents a mass destruction of asset value that appears to be a violation of property rights under the constitution."
Mr Gehrke said military collectors were not extremists nor anti-Semitic.
Multiple faith groups have sounded the alarm on culturally significant symbols, including the swastika, being captured under the laws.
The swastika is an ancient symbol of peace and prosperity for some religions.
Under the legislation, the display of Islamic State symbols will also be prohibited.
Lawyer Ramia Abdo Sultan, representing the Australian National Imams Council, said the Islamic State misappropriated important Islamic symbols adopted by early Muslims.
She said the prohibition of the Islamic State flag has unintended consequences for Muslims.
"Proposed prohibition relating to the symbols will unjustifiably restrict and marginalise Australian Muslims," Ms Abdo Sultan told the hearing.
"Many Australian Muslims have these symbols as adornments in their homes."
Ms Abdo Sultan said the bill in its current form increased the risk of vilification and targeting of Muslims, and would create an environment of mistrust between the community and law enforcement agencies.
Committee chair Peter Khalil said the inquiry evidence would inform the legislation.
The ACT is the latest Australian jurisdiction to ban the public display of Nazi symbols, joining Victoria and NSW.
Nazi salutes will not be covered under the federal bill because they fall under the jurisdiction of the states and territories.
10°C