The Shop, Distributive and Allied Employees' Association, which represents retail, fast-food and warehousing workers, wants an extra week of paid leave in addition to "roster justice".
Appearing at a hearing into national employment standards on Friday, national secretary Gerard Dwyer said industries such as retail had moved to seven-day trading and the intensification of work in the past decade.
He said many workers had described inadequate paid time-off, resulting in missed family time and lost social connection.
"Responses in our surveys repeatedly refer to being unable to attend birthdays, family gatherings, school events or significant cultural and community occasions, because leave is limited and too quickly exhausted for workers in retail with weekend work, shift work or extended trading hours," Mr Dwyer said.
"Annual leave is therefore not simply time away from work, but a key mechanism by which workers maintain meaningful connection to family unity and social life."
On the issue of rostering, Mr Dwyer said workers faced late roster changes, short shifts and unpredictable hours.
This undermined income stability, the ability to plan caring responsibilities and the capacity to identify underpayments where they're occurring, he added.
"Ongoing pressure exists because of the fear of losing hours actually discourages workers, especially young and casual workers, from raising concerns about their rosters," Mr Dwyer said.
"They simply suffer in silence. What roster justice means ... is we need advance notice of rosters, we need stable rosters, we need compensation for late changes or the cancellation of shifts."
The Australian Council of Trade Unions has called for the weekly number of hours worked by full-time employees to be reduced from 38 to 35.
If a four-day work week is not possible, sector-specific alternatives should be proposed, the peak body argues.
ACTU president Michele O'Neil said given how rapidly the environment was changing for workers, minimum standards should be regularly reviewed.
"We want to make sure that when we think about improving productivity, that workers get their fair share of it, and time is a really critical issue for working people," she told the inquiry.
The council has also called for an increase in annual leave for workers, which it said would make up for the unpaid leave employees performed.
Under the proposal, this would be up from four to five weeks per year - and from five to six weeks for regular shift workers.
Business groups strongly opposed the proposed annual leave increase, with the Business Council's chief executive Bran Black saying a one-size-fits-all approach was wrong.
"With weak productivity growth, inflation rising and living standards under pressure, this proposal does nothing to address these real challenges," he said.
"This proposal assumes businesses can simply absorb the cost of extra leave without any improvement in productivity."