But 24 days later, Pallant allegedly fatally struck the Melbourne woman's toddler with a metal table while babysitting.
Jaidyn Gomes-Sebastiao, affectionately known as Snuffles, was just two years old when he died on September 2, 2019.
Pallant has pleaded not guilty to the boy's murder and is facing a trial in Victoria's Supreme Court.
Prosecutor Mark Gibson told the jury on Wednesday the 35-year-old was frustrated, angry and lost self-control when he allegedly entered the toddler's bedroom and hit him in the head with a 4.5 kilogram table.
However, the case will rely on circumstantial evidence as there were no direct witnesses or video evidence to prove Pallant struck Jaidyn.
"It's a bit like a jigsaw puzzle," Mr Gibson said. "The strength of the case does not depend on any particular piece of the puzzle, but depends on the clarity of the picture when the jigsaw puzzle is completed."
Ms Saggers was introduced to Pallant by a mutual friend in August 2019 and within days they moved in together as she needed help looking after her two children while she was at work.
She was still mourning the death of her partner and father to two of her children in a motorcycle crash nine months earlier.
The day before Jaidyn died, Ms Saggers noticed he had a bruise over his eye and took photos to document it. She believed it may have been caused by a fallen bookshelf.
The following day, she left Pallant to babysit her children while she went to a cleaning job.
When Ms Saggers was on her way home, Pallant called her and said Jaidyn was asleep in his room.
He told her not to "rock the boat" and disturb the toddler when she got home.
The couple went for a nap in their room, then Pallant checked on Jaidyn just after 4pm.
Pallant told police there was resistance when he opened Jaidyn's bedroom door and that he found the toddler lying on the floor a multi-coloured activity table on top of him.
Pallant scooped the little boy up, brought him into the lounge room and laid him on the floor.
Ms Saggers was hysterical upon seeing her injured child, emergency services were called and a neighbour rushed over after hearing her screaming.
Jaidyn was pronounced dead at the scene by paramedics, due to injuries to his brain and skull. His body was covered in dozens of bruises and scratches.
Pallant's lawyer will address the jury this afternoon, as the trial before Justice Jane Dixon continues.