The veteran media panellist and Canterbury football boss called the NRL and its rules "stupid" five times during a rant on Channel Nine last Monday.
Gould criticised the decision not to award a try to Wests Tigers captain Api Koroisau in the round-eight loss to Brisbane and also took aim at the decision to scrap penalties for incorrect line dropouts this season.
"We are so stupid with our rules. Our game is stupid," Gould, who threw a water bottle on the ground in apparent disgust at the end of his diatribe, said.
Gould has long walked a fine line between juggling his opinionated media punditry and roles with NRL clubs, but last Thursday's fine marked his first financial penalty from the league.
On Monday morning, Gould insisted he hadn't "done anything wrong" and said he intended to dispute the fine - half of which is suspended for 24 months.
"I'll certainly be challenging this breach notice, I can let you know that," Gould said on Today, broadcast by his media employer Channel Nine.
"I haven't done anything wrong. I've got to go through a process, they've given me a breach notice.
"I've been too busy to deal with it over the weekend ... I'm going to have to get a continuance so I can get my lawyer onto it."
The NRL on Thursday said Gould's comments were detrimental to the best interests of the game.
"We welcome constructive criticism and passionate opinions," NRL chief executive Andrew Abdo said.
"But registered officials cannot overstep the mark and make comments that are considered detrimental to the game or NRL competition.
"Destructive attacks on the game itself will not be tolerated."
Gould has until Thursday to formally respond to the NRL's breach notice.
13°C