Labour MPs voted to remove Dr Sharma, already suspended from the caucus, at a partyroom meeting on Tuesday in Wellington.
Prime Minister Ardern said Dr Sharma was expelled for "repeated and calculated breaches of caucus rules" which "resulted in the complete loss of trust by his fellow Labour MPs".
The Hamilton West MP will now consider whether to sit as an independent or leave parliament.
The vote came at the end of a politically messy week for Ms Ardern, who Dr Sharma accused of lying and covering up a bullying culture.
The Labour member began a series of attacks on his own party on August 11 when he published a cryptic opinion piece in the NZ Herald.
From that point he sharpened and broadened his attacks, taking aim at Parliamentary Services for mismanagement, and Labour's leadership for what he described as a culture of bullying.
His chief target was former whip Kieran McAnulty, who Dr Sharma alleged had bullied and degraded him, including one episode where he said he was left outside his office for two hours while Mr McAnulty drank beer and watched the America's Cup.
Mr McAnulty dismissed the claims as works of fiction, and the prime minister agreed.
"He has certainly made statements that are blatantly incorrect," Ms Ardern said on the way into Tuesday's caucus meeting.
"That makes me deeply uncomfortable."
The Hamilton West MP had been frustrated at a hiring freeze placed on him by Parliamentary Services and supported by Labour, after previous employees filed complaints against him.
He wants an independent investigation into the matter, but Ms Ardern argues his claims do not meet the threshold making them worthy of a probe.
On August 16, he was suspended from the caucus but offered a lifeline of mediation, with a review at year's end, if he ceased his outbursts.
Instead, Dr Sharma gave a series of interviews and leaked private communications from MPs and ministers hinting at wrongdoing.
That left Labour MPs, including Ms Ardern, with no choice but to vote for his expulsion.
Speaking outside the meeting, Dr Sharma said there was one vote against the motion, and an abstention.
"I've been bullied. I've raised genuine claims," he said.
"An independent investigation would clear all of that up."
Speaking ahead of the caucus meeting, Mr McAnulty appeared chastened by Dr Sharma's allegations.
"It's really awful to be accused of something that isn't true in any instance," he said.
"To have that play out in public - it's been really, really tough."
Prior to Tuesday's vote, the most recent Labour MP to be expelled from the caucus was Chris Carter in 2010.
