Ukraine's ambassadors to Norway, the Czech Republic, Hungary and India were also dismissed, the decree said. It did not state why, or what they would do in the future.
"Today, I signed decrees on the dismissal of some ambassadors of Ukraine," Zelenskiy said later on Saturday.
"This rotation is a normal part of diplomatic practice. New representatives of Ukraine will be appointed to the Czech Republic, Germany, Hungary, Norway and India. Candidates are being prepared by the Ministry of Foreign Affairs," he said in a video message.
The Ukrainian embassy in Berlin did not comment on the decree. "The Foreign Office has not yet been notified of the ambassador's dismissal," a spokeswoman for the Foreign Office said upon request.
Melnyk became known for his repeated demands for more support from the German government, in the form of heavier weapons, as Russia's attacks continued in the war launched by Moscow on February 24.
Recently, Melnyk came under pressure for defending the Ukrainian nationalist leader Stepan Bandera, who led a group responsible for ethnic cleansing and the massacre of tens of thousands of Poles and Jews during World War II.
Melnyk denied that Bandera was a mass murderer, saying he had been deliberately demonised by the Soviet Union, in comments during an interview.
The Israeli embassy accused Melnyk of "distorting historical facts, trivialising the Holocaust and insulting those who were murdered by Bandera and his people".
Melnyk initially did not respond but later, he rejected the accusation.
"Everyone who knows me, knows: I have always condemned the Holocaust in the strongest possible terms," Melnyk tweeted, calling the accusations "absurd".
Melnyk has served as ambassador to Germany since January 2015, longer than diplomats usually remain in a post. Commentators in Kiev also noted that this was twice the usual posting time.
Bundestag deputy head Katrin Göring-Eckardt expressed her respect for Melnyk after news of his dismissal emerged.
"Andriy Melnyk has worked for his country with all his might. He is an unmistakable and tireless voice for a free Ukraine," the politician said.
However, she stressed that she disagreed with Melnyk regarding Bandera.
"Regardless, I wish him all the best personally, for his future service and above all for his country."
Some German media, including Bild and Süddeutsche Zeitung newspapers, reported that Melnyk is to be moved to the Foreign Ministry in Kiev, citing Ukrainian sources.
