The incident, during the Muslim holy month of Ramadan and on the eve of the Jewish Passover, sparked a furious reaction in the occupied West Bank and cross-border strikes in Gaza.
It came amid fears tensions built up during a year of escalating Israeli-Palestinian violence could be unleashed at the Al-Aqsa mosque, known to Jews as the Temple Mount.
Palestinian militants fired at least nine rockets from Gaza into Israel overnight, prompting air strikes from Israel that hit what it said were weapon production sites for the Islamist group Hamas, which controls the blockaded coastal enclave.
Hamas did not claim responsibility for the rocket attacks but said these were in response to the Israeli raid on Al-Aqsa, where clashes in 2021 set off a 10-day war with Gaza.
As ground-shaking explosions from the air strikes rocked Gaza, witnesses said Israeli tanks also shelled Hamas positions.
"We are not interested in an escalation, but we are ready for any scenario," IDF spokesman Daniel Hagari said.
As day broke, with international efforts under way to de-escalate the situation, tensions appeared to have calmed in Al-Aqsa, where large crowds of worshippers spent the night as is common during Ramadan.
The Palestinian Red Crescent said 12 Palestinians had sustained wounds during the raid, including from rubber-tipped bullets and beatings, in clashes with police.
It said Israeli forces had prevented its medics from reaching the area.
"In the yard to the eastern part of the compound, the police fired tear gas and stun grenades. It was a scene that I can't describe," said Fahmi Abbas, a worshipper at the mosque.
"Then they stormed in and started beating everyone. They detained people and put the young men face down on the ground while they continued beating them."
Videos circulating on social media, which Reuters could not independently verify, showed fireworks going off and police beating people inside one of the mosque buildings.
Police video showed police entering the building while firecrackers exploded in the darkness.
Israeli police said in a statement security units were forced to enter the compound after what it called masked agitators locked themselves inside the mosque with fireworks, sticks and stones.
"Following many continuous attempts to remove the individuals from the mosque using dialogue failed, police were forced to enter the compound in order to remove the individuals," police said.
"Throughout the presence of police forces in the compound, stones were thrown and multiple firecrackers were set off inside the mosque by many law-breaking individuals and rioters," the statement said, adding two police officers were wounded.
It said more than 350 people who had barricaded themselves inside were arrested and removed.
Thousands of worshippers spent the night in the mosque compound, amid fears of possible clashes with Jewish visitors to the site, which they revere as the Temple Mount, the site of Judaism's two ancient Temples.
Under the longstanding "status quo" arrangement governing the area, which Israel says it maintains, non-Muslims can visit but only Muslims are allowed to worship in the mosque compound.
Jewish visitors have increasingly prayed more or less openly at the site in defiance of the rules.
The Waqf, the Jordanian-appointed Islamic organisation that manages the complex, considered the third holiest site in the Muslim world, described the police actions as a "flagrant assault on the identity and the function of the mosque as a place of worship for Muslims alone".
13°C