Two people were killed in Benguet province, one in Abra province, and one more in another province, Interior Secretary Benjamin Abalos told a news conference.
Sixty people were injured, he said.
The quake struck about 11 kilometres southeast of the town of Dolores on Wednesday at a shallow depth of 10km, according to US Geological Survey data.
"Despite the sad reports about the damages caused by the earthquake, we are assuring quick response to those in need and affected by this calamity," President Ferdinand Marcos Jr said on Facebook.
The quake hit close to Marcos family's political stronghold.
Renato Solidum, director of the state seismology agency, told DZRH radio station, strong aftershocks were expected.
"The focus of attention is on Abra and nearby provinces. This is a major earthquake," Solidum said.
Abalos said 173 buildings were damaged and 58 landslides reported, with 44 of the 60 injured in Abra province.
A hospital in Abra province was evacuated after the building partially collapsed but there were no casualties reported there, officials said.
Abra's vice-governor, Joy Bernos, posted photos of the damaged Abra hospital on her Facebook account, which showed a gaping hole in its facade.
Other photos showed hospital beds, including one with a patient, being wheeled across a road and evacuated hospital staff.
Abra, home to nearly 250,000 people, is a landlocked province in the northern Philippines. Its deep valleys and sloping hills are enclosed by rugged mountains.
The quake damaged heritage buildings in the city of Vigan, known for its old Spanish colonial architecture, on the west coast of Luzon.
The quake was also felt in Manila where several buildings were evacuated, with some people forced to flee from the 30th floor of one building, and the city's metro rail systems were halted at rush hour.
The Philippines is prone to natural disasters and is on the seismically active Pacific "Ring of Fire", a band of volcanoes and fault lines that arcs round the edge of the Pacific Ocean.
Earthquakes are frequent and there are an average of 20 typhoons each year, some triggering deadly landslides.
