As a painstaking search for survivors continued, a European rescue support charity said it believed about 750 people were on board the 20-30 metre-long vessel.
The United Nations' migration agency estimated up to 400 while Greece declined to speculate on the passenger count.
By midday, 104 people had been rescued.
A media report said the boat left from Libya and a shipping ministry official who spoke on condition of anonymity said most of those on board were from Egypt, Syria and Pakistan.
Search and rescue operations were to continue through the night, with military aircraft deploying flares to light up the Mediterranean waters around the wreck site about 80km southwest of the southern Greek coastal town of Pylos.
Survivors were taken to the Greek port of Kalamata near Pylos.
Covered in blankets, they rested on mattresses at a warehouse shelter, and the migration ministry was expected to move them to a camp outside Athens.
The shipwreck was the deadliest off Greece in several years.
In February, 96 people died when their wooden boat smashed into rocks on Italy's Calabrian coast during a storm.
Greece's caretaker administration, in power between an inconclusive election on May 21 and new elections on June 25, declared three days of mourning.
Greek state broadcaster ERT said the boat was en route to Italy from the Libyan town of Tobruk, which lies south of the Greek island of Crete.
Greek authorities did not confirm the vessel's departure port.
Alarm Phone, which operates a trans-European network supporting rescue operations, said it received alerts from people on board a ship in distress off Greece late on Tuesday, with erratic contact after that.
"According to the people, there were 750 people on board... We now hear reports of a shipwreck and fear they are true," it said on Twitter.
Greek coast guard spokesman Nikos Alexiou, speaking to Greece's MEGA TV, said authorities did not know how many were on the boat, especially below deck, but reported it was crowded.
"There were too many people on the outer deck. It was full," he said.
The Greek coast guard said EU border agency Frontex first spotted the boat on Tuesday in international waters southwest of Pylos, and Italian authorities then alerted Greece to the vessel's presence.
Alarm Phone said it informed Greek authorities, Frontex and the Greek division of the UN refugee agency UNHCR late on Tuesday afternoon.
The charity said that shortly afterwards, it spoke to people on the vessel who said the captain had fled on a small boat and appealed for help.
The Greek coast guard reported its agents approached the vessel and offered help.
But migrants packed on its outer deck "refused assistance and stated their desire to continue their voyage," according to the coast guard.
In aerial images released by the coast guard, dozens of people on the boat's upper and lower decks were looking up with outstretched arms.
A few hours later, the vessel began veering from side to side before capsizing about 2am on Wednesday then sinking, a government official said.
Greece is one of the main routes into the European Union for refugees and migrants from the Middle East, Asia and Africa.
But since the previous conservative government of Kyriakos Mitsotakis introduced tougher controls at the country's migrant camps, more people have chosen to make a longer, riskier sea journey from Turkey to Italy via Greece.
The Greek migration ministry blamed international smuggling networks for putting migrants' lives at risk, while Filippo Grandi, the UN High Commissioner for Refugees, called on governments to work together on creating safe pathways for people fleeing poverty and war.
