The two leading candidates will advance to the second round of voting on October 25, after topping a field of 12 candidates in a campaign focused on concerns about the growing cost of living, corruption and insecurity in Brazilians' daily lives.
Bolsonaro, the son of right-wing former President Jair Bolsonaro, who is serving a 27-year sentence for plotting a coup after losing the 2022 election, now heads to a showdown with the leftist Lula, who picked up 44.9 per cent of votes, with nearly 99 per cent of votes reported.
Private surveys in the run-up to the election had forecast Lula leading the first-round vote by around three percentage points. Bolsonaro's stronger-than-expected performance will buoy his campaign and may bolster Brazil's currency and stock market, which have risen on the prospects of the business-friendly challenger.
A victory for the senator would extend a wave of wins across Latin America by right-wing candidates aligned with US President Donald Trump, while Lula's re-election could be a challenge for Washington, which has clashed with his government over trade and security.
Despite growing worries over Brazil's public finances and sluggish economic growth, the campaign offered meagre policy discussion, as both leading candidates skipped the major TV debates.
Early on Sunday, Bolsonaro told journalists in Rio de Janeiro he was counting on "a protest vote from people who would otherwise vote for another candidate but are opposed to Lula".
Despite falling inequality and unemployment, Lula has struggled to win over voters grappling with high living costs, heavy household debt and the threat of crime and more pervasive gang activity.
Bolsonaro has sought to capitalise on cost-of-living concerns, while Lula has moved to ban online gambling and, as the race tightened, expanded welfare and proposed fresh debt relief for families.
Rising spending and public debt above 82 per cent of GDP have stoked fiscal concerns that Brazil's next president will need to address. The government lowered its economic growth forecasts for this year and next to 2.0 and 2.3 per cent respectively.
Scandals and fears of foreign interference ahead of the vote largely overshadowed policy debates during the campaign.
Throughout the campaign, Lula's allies have warned the Trump administration might refuse to recognise election results if he wins, pointing to an aborted effort to send US officials to Brazil.
Trump has not formally endorsed a candidate, though he hosted Senator Bolsonaro at the White House and praised him publicly.
Reuters reported on Wednesday that Brazilian intelligence officials briefed a French and German delegation on alleged US and Russian interference before the election. The US has rejected claims of interference in Brazil's affairs.
On Sunday afternoon, Brazil's electoral court chief Kassio Nunes Marques said the election was running smoothly.