Herzog is scheduled to hold talks with Turkish President Recep Tayyip Erdogan in Ankara before travelling to Istanbul for meetings with members of Turkey's Jewish community.
Turkey and Israel once were close allies, but the relationship frayed under Erdogan, who is an outspoken critic of Israel's policies toward the Palestinians.
Israel also has been angered by Erdogan's embrace of Hamas, the militant group that controls the Gaza Strip. Israel considers Hamas a terrorist group.
The countries withdrew their respective ambassadors in 2010 after Israeli forces stormed a Gaza-bound flotilla carrying humanitarian aid for Palestinians that broke an Israeli blockade. The incident resulted in the deaths of nine Turkish activists.
Relations broke down again in 2018 when Turkey, angered by the US moving its embassy in Israel to Jerusalem, once more recalled its ambassador, prompting Israel to respond in kind. The two countries have not reappointed their ambassadors.
The steps toward a rapprochement with Israel comes as Turkey, beset by economic troubles, has been trying to end its international isolation by normalising its ties with several countries in the Mideast region, including Egypt, the United Arab Emirates and Saudi Arabia.
"We will not agree on everything, and the relationship between Israel and Turkey has certainly known ups and downs and not-so-simple moments in recent years," Herzog said before his departure.
"But we shall try to restart our relations and build them in a measured and cautious manner, and with mutual respect between our states."
In a step toward reconciliation, Erdogan called Herzog by phone after the Israeli head of state took office last year.
The two have held several telephone conversations since then. Erdogan has also spoken to Prime Minister Naftali Bennett following the release of an Israeli couple who were arrested in Istanbul on suspicion of spying.
During a visit to Cyprus last week, Herzog offered reassurance that Israel's warming relation with Turkey would not come at the expense of ties with Nicosia. Herzog made similar remarks in Greece last month,, insisting Israel would continue to expand its cooperation with Greece and Cyprus, which both have tense relations with Turkey.
Turkey has said there would be no change to Ankara's position toward the Palestinians despite the normalisation efforts with Israel.
