The government, with the backing of the coalition opposition, rushed legislation through both houses of parliament on Thursday to block the move, passing in a little more than an hour.
"Today's decision is one taken in the national security interests of Australia," Anthony Albanese told reporters in Canberra.
"The government has received very clear security advice as to the risks presented by a new Russian presence so close to Parliament House.
"We are acting quickly to ensure the lease site does not become a formal diplomatic presence."
The Federal Court in May overruled a decision by the National Capital Authority to evict Russia from a site within 500 metres of Parliament House.
Home Affairs Minister Clare O'Neil said the government had made a decision not to allow the site to be used for a future diplomatic presence by any country.
"The government has received clear national security advice that this would be a threat to our national security," she said.
"That is why the government is acting decisively today to bring this long-standing matter to a close."
Russia's existing embassy in the inner-south suburb of Griffith will not be affected by the decision.
"The Australian government maintains diplomatic relations with the Russian Federation, but we will always, always act in Australia's national interest," Ms O'Neil said.
Asked if the Chinese embassy, which is also located close to Parliament House, posed a security risk, Mr Albanese said the government was acting on specific advice which applied to the location of the site previously leased to Russia.
Opposition defence spokesman Andrew Hastie said the coalition would stand with the government on the issue of national security.
"Russia has not acted in good faith towards its neighbours in recent times ... it continues its campaign (in Ukraine) trashing the principles of territorial and political sovereignty," he said.
"There is a trust deficit, there is a real risk to our national interest here and the security advice is that this lease must be terminated."
Opposition home affairs spokesman James Paterson said legal action had led to the laws being necessary.
"It is important on sensitive national security issues like this, that in the national interest we come together to resolve these issues," he said.
The National Capital Authority granted the lease for the Yarralumla site in December 2008. Building approvals followed in 2011.
Under the lease conditions, Russia had agreed to finish construction within three years, but the building remains partially built.
The authority decided to terminate the lease citing that "ongoing unfinished works detract from the overall aesthetic, importance and dignity of the area reserved for diplomatic missions".
The Russian embassy has been contacted for comment.
15°C