Pocock makes deepfakes of PM and Dutton to highlight the dangers of AI

ACT independent Senator David Pocock has taken to creating and releasing deepfake videos of Anthony Albanese and Peter Dutton in a bid to show the threat artificial intelligence poses to Australia’s democracy.

The AI-generated videos show the Prime Minister and the Opposition Leader – in separate videos – announcing and supporting a total ban on gambling advertising.

Senator Pocock posted the videos to social media over the weekend, explaining that they were fakes to show how easy it could be to fool the electorate.

He wants laws banning the use of AI in election materials to be passed through parliament as a matter of urgency.

There are only five weeks of parliament sitting this year and Senator Pocock says news laws must be passed before it rises for the summer break and the nation gets close to the next federal election.

“For years, Australians have called for more integrity in politics and more recently have been expressing their concern about the risks from the unregulated use of artificial intelligence,” he said.

“Independents have been pushing for truth in political advertising, including banning the use of AI in election materials since before the last election. Time is fast running out to regulate this before Australians return to the polls.

“The failure of both major parties to introduce strong truth in political advertising laws and ban the use of AI is irresponsible and putting our democracy at risk.

“We’re seeing a growing number of examples around the world of generative AI being used to try and influence elections.”

The deepfake videos Senator Pocock created to prove his point are reasonably sophisticated and realistic.

The Prime Minister is standing in a Parliament House courtyard, where many announcements to the media are made, and in his sombre manner, tells the nation his government is phasing in over three years a total ban on gambling advertising to “protect the next generation of Australians”.

In a separate video, Mr Dutton appears to be in the Coalition party room inside Parliament House where he too holds regular press conferences to confirm the Coalition’s complete support for the government’s legislation.

As welcome as such bipartisanship might be – especially on this topic – none of it is real.

Discerning viewers might notice a little jarring to the flow of the words being spoken in the videos or recognise phrases each leader has used in previous appearances.

But overall, they seem genuine and would easily convince many voters they were real policy announcements.

Senator Pocock describes his videos as pretty low budget but adds a warning that far better ones could be created by others with malicious intent for use during an election campaign.

In a recent appearance before the Senate inquiry into AI, Australian Electoral Commissioner Tom Rogers said his agency could do little to stop voters being targeted with AI-created deepfake videos and phone calls pretending to be politicians during elections.

“It is absolutely happening at an accelerated rate, particularly as our understanding of this technology increases,” Mr Rogers said.

The commissioner said political deepfakes are not prohibited under electoral laws, which was a matter for the parliament to address.

All the AEC can do is act on misinformation being spread about the electoral processes in Australia.

“Countries as diverse as Pakistan, the United States, Indonesia and India have all demonstrated significant and widespread examples of deceptive AI content,” Mr Rogers told the committee in May.

“The AEC does not possess the legislative tools or internal technical capabilities to deter, detect, or adequately deal with false AI-generated content concerning the election process.

“What we’re concerned about is AI that misleads citizens about the act of voting.”

The commissioner said the AEC was monitoring the progress and effectiveness of newly introduced legislation in South Korea banning AI deepfakes during election campaigns.

But he noted that trying for similar legislation in Australia was a matter for the Federal Parliament.

That’s what Senator Pocock and other independents are trying to bring about, having introduced private members bills and also starting petitions calling for reform.

“Now is the time for the government to act to safeguard our democracy and ensure elections are fought and won as a contest of ideas, not on the basis of who can produce the best deepfakes or tell the most convincing lies,” Senator Pocock said.

The office of Special Minister of State Don Farrell said the government is considering advice from the AEC on the issue.

“This is not technology that we can stop – it’s not going away,” the spokesperson said.

“We have to find a way where Australians can have some protection from deliberately false information and content.”