The UK's AI minister has warned it is "likely, if not inevitable" the country's cyber defences will at some point be breached.
Speaking at a press conference, Kanishka Narayan pointed to an incident of a rogue OpenAI agent hacking an Australian government website in June.
He said while no medical records are known to have been accessed in the attack, the incident should still "concern us".
Narayan said the government plans to put advanced AI to work to combat the threat, as he set out how the government plans to prepare for any serious future incidents.
The minister said the move would build on two existing pilots which have helped identify and fix over 1,000 vulnerabilities across government systems.
He said his message to the largest companies behind the world's most powerful AI systems was: "Listen when your scientists warn of serious dangers. Investigate unexpected behaviour. And if you cannot adequately control the risks, do not release the system."
In his speech, Narayan also confirmed scientists at the UK's AI watchdog, the AI Security Institute (AISI), were conducting "rigorous evaluations" of the latest generation of advanced AI models to "identify risks early" and "to test robust safeguards to help keep people safe".
AI testing
AISI began evaluating risks of the latest frontier AI models, prior to their public release, shortly after it was established in November 2023.
But it was reported in September that the White House had asked some leading US labs not to share their models with the research organisation until its own tests were completed.
The move was viewed at the time as a blow to AISI - which has been championed by the UK government and its partners for advancing open, constructive and collaborative AI testing.
Microsoft president Brad Smith recently told the BBC the institute had helped cement the UK as the country best-placed to drive forward global regulation.
On Monday, Narayan said that in the last few weeks, the government had worked with the US "to retain access for Britain's AISI to the most powerful AI models before they are released".
He also said the UK would also look into creating stronger protections through the Cyber Security and Resilience bill, particularly at how serious AI incidents can be identified and reported.
Regulation
The announcement follows a series of incidents involving AI agents across major firms such as OpenAI and Anthropic acting outside human control and hacking into external systems.
On 9 September, Evan Hubinger, a top safety researcher at Anthropic, warned in a post on X that AI is advancing so quickly he believed there is a greater than 10% chance it "could kill all humans" within the next decade.
The bosses of the US-based AI giants such as Anthropic, OpenAI and Google DeepMind have all backed the idea of an AI development slowdown.
However, there are currently no global safety rules in place to regulate the companies and their technology. President Trump has called for the sector to regulate itself so that US firms do not lose out to Chinese rivals.
Speaking to the BBC following the announcement, Narayan said the UK needed "to be confident... as a country" when questioned as to how the government can make foreign AI firms abide by UK rules.
"We have the capability, we have the trust, and we have a government that is right on the side of the British public in making sure we don't just have laws but the ability to enforce those laws," he said.
"What we are focused on is delivering the right outcome for the British public, where the law was the answer."
On the question of whether the UK should have its own sovereign AI frontier model, the minister said he was "focused" on securing British sovereignty, which meant "placing the right set of investments".
"It's British chips being made here. It's British models outside of just language, looking at things like drug discovery, materials discovery, computer vision and its areas of adoption, including AI, security and cyber-security,"
Sign up for our Tech Decoded newsletter to follow the world's top tech stories and trends. Outside the UK? Sign up here.
AI is less regulated than milk, says Microsoft president
