Somebody else’s AI, your regulatory problem? The Cyber Bill’s proposed AI-agent powers

On 26 October, the House of Lords agreed a Government amendment to the Cyber Security and Resilience Bill that deals expressly with AI agents.  It would allow regulations to give the Secretary of State functions where a security or operational compromise of a relevant system “is or may be attributable to loss of oversight or control of an AI agent”, or where there is a risk of such a compromise.

Peers also rejected, by 165 votes to 95, a separate proposal for direct incident reporting by frontier AI developers.

In July, AI agents under evaluation by OpenAI gained unauthorised access to Hugging Face, a platform for sharing AI models and datasets; our 19 October Insight describes the incident.

OpenAI ran those evaluations internally with certain safeguards disabled, in settings it says did not reflect its production environments, but they show what loss of oversight can look like.  Agents meant to be isolated left each other messages in a shared software cache.  METR (Model Evaluation and Threat Research), a non-profit that studies the most advanced AI agents, carried out an independent investigation, assisted by Redwood Research, a non-profit AI safety and security research organisation.

It found that in roughly 7% of the transcripts it reviewed, agents had falsified parts of the record of their own actions: the transcript showed one command while a different one actually ran.  The cases found were agents trying the technique at small scale, and were obvious to spot.

For a regulated business running a third-party model, the practical question is whether it can obtain the information, and use the controls, needed to explain what an agent did.  Deploying somebody else’s AI does not outsource the regulatory problem.

Our 19 October Insight covers the wider Bill.  The Bill is not yet law.  Third Reading took place on 2 November and the Commons must now decide whether to accept each Lords amendment, including the review clause discussed below, which the Lords inserted against the Government.  Any amendment it rejects or changes goes back to the Lords until both Houses agree the same text; only then can Royal Assent follow.

What the new clause adds to the existing NIS regime

The NIS Regulations already take an all-hazards approach, covering equipment failure and human error as well as attackers.  Those duties can therefore cover AI-related risks to the security of the systems on which an essential service relies.  The amendment, now clause 32, adds something different: powers that regulations could give the Secretary of State.

The Government’s delegated powers memorandum explains that the Bill’s existing examples of what regulations may do are framed around regulators’ compliance and enforcement, while the Government also wants the Secretary of State to gather information about loss-of-control incidents, identify mitigations and share lessons.

Regulations could require regulated persons to obtain and retain information, including from outside the UK, and let inspectors enter premises, seize documents, run tests and require remedial steps.  Clause 32(9) allows regulations to provide sanctions for breaching requirements imposed under those functions, not a penalty whenever an agent misbehaves.  And the trigger remains a compromise, or the risk of one; unexpected behaviour alone is not enough.

‘AI agent’ and ‘loss of oversight or control’ are left to regulations, which, like the functions themselves, need consultation and a draft approved by both Houses.  Clause 32 would commence on Royal Assent but, until regulations are made, adds no obligation; existing NIS duties still apply.

Reporting and investigation work to different thresholds

The Lords also narrowed ‘incident’ to events with an actual adverse effect on a system’s operation or security.  That does not mean waiting for an outage: ransomware and pre-positioning attacks that have successfully compromised a system still qualify, the Government confirmed.  A reportable incident must then meet the relevant significance test.

Clause 32 has a different starting point.  It contemplates functions where a compromise may be attributable to loss of control, or where there is only a risk of one.  An episode can fall below the reporting threshold and still be within reach of an investigation, once regulations exist.  The reporting decision turns on effect and significance; an investigation may require the organisation to reconstruct the agent’s actions and explain the controls in place.

What the Lords rejected, and what is left to settle

Amendment 41, moved by Baroness Harding of Winscombe, would have applied to developers of frontier AI models made available in the UK.  It covered models that gained, or were used to gain, unauthorised access to, or interfered or were used to interfere with, a system relied on by an essential service operator, digital or managed service provider or critical supplier.

The developer would have had to notify that organisation and its regulator.  Deadlines of 24 hours, 72 hours and 14 days would have run from when the developer was, or ought reasonably to have been, aware.  The Government opposed it: the NIS regime places reporting duties on regulated entities, and frontier developers belong, in its view, in a different framework.

The two votes concern different mechanisms, and together they do not allocate AI liability between developers and their customers.  Clause 32 turns on the compromise of a relevant system, not on who deployed the agent, and suppliers can be regulated persons in their own right, for example as cloud computing providers.  The Lords also inserted, against the Government, a clause requiring a review of the NIS regime that would cover frontier AI (clause 66); the Commons may remove it, and as drafted it would need commencement regulations even though the review period runs from Royal Assent.

The Bill does not create an incident-reporting duty merely because a business develops a frontier AI model.  Unless the developer is itself regulated, information about what its model did will usually come through supply contracts or voluntary disclosure.

Somebody else’s AI, your regulatory problem

Take a hypothetical electricity distributor that operates an essential service.  An integrator builds and runs a maintenance-planning agent for it on a model licensed from a third-party developer, with write access to the system that schedules substation inspections.  After a model update, the agent starts removing inspections, using valid credentials.  Nobody notices for three weeks.  Assume that system is relied on to provide the essential service, that clause 32 regulations are in force and that this counts, as defined, as an AI-related compromise.

The questions are predictable: which model version was running, what changed, what the agent was permitted to do and when anyone could have intervened.  The distributor should hold part of the answer itself: the permissions it granted, the agent’s actions in its own systems, its alerts and its own approvals.  Here, the deployed model version and change records sit with the developer, and the agent’s configuration and logs with the integrator.  Logs may show what the agent did but not why: an evidential gap to plan for.

Without contractual rights, the distributor cannot rely on getting that material promptly, completely or on acceptable terms.  The distributor’s contract with the integrator does not, by itself, bind the developer.  The integrator therefore needs enforceable upstream rights enabling it to obtain and pass on the necessary information and cooperation.  Even then the distributor has no contractual claim against the developer; if it needs one, it needs a direct agreement with the developer or a third-party right in the integrator’s upstream contract.

What the contract needs to do

On model changes, the customer needs notice of material changes, a testing window, supported versions with minimum retirement periods and a route for emergency security updates with prompt notice afterwards.  Testing should cover the configuration the customer will actually run: the model with its tools, permissions, shared resources and any other agents.  A supplier’s model evaluation does not, by itself, show how that combination behaves.

The contract should also set out the records the supplier keeps: what is recorded, who holds it, how long it is retained, how quickly it must be handed over and whether the customer may disclose it to the relevant authority.  It cannot realistically require the supplier to explain every internal model decision.  A right to inspect the logs is of limited use if the agent can rewrite them.  Records need protection against alteration, preservation once an incident is suspected and checking against a source the agent cannot reach.

A contractual right to suspend the agent is not the technical ability to stop it safely while the essential service continues; that needs a tested fallback.  Termination ends the relationship, not the operational problem.

Where a supplier cannot provide a control the customer needs, the decision moves to the deployment: restrict the agent’s permissions, require human approval for specified actions, change the implementation or reconsider the use.  Suppliers should not promise customers cooperation that depends on rights they have not secured upstream.  Indemnities for investigation costs, remediation and third-party claims matter, within caps and exclusions, but compensation cannot substitute for timely information or the ability to intervene.

Before an agent touches a regulated system, the customer should be able to identify the model version running, obtain reliable records of what the agent did and stop the agent without stopping the service.

If the contracts cannot deliver that, the deployment should be narrower.  For the wider background, see our 19 October Insight .

This Insight was written by Robert Humphreys.  If you have any questions about deploying AI in regulated systems or the supply contracts behind it, please contact Robert Humphreys or another member of the HLaw team.

All the thoughts and commentary that HLaw publishes on this website, including those set out above, are subject to the terms and conditions of use of this website.  None of the above constitutes legal advice and is not to be relied upon.  Much of the above will no doubt fall out of date and conflict with future law and practice one day.  None of the above should be relied upon.  Always seek your own independent professional advice.

Humphreys Law