AI incident reporting and the Cyber Security and Resilience Bill: current duties, proposed changes and the Lords’ amendments

An AI agent, a system that can log in and run code with little human supervision, signs in to a water company’s maintenance system without permission.  It changes nothing.  Under an amendment now before the House of Lords, the developer of the underlying model could still have to report the incident within 24 hours, even where the water company is not required to notify it under the NIS regime.

On 12 October 2026 the government said it would consider, through the Cyber Security and Resilience Bill and the wider regime, “how serious AI incidents and near misses can be identified and reported”.  For businesses already within the NIS regime, that would build on reporting duties that apply whatever caused an incident.  The Commons Business, Innovation, Science and Trade Committee has opened an inquiry that will examine the proposals.  What the amendment adds is a duty on the party that may be the last to hear what its model did.

The incidents behind the proposals

In July, OpenAI reported that models under internal evaluation with reduced cyber safeguards, including an internal-only prototype, had worked their way out of a restricted test environment and compromised parts of Hugging Face’s production infrastructure.  In August, the AI Security Institute (“AISI”) reported that in ten of 122 test runs, with internet access deliberately enabled and some cyber safeguards disabled, agents took “unsanctioned action on the live internet, targeting real people and organisations”, including sending messages with harmful payloads and trying, unsuccessfully, to get malicious code into an open-source project.  Neither was an ordinary customer deployment.  The first was a completed compromise of a third party’s systems; the second was largely attempted compromise and social engineering, which matters because the proposed developer duty depends on access or interference actually occurring.

Duties that already apply, whatever the cause

No UK legislation imposes an AI-specific incident reporting duty on model developers, and AISI has no statutory footing.  Existing duties, however, are technology-neutral.  Under the Network and Information Systems Regulations 2018 (the “NIS Regulations”), an operator of an essential service, such as an energy or water company, must notify its regulator without undue delay, and within 72 hours, of an incident with a significant impact on the continuity of that service.  Providers of cloud computing services, online marketplaces and search engines have a similar duty.  Personal data breaches engage the UK GDPR whatever their cause.

What the Bill would change, once in force

The Bill returns to the Lords for Report stage on 26 October.  Royal Assent would not bring it all into force: the government plans phased implementation, with incident reporting among the measures needing secondary legislation.

Once the relevant provisions commence, the Bill as drafted would:

• widen ‘incident’ to events “capable of having” an adverse effect, while keeping significance tests for reporting (clause 15);
• require an initial notification within 24 hours of awareness and a full notification within 72 hours, copied to the national computer security incident response team (the “CSIRT”) (clause 15);
• bring data centre operators and managed service providers within the regime (clauses 4 and 9);
• require data centre operators, digital service providers and managed service providers, as soon as reasonably practicable after a full notification, to identify and notify UK customers likely to be adversely affected (clause 16); and
• let regulators designate critical suppliers, whose duties are to be set in secondary legislation (clause 12).

Developing AI is not itself a regulated category, although an AI business may be regulated as a cloud computing or managed service provider, or designated as a critical supplier.

Clauses 29 and 30 point to a further possibility.  Regulations made by the Secretary of State could impose requirements, including “requirements relating to the reporting of specified matters”, on persons providing an ‘activity-critical supply’, meaning “a supply of goods or services without which the carrying on of an essential activity would be at risk of disruption”.  On our reading, if the Bill is enacted in this form, a model provider on which an essential activity depends could be brought in by regulations, with duties beyond reporting.  That test turns on what the customer relies on rather than how capable the model is, so it would reach different developers from a general duty on frontier developers.

What the amendments would add

Baroness Harding of Winscombe and Lord Clement-Jones have tabled two new clauses for Report stage.  The first concerns frontier AI models, meaning highly capable general-purpose models matching or exceeding the most advanced contemporary ones.  Where such a model has gained, or been used to gain, unauthorised access to a system relied on by a regulated person, or has interfered or been used to interfere with one, a developer that is aware, or “ought reasonably to be aware”, of it would have to notify that person and its regulator.  It would have 24 hours for an initial notification, 72 hours for a full notification and 14 days for a final report, each measured from that actual or constructive awareness, with copies to the CSIRT and AISI.  Penalties would match those for operators of essential services.

The second would let regulated persons notify the CSIRT or their regulator voluntarily of “an incident, a cyberthreat or a near miss”, without the notification of itself increasing their liability.  Both are backbench proposals.

• The wording may also catch unreleased models. A developer is within the clause if it develops a frontier AI model made available in the UK, but the incident need only involve ‘a frontier AI model of’ that developer.  On that wording an unreleased research model could be caught, unless the phrase is read as limited to models made available in the UK, which the clause does not say.  In the EU, the AI Act excludes models used for research, development or prototyping before they are placed on the market (Article 3(63)), although the Commission treats internal use essential to providing a product or service to third parties as placing on the market.
• ‘Near miss’ can describe different circumstances. The developer duty requires access or interference to have occurred, so an attempt involving neither would fall outside it.  A successful intrusion whose damaging consequences were prevented, which many businesses would also call a near miss, could fall within it.  The voluntary clause is wider: its ‘near miss’ is an event that could have compromised data or services but was prevented or did not do so.
• The developer may not be first to know. In July, Hugging Face’s own security team detected and stopped the activity on its systems.  A developer may learn of an incident from its monitoring, its customer or the affected party, and the ‘ought reasonably to be aware’ test makes monitoring a compliance question.  Contracts can require prompt information, preserve access to records and allocate cooperation.  They cannot discharge a statutory duty, guarantee detection or bind an affected party with no contract with the developer.

How the EU and California regimes differ

The developer clause is drawn around systems: unauthorised access or interference is enough, whether or not harm follows.  The EU AI Act defines a ‘serious incident’ by consequence, from death or serious harm to health to infringement of obligations protecting fundamental rights (Article 3(49)).  Providers of general-purpose models with systemic risk must report serious incidents “without undue delay” to the AI Office and, as appropriate, national competent authorities (Article 55(1)(c)).  That obligation has applied since 2 August 2025, but models already on the market before then have until 2 August 2027 to comply (Article 111(3)).  Our EU AI Act piece covers the Act’s other obligations.

California’s Transparency in Frontier Artificial Intelligence Act (SB 53), enacted in September 2025, applies to developers of models trained using more than 10²⁶ operations.  Most of its ‘critical safety incident’ limbs require death, bodily injury or catastrophic harm, but one covers a model deceptively subverting its developer’s controls outside an evaluation designed to elicit that behaviour, where that shows materially increased catastrophic risk (section 22757.11).  Reports go to the state’s Office of Emergency Services within 15 days, or within 24 hours to an appropriate authority where there is an imminent risk of death or serious physical injury.  The UK clause could therefore catch an intrusion that does not meet either regime’s reporting test, while missing a rights infringement or consumer harm that involves no relevant system.

Worked example

Take a hypothetical UK managed service provider with 120 staff whose customers include a regional water company serving 1.5 million people.  Its patching agent, built on a frontier model, fails to finish a patch at 02:00 on a Saturday and uses credentials found in a configuration file to log in to the water company’s maintenance-scheduling system for its treatment works.  The account could reschedule maintenance.  The water company’s security team flags the login at 07:30 and revokes the credentials at 08:10.  By mid-afternoon, with the provider’s logs, it has established that the agent viewed the schedule for 11 minutes, changed nothing and reached no other system.  Assume the Bill’s relevant provisions have commenced, the provider qualifies as a relevant managed service provider and the system supports the essential service.  Each party must assess its reporting obligations as the facts emerge; the afternoon’s findings can support or revise that assessment, but do not postpone a deadline that has already started.

• The water company. Under the current NIS Regulations, an incident with no impact on continuity of supply is not reportable.  Under the Bill, the login is an incident affecting systems relied on for the essential service, and it is reportable if its impact has been, is, or is likely to be, significant, judged by disruption, users affected, duration, area and any compromise of the confidentiality, authenticity, integrity or availability of data.  The Bill is meant to catch some attacks likely to have significant impacts before they have had any, and write access to maintenance schedules for a large supply area could have produced them.  Whether anything was changed, or supply disrupted, does not settle the question.  On the investigation’s findings, the operator could reasonably conclude that the impact has not been, is not and is not likely to be significant, but that depends on its assessment of all the factors.  It should record why.
• The provider. Its test turns on the systems relied on to provide its managed service, which include the agent, and adds the impact on customers’ systems and on the economy or society.  The size of the water company weighs towards significance; the investigation’s findings weigh against it.  If the incident does not meet its reporting threshold, no full notification is due and the customer-notification duty that follows one does not arise, although its contract may require prompt disclosure.
• Personal data. If the login gave the agent unauthorised access to engineers’ names and phone numbers, there would also be a personal data breach.  The water company, as controller, would have to notify the Information Commission (which took over the Information Commissioner’s functions on 30 September 2026) without undue delay and, where feasible, within 72 hours, unless the breach is unlikely to result in a risk to individuals’ rights and freedoms, and must document it either way; the provider, if a processor, must tell the water company without undue delay.
• The developer. Were the developer clause in force, its trigger would be met whatever the significance, and the developer would have 24 hours from when it was, or ought reasonably to have been, aware to notify the water company and its regulator.

The developer could therefore have a reporting duty under the proposed clause even where neither the operator nor the provider has a reportable incident under its NIS test.  Its report would reach a regulator the operator had, on reasonable grounds, decided not to notify, and the operator would need to be ready to explain that decision; under the voluntary clause, it might choose to notify first.  Because the developer’s deadline can run from when it ought reasonably to have been aware, much turns on what its own monitoring shows and how quickly the provider tells it.

What to do now

• Establish your category. Existing operators and digital service providers face changed duties; managed service providers, data centre operators and designated critical suppliers face new ones; AI suppliers and customers outside the regime may face related contractual requirements.  Investors should ask portfolio companies the same question.
• Give someone the decision. Name who decides whether an incident is reportable, with a deputy and weekend cover, and set internal escalation thresholds below the statutory ones.  An agent exceeding its task instructions has not necessarily compromised security; one using credentials it was never given probably has.
• Keep accurate records and act on them. Preserve agent logs (instructions, tool calls and outputs), access logs and the reasons for each notification decision, and record what was learned and fixed.  Records can show prompt remediation.  They can equally show warnings that nobody acted on.
• Arrange supplier cooperation in advance. Terms with AI providers and customers should require notice within hours on initial facts, without waiting for root cause, plus access to logs, named contacts, regulator cooperation and alignment with insurance notification requirements.  On the UKJT’s analysis, the same contracts are the primary mechanism for allocating liability along an AI supply chain, but they will not detect an incident or make the reporting decision.

To discuss where your business sits under the Bill, or how your contracts deal with AI incidents, contact a member of the HLaw team.

All the thoughts and commentary that HLaw publishes on this website, including those set out above, are subject to the terms and conditions of use of this website.  None of the above constitutes legal advice and is not to be relied upon.  Much of the above will no doubt fall out of date and conflict with future law and practice one day.  None of the above should be relied upon.  Always seek your own independent professional advice.

Humphreys Law