Ensure High-Risk AI Accuracy, Robustness and Cybersecurity

Sub-obligations

These are independently assessable parts of the parent requirement.

1. Article 15(1)-(3)

   Upcoming

   Define, Validate and Declare Appropriate Accuracy Levels and Metrics

   Tracker Guidance

   Identify and use appropriate accuracy levels and metrics for the high-risk AI system and state the relevant accuracy metrics and levels in the accompanying instructions for use.

   Official text

   1. High-risk AI systems shall be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness, and cybersecurity, and that they perform consistently in those respects throughout their lifecycle. 2. To address the technical aspects of how to measure the appropriate levels of accuracy and robustness set out in paragraph 1 and any other relevant performance metrics, the Commission shall, in cooperation with relevant stakeholders and organisations such as metrology and benchmarking authorities, encourage, as appropriate, the development of benchmarks and measurement methodologies. 3. The levels of accuracy and the relevant accuracy metrics of high-risk AI systems shall be declared in the accompanying instructions of use.

2. Article 15(4)

   Upcoming

   Design High-Risk AI for Resilience to Errors, Faults and Inconsistencies

   Tracker Guidance

   Design the high-risk AI system to be as resilient as possible to errors, faults and inconsistencies that may occur within the system or its operating environment. Use appropriate technical and organisational measures, including redundancy or fail-safe measures where relevant, and address harmful feedback loops in systems that continue learning.

   Official text

   4. High-risk AI systems shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems. Technical and organisational measures shall be taken in this regard. The robustness of high-risk AI systems may be achieved through technical redundancy solutions, which may include backup or fail-safe plans. High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way as to eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops), and as to ensure that any such feedback loops are duly addressed with appropriate mitigation measures.

3. Article 15(5); Article 42(3)

   Upcoming

   Protect High-Risk AI Against Cybersecurity and AI-Specific Attacks

   Tracker Guidance

   Apply cybersecurity measures appropriate to the risks and circumstances of the high-risk AI system, including where relevant measures addressing data poisoning, model poisoning, adversarial examples or model evasion, confidentiality attacks and model flaws. Where the system falls within Regulation (EU) 2024/2847 and satisfies the conditions in Article 12(1) of that Regulation, Article 42(3) of the AI Act provides a presumption of compliance with the AI Act's Article 15 cybersecurity requirements.

   Official text

   5. High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities. The technical solutions aiming to ensure the cybersecurity of high-risk AI systems shall be appropriate to the relevant circumstances and the risks. The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training data set (data poisoning), or pre-trained components used in training (model poisoning), inputs designed to cause the AI model to make a mistake (adversarial examples or model evasion), confidentiality attacks or model flaws.

   3. Where high-risk AI systems fall within the scope of Regulation (EU) 2024/2847 and the conditions laid down in Article 12(1) of that Regulation are fulfilled, such systems shall be deemed to comply with the cybersecurity requirements set out in Article 15 of this Regulation.