Maintain a Quality Management System for High-Risk AI

Sub-obligations

These are independently assessable parts of the parent requirement.

1. Article 17(1)(a)

   Upcoming

   Maintain a Regulatory Compliance and High-Risk AI Modification Management Strategy

   Tracker Networks Guidance

   Maintain a regulatory-compliance strategy for the high-risk AI system, including the applicable conformity-assessment approach and a process for managing modifications to the system.

   Official text

   (a) a strategy for regulatory compliance, including compliance with conformity assessment procedures and procedures for the management of modifications to the high-risk AI system;

   Suggested controls

   • High-Risk AI Quality Management System

     Maintain a documented QMS integrating regulatory strategy, lifecycle controls, data, risk, monitoring, incidents, records and accountability.

   Related risks

   • Weak High-Risk AI Quality Management

     Provider governance may not consistently control AI design, development, testing, data, risk, incidents, records and accountability.

2. Article 17(1)(b)-(d)

   Upcoming

   Control High-Risk AI Design, Development, Quality Assurance, Testing and Validation

   Tracker Networks Guidance

   Define and operate systematic procedures for design, design control, design verification, development, quality control, quality assurance, examination, testing and validation. Specify when and how often required tests and validations are performed.

   Official text

   (b) techniques, procedures and systematic actions to be used for the design, design control and design verification of the high-risk AI system; (c) techniques, procedures and systematic actions to be used for the development, quality control and quality assurance of the high-risk AI system; (d) examination, test and validation procedures to be carried out before, during and after the development of the high-risk AI system, and the frequency with which they have to be carried out;

   Suggested controls

   • High-Risk AI Testing and Validation

     Test high-risk AI against predefined performance and risk metrics before release and at appropriate lifecycle points.

   • High-Risk AI Quality Management System

     Maintain a documented QMS integrating regulatory strategy, lifecycle controls, data, risk, monitoring, incidents, records and accountability.

   Related risks

   • AI Accuracy, Robustness or Cybersecurity Failure

     High-risk AI may perform inaccurately, fail under faults or be manipulated through conventional or AI-specific attacks.

   • Weak High-Risk AI Quality Management

     Provider governance may not consistently control AI design, development, testing, data, risk, incidents, records and accountability.

3. Article 17(1)(e)

   Upcoming

   Define Applicable Technical Specifications, Standards and Alternative Compliance Measures

   Tracker Networks Guidance

   Identify the technical specifications and standards used to demonstrate compliance. If relevant harmonised standards are not fully applied or do not cover all applicable requirements, document the means used to ensure the high-risk AI system still meets those requirements.

   Official text

   (e) technical specifications, including standards, to be applied and, where the relevant harmonised standards are not applied in full or do not cover all of the relevant requirements set out in Section 2, the means to be used to ensure that the high-risk AI system complies with those requirements;

   Suggested controls

   • High-Risk AI Quality Management System

     Maintain a documented QMS integrating regulatory strategy, lifecycle controls, data, risk, monitoring, incidents, records and accountability.

   Related risks

   • Weak High-Risk AI Quality Management

     Provider governance may not consistently control AI design, development, testing, data, risk, incidents, records and accountability.

4. Article 17(1)(f)

   Upcoming

   Maintain High-Risk AI Data Management Procedures

   Tracker Networks Guidance

   Maintain procedures governing the data operations relevant to the high-risk AI system, including acquisition, collection, analysis, labelling, storage, filtration, mining, aggregation, retention and other applicable data-management activities.

   Official text

   (f) systems and procedures for data management, including data acquisition, data collection, data analysis, data labelling, data storage, data filtration, data mining, data aggregation, data retention and any other operation regarding the data that is performed before and for the purpose of the placing on the market or the putting into service of high-risk AI systems;

   Suggested controls

   • High-Risk AI Data Governance and Bias Management

     Govern training, validation and testing data for suitability, quality, representativeness and bias management.

   • High-Risk AI Quality Management System

     Maintain a documented QMS integrating regulatory strategy, lifecycle controls, data, risk, monitoring, incidents, records and accountability.

   Related risks

   • AI Data Quality, Representativeness or Bias Failure

     Training, validation or testing data may be unsuitable, poor quality, unrepresentative or biased for the intended high-risk AI use.

   • Weak High-Risk AI Quality Management

     Provider governance may not consistently control AI design, development, testing, data, risk, incidents, records and accountability.

5. Article 17(1)(g)-(i)

   Upcoming

   Integrate Risk Management, Post-Market Monitoring and Serious Incident Reporting Into the QMS

   Tracker Networks Guidance

   Ensure the quality management system incorporates the Article 9 risk-management process, Article 72 post-market monitoring and Article 73 serious-incident reporting processes.

   Official text

   (g) the risk management system referred to in Article 9; (h) the setting-up, implementation and maintenance of a post-market monitoring system, in accordance with Article 72; (i) procedures related to the reporting of a serious incident in accordance with Article 73;

   Suggested controls

   • High-Risk AI Risk Management Process

     Operate a documented lifecycle risk-management process for high-risk AI systems.

   • High-Risk AI Quality Management System

     Maintain a documented QMS integrating regulatory strategy, lifecycle controls, data, risk, monitoring, incidents, records and accountability.

   • AI Incident Escalation and Notification

     Detect, triage and escalate serious AI incidents and relevant risk conditions within required timelines.

   Related risks

   • Inadequate High-Risk AI Risk Management

     High-risk AI risks may not be identified, assessed, treated, tested or monitored effectively through the lifecycle.

   • Weak High-Risk AI Quality Management

     Provider governance may not consistently control AI design, development, testing, data, risk, incidents, records and accountability.

   • Failure to Correct or Report Non-Conforming High-Risk AI

     Non-conformity or material risk may not trigger timely containment, investigation, corrective action and notifications.

6. Article 17(1)(j)-(k)

   Upcoming

   Maintain QMS Procedures for Regulatory Communications and Record-Keeping

   Tracker Networks Guidance

   Maintain procedures for communications with competent authorities, notified bodies, other operators, customers and other relevant parties, together with systematic record-keeping for compliance documentation and information.

   Official text

   (j) the handling of communication with national competent authorities, other relevant authorities, including those providing or supporting the access to data, notified bodies, other operators, customers or other interested parties; (k) systems and procedures for record-keeping of all relevant documentation and information;

   Suggested controls

   • High-Risk AI Quality Management System

     Maintain a documented QMS integrating regulatory strategy, lifecycle controls, data, risk, monitoring, incidents, records and accountability.

   • Regulatory Information and Cooperation Process

     Respond to reasoned regulator requests with controlled, complete and retrievable compliance information and logs.

   Related risks

   • Weak High-Risk AI Quality Management

     Provider governance may not consistently control AI design, development, testing, data, risk, incidents, records and accountability.

   • Regulatory Cooperation Failure

     The organisation may be unable to provide complete, timely and understandable information, records or cooperation to competent authorities.

7. Article 17(1)(l)-(m)

   Upcoming

   Maintain Adequate Resources and Management Accountability for High-Risk AI Compliance

   Tracker Networks Guidance

   Define how the provider manages resources needed for high-risk AI compliance, including security-of-supply considerations where relevant, and establish clear accountability for management and staff responsibilities within the quality management system.

   Official text

   (l) resource management, including security-of-supply related measures; (m) an accountability framework setting out the responsibilities of the management and other staff with regard to all the aspects listed in this paragraph.

   Suggested controls

   • High-Risk AI Quality Management System

     Maintain a documented QMS integrating regulatory strategy, lifecycle controls, data, risk, monitoring, incidents, records and accountability.

   Related risks

   • Weak High-Risk AI Quality Management

     Provider governance may not consistently control AI design, development, testing, data, risk, incidents, records and accountability.