TechnicalReport
Firstedition2024-01
Artificialintelligence-Functional safetyandAlsystems
Intelligence artificielle-Securite fonctionnelle et systemesd’intelligence artificielle
COPYRIGHTPROTECTEDDOCUMENT
ISO/IEC 2024
All rights reserved. Unless otherwise specified or required in the context of its implementation no part of this publication may be reproduced or utilized otherwise in any form or by any means electronic or mechanical including photocopying or posting onthe intermet or an intranet without prior written permission Permission can be requested from either ISO at the address below or ISO's member body in the country of the requester.
CP 401 • Ch. de Blandonnet 8CH-1214 Vernier Geneva Phone: 41 22 749 01 11Email copyright@ Website:
ISO/IEC TR 5469:2024(en)
Contents
Use of AI technology in E/E/PE safety-related systems. 6.1 Problem description. .66.2 AI technology in E/E/PE safety-related systems. .6
Al technology elements and the three-stagerealization principle. 10
7.1 Technology elements for Al model creation and execution. 107.2 The three-stage realization principle of an Al system. 127.3 Deriving acceptance criteria for the three-stage of the realization principle. 12
Properties and related risk factors of Al systems. 8.1 Overview. 138.1.2 8.1.1 General Algorithms and models. 13 138.2 Level of automation and control. 148.3 8.4 Aeuedxa pue Asuaedsuen jo aaa Issues related to environments. 15 178.4.2 8.4.1 Issues related to environmental changes. Complexity of the environment and vague specifications. 17 178.4.3 Issues related to learning from environment. 188.5 8.5.1 Resilience to adversarial and intentional malicious inputs. Overview. 19 198.5.2 8.5.3 Al model attacks: adversarial machine learning. General mitigations. 19 198.6 Al hardware issues. 208.7 Maturity of the technology 21
Verification and validation techniques.9.1 Overview. 219.2 9.2.1 Problems related to verification and validation. Non-existence of an a priori specification. 22 229.2.2 Non-separability of particular system behaviour. Limitation of test coverage. 229.2.3 9.2.4 Non-predictable nature. 22 229.3 9.2.5 Possible solutions. Drifts and long-term risk mitigations. 22 239.3.1 General. 239.3.2 9.3.3 Relationship between data distributions and HARA. Data preparation and model-level validation and verification. 23 249.3.4 9.3.5 System-level testing. Choice of Al metrics. 25 259.3.6 Mitigating techniques for data-size limitation. 269.4 9.3.7 Virtual and physical testing. Notes and additional resources. 26 269.4.1 9.4.2 Considerations on virtual testing General. 26 26
@ ISO/IEC 2024 All rights reserved
ISO/IEC TR 5469:2024(en)
9.4.3 9.4.4 Considerations on physical testing. Evaluation of vulnerability to hardware random failures. 28 299.5 A note on explainable A1. Monitoring and incident feedback. 2910 Control and mitigation measures. 9.6 30 2910.1 Overview. 3010.2 10.2.1 Overview. Al subsystem architectural considerations. 30 3010.2.2 Detection mechanisms for switching. 10.2.3 Use of a supervision function with constraints to control the behaviour of a 30system to within safe limits.. 3310.2.4 Redundancy ensemble concepts and diversity. 10.2.5 AI system design with statistical evaluation. 34 3510.3 Increase the reliability of ponents containing AI technology. 10.3.1 Overview of Al ponent methods. 35 3510.3.2 Use of robust learning 3510.3.3 Optimization and pression technologies. 10.3.4 Attention mechanisms. 36 3710.3.5 Protection of the data and parameters. .3711 Processes and methodologies. 11.1 General. 38 3811.2 Relationship between Al life cycle and functional safety life cycle 3811.4 11.3 Al phases. Documentation and functional safety artefacts. 39 3911.5 Methodologies. 11.5.1 Overview. 39 3911.5.2 Fault models. 3911.5.3 PFMEA for offline training of AI technology 4054 41Annex C (informative) Possible process and useful technology for verification and validation. 59Annex D (informative) Mapping between ISO/IEC 5338 and the IEC 61508 series. 62Bibliography. 65
ISO/IEC TR 5469:2024(en)
Foreword
ISO (the International Organization for Standardization) and IEC (the International ElectrotechnicalCommission) form the specialized system for worldwide standardization. National bodies that aremembers of ISO or IEC participate in the development of International Standards through technical mittees established by the respective organization to deal with particular fields of technical activity.ISO and IEC technical mittees collaborate in fields of mutual interest. Other international organizations governmental and non-governmental in liaison with ISO and IEC also take part in the work.
The procedures used to develop this document and those intended for its further maintenance are describedin theIECDirectivesPar1Iparticula thedifferent approvalriteriaeeded frthdiffrent tsIEC Directives Part 2 (see or of document should be noted. This document was drafted in accordance with the editorial rules of the ISO/
ISO and IEC draw attention to the possibility that the implementation of this document may involve thereceived notice of (a) patent(s) which may be required to implement this document. However implementersdatabase available at /patents and ISO and IEC shall not be heldresponsible for identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does notconstitute an endorsement.
For an explanation of the voluntary nature of standards the meaning of ISO specific terms and expressions related to conformity assessment as well as information about ISO's adherence to the World TradeIn the IEC see
This document was prepared by Joint Technical Committee ISO/IEC JTC 1 Information technology Submittee SC 42 Artificial intelligence.
Any feedback or questions on this document should be directed to the user's national standardsbody. A plete listing of these bodies can be found at /memhers.htmland
