Cyber & IT Supervisory Forum - Additional Resources

Measure 2.4 The functionality and behavior of the AI system and its components – as identified in the MAP function – are monitored when in production. About AI systems may encounter new issues and risks while in production as the environment evolves over time. This effect, often referred to as “drift”, means AI systems no longer meet the assumptions and limitations of the original design. Regular monitoring allows AI Actors to monitor the functionality and behavior of the AI system and its components – as identified in the MAP function - and enhance the speed and efficacy of necessary system interventions. Suggested Actions Monitor and document how metrics and performance indicators observed in production differ from the same metrics collected during pre-deployment testing. When differences are observed, consider error propagation and feedback loop risks. Utilize hypothesis testing or human domain expertise to measure monitored distribution differences in new input or output data relative to test environments Monitor for anomalies using approaches such as control limits, confidence intervals, integrity constraints and ML algorithms. When anomalies are observed, consider error propagation and feedback loop risks. Verify alerts are in place for when distributions in new input data or generated predictions observed in production differ from pre deployment test outcomes, or when anomalies are detected. Assess the accuracy and quality of generated outputs against new collected ground-truth information as it becomes available. Utilize human review to track processing of unexpected data and reliability of generated outputs; warn system users when outputs may be unreliable. Verify that human overseers responsible for these processes have clearly defined responsibilities and training for specified tasks. 122