Cyber & IT Supervisory Forum - Additional Resources

Measure 3.2 Risk tracking approaches are considered for settings where AI risks are difficult to assess using currently available measurement techniques or where metrics are not yet available. About Risks identified in the Map function may be complex, emerge over time, or difficult to measure. Systematic methods for risk tracking, including novel measurement approaches, can be established as part of regular monitoring and improvement processes. Suggested Actions Establish processes for tracking emergent risks that may not be measurable with current approaches. Some processes may include: Recourse mechanisms for faulty AI system outputs. Bug bounties. Human-centered design approaches. User-interaction and experience research. Participatory stakeholder engagement with affected or potentially impacted individuals and communities. Identify AI actors responsible for tracking emergent risks and inventory methods. Determine and document the rate of occurrence and severity level for complex or difficult-to-measure risks when: Prioritizing new measurement approaches for deployment tasks. Allocating AI system risk management resources. Evaluating AI system improvements. Making go/no-go decisions for subsequent system iterations.

Organizations can document the following: Transparency & Documentation

Who is ultimately responsible for the decisions of the AI and is this person aware of the intended uses and limitations of the analytic? Who will be responsible for maintaining, re-verifying, monitoring, and updating this AI once deployed?

159