Cyber & IT Supervisory Forum - Additional Resources

Identify and document processes to understand and trace test and training data lineage and its metadata resources for mapping risks. Document known limitations, risk mitigation efforts associated with, and methods used for, training data collection, selection, labeling, cleaning, and analysis (e.g., treatment of missing, spurious, or outlier data; biased estimators). Establish and document practices to check for capabilities that are in excess of those that are planned for, such as emergent properties, and to revisit prior risk management steps in light of any new capabilities. Establish processes to test and verify that design assumptions about the set of deployment contexts continue to be accurate and sufficiently complete. Work with domain experts and other external AI actors to: Gain and maintain contextual awareness and knowledge about how human behavior, organizational factors and dynamics, and society influence, and are represented in, datasets, processes, models, and system output. Identify participatory approaches for responsible Human-AI configurations and oversight tasks, taking into account sources of cognitive bias. Identify techniques to manage and mitigate sources of bias (systemic, computational, human- cognitive) in computational models and systems, and the assumptions and decisions in their development. Investigate and document potential negative impacts due related to the full product lifecycle and associated processes that may conflict with organizational values and principles.

Organizations can document the following: Transparency & Documentation

Are there any known errors, sources of noise, or redundancies in the data? Over what timeframe was the data collected? Does the collection timeframe match the creation time frame

77