Cyber & IT Supervisory Forum - Additional Resources

Measure 2.2 Evaluations involving human subjects meet applicable requirements (including human subject protection) and are representative of the relevant population. About Measurement and evaluation of AI systems often involves testing with human subjects or using data captured from human subjects. Protection of human subjects is required by law when carrying out federally funded research and is a domain specific requirement for some disciplines. Standard human subjects protection procedures include protecting the welfare and interests of human subjects, designing evaluations to minimize risks to subjects, and completion of mandatory training regarding legal requirements and expectations. Evaluations of AI system performance that utilize human subjects or human subject data should reflect the population within the context of use. AI system activities utilizing non-representative data may lead to inaccurate assessments or negative and harmful outcomes. It is often difficult – and sometimes impossible, to collect data or perform evaluation tasks that reflect the full operational purview of an AI system. Methods for collecting, annotating, or using these data can also contribute to the challenge. To counteract these challenges, organizations can connect human subjects' data collection, and dataset practices, to AI system contexts and purposes and do so in close collaboration with AI Actors from the relevant domains. Suggested Actions Follow human subjects research requirements as established by organizational and disciplinary requirements, including informed consent and compensation, during dataset collection activities. Analyze differences between intended and actual population of users or data subjects, including likelihood for errors, incidents or negative impacts. Utilize disaggregated evaluation methods (e.g., by race, age, gender, ethnicity, ability, region) to improve AI system performance when deployed in real world settings. 115