Cyber & IT Supervisory Forum - Additional Resources

What is the variable selection and evaluation process? How was the data collected? Who was involved in the data collection process? If the dataset relates to people (e.g., their attributes) or was generated by people, were they informed about the data collection? (e.g., datasets that collect writing, photos, interactions, transactions, etc.) As time passes and conditions change, is the training data still representative of the operational environment? Why was the dataset created? (e.g., were there specific tasks in mind, or a specific gap that needed to be filled?) How does the entity ensure that the data collected are adequate, relevant, and not excessive in relation to the intended purpose? Datasheets for Datasets. WEF Model AI Governance Framework Assessment 2020. WEF Companion to the Model AI Governance Framework- 2020. GAO-21-519SP: AI Accountability Framework for Federal Agencies & Other Entities. ATARC Model Transparency Assessment (WD) – 2020. Transparency in Artificial Intelligence - S. Larsson and F. Heintz – 2020. Challenges with dataset selection Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kiciman. 2019. Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries. Front. Big Data 2, 13 (11 July 2019). Amandalynne Paullada, Inioluwa Deborah Raji, Emily M. Bender, et al. 2020. Data and its (dis)contents: A survey of dataset development and use in machine learning research. arXiv:2012.05345. Catherine D'Ignazio and Lauren F. Klein. 2020. Data Feminism. The MIT Press, Cambridge, MA. Miceli, M., & Posada, J. (2022). The Data-Production Dispositif. ArXiv, abs/2205.11963. Barbara Plank. 2016. What to do about non-standard (or non-canonical) language in NLP. arXiv:1608.07836. URL References AI Transparency Resources

78