Cyber & IT Supervisory Forum - Additional Resources

Transparency & Documentation Organizations can document the following:

Who is responsible for documenting and maintaining the AI system inventory details? What processes exist for data generation, acquisition/collection, ingestion, staging/storage, transformations, security, maintenance, and dissemination? Given the purpose of this AI, what is an appropriate interval for checking whether it is still accurate, unbiased, explainable, etc.? What are the checks for this model? GAO-21-519SP: AI Accountability Framework for Federal Agencies & Other Entities. Intel.gov: AI Ethics Framework for Intelligence Community - 2020. References “A risk-based integrity level schema”, in IEEE 1012, IEEE Standard for System, Software, and Hardware Verification and Validation. See Annex B. Off. Comptroller Currency, Comptroller’s Handbook: Model Risk Management (Aug. 2021). See “Model Inventory,” pg. 26. VertaAI, “ModelDB: An open-source system for Machine Learning model versioning, metadata, and experiment management.” Accessed Jan. 5, 2023. AI Transparency Resources

15