Cyber & IT Supervisory Forum - Additional Resources

Manage 3.2 Pre-trained models which are used for development are monitored as part of AI system regular monitoring and maintenance. About A common approach in AI development is transfer learning, whereby an existing pre-trained model is adapted for use in a different, but related application. AI actors in development tasks often use pre-trained models from third-party entities for tasks such as image classification, language prediction, and entity recognition, because the resources to build such models may not be readily available to most organizations. Pre-trained models are typically trained to address various classification or prediction problems, using exceedingly large datasets and computationally intensive resources. The use of pre-trained models can make it difficult to anticipate negative system outcomes or impacts. Lack of documentation or transparency tools increases the difficulty and general complexity when deploying pre-trained models and hinders root cause analyses. Suggested Actions Identify pre-trained models within AI system inventory for risk tracking. Establish processes to independently and continually monitor performance and trustworthiness of pre-trained models, and as part of third-party risk tracking. Monitor performance and trustworthiness of AI system components connected to pre-trained models, and as part of third-party risk tracking. Identify, document and remediate risks arising from AI system components and pre-trained models per organizational risk management procedures, and as part of third-party risk tracking. Decommission AI system components and pre-trained models which exceed risk tolerances, and as part of third-party risk tracking.

198