Large Bank Examination Workshop February 2026

Vendor Products • Banks are expected to validate their own use of vendor products. • Outsourcing model development does not eliminate the need to apply a similar vetting, approval, validation and decommissioning process as would be conducted for models developed in-house. • Restricted access to proprietary intellectual property (e.g. computer code) may mean banks using vendor product have to rely more on sensitivity analysis and benchmarking.

• But models should still be fully documented, including testing results, ongoing performance monitoring and outcomes analysis.

47

Backtests • Backtests compare actual outcomes with model forecasts during a time period not used for model development.  Using “out of sample” data • Model forecasts (usually expressed as an expected range or confidence interval around a central estimate) are compared with actual values for the back-test period, at a frequency and forecast horizon corresponding to the model’s intended use. • Differences between actual and forecast values are analyzed to see if they are significant in magnitude or frequency. • Variances do not necessarily indicate a problem with the construction of the model, as the future is impossible to accurately predict. • But – management should be able to explain and reconcile variances between the model forecast and actual events.

48

24

© Global Financial Markets Institute Inc.