With advances in data analytics, machine-learning models and the creation of vast amounts of data, there has been an explosion in model development and deployment across every industry — financial services is no exception. Banks and other financial organizations are using machine learning to automate call centers, make personalized financial recommendations through mobile apps and identify financial fraud.
Unfortunately, banks are at a disadvantage relative to other industries in three interrelated areas: the hiring of talent, the efficient development and deployment of advanced analytics and compliance with regulatory expectations. Without talent, it is difficult to create the appropriate analytics to better serve bank customers. Banks are competing for talent with Netflix, Google, Facebook and Uber, among others. Aside from the cachet of working for a big tech firm, top talent will want to work in the fast-paced environment of a modern company rather than at a legacy institution. Banks also suffer from an aging technology infrastructure. Technology is improving at an increasing rate, and most banks’ infrastructure is ill-equipped for rapid implementation.
Banks are also at a disadvantage due to the length of time it takes to develop and deploy models. Banks are familiar with the semiannual cadence of regulatory stress tests. The processes around models including governance, controls, validation and promoting models to production were created to align with this six-month schedule. Developing, testing and validating models often takes months at banks. These model activities typically occur sequentially rather than in parallel, further increasing the time to market. In contrast, Netflix software to production
One of the reasons that banks are slow to develop and deploy models is due to heightened regulatory expectations around model development and independent model review. In some cases, large regional banks’ primary introduction to modeling was through the regulatory CCAR and DFAST stress test exercises. Soon after the CCAR and DFAST models were introduced, the Federal Reserve prescribed additional requirements for independent model validation. Some banks scrambled to comply with the new modeling requirements, resulting in suboptimal workflows around model development and validation.
The good news is there’s a solution to these disadvantages: Banks need to create a framework or ecosystem for efficient and compliant model development and validation. Such an ecosystem is necessary to compete in the new world of big data and analytics while satisfying the needs of customers, developers and regulators. Such an ecosystem must allow for the efficient deployment of high-value analytics while still retaining the governance and controls expected of a financial institution. To achieve this goal, the ecosystem must deliver on a number of fronts.
The model development environment is vital for attracting and retaining quantitative talent. The environment should support familiar and popular development platforms such as Jupyter notebooks or markdown. The environment should support a wide variety of programming languages so that developers can use their favorite libraries. The environment should integrate with enterprise source control management systems such as source control management for code and model versioning. Frequently, developers will iterate on models and discard models that do not perform adequately. These alternative designs provide key developmental evidence and should be archived. The environment should automatically archive these artifacts for use by other model developers, model validators, internal auditors and others. Finally, the environment should allow for every model and its dependencies to be delivered in a self-contained package.
A challenge for many developers at banks is the deployment of a trained model into production. The environment should enable simple one-click deployment of the model into a test or quality assurance environment for model consumers to assess. Efficient deployment to a quality assurance environment is vital for obtaining fast feedback and for allowing developers to iterate on models to meet model consumers’ needs. For a move to production, the environment should integrate with the enterprise software delivery system pipelines. Such pipelines allow model developers, model validators and IT to create tests that run automatically on every model change. Rather than wait weeks for technology and risk to be comfortable with a change, the technology and risk requirements can be built into the deployment pipeline. The environment should also support various types of deployment strategies such as A/B testing so that model performance can be tracked against alternative processes.
Once in production, models must be monitored for performance. Such monitoring should be automated to the extent possible. The environment should support monitoring the distribution of model inputs to ensure that the data that the model is scoring is similar to the data that the model was trained on. The environment should support triggers or alerts when performance becomes unacceptable, including automated emails or texts to key stakeholders in technology and risk divisions. The environment should support custom performance criteria. For example, for loan origination models, one criterion may be a metric that tracks the percentage of loan approvals by ZIP code to ensure compliance with fair banking laws.
Finally, the model environment should support governance by supplying model metadata such as a model inventory, documentation archiving and reporting. The model inventory should automatically track the downstream impacts of poor model performance. When a platform that developers actually want to use is provided, the inventory is likely to be more complete than traditional model inventories. The platform’s inventory helps model consumers and developers explore the universe of models. This reduces the amount of duplicate efforts around models. This inventory also encourages a healthy competition around model performance. Model developers can transparently compete to create higher-performing models.
Financial institutions must be innovative to meet the needs and expectations of their customers. They must attract the appropriate talent to develop these innovative solutions. They must be nimble, iterating quickly and efficiently while remaining compliant with supervisory guidance, internal model validation policy, and information technology standards. Model platforms enable banks to safely and transparently develop and deploy advanced analytics. Automated alerts and rollbacks provide a safety net for analytics to be efficiently deployed. Not only do these platforms improve time-to-market, but the fast-paced environment helps banks compete in the marketplace for quantitative talent.
