Groupthink Is Defeating the Purpose of Stress Tests

Recent discussion has focused on whether the Federal Reserve's stress testing regime could add risk to the financial system. Some have argued market participants are working to reverse-engineer the Fed's loss estimate models across a narrow range of potential future economic conditions, thereby introducing additional risk.

The concern, simply put, is that banks will practice the risk-management equivalent of "teaching to the test" by trying to second-guess the Fed rather than make fundamental improvements to their risk and finance infrastructure and cash-flow analytics. Such changes are now required if a firm is to convert the stress test from a chore, and a compliance nightmare, into a real opportunity for enriching the firm's planning and balance sheet risk analytics.

However, it is important to look at the history of stress tests, their promised utility, and recommend a solution to the groupthink. Simply identifying the known industry problem is not enough.

Stress testing is nothing new. While the current framework adopted by the Fed­—the Dodd-Frank Act Stress Test and the Comprehensive Capital Analysis and Review —is an innovation over historical supervisory risk analysis requirements, banks have been conducting stress tests for decades, some more proactively and more effectively than others.

Historically, stress tests were conducted by risk type rather than in a unified fashion across categories of risk. For instance, credit default and loss emergence modeling was performed independently of accrual book interest rate risk, trading risk, operational risk, capital risk and liquidity risk. If there is one lesson learned from the financial crisis, it is that these risks are correlated, and looking at exposures in isolation tends to understate potential spillover effects across risk types, at the firm and the system level.

The unique feature of the stress testing framework mandated by the Dodd-Frank Act is that it requires banks to model, in a forward-looking fashion, the entire firm's balance sheet and income statement across multiple risk types. This is done to assess capital adequacy under plausible but severe economic downturn conditions. While only a reduced set of scenarios are currently required, with sound design, numerous scenarios can be – and in time should be – performed, particularly more tailored, idiosyncratic scenarios.

In order to conduct an enterprise-wide stress test, a bank must possess or develop strong internal risk management processes. The bank must be able to integrate a complex pro-forma risk simulation across credit, interest rate, operational, capital (economic and regulatory) and – if designed properly – liquidity risk (although the Dodd-Frank tests and CCAR, oddly, explicitly ignore liquidity risk), and ensure that inputs and assumptions are consistent with planning and risk systems.

Performing this task is no easy chore and requires an unprecedented level of coordination, data, econometric and statistical models, and cross-functional integration of business processes. Very few banks would opine – nor is it likely that their stakeholders, creditors and supervisors would believe – that having better infrastructure to support cross-risk transparency and analytical capability is a socially or financially bad outcome. In fact, the rigorous data requirements mandated by the Fed may be one of the more important outcomes of the stress testing exercise, and not just for purposes of the stress test.

It would be a shame if the granular data required by the Fed isn't leveraged by the regulators (mostly the FDIC)  for recovery and resolution planning; by the Office of Financial Research for its annual report to Congress; by the FDIC to refine pricing of FDIC deposit insurance and – in time perhaps – by the Fed itself to provide needed collateral transparency for discount window crisis lending programs and to support, rather than detract, from the upcoming Comprehensive Liquidity Assessment and Review, among other things.

Perhaps most significantly, even beyond the multiplicity of uses for the consolidated industry data for supervisory purposes, once a bank develops the proper cross-risk integration platform, the analytical capabilities will clearly enhance its planning, risk assessment and decision-making apparatus and capabilities.

If the solution platform is well architected and designed, the stress testing mandate becomes merely a special instance – the regulatory reporting instance – of integrated financial planning and risk analysis. If the Fed merely achieves a groupthink outcome wherein all banks seek to target the Fed's loss and pre-provision net revenue estimates, as well as other aspects of the modeling architecture, the Fed will have failed in its mandate, banks will view the stress tests as a regulatory compliance exercise, and, we will have a more fragile system.

But why is the groupthink happening if, as stated above, there are so many positive benefits to a more integrated approach toward pro-forma financial planning and risk analysis? Primarily because of regulatory over-specification and continued over-prescription.

Under Dodd-Frank's section 165(i)(B)(v), the Fed is required to publish its loss estimates. This could be a mistake; the law is probably over-specified, but the Fed certainly needs to obey the law. It is this law and its implementing policy that is driving the groupthink behavior, as well as a particularly profound orientation of certain policy and senior banking leadership to focus too heavily on the Fed's stress test design, and less on the business processes, technology, governance and cross-risk integration needs.

The mandate to disclose supervisory loss estimates is unique to the United States. In Canada, for example, the Office of the Superintendent of Financial Institutions has required stress tests for many years, although the detail of its data request doesn't allow nearly as granular an analysis as the Fed's required data submission. The Canadian regulator has kept its own analysis confidential, and the banks have no requirement to publish their stress test results. The U.K.'s (oddly) "closely guarded" Financial Data Submission Framework is similar, albeit, like the Canadian stress test, not as comprehensive, granular or transparent as the evolving CCAR and Dodd-Frank requirements.

While the Fed has made dramatic improvements in its modeling and validation capabilities since the original 2009 stress tests – for losses, pre-provision net revenue and capital – the Fed's internal models only should be used as a very rough approximation in ongoing supervisory discussions. After all, the Fed's models will never be as sophisticated and informed as the actual "work-flow" analytics used by the banks. And while it is helpful and prudent for the firms themselves to publish their own loss and revenue estimates, there is dubious value in the law that requires the Fed to publish its own internal model estimates; estimates which almost by design will, in time, be significantly inferior to the bank's internal models. After all, it is the firm's responsibility to assess and manage its risks, and the Fed's role to verify acceptable adherence to risk management norms. Given our national experience and policy lessons learned through the implementation of internal models for interest rate risk, a direct response to the 1991 FDIC Improvement Act (Section 305), we know the risks and policy rationale against a regulatory model that is narrowly prescribed and over-specified.

Encouraging firms to improve their conditional new business planning, forecasting, balance sheet management and credit analytic processes will help reduce financial fragilities, but only if the bank supervisors also focus on needed improvements to governance, infrastructure and enhanced business process management. Before such an outcome, we should – collectively – do what is needed to end the groupthink and focus more attention to the banks' estimates, needed infrastructure and analytical capabilities rather than targeting the Fed's models.

Thomas Day is a senior director at Moody's Analytics.