How the Fed Might Change Stress Tests in Version 2.0

WASHINGTON — The Federal Reserve is looking for advice and insight into how it might improve its annual stress-testing regime, but banks and industry observers say the central bank is unlikely to touch the aspect of the tests that bothers them most: its secrecy.

Bankers remain concerned about the models the Fed uses in its Comprehensive Capital Analysis and Review, arguing it is a black box that makes it hard to evaluate.

"There is a general concern within the banking industry that what takes place inside of the Fed and how the Fed does its own forecasting for individual banks is not as transparent as they would wish it to be," said Bruce Stevenson, managing director of the professional-services firm Alvarez & Marsal. "The Fed's own models are known only to the Fed, and that is worrisome to some banks because they would like to have the same transparency on the Fed models as the Fed has on the banks' own models."

Hugh Carney, senior counsel at the American Bankers Association, said that banks' top priority in any review of CCAR is the lack of transparency on how the models are designed and economic scenarios used in the test are developed.

"The lack of clear standards that banks are supposed to meet is a primary focus, and has been for years," Carney said. "Effectively, people are being judged by a standard they don't know — that sounds a lot like secret law. When there's such a stiff penalty and you don't know what standard you're being held to, that clearly has to be a big focus."

The Fed quietly announced last month that it has been engaged in a broad review of virtually every aspect of its CCAR program alongside a notice of the finalization of smaller technical changes to the 2016 process. The review began in April, and Fed Gov. Daniel Tarullo — who heads the board's supervisory committee and has an outsize role in steering the agency's regulatory direction — attended a handful of meetings in June with representatives of the largest global systemically important banks, academics, market analysts and other interested stakeholders on what changes should be made and why.

The Fed has not indicated when its review will be complete, what will be included or what form the revisions will take. But Tarullo has dropped some hints about where he intends to go as the agency crafts CCAR 2.0.

In an interview on Bloomberg TV last month, Tarullo said that he envisions the stress-testing process moving beyond just examining the performance of individual banks' capital levels under stress and also include the interconnection of institutions under stress as well.

"Although we haven't decided exactly how to do that, I think there's a pretty good chance … that at the end of the day, whether through the incorporation of some or all of the capital surcharge as a post stress minimum, or some other mechanisms, such as … more emphasis on shared counterparties, that there will be some net increase in the post-stress minimum capital requirements," he said.

In a speech at the Brookings Institution last month, Tarullo added that the review should result in a CCAR that will "reflect better the range of risks" facing banks — especially global systemically important banks, or G-SIBs.

There's also reason to believe that the stress testing will expand over time. The Fed finalized a rule in July laying out what enhanced prudential standards it would require of GE Capital — one of only four nonbanks to be designated as systemically important financial institutions — and those requirements effectively amounted to meeting the same capital and regulatory requirements that other bank SIFIs had to meet, including annual CCAR stress tests. While GE has begun a program to shed its financial assets and its SIFI status, other nonbank SIFIs may ultimately be subject to a similar form of stress testing.

The Committee on Payments and Market Infrastructures and the International Organization of Securities Commissions also launched a review of whether central counterparties, or CCPs, should be subject to the same kind of independent regulatory stress testing that banks are subject to in CCAR.

CCPs are regulated by the Commodity Futures Trading Commission, not the Fed, but the clearing houses could be subject to a similar regime in the future.

Brett Hintz, professor of finance at New York University's Stern School of Business, said the stress tests — particularly CCAR — has evolved since the Fed used them in the wake of the financial crisis to assuage a nervous public about banks' health.

But he said keeping the Fed's internal models confidential is a critical part of the test. If those models were known to the banks, they could simply allocate their capital to pass the test rather than conservatively shore up risk.

"CCAR is the Swiss Army knife of the Fed," Hintz said. "They use it as a public relations tool, because it shows that the banks are strong and it encourages the banks to be stronger. It's also used to keep the street and the banks from gaming the system."

Still, the secrecy means there is no way for banks to trust that the Fed is using credible models.

The central bank's curation of its models recently came under fire from the Fed's Office of Inspector General, which found that the agency's own practices fell short of what it requires from the banks it supervises. The Oct. 29 report found, for example, that the Fed's "staffing approach was not consistent with industry practice" in that it relies on directing staff to examine the models on an ad hoc basis leading up to the annual exams rather than having full-time model-validation staff working on the models year-round. The report also found deficiencies in the time spent on model review and the time spent evaluating any particular model.

The Office of Inspector General concluded that the limited number of model evaluators could lead to "personnel-dependency" risk at the Fed; the model-evaluation function did not communicate ways to improve the models to its reviewers; and the use of dedicated reviewers could pose a risk that the models will lack fresh scrutiny.

The Fed said that it was already planning to have the Federal Reserve Bank of Minneapolis "lead the supervisory stress-testing model validation function on a permanent basis" by the end of 2016, according to the report. The central bank also said it will address the remaining concerns, including identifying the skills needed to better test its models and retain necessary staff with those skills; reduce the use of supplemental reviewers and have reviews conducted "primarily by permanent staff."

Marcus Stanley, policy director for Americans for Financial Reform, said that the idea of having CCAR more closely examine the interconnectedness of banks or incorporating the G-SIB surcharge could be positive. But any effort to reveal the secret sauce in the Fed's models would effectively gut the most transformative bank supervisory tool that the agency has ever had.

The models are supposed to be unknown — or at least substantially unknown — to the banks, he said, because if they were perfectly understood then banks would simply allocate capital to pass the test. He pointed to the former Office of Federal Housing Enterprise Oversight, which had publicly available stress tests that failed to predict the collapse of both Fannie Mae and Freddie Mac.

"OFHEO made their stress test fully public to everybody, and the stress-testing process for Fannie and Freddie was a dismal failure," Stanley said. "That OFHEO example is very telling to us."

The Fed said that all options are still on the table concerning the five-year review, but Tarullo raised concerns in 2014 that such a revelation would undermine the effectiveness of the tests.

"We do not want firms simply to copy our modeling in their own assessment of risks and capital needs," Tarullo said. "And we certainly do not want them to construct their portfolios in an effort to game the model--a kind of analogue to teaching to a test."

There are other aspects of the test, however, where the Fed could conceivably give some ground, Stanley said. The CCAR model includes various assumptions about how banks might behave in a crisis — assumptions that intentionally envision a more severe scenario than would be expected.

Those assumptions are in place to counter the pre-crisis attitude of assuming best-case scenarios, Stanley said, and serve the purpose of preparing banks to weather a more severe stress event than they might otherwise be prepared for.

Banks, for their part, have argued that those assumptions are duplicative or not representative of banks' behavior. Easing those assumptions — especially if the capital surcharge for G-SIBs were added to the tests — could cushion the blow.

"The banks want the Fed to relax some of those conservative assumptions, and I think if Fed adds the G-SIB requirements to stress test, then they may be a little open to some of the assumptions," Stanley said, even though those assumptions "are crucial to the macroprudential aspect of the testing process."

Banks may not support such a deal, however. One banking industry source who spoke on the condition of anonymity said that the point of CCAR is not to move around the capital requirements year by year, but to examine the industry and find out where the sources of risk might be and to allocate capital accordingly.

Putting aside the practical challenges of incorporating the G-SIB surcharge into CCAR, the source said, changing the assumptions as a sort of concession would turn a critical supervisory exercise into a political one.

"Everyone thinks CCAR is a good idea — it's the right thing to do," the source said. "I think it's a bad thing to pollute that by saying, 'Now we're going to add a surtax that doesn't really have anything to do with how well you do under the scenarios, but we just want more capital, and then we'll … change some other parameters that partially offset it.' "

Other stakeholders say there are other ways that the Fed could improve CCAR that have nothing to do with models or capital levels.

The ABA's Carney said, for example, that the Fed has put too much emphasis on evaluating the stress tests and banks' performance in the context of a slowly, but steadily improving, economy. Much less attention has been put into how the tests are supposed to operate in the downward side of the cycle.

If the economy is in a downward swing, the purpose of holding larger levels of capital is to keep banks lending. If the banks have to hold that capital for regulatory reasons, it might be counterproductive from a recovery standpoint.

"As I see it, we need to seriously start thinking about what the role of stress testing is on the downward side of the economic cycle," Carney said. "This depends on how they set scenarios and whether they take into account where we are in the economic cycle."

Paul Tucker, incoming chairman of the Systemic Risk Council and former deputy governor of the Bank of England, said that one blind spot of the stress tests in the United States — and elsewhere — is that the regulatory models aren't shared among each other internationally. If the industry and lawmakers aren't allowed to look at the Fed's models, they might at least take comfort if other regulators could, he said.

"If you're a U.S. supervisor, you can't do your job of ensuring that the U.S. banking system is sound without having a reasonable degree of confidence that the German, French, Japanese, U.K. banking system is sound, and vice versa," Tucker said. "Supervisors can see each others' published results, but can they trust them? Yeah, maybe. But why not let U.K. people observe the U.S. stress tests, including the models? And U.S. people observe the German stress tests? I don't think that should be fanciful."

Stevenson echoed similar concerns in suggesting that a disinterested third party — say, a federal agency with economic and accounting experience that isn't a bank regulator, such as the IRS, NSA or SEC — can be given an opportunity to evaluate the models.

"If the Fed were to allow a third party, who is not a bank, to do that scrutiny, then the banks could take confidence that a qualified third party has done the same due diligence on the Fed that the Fed does on them," Stevenson said. "While the banks can't then see the result, they can take confidence in the conclusions made by the third party."

But Stanley said that idea would be a nonstarter, especially at the Fed. The stress tests are already reviewed by a Model Validation Council populated with many of the leading economists in academia. And the Fed itself is a fount of expertise on economics and finance — and its prized independence as a bank supervisor is not something it is likely to give up.

"The notion that you're going to have the IRS or SEC auditing the Fed's stress tests for the institutions it supervises is pretty much a transparent effort to rein in the Fed's ability," Stanley said.