Stress Tests? Baloney

By government fiat, bank stress tests are becoming more complex and demanding and are being imposed on larger numbers of banks. Remarkable soothsaying powers are wishfully attributed to these recently built and constantly repainted totem poles.

If you want to avoid catastrophic outcomes, don't rely on stress test models with their hundreds of variables. Focus on finding the small number of causative factors that could individually precipitate failure.

It's said that generals always prepare to refight the last war, rather than envisioning the next one. Stress tests as now practiced are still more retrograde, since they couldn't even have helped avoid the 2007-9 crisis.

That time, failure of individual institutions and massive systemic risk were primarily the consequence of just four major factors:

Mortgage credit quality dropped precipitously, with degeneration of terms and underwriting; home prices fell sharply when the resulting price bubble burst; sales of the defective mortgages generated immense legal liabilities; and off-balance-sheet activities caused heavy losses.

All four exposures were unrecognized by regulators, accountants and most banks until after the crisis was well under way. Recently released transcripts of Federal Reserve Board meetings in 2007 confirm this. Similarly for Ireland, Iceland (complete with timely stress tests) and Spain. No one asked the keystone question: How could we fail?

The mandated stress tests, instead of aiming at identifying pivotal factors with massively destructive power, mandate a bottom-up  approach, assuming environmental factors—interest rates, GDP changes and overall declines in credit quality—are the most likely precipitators of bank and systemic failure and will affect wide ranges of asset classes. Maybe that would have been a tolerably lazy and insight-free approach in 1928, but not now. More recently, mismanagement rather than macroeconomic factors caused the disasters.

In recent decades we've had a succession of crises, each triggered by balance sheet damage from a very few primordial causes, not from the interaction or cumulative effect of great numbers of variables. Asset values weren't chopped across the board in a death of 1,000 cuts. Rather, big craters appeared at specific places in bank balance sheets—for instance, residential mortgages. Cataclysm in the residential market then undercut the value of construction, development and commercial real estate loans.

Failure caused by a few endogenous rather than many environmental factors is not limited to banking and finance. It is typical. If a large nonfinancial corporation wants to assess and control the risk of suffering catastrophic results, it needn't assemble a team of statisticians to build a huge model. Rather, it must identify, understand and mitigate the few internal, competitive and macro variables with the potential to upend its balance sheet. As at Netflix, Yahoo and Kodak. A capable chief risk officer focuses on points of potential critical impact and lines of cleavage radiating from them—rather than assigning probabilities to large numbers of ambient uncertainties and then to the correlations between them.

A routine stress test would not tell JPMorgan Chase whether the next “London Whale” could cost the bank $60 billion.

If you were budgeting to achieve targeted earnings in 2013, then you'd have to work from the top down through major business lines and markets, rather than from the bottom up—income and expense item by item and branch by branch. Likewise, if you wanted to assess and reduce the probability of corporate disaster in 2013, you'd also have to go top down. This requires fault-tree logic, rather than flow diagrams integrating large numbers of small revenue, cost and valuation slivers.

At the top of the fault tree goes the final result: liabilities at end of year exceed assets. How could that happen? We draw lines from the top to the next level of the tree. Possibilities might be: shrinkage of loan asset values; trading and securities losses; and off-balance-sheet liabilities. Hence, three points at this level of the tree. Down from the point labeled shrinkage of asset values might be lines leading to nodes for decline in mortgage portfolio valuation; decline in other consumer credit values; and commercial portfolio losses. The lines downward from the mortgage node might go to interest rate risk and credit risk on first mortgages, and credit risk on seconds.

Details will depend on scale of the respective activities in a particular institution. But no hindsight is needed to see that this approach would have been far more likely to illuminate the scenarios that were fatal in 2007-9 than would the stress tests that are now mandated.

Let's focus on locating potentially disastrous vulnerabilities, rather than on mechanically cranking numbers to convince regulators that typhoon-driven waves won't knock too many small holes in our yachts.

Andrew Kahr is a principal in Credit Builders LLC, a financial product development company, and was the founding chief executive of First Deposit, later known as Providian.