Regulators Need Schooling on Measuring Lending Bias

For almost two decades, out of concern over disparate impact of standard lending practices on minority loan applicants, regulators have been pressuring lenders to modify their practices in ways that will reduce the frequency of mortgage denials and other adverse lending outcomes. But regulators have continued to monitor lending disparities on the basis of the size of relative differences in those outcomes. Thus, as previously discussed, by responding to regulator pressures to reduce adverse lending outcomes, lenders increase the chance that the government will sue them for discrimination.

An important related issue concerns efforts to determine whether a particular difference in outcome rates is a result of racially biased decisions by a lender. Federal regulators do not understand this issue either.

The issue can be illustrated with the same type of test score data I used to show that reducing the frequency of adverse outcomes tends to increase relative differences in experiencing them.  I posited a situation where, at a particular cutoff point, pass rates are 80% for an advantaged group and 63% for a disadvantaged group.  At that point, the advantaged group's pass rate is 27% higher than the disadvantaged group's pass rate, while the disadvantaged group's failure rate is 1.85 times the advantaged group's failure rate (37%/20%). 

I then explained that if the cutoff is lowered to a point where 95% of the advantaged group passed the test, assuming normal test score distributions, the disadvantaged group's pass rate would be about 87%.  With the lower cutoff, the advantaged group's pass rate would be only 9.2% higher than the disadvantaged group's pass rate, but the disadvantaged group's failure rate would be 2.6 times the advantaged group's failure rate (13%/5%).  Thus, lowering the cutoff reduced relative differences in pass rates, but increased relative differences in failure rates. 

Let us now assume that the pass and fail rates before and after the test cutoff is lowered actually reflect the loan approval and denial rates of two lenders and we have to try and determine which lender is more likely to have made biased lending decisions.  Those examining relative differences in favorable outcomes, as typically would be done in an employment discrimination case involving hire or promotion, would find the first lender more likely to be biased.  Those examining relative differences in adverse outcomes, as typically would be done in a lending discrimination case, would find the second lender more likely to be biased.

Obviously, however, it would be absurd to regard one lender as more likely to be biased with respect to approving applications and the other as more likely to be biased with respect to rejecting applications, and I would discourage the reader from wasting time searching for a rationale to support one approach over the other.  The opposite conclusions simply illustrate that neither approach can reveal which lender is more likely to be biased. 

So which lender is more likely to be biased against mortgage applicants from the disadvantaged group? As far as can be determined from the figures above, the only answer is that each lender is exactly as likely to be biased as the other.  Each pair of approval or rejection rates for advantaged and disadvantaged groups is based on a situation where the means of underlying, normal distributions of factors associated with loan approval (or rejection) differ by half a standard deviation.

Implicit in that answer is the only supportable methodology for appraising the size of a racial difference in lending outcomes (or any other outcome where demographic differences are matters of concern in the law or the social and medical sciences).  Just as knowledge of the difference between the means enables us to derive pairs of rates of advantaged and disadvantaged groups at particular cutoff points, we can derive from any pair of rates of an advantaged and disadvantaged group (whether for the approval or rejection) the difference between means of the underlying, hypothesized distributions of factors associated with experiencing an outcome.  For example, favorable outcome rates of 70% and 60% (with corresponding adverse outcome rates of 30% and 40%) reflect a difference between means of .27 standard deviations. Favorable outcome rates of 98% and 95% (with corresponding adverse outcome rates of 2% and 5%) reflect a difference between means of .41 standard deviations.

This method, to be sure, is an imperfect one. Among other things, it is based on assumptions about the shapes of the underlying distributions that commonly cannot be verified.  But it remains clearly superior to reliance on standard measures of differences between outcome rates without consideration of the way those measures tend to change simply because the frequency of an outcome changes.  And, for example, it enables us to recognize that there exists no rational basis for concluding that either of the two hypothetical lenders is more likely to be biased than the other. 

A more complete explanation of this method may be found in Section B of the Appendix to my letter explaining to the Federal Reserve Board a number of statistical issues pertaining to fair lending enforcement. 

James P. Scanlan is a lawyer in Washington. He specializes in the use of statistics in litigation.