As bankers play with new data types, beware of ingrained bias

The philosopher John Locke posited in the 17^th century that we are rational creatures who use our intelligence to draw rational conclusions based on the evidence of our eyes, ears and other senses. Given that assumption, a visitor from outer space would probably be surprised at the extent to which people today disagree on basic facts as wide-ranging as the state of the climate and the state of the economy.

In opposition to Locke, social scientists’ research suggests that our biases can make us believe things that are objectively false. It is important to be aware of these biases as they can lead to poor decision-making. Anecdotally, we hear stories about job-searchers stymied by the fact that their name conveys a Hispanic or African-American background. A natural predilection to follow our own biases can lead to poor business moves, such as a faulty decision on whether to extend credit to an applicant. The question is: How do we overcome our innate prejudices?

I wrote an article in BankThink two years ago about how social-media data informs decision-makers at banks in all types of areas from hiring new employees to making loan decisions. I thought, and still do, that such data can potentially be beneficial for individuals locked out of the financial system’s more conventional data types, such as credit scores. At the same time, it is clear from recent fake news stories on social media that such data is open to manipulation or biased interpretation.

Sasho Bogoev/benjaminec - stock.adobe.com

A recent episode of the British speculative fiction TV show, “Black Mirror,” takes how people today “like” photos and posts on Facebook and other social media sites to a new level. The show imagines a future world where people themselves are “liked” by others based on their everyday interactions. Each person is tagged by a likeability score which is then used by real estate agents, banks and so on, to make decisions as to whether or not they will extend loans and sell products to their applicants. People with a low likeability score find that their applications are denied and so change their behaviors, for example, by smiling constantly, by answering rudeness with graciousness, or by friending “popular” people, to improve their scores.

The show is pure fiction. But when we make loan-approval or hiring decisions in part based on LinkedIn or Facebook profiles, we run the risk of becoming unduly influenced by aspects of an applicant’s persona — their comments, likes, interests, community memberships etc. — that are unrelated to assessing relevant qualifications. As a result, such decisions on extending credit or employment become less based on rational, objective thinking.

To describe this risk in plainer terms, relying too much on such factors from an applicant’s social media profile can incorporate the decision-maker’s personal bias into the process. The decision could be driven by “motivated reasoning” — a term cognitive and social psychologists developed to refer to a motivation to use information at one’s disposal to arrive at a certain, desired conclusion.

Julia Galef, co-founder for the Center of Applied Rationality, in a recent Ted Talk looked at a scandal that rocked France in the late 19th and early 20th centuries — the Dreyfus affair — to illustrate the concept. In the 1890s, a Jewish artillery captain in the French army, Alfred Dreyfus was falsely convicted of passing military secrets to the Germans and sentenced to life behind bars. The evidence against him included a torn-up piece of paper in the trash, with writing tied to Dreyfus. Galef said investigators’ anti-Semitic bias concealed how thin their case was.

“He had a sterling record, no past history of wrongdoing, no motive as far as they could tell,” Galef wrote of Dreyfus. “However, Dreyfus was the only Jewish officer at that rank in the army, and unfortunately, at the time the French Army was highly anti-Semitic. The other officers compared Dreyfus’s handwriting to that on the paper and concluded it was a match, even though outside professional handwriting experts were much less confident about the similarity.” After he was imprisoned, an intelligence officer, Georges Picquart, uncovered evidence pointing to a different suspect, and Dreyfus was ultimately pardoned.

Galef defines Picquart’s subsequent investigation as the “scout mindset,” reflecting a search for facts and information versus someone who looks only for clues that confirm their own bias. She describes this mindset as “the drive not to make one idea win or another lose, but to see what’s there as honestly and accurately as you can even if it’s not pretty, convenient or pleasant.” In contrast to the scout is the soldier, who, Galef says, is influenced more by emotional responses and motivated reasoning. “Our judgment is strongly influenced, unconsciously, by which side we want to win — and this is ubiquitous,”

Motivated reasoning and biased thinking are particularly timely topics in the financial world. One issue that came out of the financial crisis of 2008 was the fact that the right mortgage products were not necessarily sold to the right people. People without the means to pay were saddled with so-called exploding mortgages concealed behind the façade of an apparently benign set of payment terms. While such sales were viewed by the public and commentators as the result of cynical, even criminal activity by bank sales people, it is also possible that they were the result of poorly thought through, motivated thinking. The motivation behind the reasoning was the sale of a product to a client that otherwise would not be able to afford to purchase the product. The reasoning behind it was that the housing market would continue to go up and refinancing would be possible at a later date. What is needed, however, is the scout mindset that Galef refers to in the Dreyfus case.

This mindset is especially necessary in the current environment of deregulation. As the Trump administration installs new leadership at the Consumer Financial Protection Bureau and other agencies, and regulators signal a more relaxed approach to investigating cases that require consumer protection, one hopes that more banks’ loan and mortgage officers can adopt a scout mindset as they set out to identify objectively the right product for their clients.

As Galef put it, “the scout’s job is not to attack or defend; it’s to understand. The scout is the one going out, mapping the terrain, identifying potential obstacles. Above all, the scout wants to know what’s really out there as accurately as possible.”

The scout is not tied to a particular outcome but is only aligned with finding the truth. For example, a scout may posit that capital punishment works to deter crime but if studies come out that show the contrary, he or she will accept those findings. This cluster of traits is what researchers have found — and I’ve found anecdotally — will predict good judgment. When being aggressively pushed to meet certain financial targets, bankers and financial consultants will be highly motivated to meet those targets and so rational and objective thinking can fall by the wayside as they look to fit potential customers into certain products that may not fit all the objective criteria.

Loan officers, hiring managers and other banking employees should approach any new decision with a curiosity, withdrawing their prejudices from the decision-making framework, so that they can focus on making the decision that is right for the bank in an objective sense.