A Wall Street regulator is opening a probe into Goldman Sachs' credit card practices after a viral tweet from a tech entrepreneur alleged gender discrimination in the new Apple Card's algorithms when determining credit limits.
A
The same thing happened to us. I got 10x the credit limit. We have no separate bank or credit card accounts or any separate assets. Hard to get to a human for a correction though. It's big tech in 2019.
— Steve Wozniak (@stevewoz)
November 10, 2019
Hansson didn't disclose any specific income-related information for either of them but said they filed joint tax returns and that his wife has a better credit score than he does.
"The department will be conducting an investigation to determine whether New York law was violated and ensure all consumers are treated equally regardless of sex," said a spokesman for Linda Lacewell, the superintendent of the New York State Department of Financial Services. "Any algorithm, that intentionally or not results in discriminatory treatment of women or any other protected class of people violates New York law."
Apple Card only offers individual accounts and it is possible for two family members to receive significantly different credit decisions, a Goldman spokesman said. "In all cases, we have not and will not make decisions based on factors like gender," he said.
Hansson said Goldman's response doesn't explain what happened after he started airing his issues on social media.
"As soon as this became a PR issue, they immediately bumped up her credit limit without asking for any additional documentation," he said in an interview. "My belief isn't there was some nefarious person wanting to discriminate. But that doesn't matter. How do you know there isn't an issue with the machine-learning algo when no one can explain how this decision was made?"
I’m surprised that they even let her apply for a card without the signed approval of her spouse? I mean, can you really trust women with a credit card these days??!
— DHH (@dhh)
November 7, 2019
This is the second such action from the regulator in recent weeks. NY DFS opened a probe against health care giant UnitedHealth Group after a study found an algorithm favored white patients over black patients.
Goldman's growing ambitions for main street is bringing increased scrutiny and a new set of challenges it hasn't faced previously. The Apple Card is a joint venture between Apple Inc. and the New York-based bank, which is responsible for all the credit decisions on the card. The card was rolled out earlier this year and executives at both firms hailed it as the most successful launch ever.
Traditional lenders are upping their use of machines to decide who gets how much credit as part of a strategy to reduce costs and boost loan applications. Meanwhile, technology companies are moving in on the financial services industry's turf, with businesses such as Amazon, Apple, Facebook and Google offering loans and payment options.
Black-box algorithms
Hansson said his posts had led to an internal review and that he was hopeful it would spark a conversation about black-box algorithms and the inherent biases in those systems.
The 40-year-old Dane is known for being the creator of the popular programming tool Ruby on Rails. He's a partner at Basecamp, a web-based software development firm, and also known to regularly take part in automobile endurance races, including the 24 hours of Le Mans in France.
"Goldman and Apple are delegating credit assessment to a black box," Hansson said. "It's not a gender-discrimination intent but it is a gender-discrimination outcome."
The use of algorithms by lenders in credit decisions has drawn scrutiny in Congress. In June, the House Financial Services Committee heard about examples of algorithmic decision-making where researchers have found instances of bias targeting specific groups even when there was no intent to discriminate.
Some lawmakers already are demanding a federal response. Sen. Elizabeth Warren, a Massachusetts Democrat and contender to challenge President Donald Trump in the 2020 presidential election, told federal regulators in June that the government "will have to take action to ensure that anti-discrimination laws keep up with innovation.
