A recent analysis of large banks' efforts to develop and deploy artificial intelligence technology gives JPMorgan Chase top marks, followed by Royal Bank of Canada and Citigroup. UBS Group, Wells Fargo, TD Bank, ING Group, Bank of America, BNP Paribas and Morgan Stanley round out the top ten.
The
"Banks are spending billions on digital transformation, and AI has become a larger and larger component of that," said Alexandra Mousavizadeh, CEO and co-founder of Evident, which is based in London. They haven't had a way to compare how their AI talent, innovation, leadership and organizational structure and responsible AI stacks up against their peers, she said. Banks can use the index to gauge their strengths and weaknesses as well as to attract talent.
A look at the ranking, the analysis and how those at the top of the list got there could be relevant to any bank that's thinking about its AI strategy — in other words, all banks.
JPMorgan Chase uses machine learning in risk monitoring, prospecting, marketing, customer experience, fraud prevention, payments processing and money movement, according to Teresa Heitsenrether, chief data and analytics officer at JPMorgan Chase. Heitsenrether got the job of overseeing AI across the organization in June. Before that, she was the bank's global head of securities services for eight years.
Machine learning "helps us significantly decrease risk in our retail business by reducing fraud and illicit activity, and it improves trading optimization and portfolio construction by providing optimal execution strategies, automating forecasting and analytics and improving client intelligence," she said. "The rise of generative AI and large language models is the next frontier, and we are exploring how it can deliver value to the firm."
Heitsenrether declined to say how much JPMorgan Chase is spending on AI. The bank has estimated that in 2023
"JPMorgan Chase seems to have created a Big Tech culture within the bank, which is probably why it does so well in the index," said Aaron McPherson, principal at AFM Consulting. "This has been a clear priority for the CEO for many years."
Ranking the banks
The first set of measures in Evident's scorecard are of AI-related talent, including how many data engineers, AI implementers, risk modelers and AI researchers a company has, Mousavizadeh said.
"We also look at talent development because there's an enormous amount of poaching that goes on among the banks," Mousavizadeh said. "We see who's moving from where. Wells Fargo took a very large chunk of Bank of America's talent. And then we've seen recently lots of moves from HSBC to JPMorgan in the U.K. markets. Hiring initiatives and training and development and all of those things that keep your AI talent in place are very important."
JPMorgan has more than 900 data scientists and 600 machine learning engineers, Heitsenrether said.
"For years, we have been hiring the best and the brightest in this space — PhDs from the top universities around the world," she said. "We have rich datasets, interesting use cases and a firmwide commitment to investing in this space."
The next category Evident considered is leadership.
"Jamie Dimon is a good example of expressing a narrative of the bank's vision for using AI," Mousavizadeh said. In addition to CEOs' public statements, her team also looked at how much a bank engages publicly with regulators, how well its organizational structure is set up for AI, and how it handles responsible AI.
"A big part of leading AI in a large organization is prioritization and connecting the dots across the firm," Heitsenrether said. "Oftentimes, the solution we are developing in one part of the bank can be leveraged to help solve a problem in another."
Another category Evident looked at, innovation, "is all about pure and applied research," Mousavizadeh said. This includes AI-related patents, strategic investments in AI companies, partnerships with universities and Big Tech, and anything else that could accelerate AI implementation. Capital One Financial and JPMorgan Chase have published a lot of AI-related research, for instance, she said.
Birth of an AI benchmark
Mousavizadeh started Evident a year and a half ago. An economist, she has spent much of her career building indices, first at Moody's Investors Service in New York, where she measured sovereign risk in Russia, Central Asia and Africa.
"Doing sovereign risk is all about building the measurement that can best tell you about risk of default of collapse of the economy or collapse of society, and that's all based on an index," said Mousavizadeh. She ran the country risk team at Morgan Stanley, again building indices for measuring sovereign risk of default. She moved on to be CEO of the credit ratings provider Arc Ratings.
In 2018, James Harding, who at the time was director of news and current affairs at BBC News (before that, he was editor of The Times of London), was setting up a media platform in England called Tortoise Media and asked Mousavizadeh to lead the intelligence team. Shortly after she started, the U.K. government asked her to create an index rating countries on their development and deployment of AI, and she made that her first index at Tortoise.
She started getting requests from banks, as well as pharmaceutical and manufacturing companies, that wanted to be able to compare their AI adoption against their peers, she said.
Evident uses only public data for the benchmark, Mousavizadeh said, because she doesn't trust surveys, which can vary depending on who in a company fills them out and their mood that day.
"In my view, you get a much, much more accurate picture if you build the benchmark on many, many data points that take the footprint of the bank's AI activities," she said. "And actually you can read a lot of what's going on inside the bank when you do it that way. But that also means that you don't rely on input from the banks."
Data sources used for the index include Crunchbase, the banks' LinkedIn job postings, patent repositories, conference paper submissions, citations of patents in research papers, the software developer platforms GitHub and Kaggle, as well as comment letters to regulators.
Some industry observers give the Evident AI Index high marks.
"I do think it is a useful tool, if only to establish a framework for what it means to be leading at AI implementation," McPherson.
Executives at Citizens Financial Group and community banks such as Seattle Bank and Citizens Bank of Edmond say there are potential use cases for large language models. Yet they have concerns that impersonal service and likely errors could erode customers' faith in their financial institutions.
But they also point out shortcomings.
"Relying only on public data has the limitation that it can be gamed," McPherson pointed out. "Banks with superior marketing teams can make sure that AI is prominent in all their communications, creating the impression that they are doing more than they really are. Also, banks tend to hire loudly and fire quietly, meaning that if their investments don't pan out, they will not keep the expensive resources around."
A good addition to the index would be a measure of AI-based products, because this would show that the banks are doing more than just conducting research and registering patents, but actually applying the results to their business, he said.
For the next iteration of the index, Evident may get at this, by setting up a system where banks submit their more sensitive data, Mousavizadeh said.
"Seven banks have raised their hand to say they'd like to take part in that," she said. Over time, her team plans to track banks' performance metrics against how they handle AI.
