Sprawling is a word often used to describe the disparate technology organizations that reside in the many banking companies that have been assembled via acquisition.
But it's a word that doesn't begin to describe the organization Bank of America's Catherine Bessant presides over, an IT operation with dozens of data centers, thousands of different software applications, and more than 100,000 employees around the world.
Nor does the word streamlining do any kind of justice to the task that CEO Brian Moynihan has handed to Bessant: turning that far-flung empire into a truly integrated global technology and operations division.
Some 18 months into the task, she recalls how Moynihan described her then newly assigned mandate: "Think of GT&O as the spine of the company; your job is to make sure the spine of the company works in an integrated, cohesive fashion for customers."
Importantly, that means that getting smaller isn't the only task on the to-do list - along the way Bessant is also aiming to optimize resource allocation and usage, tighten up operations risk monitoring and controls, and improve energy efficiency.
Given the myriad regulatory, capital, and other challenges that have buffeted BofA in that time, the project perhaps understandably has flown under the radar of people outside the Charlotte company. But there is widespread agreement that such a project was, if anything, overdue.
"At Bank of America right now, because they have such a massive cost-cutting program under way, it's important for them to rethink technology, what they want and what's good for them. It's a major issue for the industry and for this bank," says Richard X. Bove, an analyst at Rochdale Securities.
For staffers, the sheer magnitude of the change underway - implementation is scheduled to be complete by 2014 - of course brings anxiety. And it's hard to see how operations would be immune from reductions in staffing that are occurring across the company. (Plans to reduce staffing by 3,500 were announced subsequent to the interview.) But not all on the sentiment front has been negative. A recent survey by a technology publication rated Bank of America among the best places to work in IT; the bank did not make the list last year.
In talking to Bank Technology News, Bessant, whose official title is global technology and operations executive, offered the first public under-the-hood look at the project, articulated a vision for where the IT operation fits in BofA's broader objectives, and discussed how even small decisions like restricting access to services like YouTube freed up surprising amounts of network bandwidth.
Centralizing Platforms
The first piece of BofA's IT transformation - centralizing IT - is a particularly daunting task at the bank, which is still dealing with the technology it inherited with its acquisitions of Merrill Lynch, Countrywide, MBNA and LaSalle. The global tech and ops organization comprises 109,000 employees in 40 countries. For most major activities - card processing, payment processing, foreign exchange trading - Bank of America would like to use one platform across the entire organization.
In choosing which platforms to keep and which to discard, the tech team hasn't necessarily sought the most state-of-the-art systems. "The lens is, how can we bring the companies together with minimum customer disruption," Bessant says. The card processing platform chosen, for instance, belonged to MBNA, the bank holding company BofA bought in 2006. At the time, MBNA was the largest credit card issuer in the world.
In other cases, the bank has chosen the most technologically advanced system. For example, in the corporate and commercial space, LaSalle, a Chicago regional bank BofA bought in 2007, had a payment portal that surpassed anything Bank of America was using. Although the smaller bank had only 10% of the overall customer base and it was a huge transition, "we made that decision because the capabilities allowed us to leapfrog what we would have tried to build in the old Bank of America," she says. The work of selecting which major processing platforms to standardize on will be completed by the end of 2011, Bessant says.
Certain systems that support local laws, reporting rules and cultural differences will need to remain in place. "We need to be consistent but customizable to the extent that local custom, local law, local regulation and even local ways of working require a different approach," Bessant says. "What works in Brazil doesn't work in Mexico, and vice versa."
Shrinking Data Center Footprint
Simplification at Bank of America means doing more with less: fewer data centers, fewer applications, and "fewer elements to every vertical that starts with a great idea and ends with a customer," Bessant says.
Bank of America has a network of 55 data centers, mostly in the U.S. Bessant plans to halve this number over the next 18 to 24 months. She's already shut down two data centers and 92 server rooms. The bank is a quarter of the way toward its server reduction goal. BofA is also consolidating its mainframe environment. "Our mainframe environment is one of the largest in the world, operating in over a dozen locations," says Marc Gordon, CTO. "We intend to reduce it by 15% to 30%"
One factor in deciding which facilities to keep and which to ditch will be the availability of renewable, inexpensive sources of power such as hydropower electric plants. "Since power is the number one expense in a data center, we want to use or develop alternative sources of power that ultimately should prove to be more reliable and cheaper," Bessant says.
Another factor, for data centers that house time-sensitive trading applications, is proximity to exchanges. In high-speed trading, distance from an exchange can cause minuscule delays in the transmission of data from exchange server to trading platform and back again. This is why so many Wall Street firms and large banks either have their own data centers in New York exchange-friendly locations such as Mahwah and Piscataway, New Jersey, or collocate servers in an exchange's facility or a hosted facility nearby. The New York Stock Exchange, for instance, has built a profitable side business from server hosting in New Jersey and London.
A third factor is business continuity and disaster recovery. "As we consolidate, how do we not become dependent on these mega data centers?" Bessant muses. "I've got this concept of what I call tertiary redundancy for our core platforms and our payments platform, making sure we have tertiary failover capability - will you be able to get money into your checking account, be able to pay a bill? It's different than 100% uptime; of course we want that. But if something fails, do we have 100% capability to do a total failover?" She cites the bank's Japan branches as an example. "After the earthquake and tsunami in Japan, the trading markets and the funds transfers in Japan had to function; we were proud that we settled all our transactions on time every day," she says. "That team had been brilliant over the years in the identification and elimination of risk."
Bessant won't discuss which locales are under consideration as consolidated data center sites. "The minute I talk about specific data center plans, the price of real estate in that area goes up," she notes. At press time, Bessant's attention was diverted to the bank's London data centers, two kilometers away from some of the riots in August. "We would never exit the geographies that are important to us," she noted, but she was making sure the bank's failover plans would work even with mobs wreaking havoc in several cities.
The data centers that survive the ax will be redesigned to be more efficient than the current ones. "Eighty percent of our servers operate at 5% capacity, more or less. It's mortifying," Bessant admits. Many organizations suffer from low server utilization - to accommodate periodic bursts of peak activity, powerful servers sit idle during periods of low activity.
"The trick is to create peak capacity without creating idle capacity at nonpeak times," Bessant says. Her team is implementing server virtualization and workload balancing to even out the distribution of work among servers and cut down idle time. "In the days before technology around routing and load leveling, your server capacity had to equal the most you could ever need - peak utilization, not average utilization," she says. "Now with load leveling, you can run your applications at a higher utilization because you can on a very real-time basis with no diminution of service from one server to another."
Rival money center banks, including Wells Fargo, Citigroup and JPMorgan Chase, went through the exercise of server and data center "rationalization" and server virtualization to improve usage rates several years ago. Wachovia, shortly before becoming Wells Fargo, was the first to bring this concept to a large bank and harness thousands of servers into one farm, with a software layer that decides what work will take place on which machine and lets users provision their own computing resources and get billed for what they use. The terminology changes all the time, but currently such managed server arrays are usually called internal clouds or private clouds.
Streamlining Software
Bessant is also working to cut the number of applications the bank uses from 7,000, down to 3,500 or fewer. "We often have multiple applications that do the same thing, for instance, three different apps that help us send wire transfers," she says. "Finding places where they're redundant is the easiest place to make decisions, along with finding the places where they're inefficient, where there's a different or better solution or where the apps are no longer supported. In a company as big as ours, we have more than our share of apps that are no longer supported by the people who sold them to us. The decisions are painful for some, she says. "I've learned that apps are very personal things."
The bank is moving away from building its own applications in areas that don't touch customers, such as credit card processing. "What people want is product development, speed to market and the ability to create and design credit card packages that are relevant to them," Bessant says. "How we create your statement? That's immaterial to everyone."
The bank also decided several months ago to stop producing actual credit cards. It shed several factories it had for this purpose.
"We're looking at every place where we have a system that was developed in house, some of which are 20 to 30 years old, to make a decision about buy or build because it's unsupportable to have technology and processing capabilities that are that old," Bessant says. Many of these applications were built pre-Countrywide and pre-Merrill Lynch, when the bank looked very different than it does today.
The time frame for this overall application reduction goal? "Tomorrow," Bessant says with a laugh. "Nothing stands in the way of application reconciliation except time and sheer will," she says. "It's a matter of being definite about the goal and freakish about measuring against the goal and ensuring we have the right resources to invest in application retirement. It isn't necessarily a cheap thing, but the benefits of doing it are legendary." She's eliminated 106 "non-strategic" applications so far.
In the new order, workflow software will be used more prominently, to automate manual processes and achieve straight-through processing wherever possible. "When I came into this role, the thing that I thought would keep me up at night was, how does a place of this size and scale run every day, and how do I know it's going to run every day?" Bessant says. "That's not what keeps me up at night, because we run at Six Sigma levels across our functions. What keeps me up at night is, with the volume of manual processes in that mix, how do we sustain this level of performance? This effort to modernize and simplify what we're doing is largely about reduction of manual tasks and emphasis on automation as a way to make our Six Sigma performance sustainable."
The bank is in the middle of automating its general ledger processes and reporting. "We're deploying a slightly customized version of a classic SAP solution," Bessant says. BofA is also automating its anti-money laundering compliance, and plans to have a new AML system running by the end of the year.
Reducing Risk
Another goal for the global tech and ops team has been to reduce risk, partly because Basel II requires it. "Much of what we do is a definable science, so it lends itself really well to risk reduction; everything from platform stability - no outages, zero tolerance for failure to perform - down to, do the security cameras in your cash vault work?" Bessant notes. The group is working to put the right controls in place and monitor those processes to make sure the tasks that are identified as necessary are deployed, she says. "It's one of the geekier parts of what I do, but I like it because I think banks have to be about taking risk, but taking the risks we want to take. Taking risks we don't want to take that we can't get paid for, has to be at minimum." Robosigning can't happen, for instance.
GT&O employees go through a risk boot camp in which they have to self-identify the risk issues in their areas and come up with a plan for addressing them. Bessant looks at the self identification percentage for every one of her top 250 managers. "I can tell you on a monthly basis who is self-identifying and who isn't," she says.
Computing in the Cloud
Although Bank of America has several internal or private clouds in operation today, particularly on the Merrill Lynch side where high performance is so important to capital markets clients, the bank is not interested in purely external cloud implementations.
"To me, cloud computing is a shared service with flexible capacity," Bessant says. "That's possible privately across our businesses, and it's possible with other companies. A private cloud or a shared cloud with other similarly focused companies [for instance, companies that share a deep concern for security] are viable options for us and we use both today. But the Amazon version of a cloud, where anybody can buy space and do anything with it: that is something we will never, ever sign up for." The bank does not even let employees purchase items on Amazon using work computers.
BofA has also restricted access to YouTube and Pandora, which were for a time consuming 25% of network capacity. "Think about that from a shareholder capacity, the cost of that, the inefficiency," Bessant says. "We already run a network that's the size of a telecommunications company."
Bessant gets push-back (a.k.a. whining) from employees on the Internet restrictions, but she shrugs it off. "Before I was in technology and operations, I always thought of the technology people as 'they': 'they said this or that.' Now I'm 'they.' I don't know what to do with that except to embrace it, own it, make it mine." The bank does let employees use most other sites at work.
Being Green
Bessant chairs Bank of America's Environmental Council and oversees the bank's 10-year, $20 billion commitment to address global climate change. In 2004, the bank stated a goal of reducing greenhouse gas emissions by 9%; by 2009, it had reduced emissions 18%. In May, the bank set a new goal: to reduce absolute greenhouse gas emissions by 15% from 2011 to 2015, based on its 2010 baseline. Bessant sees this as a civic responsibility and a practical plan. "Power, water and waste are three of the most expensive things we deal with," she says. "Our overall permanent cost and unit cost run rate is largely determined by our physical footprint and carbon footprint." The bank has won LEED certification for many of its office buildings and data centers, and measures itself "relentlessly" against its environmental goals, she says.
The bank's custom-built tower in New York City's Bryant Park was the first skyscraper to be platinum LEED certified. It features insulated floor-to-ceiling glass walls that retain heat and maximize natural light. The cooling system produces ice during off-peak hours, and allows the ice to melt during the day to cool the building during peak energy use.
A building and hotel complex the bank built in Charlotte, N.C., is also LEED certified. In June, Bank of America Merrill Lynch, Prologis and NRG Energy announced a conditional commitment from the U.S. Department of Energy's loan programs office to help finance the largest distributed rooftop solar generation project in the world. The loan guarantee supporting $1.4 billion of debt facilitates a total project size of about $2.6 billion, which is being financed by the private sector over the next four years.
These efforts get mixed reviews from green IT expert Bill Tschudi, program manager at Lawrence Berkeley National Laboratory. "We have worked with BofA here in California and recognize that their corporate energy reduction goals are good," he says. "But like most financial institutions, they are very set on operating the way they always have as long as it's reliable. Unfortunately, this leaves a lot of energy efficiency opportunity on the table." While the IT industry is moving to allow equipment to operate at higher temperatures (and thus reduce the amount of energy consumed to cool it) up to 80 degrees, bank data centers tend to be cooled to the 60-degree range.
In addition to its ongoing data center consolidation, application streamlining and green IT work, Bank of America has several mobile banking applications on its drawing board; the bank has seven million mobile banking customers to date and was among the first to offer iPhone and iPad apps. It also has plans to integrate customer-facing banking and brokerage applications, to provide a joint Bank of America/Merrill Lynch experience to mass affluent clients.
"Right now, the work we're doing is not glamorous but integral to driving out our vision for the company, which is to deliver all our capabilities to each client and system," Bessant says.
