Application modernization using GenAI (LegacyX)

Is your institution struggling to modernize legacy applications? Are your business stakeholders frustrated with the speed of application delivery because of technical debt? Come and hear how leading institutions are tapping into the power of GenAI to accelerate legacy modernization and transform business processes in tandem, to finally bend the cost and develop a compelling ROI for modernization. 

Hear from Aaron Bawcom and Leandro Santos on how GenAI is accelerating the entire modernization lifecycle from discovery to design and final conversion. 


Transcription:

Leandro Santos (00:10):
Welcome. Thanks for joining us. I'm sure more folks will come in, but just maybe a quick introduction. We'll talk today about using gen AI as an accelerator, as a way of working on your technology modernation efforts, especially as it comes to legacy core mainframe systems, things of that nature, which oftentimes gets in the way of all of the other fun stuff that we like to do. So cover that today just as a means of quick introduction. I'm Leandro Santos, Senior Partner with McKinsey. Do most of my work in the intersection of technology and businesses, work across industries, but banking, obviously, it's a big part of my focus and I lead our cloud and technology modernization service line, which is where we do a lot of this type of work. And then Aaron,

Aaron Bawcom (01:01):
Hi, Aaron Bawcom. I'm a Partner out of the Atlanta office. Been with McKinsey for about three and a half years. I came to McKinsey via acquisition. McKinsey acquired the company I was with to be able, what we focused on was largely technology modernization, cloud data and AI. I'm a lifelong technologist. I've been coding since I was very young and most of my days on the sector side, I spend with top 20 banks, insurance payments, capital markets based clients to help be able to modernize their technology systems. And on the functional side, I lead our cloud platform as well as our gen AI powered technology modernization effort, which we're going to dive into a lot here today.

Leandro Santos (01:44):
So a small group, I know you all seemed a little scared and stayed on your back, but let's just make it an interactive discussions since it's cozy. So that's the topic just to get started, I think that legacy similar organization oftentimes gets in the way, like I said, of all of the other cool stuff we're going to do. So if you look at some of the themes that we've been hearing around the conference, whether you're trying to kind of grow revenues or improve customer experience, whether it's around setting up a digital and physical collaboration, improving that, or if it's about automating what you're doing on fraud, oftentimes what we hear from our clients is that their legacy systems become an impediment to do that, right? Become certainly an impediment to be able to move fast. And oftentimes they just outcompete for funding because the importance of keeping these things resilient, operating and all of that, they just kind of suck away a lot of the investments that you'd have required elsewhere.

(02:44):
And it makes it challenging to innovate on your friends. You can do it to an extent, but at some point you kind of got to tackle what's there on legacy. And we look at it as a set of problems or difficulties that are created both on the business side and on the technology side. So like I said, on the business side, this is something that legacy ends up being something that slows down innovation, prevents sort of that digital transformation and just kind of hampers all of your overall modernization initiatives oftentimes. And then on the technology side, there is usually increased risk in terms of cyber resiliency on a lot of the systems. More and more the workforce availability is becoming an issue. And then if you just look at sort of VMware cost licenses and software licenses in recent years, vendors are realizing that companies will eventually get out of those platforms and therefore now is the time to jack up prices.

(03:44):
So they used to be not as bad from a run cost standpoint, but more recently we're seeing a significant inflation, especially in those areas. So those are some of the challenge. Now, why can't we overcome them? Why is it so hard to do it? Again, I think on the business side there is always the opportunity cost because if you're putting a dollar on legacy, you are not putting a dollar kind of directly on innovation, therefore it's hard to create the business case. And ultimately it's very difficult to build the buy-in across the stakeholders. Again, on the tax side, it is not always willing to push for that either because sometimes we're scared that there is a lack of documentation, there is a lack of resource that actually understand what was there before. And the experts and ultimately all of the solutions that have been available so far, they're suboptimal, right?

(04:37):
This is going to be a lot of hard work longs log, not sure if I'm going to succeed. So if the business is not calling for it, we better just kind of let it be and we'll tackle it in the future and it'll always be a problem for the future. Now we see gen AI as being a really strong force for change. Here again, a lot of the legacy challenge that we had can be aided with gen ai. So Aaron will talk a little more in terms of how we've been approaching that and success we had.

Aaron Bawcom (05:07):
Yeah, so when you look at many legacy modernization efforts to fund them requires a lot of people like you see on the left-hand side, there's a lot of different types of engineers, requirements analysts, product owners, test engineers, data analysts, architects that are all involved to be able to modernize. And we've all heard of the copilot use case, which is really accelerating what those types of people are doing within that journey. But what we've seen is this new pattern emerge of using what's called a multi-agent architecture where gene AI isn't used to be able to do just one task, but agents are actually teamed together into agent teams, and those agent teams actually work together as a part of a central agent orchestration with human oversight at the top. So if you're familiar with the concept of Tesla full self-driving, the concept is that your hand's on the wheel, but the car's doing most of the driving.

(06:07):
So this is where we're seeing significant acceleration in the ability to use gene AI to decrease the software engineering effort literally by automating things that software developers and engineers used to have to do themselves. And the impact of doing that is at minimum a decrease in the modernization speed of at least 50% and on the cost reduction, at least 30 to 40% in the modernization cost efforts. So it's very conservative numbers. We've seen these numbers be much higher, but these are bankable numbers, being able to use gene AI in this type of methodology. So if you talk a little bit more about, okay, we've heard how gene AI can be used within software development for technology, but there's different ways that it can be used. So what we refer to at the top is really the magic button approach, which is, Hey, I'm just going to put my code into a foundational model, into a prompt and ask it to be able to translate code.

(07:06):
And that works to a certain extent, but there's a lot of accuracy problems, hallucinations, different types of issues that still have to be handled manually. So what we've seen as an alternative to that is this three-step process where there is a possibility to use generative AI to reverse engineer an existing system, whether that's the existing technology processes, business processes, operations support teams, software engineering, requirements analysis, database systems, reverse engineer that into almost a software digital twin that represents how an existing system runs and operates. And that can be used to shape what the target state is. We all would love to be able to start from scratch. There was a conversation earlier where the ability to build a new system from scratch is ideal, but many are not able to do that because of the business rules and capabilities that have been built into systems for 10 or 20 years.

(08:07):
So what we're seeing from this type of approach is the ability to have your cake and eat it too. The ability to build a net new modern system but not have to throw away all of the legacy that exists that really has a lot of business context. So on the next slide, go into a bit more detail on what that looks like. So there's really that first step of reverse engineering we get asked a lot is, well, what do you reverse engineer? And the best answer is the kitchen sink. So basically every single aspect of how existing system runs. So documentation, observability, data, architectural wikis unit, everything that could be possibly documented about existing system. In addition, augmented by interviews with subject matter experts. If you've worked with legacy systems, there's usually that one or two people that have maybe been around for 10 or 20 years, they have everything in their head, but it's really hard to be able to get it out.

(09:04):
So using documentation as kind of the first step and then augmenting the gaps with those subject matter experts allows for you to form a complete picture very quickly for that existing system to be able to create these types of documentation for the system. But now you can use that to rationalize, 10 years ago we had to do it this way for these reasons, we don't need to do that anymore. We can now take that out and actually put this new process in or new technical architecture capability. Or maybe there's multiple systems that were built to the same thing or acquired systems that came in. They do very much the same thing. So being able to rationalize us down to a set of services that can be used and that those detailed technical specifications are the actual business context. Why does this system need to work and operate whenever a developer looks at a piece of code that developer wonders, why is this here?

(09:57):
Why am I developing this? How does this actually contribute to the business? So having that business context, the rules, and also the tests to be able to then use gen AI to be able to generate that target state architecture which could actually jump multiple generations to be able to get to let's say a cloud native state for being able to work and operate. So that's the full process all the way through. On the next slide, even though there's not this magic button of just, I do this one thing and everything works, you have to work through this in a very highly complex workflow. There are a series of breakthroughs that have actually happened that are yielding in a lot of benefit here. So I won't call these out, but I'll call attention to a few of these is being able to decompose a highly mesh set of code calling patterns.

(10:49):
So reflect on one client that had let's say 10,000 functions and each function called 2000 functions. This is a massive mesh of a lot of code that's just supposed to talk together. So being able to use generative AI to be able to turn those into hierarchies, be able to organize them in a standardized way, and then use similarity scoring to be able to pull out reusable components, that's the basis of microservices is really being able to do that. So we've now seen that it's possible to be able to do that with generative ai, being able to actually change data models from let's say a relational to a document document-based transaction type data model. So there's a lot of breakthroughs that have actually happened within the last year as this area tends to move very quickly. Here on the next slide, it's just going to show a little demo.

(11:35):
This approach, this modernization factory can really be used for any legacy technology.net or Java or VB script or PSLs toward procedures. But one of our favorites here for legacy is mainframe systems. So this is a demonstration of this approach being used for a mainframe card data processing based demonstration. So click one. Yeah, so we'll start the demo here. So now this is going through, the first thing is just what's all the code that exists? Okay, so here's all the COBOL files, the BMS maps, the JCL. So we're going to go look at one particular source code file. It's a lot of very, very legacy COBOL, a lot of copy books and a lot of very sophisticated nested if logic that can be very difficult to understand. So the first step is what does this code do at a high level? And we're going to come back to this documentation.

(12:32):
Okay, well, it's going to be able to process transactions on accounts and be able to post that to an index vs a m file. So here's now a high level description of what that code does, but as a developer, I want to change, I want to improve it a little bit. So I'm going to edit the description and say, well, I also want it to be able to improve the observability. I want transactions to be logged in a particular place that meets my new organizational standards and maybe didn't exist there. So just documents that and now from the documentation test cases can be generated to prove that any type of new implementation matches the old implementation. Once that's done, now here's the core. This was Java was the conversion. So now instead of flat files, this is actually converted to SQL base relational database. There's callouts for the application endpoints, the actual rest API endpoints, the data models have now been extracted in a modernized Java type of structure and that the developer can now go through that and just go through that workflow. The developer has their hand on the wheel and can modify the process at any given point in time, but they're not having to do the blocking and tackling of every single function, every single variable through the process. And so the intent is to turn a software developer from let's say a brick layer into a crane operator to be able to have much bigger impact on the decisions they make and where this modernization may have taken months using this type of process. It can happen in hours or days. So

Leandro Santos (14:02):
Yeah, on that, I mean we've been working with a few clients and banks of different size and other institutions, so we'll talk about one today. But like Aaron said, the technology obviously is still recent and it's evolving fast. But given how maturity LMS are getting in terms of various different combinations from two, it's just constantly opening up new opportunities and we're seeing a real impact already with the state of technology now and something that you can design, if you think of it as the approach that will continue to evolve and mature as improvements are made on the gen AI engine. So Aaron will talk about the one case here on the left.

Aaron Bawcom (14:41):
Yeah, it's kind of the milestone of the industry of let's try this out and see if it works, is kind of passed, is that there are now some pretty significant examples of being able to do this. So first example here is a top 20 bank with over hundreds of millions of wines of COBOL and it was absolutely amazing to see how fast gen AI was used, not just to modernize the code but also improve how well it performed. So being able to achieve a 500% performance improvement for being able to process transactions that formally the client thought that there was no way it could actually operate any faster than the mainframe. So going through the modernization process running on X 86 and it was a mix of batch as well as transaction processing. You can tell some of the API endpoints that were actually introduced from the kicks transaction to a modern angular user interface to be able to produce by provide the same outputs that were processed before using this type of approach.

(15:44):
And as far a big question we always get is, okay, well yeah, but the code that comes out, how does it look if you've used code translators before that actually go line per line? Sometimes what you get is referred to as Joe Ball or it's very difficult to read. It still looks like co ball, but it's Java here. We actually used an type of testing to be able to have a developer who would've done it manually versus what was done with gen AI to be able to score it. So these weren't our numbers, these were the actual client's grades for the type of output that ended up coming out. And you can see that in a very small timeframe, this was eight weeks of effort and eight weeks of data science to be able to get to this level is that with further data science, it continues to mature to very, very optimal levels in addition to this. And back to the so what or why go through this type of reverse engineering and to be able to generate that modern system to be able to reduce the cost to be able to lower the effort to modernize those legacy systems, whether it's data processing, whether it's a customer experience, whether it's calculating loan premiums or fraud, it's that the cost associated with that modernization needs to be able to be lowered using gen ai. So we've actually seen these benefits come, so it's real, the tangible benefits are absolutely there.

Leandro Santos (17:12):
So like I said at the beginning, we look at this as sort of the next step in evolution on how companies are developing software and that applies to legacy modernization. Also, if you think about your legacy systems and how you've looked at modernizing them in the past, then they usually have a Pareto in terms of the ROI that you got and companies, if you can justify the ROI, if it's only black, then those may get funded and may be executed. Perhaps the competition is with what are alternative use of funding, but at some point that the ROI is just not clear, it's just not there. It's not worth our time and it stops. So by accelerating the migration, by being able to therefore reduce the total cost to execute it, and then as Aaron said, even be able to improve upon some of the things that are there, that math sort of changed and what used to be underwater now becomes possible.

(18:12):
Again, not in a magic button type of approach, but still requires work but a lot less than what they did before. So if you think about your operator distribution, those that were on the right to where you had drawn that line before, now the line moves to the right and there are a few more that become viable. And especially if you are already using agile development methodology, if you're already leveraging things like public cloud, therefore instantiating infrastructure and new technology, it's a little more straightforward, especially if you're already doing that SecOps, you can shift a lot of Theas and whatnot to the left and kind of move faster, then these things compound on each other. And that's where GNI then can really make a huge difference here. And ultimately we see this as becoming part of the software development factory that you're going to have in place.

(18:59):
So the way that most companies start this, and we actually feel pretty strongly about this, is updating that business case upfront both with what are the additional benefits if I can do that bit of modernization we talked about, but also how can we actually test and make sure that we got a good sense of how much faster and how much cost this is going to be. Once you look at that and then you prioritize the areas that there is clear alignment to the business that there's clearly clear ROI on what you do quickly, A POC becomes the next step because the from twos are very, very unique to each organization what you have in terms of documentation, expertise, available language from to all of that. And that's where then playing of the toolkit bringing, like Aaron talked about those 20 breakthroughs on gen ai, that's part of the IP that say we would bring, but then putting that in the context of the actual tooling you're going to use for your particular organization and kind of tasking that tends to then reinforce or refine the business case and build the conviction that we can actually do that.

(20:03):
And then from there, usually it's around scaling 3D factories, putting the right governance in place and then eventually accelerating because as you start doing this, then both the tooling is going to get better, but most importantly your muscle becomes stronger in doing it. So even if you go back to that Pareto, the next tranche might actually go faster and be cheaper so you can actually gain some additional benefits or more applications can be modernized afterwards. So that's a bit of what we're seeing. We wanted to make sure we could pause and do Q&A. If you want to send, this is the note will go out to us. So if you want to save and do follow ups afterwards, but let's just do show of hands and we'll tackle questions live. Go ahead. Yeah,

Audience Member 1 (21:01):
As you get to the output you mentioned about what is the significance of how would you them to be able to verify the,

Aaron Bawcom (21:15):
Oh, it's standard engineering, right? Is that when you look at, if you were to manually rewrite something, there's a set of engineers of how they would figure out what the requirements were based off an existing system, introduce new requirements and then be able to build a new structure. And so that same process can now be done through gene ai. So what those engineers are now doing is just looking and saying, oh yeah, this is exactly how I would've done that and what I've done that and probably want to improve this and change this, how this is done. And then that is fed back into usually a central team that then makes those tweaks to the data science so that the next engineer that sees that particular situation is now going to have that be present. And so that's where the, for every hour that goes into data science in these types of modernization factories yields hundreds of hours of actual software engineering effort.

Audience Member 1 (22:12):
Are there certain systems within business units of the institutions or certain programming systems that are available, programing languages that work better than others With this?

Aaron Bawcom (22:24):
We keep hunting for something that doesn't work

(22:28):
And we haven't found it yet, is that it's absolutely amazing the level of the billions of dollars of investment that have gone into the foundational models is yielding fruit. Is that the most exotic systems you can possibly imagine can be used to be able to modernize this. We worked with a client to modernize something called E SQL, which is a proprietary IBM language, if you're familiar with IBM middleware to be able to process HTP messages that are going to go to message queues and go through some type of prostate, no one's heard of it and the foundational models actually understand it. Yeah, it can be able to reverse engineer it into a human English readable form of like, okay, it's going to receive this message, it's going to do this that can be shaped and then let's go express that within a Java container running in the cloud as opposed to doing it this legacy way.

Leandro Santos (23:24):
And I think part of why, so again, this is also where as the LMS progressed and more training data gets fed and whatnot, it's even going to broaden a little more. But I think what's unique about what we are talking here is that first we look at it as a approach, not a tool. Again, there is a bunch of tools out there and those can be incorporated. That's what you're going to use. But ultimately for each one of you to approach this as say, Hey, this is a methodology, this is a capability we're building within the organization because now there is the possibility to go and modernize and I want to kind of create that muscle within the organization to do. So. That's one thing is really kind of approaching it as an approach, as a methodology, not just as a tool that's going to solve it.

(24:06):
I think the second thing then is the reverse engineering, the ability to actually work on that pseudo language that kind of gets created. Being able to interfere so that you're not just replicating all of the legacy in a modern way. You can actually interact with it. And that's something that's a bit different than what a lot of others are doing there. And then the last point is when Aaron talked about those 20 different kind of breakthroughs, that's where a lot of the proprietary IP and the experimentation is happening. We keep adding to it because we look at how engineers would go about doing modernization and then how can we assess where can a machine completely replace, where can it be a copilot? And that's where I feel like when we first started, I think we had less than a dozen. We're at about 20 or so. So those will keep getting added. And that's what I meant by over time we're also getting better as an organization to do it.

Aaron Bawcom (24:52):
I think the real key here is the difference between developer optimization, which is helping developers go faster versus this approach which is actually automating what developers would otherwise have to do. And that's why it's a kind of a step change from the copilot use case is actually eliminating developers having to do it. I was talking to somebody here earlier today that I've seen an example doing data analysis where two software agents argued with each other over the quality of the data to be able to get to the right answer. The goals were set. Clearly this is the goal we want to get to. And so when the goals weren't being met, the agents would go back and forth to each other until they finally got to the right answer. It's amazing like this type of progression. Sorry, is there a question in the back?

Audience Member 1 (25:34):
I was just this sidebar, just directly systems technology to the extent that a system has encoded a part, but there's a bigger context. Oh yeah. Developer working with business analysts, there's an entire workflow in the middle. Does the AI have the capability to deal with that broader context and to actually map it out?

Aaron Bawcom (26:09):
I think the best way to answer it is the AI doesn't itself. This type of approach that we are talking about, which is a structure on top of the foundational models could, and we've actually seen it where it can. So being able to decompose the business process

Audience Member 2 (26:28):
So it can understand, for example, all the systems documentation and then make observations, for example, a person interacting with the documents associated with the actions that people are doing,

Leandro Santos (26:49):
The operational procedures.

Audience Member 2 (26:52):
Server at a very high level to observe that I know this person A, they've done these documents,

Aaron Bawcom (27:01):
Correct,

Audience Member 2 (27:02):
And it interacts with this system. I know with the documentation of the system together, does it actually have the ability to do that?

Aaron Bawcom (27:14):
The answer is yes, they have limited on time, but there's a separate demonstration that we can actually show you of going through all the way from a call center and what call center agents are supposed to do, the call center logs of the type of issues that are coming in and basically the problems that people are calling about to be able to then understand the architecture of the existing system with that to be able to then suggest how to change it so that those type of call center problems stop coming in because the technology changes to be able to adapt to it. And then generating that new target state for the architecture, the implementation as well as the process. It's absolutely possible to be able to do that. And we've seen it done include

Audience Member 1 (27:58):
Ux.

Aaron Bawcom (27:59):
Absolutely.

Leandro Santos (27:59):
You could. Yeah. And I think this is again how you want to design the scope, right? Because you could use it the more straightforward way to don't do any of that, just look at the system, replicate the system, maybe prune areas that you don't need anymore, but don't really change the processes. The other way is to kind of put the process overlap on top and then how can we also optimize there? That's obviously a little, again, there are agents that will help you do that, but you need done all of those skills within the team also because the change management AI can't quite do it yet. So you need to be able to make those judgment calls and adjust. And then it's just again, how you define the scope of the change you want to do, but the assistance are there, kind along the way.

Audience Member 1 (28:41):
Create new documentation then.

Leandro Santos (28:43):
Yeah, I think that's the better. Like I said, on that reverse engineering state with the pseudo language and the documentation, you can then add, you can change, that becomes documentation for the two B state essentially. So once you're done with the conversion, then obviously you forget about legacy and that becomes your documentation and this becomes the new system. So you're kind of generating as well as we move through it quickly, but in the documentation we're getting down to what are the task case to actually test the system afterwards, which again, you can also augment those. So the tool can generate a set by reverse engineering what's there on the code. You can always kind of come back with others.

Audience Member 1 (29:22):
Are there concerns that the test cases it develops are

Leandro Santos (29:27):
Not robust? Not robust

Audience Member 1 (29:28):
Test? What if itself also created?

Leandro Santos (29:31):
Yeah, yeah.

Aaron Bawcom (29:32):
And that's why there's a human in the loop for every single step of the process. The human has their hand on the wheel no matter what, every single step.

Leandro Santos (29:40):
Right. But that's why we're always skeptical of the No. Yeah, the thing can do everything. We feel strongly that you still need to apply the judgment call on what you're doing. The key is, like Aaron said, most of it gets automated. You do a course correction here and there and you add, and again, I feel strongly that as you're creating a new system, part of what you want to eliminate is the knowledge gap that we had from people that maybe left. So it's very important to have those skills kind of going forward, right? Yeah. How are doing time, any issues

Audience Member 3 (30:18):
Around some of that stuff around moderation of code from applications?

Aaron Bawcom (30:24):
Great question. I'll have two answers for that. So the first is that the enterprise LLMs, like let's say like Azure, OpenAI or Gemini, that can be provider clawed that are provided within a virtual network environment that you and the client, it's your environment that doesn't leave. So most of those agreements actually prevent any of the data that actually goes to be used for training and there's limits for how long they're cached in certain circumstances. So that's one. The second method for handling that is that we've seen this as well is that instead of sending data to the LLM, that the data is, the agents are used to write code that will process the data according to the goals that are set at the high level so that the data never hits, it never even goes to the LLM, right? So you're basically doing higher level analysis that is being instrumented and automated through LM to write the code so that you don't have to have developers to go and do it. So it's like multiple different abilities to be able to control IP.

Leandro Santos (31:43):
I can't quite see the back

Aaron Bawcom (31:44):
Is any more questions. Okay.

Leandro Santos (31:46):
Alright. Well thanks for the time. Yep.

Aaron Bawcom (31:49):
Any more questions? Yeah, is the email addresses there.