CFE Federal Credit Union’s inaugural Lending Analytics Competition challenged University of Central Florida graduate students to develop forward-looking predictive models that could better interpret data.
“UCF has hosted similar competitions in the past, so CFE offered to bring this competition to the next level,” said Jason Mizrahi, CFE Federal Credit Union’s manager of business intelligence. “It was our intent to give these students an opportunity to transition their classroom learning and analytical skills into real world problem solving.”
In total, seven teams comprised of 23 graduate students participated in the competition that spanned roughly four months. The winning team received a total of $5,000 in scholarships.
“Our decision process for determining a winner was broken down into two categories: validation and presentation,” explained Mizrahi. “It is important to perform a good analysis that can be validated, but you have to be able to effectively communicate your findings to decision makers. Ultimately, the winning team had the best validation results and did a good job presenting their findings. They also took the extra step of trying to forecast loan loss amounts.”
Graduate student Mingming Zhou was a member of the winning team, Eagle.
“For the vehicle loan and non-vehicle loan, gradient boosting and logistic regression modeling were selected as the champion models respectively,” said Zhou. “The champion models can be applied to new data points (new loan applications), then the risk of the loan charge-off will be predicted through the corresponding model. If the loan has a high probability to charge off, then the credit union would deny this loan in case of the loss in future.”
Dr. Shunpu Zhang, chair of UCF’s Statistics Department, explained that each team worked closely with a faculty advisor. The students learned the difference between “textbook” data and “real data,” added Zhang.
“They practiced their skills to clean the data with missing variables, to select relevant variables and to build and test different models,” said Zhang. “The students applied statistical and analytical skills they have learned in the classroom to the complex data provided by the CFE team.”
Differentiating data
The $1.74 billion Lake Mary, Fla.-based CFE Federal Credit Union supports approximately 500 employees and 155,000 members. Mizrahi explained the credit union has approximately 95,000 existing loans.
“This competition focused on auto and personal loans only, which is about 45,000 loans within CFE’s portfolio,” he said.
Over the course of competition Mizrahi explained that the credit union provided “a wealth of anonymized information” to UCF students to help them determine “which factors were important and which were not.”
The challenge for students, he said, was determining to what degree a specific piece of information might impact lending risks. The information was organized into several categories:
• Basic loan information (loan type, dates, amount, etc.)
• Borrower and co-borrower information (credit information, occupancy, employment, etc.)
• Account information (membership information, other credit union services used, etc.)
• Vehicle information (if applicable, make, model, year, value, etc.)
“We provided the students with 87 data points for approximately 20,000 loans – about 1.4 million data points in total,” said Mizrahi. “CFE withheld an additional 20,000 records that very closely mirrored the provided dataset so that CFE would have a validation dataset to run the students’ models against.”
While graduate student Michael Koller’s four-person team did not have a name, he said they tried to combine “some limited domain knowledge of the industry” with what they have been learning during via UCF’s graduate program.
“We were given some opportunity for trial and error such that we could see what was and wasn’t effective in our modeling techniques,” said Koller. “We noted a strong correlation between an applicant’s history with CFE and their probability of failure. If someone has savings products or credit cards with CFE, for example, they are much more likely to repay.”
Mizrahi said CFE Federal Credit Union is currently working with UCF to convert the winning team’s model to be part of its loan decision engine.
“CFE is committed to growing our loan portfolio, and we are always interested in finding ways to drive down charge-offs,” he said. “In 2018, charge-offs are trending at 0.89 percent, which is down from 1.1 percent in 2017. We are hopeful that leveraging our findings from this competition will bring this number down even more.”
