It was a major public relations coup, except it wasn't.
The Credit Union National Association is now saying that membership for credit unions grew by only a third of what it originally projected for the weeks before Bank Transfer Day.
In November, the trade association said that based on a survey, 650,000 people joined credit unions from Sept. 29 to Nov. 2. However, after the group collected its regular monthly data for October that covers balance sheet figures and new membership, it is now saying that only 214,000 people joined in October.
The association's scaling back of its numbers on Monday gave credence to banking industry officials and others who criticized the association's methodology last month.
"Credit unions are very optimistic about this whole process of Bank Transfer Day," says Alan Theriault, the president of CU Financial Services, a consulting firm for credit unions. "I think the enthusiasm of that concept and the recognition they were getting in all of the national media just created an exuberance that resulted in numbers looking a little better than they were."
Bill Hampel, the chief economist at CUNA, said in an interview that the group's initial data was flawed and rushed.
"We have been routinely collecting data for 30 years to make population projections," he says. "Because there was so much hubbub, we went out with a quick survey to pick up anything between those other surveys. When we came back and got more routine data, we tried to reconcile it and find out why it would be different."
The overestimation stemmed from several problems, Hampel says. The first question in the survey asked respondents, "since Sept. 29, has your credit union seen growth in membership and accounts?" The second question then asked the person to "estimate the growth of members." Even though the second question specifically asked for new members only, the wording of the first question may have caused respondents to combine the counts for new members and new accounts (whether opened by the new members or existing ones) in their estimates, Hampel says.
The data also did not specify an end date for respondents to use in their data and instead sought information from Sept. 29 to when the survey was filled out. This may have caused respondents to answer the question with different cutoff dates in mind.
Ambiguities such as these can wreak havoc on the accuracy of survey data, said Lee Kidder, a senior consultant at CCG Catalyst, a bank consulting firm.
"One of the favorite sayings of my old debate coach was 'define your terms, sir,'" Kidder says. "Survey questions that leave open any possibility of a lack of consensus of what it is trying to say can get you skewed results. The more you can drive out possible confusion or differences in definitions or scope, the better."
Finally, CUNA said in a fact sheet on Monday that the "credit unions with the strongest improvement in member growth appear to have been more likely to respond to the survey."
Mike Schenk, a vice president of economics and statistics at CUNA, had previously defended the accuracy of the data to American Banker and said that selection bias is a potential for any survey.
Hampel says that it was possible that credit unions saw 400,000 new checking accounts opened by existing members. This would help reconcile the data that CUNA originally reported with its new monthly figures, he says.
CUNA also initially reported that credit unions added about $4.5 billion in savings. However, its latest numbers show that savings decreased by $400 million in October. This September was an unusually strong month with 1.3% growth in savings from August. He attributed the strong performance to the fact that September ended on a Friday, which is a payday for some companies. October's numbers were hurt because it ended on a Monday, he says.
CUNA will not be revising the estimate that 40,000 people joined credit unions on Nov. 5, Bank Transfer Day, he says. There was no way for the group to verify data for just one day in the monthly figures it collects, he says. Hampel feels comfortable with that number as a "ballpark figure" and it supports anecdotal evidence the group had heard, he says.
Hampel noted that this was the first time CUNA has needed to dramatically revise some of its data.
"We were not precise enough in drafting the questionnaire," Hampel says. "When we got our regular numbers, they looked different. We said 'woops we have to go out and fix that.'"
