Latest outage for Mizuho another missed chance to prevent larger systemic failures

The Yomiuri Shimbun

17:01 JST, August 22, 2021

After a series of glitches took down ATMs and online banking services earlier this year, Mizuho Bank Ltd. assured customers that recurrence prevention measures introduced in June would rectify the issue. But even these failsafes failed to prevent yet another system outage on Aug. 20. To make matters worse for the beleaguered bank, the latest mishap occurred on the first Friday after the Bon holiday, which also fell on a “gotobi” — a date that is a multiple of five — when bank counters tend to be at their busiest with individual and corporate customers settling routine transactions. As the disruptions accrue, it would seem that Mizuho has still not fully taken stock of the lessons that should have been learned from past mishaps.

Backup systems

Speaking at a press conference Friday, Mizuho Financial Group Inc. Chief Executive Officer Tatsufumi Sakai said the latest system disruption as “very regrettable,” and renewed the bank’s pledge to prevent future recurrence of the issue. “We intend to reflect on these lessons going forward to eliminate the concerns of our customers as soon as possible,” he said.

But the public voicing of contrition rang hollow given the irony that the glitch occurred in the very equipment that was supposedly being overhauled to prevent such snafus in the first place.

After a glitch in March interrupted ATM service and overseas remittances, the Mizuho group had launched a comprehensive review of their back-end systems, with plans to complete the review by the end of the year.

Now, the effectiveness of this overhaul has been thrown into question.

As the product of a merger between the Industrial Bank of Japan, Fuji Bank and Dai-Ichi Kangyo Bank, the Mizuho Financial Group’s core banking systems for account and payment management were built as an amalgamation bearing the fingerprints of multiple system development firms.

At the press conference, Satoshi Ishii, a senior executive officer and head of the IT and Systems Group, cited this convolution as a complicating factor in the latest glitch. “The system failed with incredible complexity.”

Although the bank has two backup layers that were supposed to activate in the event of such outages, neither functioned in the latest incident. This raises other damning questions, given the inevitability of glitches as digitization progresses. In the words of a senior official at another megabank, “It’s common knowledge in finance that you want two or three layers of backup to support your systems.”

Ishii conceded that, in retrospect, more could have been done in restoring the system. “Insufficient cooperation with our [system] vendors was a factor,” he said.

Escalated by email

A point of scrutiny throughout the series of system outages has been the bank’s initial response to the problem at the time of discovery. The latest glitch once again highlighted how difficult it is to improve those first steps.

Although the glitch was first detected around 9 p.m. on Thursday, an hour would pass before notification was sent to Sakai and Mizuho Bank President Koji Fujiwara around 10 p.m. Moreover, as notice was only made by email, Fujiwara did not become aware of the issue until around 11 p.m.

Even after reading the memo, Sakai decided the issue was something that could be handled by Mizuho Bank as opposed to the group. He would not realize the gravity of the issue until the wee hours of the morning, when it became apparent that it would take time to bring systems back online, affecting over-the-counter services when the banks opened.

Mizuho management’s lack of a sense of urgency toward system failures has drawn criticism before. When Mizuho’s ATMs failed back in February, management did not even catch wind of the issue until the news articles began appearing online.

Last-minute notice

The bank’s response to customers was also not enough. Mizuho instructed bank employees to arrive at their branches by 7:30 a.m. However, notice of the outage was not posted to the bank’s website until just 30 minutes before they were to open to customers for the day.

Complex organization

The question remains: Why has the Mizuho Financial Group been at the center of so many system outages?

The Financial Services Agency is increasingly leaning toward the conclusion that the problem lies in the bank’s corporate governance structure.

At the group’s inception in 2000, the top posts were divvied up equally among representatives from the three big banks involved in the merger.

However, the group has since sought to dilute these former tripartite loyalties. Frequent personnel changes have been made to prevent the ​internal tug-of-war seen as behind the earlier large-scale service outages in 2002 and 2011, as well as an illicit money lending scandal to “antisocial” groups that came to light in 2013.

Currently, 6 of the group’s 13 board members are outside directors, a ratio meant to strengthen the board’s supervisory role. Furthermore, to ensure cohesion across the group’s three core subsidiaries — banking, trust banking and securities — executives have been installed to play a unifying role in departments that straddle all three, including information technology and systems.

But the messy task of triage after system failures has always been left up to Mizuho Bank and Mizuho Trust & Banking Co.

As one senior FSA official said, “The deployment of corporate officers atop departments may have resulted in a more convoluted organizational web that obfuscates where responsibilities lies.”