Unbalanced Information Diet: AI-generated deceptionAI-generated deception / Hatred of Politicians Spurred Japanese Man to Create AI Generated Realistic Fake Videos

The Yomiuri Shimbun

12:21 JST, January 25, 2024

This is the second installment in a series examining moves to spread biased information through the use of generative artificial intelligence, acts that threaten our democracy.

***

“Hold him down! Hold him down!”

Trouble suddenly erupted at a memorial event held July 8, 2023, to mark one year since former Prime Minister Shinzo Abe was fatally shot while giving a campaign speech in Nara. The event was held near the site where Abe was shot in front of Kintetsu Railway Co.’s Yamato-Saidaiji Station.

The memorial event had started solemnly, but then a man in his 20s waved around an object that looked like a gun while yelling, “Japan Teachers’ Union, Japan Teachers’ Union,” apparently referring to similar comments Abe once made when heckling an opposition party member during a Diet session. Liberal Democratic Party officials who were on security duty wrestled the man to the ground, and the Nara prefectural police arrested him at the scene on suspicion of violating the Petty Offense Law by disrupting the ceremony.

A 25-year-old man from Hyogo Prefecture was nearby. He was holding a handmade uchiwa fan bearing a cartoon portrait of Abe’s face. The man began jostling with a police officer and was requested to come voluntarily for questioning, but he quickly broke free.

This fracas had its origins in messages this man had posted on social media. Since about one year earlier, the man had been uploading fake but realistic videos of Abe he had made using generative artificial intelligence. Using AI, he very easily created a fake voice that sounded like Abe’s after letting the AI study material including online videos of Abe delivering speeches.

These fake videos all mocked Abe’s past policies and his relationship with the Unification Church, formally known as the Family Federation for World Peace and Unification. The man insisted these videos were parodies, but he also said he wanted to arouse anger toward Abe and “shape public opinion.” The man’s YouTube channel has about 2,800 subscribers and his account on X, formerly Twitter, has about 6,300 followers. Most of these people share the man’s scathing opinion of Abe.

At some point, a suggestion emerged among his X followers to hold an “offline meeting” at Yamato-Saidaiji on July 8. This real-world meeting between people who became acquainted through social media was intended to disrupt Abe’s memorial service. Four people, including the man from Hyogo Prefecture, assembled at the site and caused the commotion that resulted in an arrest.

Other cases of people interacting on social media with others who share their views, becoming radicalized and then exercising force as a group were seen in attacks on Brazilian government buildings in January 2023 and the assault on the U.S. Capitol in January 2021. This menace to democracy is now becoming closer to reality in Japan, too. Generative AI is becoming a weapon that increases this threat.

Stirring up hatred

The Hyogo Prefecture man’s motivation for regularly posting fake videos of Abe and other high-profile people stemmed from his desire to influence how people feel about them.

The man’s family often barely had enough food to eat when he was a child, and he became disillusioned with the LDP-led government while he was in high school. He developed an intense hatred for Abe because of policies he felt treated wealthy people favorably and squeezed ordinary citizens.

Fake videos generated by AI were the ideal way for the man to ridicule politicians. About one year earlier, a 26-year-old man from Osaka Prefecture who had created fake videos of Prime Minister Fumio Kishida taught the Hyogo man how to make such recordings. This opened the door to the Hyogo man making and posting fake videos of Abe and others.

These videos, which feature a voice identical to Abe’s, include crude words and comments that suggest he has connections to a specific religious organization. “The internet has mountains of audio clips of politicians’ voices,” the man said. “A single 20- to 30-minute video of a speech is enough to create a decent-quality fake voice.”

Many viewers of his fake videos that also featured Kishida and former Prime Minister Yoshihide Suga commented that his posts were “interesting.” The man believed this was proof that people hated these politicians, so he even created a smartphone app that allows users to create fake audio clips imitating the voices of Abe and others.

“Japan mustn’t produce a politician like Mr. Abe ever again,” the man said. “This is my own form of educational activity.”

Kazutoshi Sasahara, an associate professor at the Tokyo Institute of Technology and an expert in computational social science, said: “The more a video stirs up emotions and thrills its viewers, the easier it is to create an echo chamber in which polarized views become amplified. The men who disrupted Mr. Abe’s memorial service clearly showed that such a risk exists.”

Sasahara added: “The frightening thing about AI-generated fake videos is that they can easily demean the reputation of a specific person. If videos made intentionally to whip up hatred spread widely before an election, they could skew voting behavior.”

Used for criminal purposes

There also is a very real risk that fake videos could be used to commit crimes.

Chikae Ide, a 75-year-old manga artist living in Kyoto, fell victim to such a crime. “It’s impossible to tell if such a video was fake, or to even suspect it might be,” Ide said.

In 2018, Ide received a message on Facebook from a person purporting to be a Hollywood actor she admired. They sent messages to each other, and in July that year Ide received a video call from that person. The face that appeared on the screen was of a man who looked identical to the actor. On the screen of the tablet the man was holding was a digital comic Ide had drawn.

“I’ve read your work,” the man said.

Ide was convinced she was talking to that actor. “It’s really his voice. It’s actually him,” Ide recalled thinking.

The man later started asking for money from Ide. She repeatedly sent money to a designated bank account, and in total ended up giving him tens of millions of yen. Ide noticed she was being scammed only after her daughter suggested the video calls might be fake, but by then it was too late.

“It’s truly terrifying that AI technologies might be exploited for bad purposes,” Ide said.

Fighting AI with AI

Moves to use AI technologies to combat AI deep-fake videos are already emerging.

An image posted on Instagram showed a large group of people waving to a line of military personnel from the balconies of a building from which many Israeli flags were hanging.

The Yomiuri Shimbun asked Nablas Inc., a Tokyo-based venture company with origins at the University of Tokyo, to examine the image and determine its authenticity. Nablas concluded the image was a fake made with AI. It is possible this image was used to manipulate public opinion during the current Israel-Hamas conflict.

Nablas is researching AI technologies that can determine whether an image is fake. The company makes this system learn a huge volume of AI images, and is developing methods to detect the distinctive characteristics of these images. The system can spot traces of the minute flaws that remain when these images are generated.

“AI technologies that can slip past detection systems are being created one after another, so it’s a game of cat and mouse,” said Kunio Suzuki, a director at Nablas. “We’ll keep improving our technologies so that we don’t miss any harmful fake images.”

Japan lacks legal framework

Many nations around the world are building legal frameworks designed to control fake videos.

The U.S. state of Texas has banned posting fake videos of election candidates. Last year, Britain strengthened a crackdown on fake pornographic content under its online safety law. In the same year, the European Parliament adopted a draft AI law that addresses the risks posed by fake videos.

Japan, in stark contrast, has no law directly targeting fake images and videos. Moves to establish concrete rules have made little progress.

Chuo University Prof. Taro Komukai, an expert on information law, said, “The government needs to get serious about discussing countermeasures, such as by calling on platform operators to consider and report methods to combat harmful information.”

Japan’s measures are not keeping up with the advances in generative AI. The man from Hyogo Prefecture who detested Abe intends to keep posting his fake videos. “I’ll increase the number of people who are anti-Abe,” he said.