Unbalanced information diet: AI-generated deception / News May Proliferate, Despite Being Based on Fake Images

The Yomiuri Shimbun

21:00 JST, January 26, 2024

This is the third installment in a series examining moves to spread biased information through the use of generative artificial intelligence, acts that threaten our democracy.

***

“There has reportedly been an explosion near the Pentagon,” an Indian news announcer said.

It was 7:44 p.m. on May 22, 2023, and the TV screen — emblazoned with the chyron “Breaking news” — showed a plume of black smoke rising close to the Pentagon, the headquarters of the U.S. Department of Defense.

Madhav Nalapat, an international political scientist, appeared on the TV program via phone, beginning his commentary by saying that the United States was overflowing with explosives.

About an hour later, Republic, the TV station that broadcast the news, posted a message on its X account, saying it had “retracted the breaking news.” The story was based on a report distributed by RT, Russia’s state-run international broadcaster.

However, the image RT referred to as the information source was found to be fake and was likely created by artificial intelligence.

Nalapat made his comments on the news program shortly after RT reported the fake news. Upon seeing the image sent by a reporter, the tragedy of the September 2001 terrorist attacks on the United States crossed Nalapat’s mind.

The political scientist later expressed remorse, saying he should have properly checked the information before speaking out.

He also opined that the “breaking news” story had demonstrated how AI could be weaponized to trigger social panic.

In India, several news outlets, including Republic, ran stories based on the fake image. A person, 24, who works as a fact-checker in India, said that Indian media outlets are “so eager to issue breaking news that they tend to neglect fact-checks.”

Some news media report false information on social media without supporting evidence, adding “credibility” to such misinformation, and helping spread it further. Concerns are also being voiced over the power of a single fake post to potentially influence public opinion.

Aggregator amplifies fake news

In May, a post headlined “Stationing of NATO troops in Japan being mulled” was made to an online forum. The post was quickly shared by over 10 so-called “matome” sites that collect information and links from across the internet.

However, the headline did not match the rest of the post.

Readers were led to believe from the headline that the North Atlantic Treaty Organization, which is at odds with Russia, might station troops in Japan. However, the body of the post, which cites an article by a Japanese news agency, merely says, “NATO plans to establish a liaison office in Japan.”

“The headline seemed like it would irritate China,” said a man in his 40s in Tokyo, who viewed one of the aggregator websites. “I shared it because I thought it would be fun.”

Many commented on the post online, with some saying, “They’re getting ready for war” and “Do they want to turn Japan into a battlefield?”

Matome websites, which aggregate information from online forums and social media sites, are called “middle media” in Japanese, as they fill a space between social media and mass media. As in the “NATO troops” case, such websites often feed the spread of fake news.

But how many page views do such sites get?

Similarweb Ltd., a Tokyo-based private research company, analyzed websites that published the false “NATO troops” story at the request of The Yomiuri Shimbun and found that one such website got over 3.18 million hits in May alone.

The Yomiuri Shimbun also tried to find out who was running the websites, aided by a security expert called Retoro. Identifying those who had acquired the site domain names, equivalent to addresses on the internet, proved challenging, but after some three weeks of searching one matome site operator was identfied.

It was a man in his 20s living in the Kansai region. He runs two aggregator sites that get 800,000 views a month. He spends five to six hours a day gathering posts from online forums and reposting them on his sites. Asked about his criteria for selecting posts, he said, “I choose posts that I think are interesting.”

Through the sites, he earns ad reve-nue. Each month the ads bring in ¥60,000 to ¥70,000. Asked why he had shared the false headline about NATO troops, he said: “I thought it would be fine if l used the headline as it was. I’m sorry I caused trouble.” He then deleted the post.

Mass media can be culprit

Mass media outlets have also been known to spread misinformation.

In March 2022, the Chunichi Sports newspaper posted to the Yahoo News portal site an article headlined “U.S. conducting research in Ukraine like that done by Japan’s Unit 731, says Russian news agency.”

The story was based on reporting by Russia’s state-owned news agency Sputnik. The European Union bans the distribution of any content by the news agency, arguing that it spreads Russian propaganda.

Chunichi Sports cited the Sputnik story, which had gone viral on social media, without fact-checking it. Soon after, the paper deemed the article inappropriate and it was deleted.

The newspaper has been posting its articles to Yahoo News since 2019. “The website has become the main battlefield for sports papers,” a senior official at the newspaper said. If an article gets on the Yahoo Topics list of top news stories, it will garner more page views and earn more ad revenue. This leads media outlets to quickly post large numbers of articles.

“Competition to get articles onto Yahoo Topics is fierce, and this causes a lot of trouble,” the official said, explaining why the newspaper failed to fact-check the article.

After the incident, the paper again asked its staff writers to check several sources when handling unconfirmed news.

Social, middle and mass media can also create feedback loops, where information circulates between them. In a loop, misinformation is reposted or republished without sufficient checks, and news that has been exaggerated to attract attention is sometimes amplified.

At the lab of Hosei University Prof. Hiroyuki Fujishiro, his team works to educate young people on the issue. They hold workshops in which participants discuss how middle media outlets choose headlines to get more page views.

“We want people to know that sensational headlines disconnected from the facts are used to attract attention,” said a 21-year-old senior in the lab.

“Misinformation circulating on social and other media is treated as if it were true and it takes on a life of its own,” Fujishiro said. “A system must be established to prevent this.”