Generative AI Shock Wave / ChatGPT Can Be Tricked into Generating Malware, Bomb-making Instructions
The Yomiuri Shimbun
17:36 JST, April 25, 2023
The use of generative artificial intelligence tools like ChatGPT is spreading rapidly. While forecasts indicate the technology could revolutionize society, a number of problems linked to such tools have emerged. This is the first installment in a series examining issues with generative AI ahead of the Group of Seven summit in May, where international guidelines on the technology will be discussed.
***
In online forums where criminals exchange information, discussions about “ChatGPT loopholes” and “Ways to evade censorship” are common.
Use of ChatGPT for criminal purposes is prohibited under the service’s terms of use. The system is designed to not provide information to queries linked to crime. However, ways to trick the system can be found online.
Exploiting prompts that get ChatGPT to reveal information it has been designed to restrict — such as material that can be used for criminal purposes — is referred to as “jailbreaking.”
By using certain prompts, it is possible to get ChatGPT to generate computer viruses, write text for phishing emails that can be used to steal personal information, and reveal how to make explosives.
Takashi Yoshikawa, a senior malware analyst at the Tokyo-based security firm Mitsui Bussan Secure Directions, has been investigating the risks associated with generative AI technology.
By entering jailbreak prompts found online, Yoshikawa got ChatGPT to generate the source code of ransomware, a type of malicious software designed to block access to a computer system until a sum of money is paid.
When the code was run on a computer, the software encrypted the data on the machine, rendering it unusable.
Yoshikawa, 38, said: “Some beginners with limited knowledge of viruses use forums [where such information is shared]. It is a dangerous situation.”
Over 100 million users
ChatGPT responds to questions and instructions using natural language.
Developed by U.S. startup OpenAI, the service was released to the public in November, and within two months had more than 100 million users.
However, the technology is still developing. On April 11, OpenAI announced rewards of up to $20,000 for people who discover security flaws in the service, but the company is excluding jailbreaking techniques, as “addressing these issues often involves substantial research and a broader approach.”
Regarding jailbreaking, the European Union Agency for Law Enforcement Cooperation released a report in March stating, “It is of the utmost importance that awareness is raised on this matter, to ensure that any potential loopholes are discovered and closed as quickly as possible.”
Warning that ChatGPT could be used for terrorism and other criminal activities, the report also stated, “It will become easier than ever for malicious actors to perpetrate criminal activities with no necessary prior knowledge.”
Leaks of corporate secrets?
Generative AI may affect Japanese culture and entertainment.
AI image generators are expected to be used more often in the production of manga and anime, to make characters and backgrounds, leading to concerns that the copyrights of original data used for AI learning could be violated.
When it comes to expression that uses short phrases, such as haiku, tanka and other poems, quality work might get lost amid the huge volume created by AI.
There are also concerns about the leakage of confidential information, because some settings might allow information input by users to be used as data for learning.
“Your internet access is blocked.” This message appears on the screen when someone tries to browse on a work computer at the Mizuho Financial Group, Inc.
The group has been considering using ChatGPT, but at the end of 2022 it added ChatGPT to its list of websites to which access is restricted, to prevent the leakage of confidential information. “We took extra precautions,” a security official at the group said.
Honda Motor Co. and Hitachi Ltd. cautioned employees about the issue earlier this month, while SoftBank Corp. and other companies positive about utilizing ChatGPT for business operations are creating rules related to its use.
“Many people would give instructions like, ‘Summarize this internal document.’ Some restrictions are necessary,” said Ichiro Sato, a professor specializing in information public policy at the National Institute of Informatics.
An engineer working for a company in Tokyo received a notice from his company telling him not to use ChatGPT on work computers.
“ChatGPT is still immature as a service and we’re not absolutely sure about its transparency and safety,” the engineer said, expressing his understanding of the company’s policy.
The world is taking a defensive stance toward the new technology, which appeared just half a year ago.
Generative AI Triggers Concerns — Even Among Developers of the Technology
https://japannews.yomiuri.co.jp/society/general-news/20230426-105719/Popular Articles
-
Risk of Nuclear Weapons Being Used Greater Than Ever; Support Gro...
-
CPTPP Will Let Britain Offer Further Benefits to Japan, Says U.K....
-
Girl Stabbed to Death by ‘Stranger’ at McDonald’s in Southern Jap...
-
Tokusatsu Fans from Brazil Enjoy Touring Locations Across Japan U...
-
Olympic Champ Kitaguchi Wins Top Prize at Japan Sports Awards
-
Widow of Late Japan PM Abe Meets with Trump
-
Japan, Italy, U.K. Launch Body To Manage Next-Generation Jet Proj...
-
Select Asian Students to Study Medicine in Japan for Free, with N...
"Society" POPULAR ARTICLE
-
Miho Nakayama, Japanese Actress and Singer, Found Dead at Her Tokyo Residence; She was 54 (UPDATE 1)
-
Risk of Nuclear Weapons Being Used Greater Than Ever; Support Growing in Russia As Ukraine War Continues
-
Overtourism Grows as Snow Cap Appears on Mt. Fuji; Local Municipalities Hard Pressed to Establish Countermeasures
-
Companies Expanding Use of Recycled Plastic; Technological Developments Improve Production Process, Allow Incorporation in Cars, Electronics
-
Japan Star Miho Nakayama’s Death Unlikely Caused by Foul Play; Tokyo Police Make Conclusion After Autopsy (UPDATE 1)
JN ACCESS RANKING
- Mitsuoka Motor Launches Limited Run of M55 Concept Car; 100 Retro Inspired Vehicles to be Sold for ¥8.08 Million Each
- Japan’s Factory Activity Extends Declines on Sluggish Demand, PMI Shows
- Japan’s Core Inflation Steady Above BOJ Target, Service Prices Firm
- China to Test Mine for Rare Metals Off Japan Island; Japan Lagging in Technologies Needed for Extraction
- Miho Nakayama, Japanese Actress and Singer, Found Dead at Her Tokyo Residence; She was 54 (UPDATE 1)