"GPT Hacking Incident is a 'Wright Brothers' Moment... AI Could Be Exploited in Jeonse Fraud"

▲ Ram Shankar Siva Kumar, Head of Microsoft AI Red Team

Ram Shankar Siva Kumar, who leads Microsoft's "Red Team" conducting simulated attacks to strengthen AI security, recently compared the incident where a GPT model hacked Hugging Face to the moment the Wright brothers first achieved powered flight in Kitty Hawk, North Carolina, in 1903.

Just as the rudimentary airplane that could fly for less than 30 seconds eventually became popularized thanks to continuous technological advancement, this is a warning prediction that AI threats in the security domain could also become much larger due to subsequent technology.

Meeting with reporters at a Microsoft security event held in San Francisco on the 27th (local time), Kumar emphasized, "What we witnessed is a snapshot of what can happen when AI systems combine with cutting-edge risks," adding, "It is important to consider AI safety before releasing systems."

In particular, he stressed that such AI security threats are not limited to cybersecurity alone but also apply to everyday security domains, and that in such cases, AI safety systems tailored to the context of each region and culture must be in place to block threats.

He said, "When people think of 'jailbreaks' (methods bypassing AI safety measures), they always talk only about 'cyber jailbreaks,'" adding, "In fact, jailbreaks can be used to cause other harms."

For instance, he pointed out that AI could be used in crimes such as committing fraud using jeonse, South Korea's unique lease system, or turning off the mandatory smartphone camera shutter sound in countries like South Korea.

Because of this, he argued that when developing AI models, one must not think only in terms of a specific country or a specific language, but rather align with the contexts of all countries.

Kumar explained that to understand such diverse contexts, Microsoft is funding universities around the world to train AI to address this.

In fact, Microsoft announced an initiative on this day called the "External Red Teaming Alliance" (EXTRA), which supports research labs at 18 universities worldwide, including the Korea Advanced Institute of Science and Technology (KAIST).

Kumar is known to have pioneered research on adversarial machine learning (ML) and AI adversarial attacks and defenses since joining Microsoft in 2013.

(Photo: Yonhap News)

※