1. AI Begins to Attack on Its Own
On the 6th of this month, Russia launched a drone attack on Zaporizhzhia in southeastern Ukraine. It struck a large propane storage facility at a gas station, causing a massive explosion that tragically killed three civilians. Russia sometimes launches hundreds of drones a day, so it initially seemed like just another conventional drone. However, upon retrieving and analyzing the drone, experts found it was vastly different from previous models. First, the Russian drone lacked a communication antenna. Drones are supposed to have an antenna to receive instructions and commands from humans. Yet, this drone had none at all. It was not that it had fallen off; it appeared to have been manufactured without a communication antenna from the very beginning. Instead, what was found inside was an ultra-small computer from NVIDIA. It was a Jetson Orin. Upon looking into what this Jetson Orin is, it is a small computer made by mounting an NVIDIA GPU on a board slightly smaller than the palm of a hand, allowing it to operate independently without connecting to a cloud server. NVIDIA stated that it has never sold this to Russia, taking the stance that it was likely resold or resold a second time. However, because Russia left the computer unencrypted, inspectors were able to look inside. Upon doing so, they found regional images of Ukraine as well as embedded code. That code contained not only Ukrainian terrain data but also pre-trained data designed to track and strike specific objects. In other words, if Ukraine's claims are correct, Russia had trained the drone to fly autonomously to pre-input coordinates without human control, allowing the AI to select targets on its own from among the learned options and attack them. Since energy storage facilities were learned as attack targets this time, the drone apparently flew around the sky, picked one out, and struck it on its own. It is truly the arrival of an era where AI attacks on its own.
To be sure, AI has been used in operating drones up to now. However, it has played an auxiliary role in finding targets and executing operations, rather than independently setting targets and attacking in actual combat. In particular, there has never been a case before where lethal AI attacks resulted in casualties like this. Until now, humans selected the final attack target and issued the final attack command. Yet, an era has somehow arrived where machines make autonomous judgments and attack humans without human intervention.

2. Ukraine Is No Less Advanced in Drone Technology
Ukraine is by no means lagging behind Russia in drone technology. It is no exaggeration to say that they have overcome an absolute inferiority in military power by fighting a war for four years with drones. At Ukraine's Independence Day military parade held last week, various drones such as unmanned ground vehicles, unmanned aerial vehicles, and uncrewed surface vessels were showcased instead of soldiers and conventional weapons. Back in 2024, the world's first dedicated drone branch of the military was even established in Ukraine. If Ukraine sets its mind to it, they can manufacture lethal AI drones without human intervention as much as they want. If a situation arises where Ukraine and Russia engage in drone warfare against each other, a war that currently lingers at the remote-controlled drone combat stage could accelerate far past the autonomous engagement system stage into battles fought between autonomous drones controlled by AI. In other words, situations once seen only in futuristic science fiction—where robots fight on behalf of humans or even kill humans—could soon become a reality.

3. Is the DMZ No Longer Safe?
Recognizing that this needs to be reined in, the UN and the international community have been discussing regulations on it for a long time. There is an agreement known as the CCW, the Convention on Prohibitions or Restrictions on the Use of Certain Conventional Weapons. Restrictions on lethal AI are being discussed there, as it is a treaty aimed at limiting the use of conventional weapons that cause indiscriminate civilian casualties. Although lethal AI systems are not conventional weapons, the possibility of civilian casualties is extremely high, so an attempt was made to restrict the use of lethal AI weapons through this treaty. However, multiple countries did not agree. Russia's opposition in particular was fierce. Even after that, international discussions have continued to ban the use of lethal AI weapons targeting humans. Yet, they have repeatedly run up against Russia's opposition. In 2024 as well, a similar agenda was brought before the UN General Assembly. It proposed creating a platform to discuss killer robot proposals; 161 countries voted in favor, but Russia opposed it once again. North Korea also cast a dissenting vote. North Korea voted against it, several countries abstained, and major military powers such as the United States, China, India, and Israel all abstained. South Korea also maintained a reserved stance. Facing North Korea, South Korea is in a position where guard duties in the DMZ must gradually be handed over from humans to robots, making it unable to blindly agree to a ban on the development of autonomous lethal weapons. The UN Secretary-General has long strongly argued that regulations on the development of autonomous lethal weapons are necessary. Furthermore, he has warned that very little time is left to prepare preventive measures. Understandably so, given that people have already been killed by lethal AI drones in Ukraine. This coming November, the CCW conference will be held again in Geneva, Switzerland. There are plans to discuss how to regulate autonomous lethal weapons and whether to draft a treaty. The problem, however, is that this body operates on a consensus system, making it somewhat difficult to produce a binding agreement under circumstances where Russia openly opposes it. Moreover, even if an agreement is painstakingly reached, another issue is that existing drones can easily be converted into autonomous lethal weapons at any time through a simple software update. Ultimately, this current toward a war fought by AI robots replacing humans—much like how nuclear weapons could not be stopped—appears difficult to avoid. Nevertheless, fierce debates are expected to erupt every time victims of lethal AI weapons occur, centered on whether or not to entrust the final choice of pulling the trigger to machines.

(Photo: Telegram '@mod_russia')

Produced by SBS Digital News