When AI is behind cyberattacks, the defense forces must keep up

Nobel Prize winner Geoffrey Hinton, known as the “godfather of AI,” has compared the development of AI and the looming threat of artificial superintelligence to having a “super-cute tiger cub” as a pet. It feels okay for now, as long as we can guarantee that the tiger won’t want to kill us when it grows up.

AI models can write and customize malware on a large scale

After a number of incidents in 2026, the “Godfather’s” analogy from 2024 feels even more relevant. News outlets have reported on AI agents that have taken the initiative on their own, found ways out of their isolated training environments, and hacked into external platforms. 

Cyberattacks and cyber threats have traditionally involved a person creating malicious code—such as ransomware—depending on the attacker’s objective. The attacker runs some kind of script to find vulnerabilities in digital systems and network infrastructure. Until recently, the process has been relatively manual in the sense that the attacker needs to be present, run the scripts, and coordinate the attack. 

“We are now in the midst of a paradigm shift, where we have AI models that can write and adapt malicious code at high speed and on a very large scale. The human attacker is still part of the process, albeit more in the background, but we will see more cyber threats in the future where AI acts entirely on its own,” says Apostolos Pyrgelis, a senior researcher at RISE specializing in digital privacy and privacy-preserving technologies.

That's why things are changing so fast—and here's how we can keep up

Development is proceeding at a breakneck pace, driven by a massive economic race and a sharp increase in both computing power and the volume of data used for model training. In addition, AI is helping developers build AI—powerful tools can help people build the next generation of tools more quickly.

The models used for cyberattacks are already very capable today and can test many different things at the same time. Even if the AI were to make a wrong decision, it would continue to try until it finds exploitable vulnerabilities. 

What does this mean for organizations that want to implement AI solutions in their operations? Is it possible to build cyber-resilient systems that can withstand AI-driven threats and continue to function? 

– A digital system does not become resilient simply by maintaining or protecting its security as much as possible. Resilience is a broader concept. It is based on the assumption that, sooner or later, a system will be subjected to an attack and that the defenses that have been put in place will be breached. The goal is for the system to survive the attack, recover, and ultimately remain functional,” says Apostolos Pyrgelis.

A defensive AI that keeps pace with the attacking AI

The natural way forward, he says, is to first and foremost have preventive measures in place. And the best way to build a resilient system is to integrate AI on the defense side as well.

“In a way, this allows the defensive AI to keep pace with the attacking AI. Of course, it’s not easy. We have to ensure that the AI we use for defense is robust and functions correctly, and humans must oversee the process. We also have to ensure that the defense cannot be manipulated, since the attacker will likely try to exploit the fact that a system has an AI-based defense,” he says. 

But how can you be sure that the defenses you’ve built up are up to par? Penetration testing in a secure environment is an effective way to evaluate this. Under strictly controlled conditions, the systems are attacked to gain a better understanding of how cybersecurity measures can be designed. 

Reactive processes—that is, how an organization responds when a crisis arises—are also important.

“When you implement a new digital system in your organization—whether it’s an AI-based solution or not—you need to be able to track and monitor the system. Rapid detection and a quick response to threats are key elements that I think all organizations should start discussing,” says Apostolos Pyrgelis.

Addresses highly topical issues through research

Researchers at RISE are actively working on issues such as how to make AI models more robust and more resilient to attacks, and how to ensure that AI is not manipulated into doing things it was not originally intended to do. RISE is also one of the main partners in RESIST, a research center for resilient AI systems. 

“Here, we’re exploring, among other things, how we can verify that AI systems are functioning as intended, how we can ensure that AI-generated software or code doesn’t introduce new vulnerabilities or attack surfaces, and how we can detect attacks more quickly so we can adapt our defenses and recovery strategies,” says Apostolos Pyrgelis. “The plan moving forward is to work with partners to demonstrate how our solutions work in real-world applications,” says Apostolos Pyrgelis.

So, there’s a lot of activity in the research community—but the level of knowledge in the private and public sectors varies. A report from IVA, the Royal Swedish Academy of Engineering Sciences, points out that many organizations lack adequate procedures to reduce unnecessary exposure to cyber threats and to detect incidents in a timely manner. The report, much like Apostolos Pyrgelis, emphasizes the importance of responsibly integrating AI into cybersecurity.

“Now is the time to prepare at the organizational level—in administration and management—as well as on the technical level. We must keep up with developments, even if they may feel like a runaway train. Understand the potential threats that could target your own organization and try to make decisions that minimize the risks,” says Apostolos Pyrgelis, continuing: 

– For example, if an organization doesn’t need AI for a specific task, it shouldn’t implement the technology in its systems just because it’s trendy. As I said, implementing AI can create a new point of vulnerability. 

A strategic decision can therefore begin with the question, “Do we need AI here?” If the answer is yes, then the next question is: “How do we implement the technology in a way that allows us to put safeguards in place so that the AI does not behave incorrectly or be exploited by malicious actors?” Those who develop a basic understanding of the risks are in a much better position to recover from an AI-driven threat than organizations that have put off addressing the problem.