The opportunities and pitfalls of AI agents

AI agents and agent systems are all the rage. But what exactly are they?

"An AI agent is an AI system that can act independently to achieve specific goals. While agents have been around for a long time, what is new are agent systems consisting of new types of AI. These can be language models that work together, orchestrated by a conductor. Sometimes the conductor is a human and sometimes it is an AI controlling the others,” says Stella Riad, head of the Industrial Systems department at RISE.

AI agents – often language models, but also other types of AI – act autonomously to achieve a goal. This may involve taking instructions, breaking the goal down into sub-goals, searching the web, analysing data, and compiling reports. AI agents can handle tasks requiring multiple steps and combine different types of tools to perform them. When several agents interact, it is called an agent system.

The strategy for AI agents is to start with the needs.

This technology is ideal for automating complex, repetitive tasks such as writing code or managing email inboxes. Agent systems have also been known to replace entire teams of business analysts. Unlike human analysts, agents can gather information around the clock. However, the technology is not the solution to everything.

"If it's a straightforward challenge, you don't need an agent – a simpler model can solve it. You need the right tool for the right problem", says Stella Riad.

Several hundred employees at RISE work with AI-related issues across the entire organisation. There is also a centre focusing on applied AI, as well as several test and demonstration environments and innovation hubs. Companies and organisations wishing to explore the potential of AI agents can receive support with anything from technology and system development to prototype design and strategic guidance.

"The starting point for all AI projects should be an examination of the organisation's needs. What problems do you want to solve? What could you achieve with this technology? It's also important to consider the role of humans in relation to AI in general and agents in particular. Defining clear roles and responsibilities is important for risk management and scalability," says Stella Riad.

The black box of the agent system

The concept of AI agents is based on their ability to work autonomously, without human intervention. However, this approach raises questions about security, explainability and transparency.

"Problems such as hallucinations, which occur when using language models, risk becoming many times greater in an agent system. It's not a question of adding uncertainties, but multiplying them," says Stella Riad, continuing:

"The question of explainability requires careful consideration. We can either allow the system to remain a black box and accept that we will not know all the details about how a decision was made, or we can open the black box and accept poorer performance when the process needs to be simplified to make it understandable to us. Or we can open the black box, which will result in poorer performance when the process needs to be simplified to make it understandable to humans."

Online information retrieval agents can be compromised by malicious actors or leak data along the way. This can be prevented to some extent by limiting the domains the agent is permitted to visit and by ensuring it does not carry sensitive or business-critical information.

The message from Stella Riad is clear:

"The potential of AI agents is immense, but don't adopt the technology without first devising a clear plan."