When AI gets a body: cyber-physical systems in practice

Imagine walking past Reachy and asking whether it is warm in the meeting room, or asking it to turn on the lights. It turns its head toward you, answers and does what you ask, without anyone having prepared that particular question.

What do we see in the demo?

Reachy Mini is a small, open-source robot from Pollen Robotics and Hugging Face that can be programmed in Python. In the demo, we see how it turns toward the person speaking and moves its head and antennas to show that it is listening or processing a question.

Behind the interaction is the Reachy Mini application, which works as an AI agent. A language model handles the conversation and can call on tools. Here, a real-time model from OpenAI is used to provide fast responses. The lights and sensors are connected via IKEA’s Dirigera hub and made available as tools through MCP, Model Context Protocol.

What becomes possible when AI gets a body?

Two things. First, the body changes the conversation. Turn-taking, gaze and small movements make the interaction more natural, and the robot can function as a shared interface for several people in the same room. At the same time, we may attribute more understanding to a robot that nods than it actually has. We need to take that tension seriously.

Secondly, the robot can be connected to physical systems through open interfaces. New systems can then be added without the robot needing to be rebuilt. This is where trust becomes decisive. An AI that can control physical things needs clear permissions. We have to decide what it may do on its own and when it should ask.

Where can the technology make the biggest difference?

Voice assistants have existed for more than ten years and are mainly used for timers and music. But when we want to control the environment around us, at home, at the office or in a building, an AI in the room can become a natural interface.

The home: from commands to conversations

Take the home as an example. Today, it is often controlled through one app per supplier, pre-programmed scenes and rules like “if it is seven o’clock, turn on the light.” A voice assistant can handle “turn on the light in the kitchen.” Requests like “it’s stuffy in here” or “we’re going to watch a movie” require more. The system needs to interpret what the person means, know what equipment is available and be able to combine it: lower the lights, pull down the blind or increase the ventilation.

That is something a language model with access to tools can do. A physical robot in the room can also contribute information about where the voice is coming from.

A shared interface for care and buildings

The same need exists in other environments. In an elderly home, a robot can make it easier for those who have difficulty using apps to control their surroundings. It could also offer companionship and notice if something seems to be wrong, but such functions require clear consent and clear limits on what the robot may perceive and react to.

In offices and buildings, there is ventilation, heating, lighting, access control and energy measuring from different suppliers. To answer the question “why is it cold in the conference room?” the AI may need to combine information from several of them.

Energy use you can talk about

The home is also increasingly becoming part of the energy system, where it should be possible to shift consumption when electricity is expensive. That can be easier to explain and agree on in a conversation than through a settings menu.

Open interfaces and clear limits

Open protocols like MCP make it possible to connect the systems. Each system is made available as a tool and can then be used by AI applications that support the protocol.

This is also where the question of trust becomes concrete. Reading a temperature or turning on a light is relatively uncomplicated. Unlocking a door, turning off an alarm or changing the heat in an entire building requires other considerations. A reasonable model is that the AI may read and explain, but asks for approval before doing anything that affects safety or more people than the one asking.

How does RISE take the technology from research to something we can test?

RISE is independent. We sell neither robots, lamps nor AI models, which is why we can test how systems from different suppliers work together without having our own interest in which supplier is chosen. When AI is to control homes and buildings, the equipment will come from many directions, and someone who can show what actually works together is needed.

We also have long experience with connected systems. Contiki, one of the first operating systems for Internet of Things, has its roots here. Today, we work with everything from sensor networks to 5G and edge computing. We understand the chain from the small sensor in the wall to the model that interprets what someone says. It is often in the connections between these parts that problems arise.

At RISE, there are also labs and testbeds where AI can meet physical equipment. There, we can investigate what happens when a robot turns on lights, reads sensors and reacts to people.

We can also contribute where the questions are most difficult. A robot in the home can hear and see a lot, and in this demo the conversation is sent to a cloud service. How much can be run locally, in the robot or on an edge server in the house, so that what is said at home stays at home? What ia an AI allowed do on its own, and how do we test that it stays within those limits?

The questions concerning safety, privacy and trust need to be answered before the technology is used at a larger scale. It is because of this kind of independent work RISE exists.

Curious to learn more?

Behind the demo is research on how AI can be run close to the user and control robots safely. Two projects linked to the work are:

• 6GHype4Ind: Hybrid positioning and edge computing, so that AI knows where things are and can be run close to the user.
• EdgeWise: Safe control of robots, so that AI stays within limits that can be verified.

Want to know more about the technology? Contact researcher Joakim Eriksson på joakim.eriksson@ri.se. 

Feel free to subscribe to the AI at RISE newsletter for more examples of how AI research is put into practice.