Henrik Abrahamsson
Senior Researcher
Contact Henrik
How can cars use cameras and AI to detect obstacles and other road users? How can AI models be trained collaboratively across cars without transferring images to a data centre? And how can knowledge be transferred between car generations? These are questions that RISE and Zenseact explore in a recently published research paper.
Cars can detect other road users and obstacles in their surroundings using sensors and AI. The AI models are trained on images of different traffic situations and can learn to identify objects (such as cars, cyclists and pedestrians), as well as pinpoint where in the images these objects are located.
In simple terms, an object detector can be divided into two parts. The first part, known as the backbone, consists of a deep neural network that extracts and learns visual features from images. It builds an internal representation of shapes, structures and more complex visual patterns. The second part, the detection head, uses these features to classify and locate objects.
Training a good AI model for object detection usually requires a very large collection of images covering a variety of driving conditions and geographical locations. Collecting all these images can be problematic for several reasons: the cost of transferring and storing the images, as well as privacy concerns about your car recording its surroundings, where you are and how you drive, and sending the images to a data centre.
RISE and Zenseact are investigating whether federated learning offers a possible solution. Federated learning is a form of distributed AI that allows, for example, cars to train an AI model collaboratively without sharing raw training data.
Bringing together industrial experience and research expertise allows us to explore how vehicles can learn collectively and share knowledge without sharing raw data.
At the end of October, we presented some of our work at the IEEE International Conference on Federated Learning Technologies and Applications (FLTA 2026), which was held in Paris this year.
More than a hundred experts in distributed AI and federated learning gathered for three days to present the latest research findings, network and exchange experiences. Participants included researchers and students, as well as leading representatives from industry and organisations that develop software and frameworks for federated learning.
Want to learn more about the results we presented in Paris? Our paper, Knowledge Distillation for Heterogeneous Federated Object Detection in Autonomous Driving is available in Digitala Vetenskapliga Arkivet (DiVA). We have also made the source code publicly available on GitLab, making it possible for others to verify, reproduce, and build on our results. The paper’s lead author is Albert Bargalló i Sales, and the work builds on the Master’s thesis he completed at RISE earlier this year. The co-authors are Mina Alibeigi, AI researcher at Zenseact, and RISE researchers Sima Sinaei and Henrik Abrahamsson.
The research paper focuses on how knowledge can be transferred from one generation of cars to another, avoiding the need to start collecting data and training the AI model from scratch every time a new car model is launched. If an existing vehicle fleet has encountered and learned from a wide range of traffic situations, both common and unusual, how can that knowledge be passed on to a new generation of cars? If we have a well-performing distributed AI model that can locate and classify objects in its surroundings, how do we transfer that knowledge to the next generation of vehicles, which may use a different AI model and different hardware?
Our approach is to use knowledge distillation, in which an existing, trained AI model acts as a teacher and transfers its knowledge to a new student model with a different architecture. The models do not need to be directly compatible. For object detection, the student model receives information about how the teacher interprets images and which objects it identifies in them, and the student learns from this. We use knowledge distillation (1 in the figure above) to give federated learning (2,3,4) a better starting point.
To train and evaluate AI models for object detection in our research, we use images from the Zenseact Open Dataset (ZOD). Developed by Zenseact, it is a large, open, multimodal dataset for research and development in autonomous driving. The dataset was collected using test vehicles on European roads over a two-year period and contains images covering a wide range of traffic, road, and weather conditions. ZOD has become a widely used benchmark dataset in autonomous driving research and serves as the foundation for several of our collaborative research projects.
Our research aims to make autonomous AI systems more adaptive, efficient, and trustworthy through distributed learning. Working closely with industry allows us to develop AI methods that are not only innovative but also relevant to real-world autonomous driving.
The initial results with knowledge distillation are promising and encourage us to take the work further. The next step is to study more complex scenarios in which multiple generations of vehicles coexist and can continually learn from and exchange knowledge with one another. We also want to investigate how self-supervised learning can be used to benefit from the large volumes of unannotated images collected by vehicles. Meanwhile, Zenseact is developing a new multimodal dataset that will be important for our continued research. So far, our experiments have been conducted through simulations in data centres. An interesting next step is therefore to also test the methods on the type of hardware found in today’s cars.
The research is carried out within the DREAM and DART projects, funded by Vinnova through the Advanced Digitalisation programme. In DREAM (Distributed, Robust and Efficient AI for Autonomous Vehicles), RISE, Zenseact, Scaleout and AI Sweden collaborate on distributed AI for autonomous vehicles, focusing on self-supervised learning, knowledge distillation and communication efficiency. DART (Distributed AI for Robust and Trustworthy Autonomous Vehicles) is a Swedish–American research collaboration in which RISE and Zenseact work with the NVIDIA FLARE team in the United States. The project investigates how federated learning can be used for robust and trustworthy autonomous systems.
To learn more about the projects and the research, please visit the project websites. If you have any questions, contact researchers Sima Sinaei (sima.sinaei@ri.se) and Henrik Abrahamsson (henrik.abrahamsson@ri.se).
Presentation at the conference FLTA 2026 (pdf, 6.39 MB)