Jonas Hellgren: Reinforcement Learning – theory and applications

Reinforcement Learning (RL) is a paradigm of machine learning that focuses on training agents to make sequential decisions in an environment. This presentation, that is algorithm/math oriented, highlights the most essential concepts in RL. The concepts are needed to understand algorithm candidates for solving RL problems. The presentation also includes some toy-problems to increase the intuition of the concepts. Grid world and chart-pole are classical example toy problems. Real world applications are also presented.

The talk will cover the following concepts: Markov Decision Processes serve as the foundational framework for modeling RL problems. They define the key components: states, actions, transition probabilities, and rewards. Reward Functions: The reward function encapsulates the objective of an RL task, guiding the agent towards desirable outcomes. Value Functions: Value functions estimate the expected cumulative reward associated with a state or state-action pair, aiding in decision-making. Policy: The policy defines the agent’s strategy for selecting actions in a given state, shaping its behavior.

The following algorithms will be presented presented: Dynamic programming. Temporal Difference Learning: TD learning algorithms, such as Q-Learning and SARSA. Deep Q-Networks (DQN): Combining deep neural networks with Q-Learning enables the handling of complex, high-dimensional state spaces and has led to groundbreaking results in game playing and robotics. Policy-gradient methods.

Jonas Hellgren completed his PhD studies at Chalmers in 2004, where he acquired skills in mathematical simulation, optimization, and control theory. Subsequently, he applied these skills in the design of hybrid and electric vehicles for various OEMs. In 2016, he ventured into the field of reinforcement learning. Alongside his studies, he has also supervised around 10 master’s students, guiding them in the application of reinforcement learning. In 2021, he received the prestigious Volvo Group Inventor Award, which recognized his contributions to a patent involving the use of reinforcement learning for coordinating multiple autonomous vehicles.