Newton Mwai Kinyanjui: Improving treatment personalization with structures in sequential decision making

Personalizing treatments for patients involves a period during which different treatments from a set of available options are tried until an optimal treatment is found for particular patient characteristics. To minimize suffering and other costs, it is critical to reduce the duration of this search. When treatments have primarily short-term effects, the search can be conducted using multi-armed bandit algorithms (MABs). However, these algorithms typically require long exploration periods to guarantee optimality.

With historical data, it is possible to identify structures that incorporate prior knowledge of the types of patients that may be encountered and the conditional reward models for those patient types. Such structural priors can be used to shorten the treatment exploration period, enhancing their applicability in real-world settings. Additionally, structures are beneficial in guiding how exploration is performed—switching treatments often incurs costs for the patient. Every time a treatment is changed, the patient must wean off their current therapy and adjust to the new treatment and its potential side effects.

In this presentation, I will discuss how we leverage structures with latent bandits and batched bandits to design algorithms for treatment personalization.

Newton is a PhD student in Computer Science and Engineering at Chalmers University of Technology, within the Healthy AI Lab. 

He works in machine learning to improve sequential decision making in healthcare using historical data. 

Prior to joining Chalmers, he earned a Master of Science in Electrical and Computer Engineering from Carnegie Mellon University.