Kelsey Doerksen: EarthShift: Benchmarking Robust Geospatial AI

Geospatial Foundation Models claim to offer powerful solutions to simplify and accelerate real-world problems, enabling the capabilities to monitor, analyze, and predict changes on our planet. Current Earth Observation benchmarks to quantify the performance of these models focus on measuring performance on diverse tasks and applications, typically measuring generalization in-distribution. 

However, when models are deployed, they must generalize to many out-of-distribution scenarios, such as new time periods, geographies, and sensors; and in many contexts, these models are brittle. We introduce EarthShift: the first public testbed for benchmarking robustness across multiple realistic distribution shifts encountered in remote sensing. 

EarthShift enables users to measure distributional robustness by comparing performance in- and out-of-distribution using datasets from paired data sources, temporal windows, geographic locations, and sensors. 

Kelsey Doerksen is a joint postdoctoral fellow at the University of Cape Town's Climate Risk Lab and Arizona State University in the School of Augmented Intelligence and Computing and research affiliate at the NASA Jet Propulsion Laboratory. Her research focuses on combining her expertise in AI, Earth Observation, and Foundation Models to applied problems in the climate, food security, and biodiversity domains. She is a recent graduate from the Autonomous Intelligent Machines where she completed her thesis in the Oxford Applied and Theoretical Machine Learning Group on "Applied Machine Learning for Earth Systems Science", collaborating with the United Nations, European Space Agency and NASA during her PhD.