Skip to content

Lightweight Foundation Models for Earth Observation in Low-Data Regimes

Type of project

M.Sc.

Contact

Spyros Kondylatos (spko@dhigroup.com)

Topic

Remote sensing provides unique opportunities for observing Earth from space. Traditionally, Earth Observation (EO) workflows use satellite data such as Sentinel-1 and Sentinel-2 to extract insights. Recently, EO Foundation Models have offered an alternative to these traditional workflows. These models are pre-trained on large archives of EO data, encoding vast amounts of knowledge in their representations [1].

However, training these models typically requires enormous computational time and resources. Recent research has shown that lightweight Foundation Models can also perform well on downstream tasks [2]. Following this direction, this project will further explore whether smaller models trained on smaller datasets can achieve comparable performance to their larger counterparts.

This thesis will investigate the use of lightweight Foundation Models in low-data scenarios. The student will explore questions like: How small can we go while maintaining performance? Can methods such as supervised EO Foundation Models work effectively? [3] What techniques make lightweight models competitive?

Relevance to DHI

The DHI Earth Observation Center of Excellence frequently applies machine learning and deep learning methods to EO applications. This project explores state-of-the-art models and techniques to improve current workflows and products with limited computational resources, aligning well with the focus on sustainable AI.

Moreover, DHI focuses on water-related applications, including bathymetry, coastal monitoring, and ocean observations. However, most EO datasets are land-focused. Developing Foundation Models that excel in low-data regimes could deliver immediate impact to DHI's water-centric operations. This capability can accelerate the development of specialized Foundation Models for such applications while requiring substantially less training data.

What is the student's contribution

  • Conduct a comprehensive literature review of EO Foundation Models
  • Investigate current capabilities, performance benchmarks, and fundamental limitations of lightweight geospatial Foundation Models
  • Develop or enhance lightweight Foundation Model architectures for EO applications through systematic experimentation
  • Evaluate downstream task performance across multiple domains, including comprehensive benchmarking on 2-3 water-related applications to assess real-world applicability

References

[1] Zhu, X.X., Xiong, Z., Wang, Y. et al. On the foundations of Earth foundation models. Commun Earth Environ 7, 103 (2026). https://doi.org/10.1038/s43247-025-03127-x

[2] Hackel L, Burgert T, Demir B. How Much of a Model Do We Need? Redundancy and Slimmability in Remote Sensing Foundation Models. arXiv preprint arXiv:2601.22841. 2026 Jan 30.

[3] Adorni P, Pham MT, May S, Lefèvre S. EoS-FM: Can an Ensemble of Specialist Models act as a Generalist Feature Extractor?. arXiv preprint arXiv:2511.21523. 2025 Nov 26.