Gilberto Câmara

Machine Learning and AI for Remote Sensing Application: Promises and Challenges

Satellite imagery has become invaluable for many applications, ranging from environmental monitoring and urban planning to disaster response and agriculture. Two recent major developments have broadened the use of satellite data: (a) availability of cloud services with large collections of open data, with an emphasis on the Sentinel and Landsat programmes; b) advancement of machine learning (ML) methods to analyse and extract meaningful information from images. The combination of big Earth Observation (EO) data and advanced machine learning for satellite data (SatML in what follows) has substantial potential for supporting evidence-based policy to address global environmental challenges.

There are important shortcomings in most current SatML models. Most techniques do not consider important differences between satellite images and natural ones. Many SatML algorithms, inspired by the U-net paradigm, rely on the distinction between foreground (things) and background (stuff). While this design suits high spatial resolution images with 3-meter or smaller pixels, it is unsuited for mid to low-resolution images (10-meter or larger pixels). Mid and low-resolution images are continuous distributions of radiance values and are better described as fields than as collections of objects. Human-sized, everyday objects depicted in natural photos differ from continuous landscapes captured in satellite images. All pixels matter when working with Sentinel data for land mapping and similar broad-area applications; the foreground/background distinction is not applicable. Arguably, no proper “objects" exist in mid to low-resolution images; image classification identifies compact regions of similar values in multidimensional spaces. While domain scientists may believe they recognise objects in a remotely sensed image, they are actually measuring fields.

A further challenge to SatML models is dealing with satellite image time series. These are calibrated and comparable measures of the same location on Earth at different times. Due to frequent revisits, time series capture both gradual and abrupt changes. Researchers have used time series in applications such as forest disturbance, agricultural production, and land cover mapping. To work with satellite image time series, SatML models must include a temporal component, which is not present in most natural image collections.

This presentation analyses whether the current generation of SatML addresses the ontological challenges of dealing with images of continuous landscapes in different parts of the planet. One case of particular interest is using satellite images to address global environmental change. By covering the same location multiple times, satellites are unique in surveying large areas that are difficult to observe from the ground. Images of tropical and boreal forests, polar regions, and areas at risk of desertification require data and analysis methods that are adequately designed and properly tested. We consider the current status of SatML benchmarks for global change applications. We identify gaps and missing features and propose a way forward so that the community can produce a new generation of SatML datasets and methods adequate for global environmental change.