Explaining the decisions and the functioning of a convolutional spatiotemporal land cover classifier with channel attention and redescription mining - Réseau télédétection INRAE Access content directly
Journal Articles ISPRS Journal of Photogrammetry and Remote Sensing Year : 2024

Explaining the decisions and the functioning of a convolutional spatiotemporal land cover classifier with channel attention and redescription mining

Abstract

Convolutional neural networks trained with satellite image time series have demonstrated their potential in land cover classification in recent years. Nevertheless, the rationale leading to their decisions remains obscure by nature. Methods for providing relevant and simplified explanations of their decisions as well as methods for understanding their inner functioning have thus emerged. However, both kinds of methods generally work separately and no explicit connection between their findings is made available. This paper presents an innovative method for refining the explanations provided by channel-based attention mechanisms. It consists in identifying correspondence rules between neuronal activation levels and the presence of spatiotemporal patterns in the input data for each channel and target class. These rules provide both class-level and instancelevel explanations, as well as an explicit understanding of the network operations. They are extracted using a state-of-the-art redescription mining algorithm. Experiments on the Reunion Island Sentinel-2 dataset show that both correct and incorrect decisions can be explained using convenient spatiotemporal visualizations.
Fichier principal
Vignette du fichier
ISPRS_Journal.pdf (35.25 Mo) Télécharger le fichier
Origin Files produced by the author(s)
licence

Dates and versions

hal-04650672 , version 1 (16-07-2024)

Licence

Identifiers

Cite

Enzo Pelous, Nicolas Méger, Alexandre Benoit, Abdourrahmane Atto, Dino Ienco, et al.. Explaining the decisions and the functioning of a convolutional spatiotemporal land cover classifier with channel attention and redescription mining. ISPRS Journal of Photogrammetry and Remote Sensing, 2024, 215, pp.256-270. ⟨10.1016/j.isprsjprs.2024.06.021⟩. ⟨hal-04650672⟩
0 View
0 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More