FAST-EO Use Case 2 - Methane Detection
This directory contains all data and code necessary to recreate experiments conducted for fine-tuning Terramind-Base to detect methane in satellite images. It includes five distinct experiments along with their corresponding datasets. The attached Methane_benchmark_patches_summary_v3.xlsx file provides descriptions for every patch extracted from the Methane Benchmark Dataset (MBD) and defines the fold splits to ensure non-overlapping data. This Excel file is used by the runner scripts to partition the data, typically reserving one fold for testing.
Each script includes usage instructions which can be accessed by applying the --help (or -h) flag.
Important: Ensure the Terramind package is installed before running any experiments.
Experiment 1: Fine tuning on Methane Benchmark Dataset
The first experiment is fine tuning the model on Methane Benchmark Dataset. The dataset has been attatched in the directory MBD_nan_S2_zscore, and has been already normalized. The code for running the training is located in the classification directory, along with neccessary dataset and dataloader classes.
Experiment 2: Fine tuning on MBD with text captions
This experiment contains a modified verion of the Terramind-Based model, which concatinates the textual embeddings of the text captions for every image, with the visual embeddings of the base model. The text embeddings are calculated using the all-MiniLM-L6-v2 model. All the code, along with embeddings calculation, and data, is available in the classification_with_text directory. The original captions are located in classification_with_text/MBD_text, and the embeddings are located inside the combined_caption_embeddings.csv file.
Experiment 3: Fine tuning and inference on Sentinel 2 with simulated atmospheric conditions
This experiment checks how the Terramind-Base behaves on the Sentinel-2 data with simulated atmospheric conditions. The simulated data is both in the Top-of-Atmosphere and Bottom-of-Atmsphere variants. The model can be both trained on this data, or only run on it, to test how good it is at generalization when trained on different data.
Experiment 4: Fine tuning and inference on Intuition 1 with simulated atmosphric conditions
This experiment checks how the Terramind-Base behaves on the Intuition-1 data with simulated atmospheric conditions. The simulated data is both in the Top-of-Atmosphere and Bottom-of-Atmsphere variants. The model can be both trained on this data, or only run on it, to test how good it is at generalization when trained on different data.
Experiment 5: Testing the detector on urban dataset without methane
The urban dataset has been prepared to check whether the models really learned to detect methane from multispectral data or just look for urban signatures in the images. All of the images in this dataset do not contain methane, the goal is to run the models and see how many false positives are returned. Python script for loading and running the models were attatched.