DeepTreeAttention

Tree Species Prediction for the National Ecological Observatory Network (NEON)

Implementation of Hang et al. 2020 Hyperspectral Image Classification with Attention Aided CNNs for tree species prediction.

Model Architecture

Road map

• Data Generation: Convert NEON field data into crowns -> pixels for mapping
• Baseline Model: A vanilla 2D CNN score for computing comet environment, metrics and figures
• 3D CNN with Attention

After this there are many different routes including weak label learning, self-supervised contrastive learning and other psuedo-labeling approaches

Project Organization

├── LICENSE
├── README.md          <- The top-level README for developers using this project.
├── data
│   ├── interim        <- Intermediate data that has been transformed.
│   ├── processed      <- The final, canonical data sets for modeling.
│   └── raw            <- The original, immutable data dump.
│
├── environment.yml   <- Conda requirements
│
├── setup.py           <- makes project pip installable (pip install -e .) so src can be imported
├── src                <- Source code for use in this project.
│   ├── __init__.py    <- Makes src a Python module
│   ├── data           <- Pytorch Lighting data module for creating dataloaders for model training
│   ├── dataset        <- Pytorch dataset for generating batches of data
│   ├── generate       <- Convert csv of point files to tree crowns
│   ├── main           <- Pytorch Lightning Module for model training
│   ├── neon_paths     <- Utilities for getting paths and metadata from NEON HSI data
│   ├── patches        <- Convert tree crowns into a set of pixels with overlapping windows
│   ├── start_cluster  <- dask utilities for SLURM parallel processingt

---

Pytorch Lightning Data Module (data.TreeData)

This repo contains a pytorch lightning data module for reproducibility. The goal of the project is to make it easy to share with others within our research group, but we welcome contributions from outside the community. While all data is public, it is VERY large (>20TB) and cannot be easily shared. If you want to reproduce this work, you will need to download the majority of NEON's camera, HSI and CHM data and change the paths in the config file. For the 'raw' NEON tree stem data see data/raw/neon_vst_2021.csv. The data module starts from this state, which are x,y locations for each tree. It then performs the following actions.

1. Filters the data to represent trees over 3m with sufficient number of training samples
2. Extract the LiDAR derived canopy height and compares it to the field measured height. Trees that are below the canopy are excluded based on the min_CHM_diff parameter in the config.
3. Splits the training and test x,y data such that field plots are either in training or test.
4. For each x,y location the crown is predicted by our tree detection algorithm (DeepForest - https://deepforest.readthedocs.io/).
5. Crops of each tree are created and divided into pixel windows for pixel-level prediction.

Project based on the cookiecutter data science project template. #cookiecutterdatascience

Pytorch Lightning Training Module (data.TreeModel)

Citation