Skip to content

Latest commit

 

History

History
218 lines (197 loc) · 6.92 KB

README.md

File metadata and controls

218 lines (197 loc) · 6.92 KB
Raw

Installation

Requirements

  • Linux or macOS with Python ≥ 3.8
  • CUDA>=11.7, lower CUDA versions may result in not successfully built on detectron2
  • pip install -r requirements.txt

Example virtualenv environment setup for kMaX-DeepLab

pip3 install virtualenv
python3 -m virtualenv kmax_deeplab --python=python3
source kmax_deeplab/bin/activate

pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2
pip install -r requirements.txt
unzip detectron2.zip
cd detectron2
pip install -e .
pip install git+https://github.com/cocodataset/panopticapi.git

Example test model checkpoint of kMaX-DeepLab

  1. Download the checkpoint.

  2. Dataset preperation and structure for evaluation as below. You need to download 'relabeled_COCO_val' and rename it to 'panoptic_val2017', and rename the corresponding json file and download the images from COCO dataset.

datasets
└── coco
    ├── annotations 
    │   └── panoptic_val2017.json # relabeled_coco_val.json
    ├── panoptic_val2017  # relabeled_coco_val
    ├── val2017 # original COCO dataset val set images
  1. Use the script below to evaluate the model.
export DETECTRON2_DATASETS=YOUR_DATA_PATH
python3 train_net.py --num-gpus 8 --dist-url tcp://127.0.0.1:9999 \
--config-file configs/coco/panoptic-segmentation/kmax_convnext_large.yaml \
--eval-only MODEL.WEIGHTS YOUR_MODEL_PATH
  1. The provided checkpoint should give a PQ of 64.4 on relabeled COCO-val.

Example demo

Use the following script to demo images in a directory.

python3 demo.py --config-file ../configs/coco/panoptic-segmentation/kmax_convnext_large.yaml     \
--input YOUR_IMG_FOLDER_PATH \
--output  OUTPUT_VIS_FOLDER  \
--opts MODEL.WEIGHTS  YOUR_MODEL_PATH

Distributed training

Need to set up your environment variables to run the training script below.

export DETECTRON2_DATASETS=YOUR_DATASET_PATH
python3 train_net.py --num-gpus 8 --num-machines $WORKER_NUM \
--machine-rank $WORKER_ID --dist-url tcp://$WORKER_0_HOST:$port \
--config-file configs/coco/panoptic-segmentation/kmax_convnext_large.yaml

Model zoo

coco-val relabeled coco-val coconut-val checkpoint
backbone training set PQ AP_mask mIoU PQ AP_mask mIoU PQ AP_mask mIoU gdrive huggingface
ResNet 50 COCO 53.3 39.6 61.7 55.1 40.6 63.9 53.1 37.1 62.5 download download
COCONut-S 51.7 37.5 59.4 58.9 44.4 64.4 56.7 41.2 63.6 download download
COCONut-B 53.4 39.3 62.6 60.2 45.2 65.7 58.1 42.9 64.7 download download
coco-val relabeled coco-val coconut-val checkpoint
backbone training set PQ AP_mask mIoU PQ AP_mask mIoU PQ AP_mask mIoU gdrive huggingface
ConvNeXt-Large COCO 57.9 45.0 66.9 60.4 46.4 69.9 58.3 44.1 66.4 download download
COCONut-S 55.9 41.9 66.1 64.4 50.8 71.4 59.4 45.7 67.8 download download
COCONut-B 57.8 44.8 66.6 64.9 51.2 71.8 61.3 46.5 69.5 download download