Skip to content

WJ-Chang-42/ASTransformer

Repository files navigation

Transformer-based Monocular Depth Estimation with Attention Supervision

Pretrained models

You can download pre-trained model

Evaluation

Evaluate on KITTI

python3 eval.py --model_dir "PATH/KITTI.pkl" --evaluate --batch_size 1 --dataset KITTI --data_path "PATH" --gpu_num 0 --encoder AST

Evaluate on NYU2

python3 eval.py --model_dir "PATH/NYU2.pkl" --evaluate --batch_size 1 --dataset NYU --data_path "PATH" --gpu_num 0 --encoder AST

DPT-Large trained by us

python3 eval.py --model_dir "MODEL_PATH" --evaluate --batch_size 1 --dataset "DATASET" --data_path "PATH" --gpu_num 0 --other_method DPT-Large

Adabins trained by us

python3 eval.py --model_dir "MODEL_PATH" --evaluate --batch_size 1 --dataset "DATASET" --data_path "PATH" --gpu_num 0 --other_method Adabins

Dataset Preparation

We referred to LapDepth in the data preparation process. Thanks for sharing the processed dataset.

KITTI

1. Official ground truth

  • Download official KITTI ground truth on the link and make KITTI dataset directory.
    $ cd ./datasets
    $ mkdir KITTI && cd KITTI
    $ mv ~/Downloads/data_depth_annotated.zip ./datasets/KITTI
    $ unzip data_depth_annotated.zip

2. Raw dataset

  • Construct raw KITTI dataset using following commands.
    $ cd ./datasets/KITTI
    $ aria2c -x 16 -i ./datasets/kitti_archives_to_download.txt
    $ parallel unzip ::: *.zip

3. Dense g.t dataset
We take an inpainting method from DenseDepth to get dense g.t for gradient loss.
(You can train our model using only data loss without gradient loss, then you don't need dense g.t)
Corresponding inpainted results from './datasets/KITTI/data_depth_annotated/2011_xx_xx_drive_xxxx_sync/proj_depth/groundtruth/image_02' are should be saved in './datasets/KITTI/data_depth_annotated/2011_xx_xx_drive_xxxx_sync/dense_gt/image_02'.
KITTI data structures are should be organized as below:

|-- datasets
  |-- KITTI
     |-- data_depth_annotated  
        |-- 2011_xx_xx_drive_xxxx_sync
           |-- proj_depth  
              |-- groundtruth            # official G.T folder
        |-- ... (all drives of all days in the raw KITTI)  
     |-- 2011_09_26                      # raw RGB data folder  
        |-- 2011_09_26_drive_xxxx_sync
     |-- 2011_09_29
     |-- ... (all days in the raw KITTI)  

NYU Depth V2

1. Training set
Make NYU dataset directory

    $ cd ./datasets
    $ mkdir NYU_Depth_V2 && cd NYU_Depth_V2
  • Constructing training data using following steps :
    • Download Raw NYU Depth V2 dataset (450GB) from this Link.
    • Extract the raw dataset into './datasets/NYU_Depth_V2'
      (It should make './datasets/NYU_Depth_V2/raw/....').
    • Run './datasets/sync_project_frames_multi_threads.m' to get synchronized data. (need Matlab)
      (It shoud make './datasets/NYU_Depth_V2/sync/....').
  • Or, you can directly download whole 'sync' folder from our Google drive Link into './datasets/NYU_Depth_V2/'

2. Testing set
Download official nyu_depth_v2_labeled.mat and extract image files from the mat file.

    $ cd ./datasets
    ## Download official labled NYU_Depth_V2 mat file
    $ wget http://horatio.cs.nyu.edu/mit/silberman/nyu_depth_v2/nyu_depth_v2_labeled.mat
    ## Extract image files from the mat file
    $ python extract_official_train_test_set_from_mat.py nyu_depth_v2_labeled.mat splits.mat ./NYU_Depth_V2/official_splits/

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages