Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator

This is the code that we use to train and evaluate. The following figures are the the evaluation results. (first row: DroNet [1]; second row: TrailNet [2]; Third row: our UAVPatrolNet )

Get Start

The most current dataset and videos

www.yfan.site/UAVPatrol/html

Requirements

This code has been tested on Ubuntu 18.04, and on Python 3.6.

Dependencies:

• TensorFlow 1.15.0
• Keras 2.2.4
• Keras-contrib 2.0.8
• tensorflow-estimator
• tensorflow-probability
• tensorflow-tensorboard
• NumPy
• OpenCV
• scikit-learn
• Python gflags

Usage

0. Download the model:

   https://drive.google.com/drive/folders/19kA3OYej9iVE7fa6RWJ9X2Trh1kumlYj?usp=sharing
   

1. Run the test code, predict_UAVPatrolNet.py,predict_DroNet.pyandpredict_TrailNet.py

   python predict_whichNet.py [Flags]
   

   Check configuration parameters in common_flags.py

2. Run the accuracy evaluation code, evaluate_UAVPatrolNet.py,evaluate_DroNet.pyandevaluate_TrailNet.py

   python evaluate_whichNet.py [Flags]
   

3. Format of our Patrol Dataset (download at yfan.site/UAVPatrol.html ).

   validation_dir/
       direction/
           images/
           direction_n_filted.txt
       translation/
           images/    
           translation.txt
       ...
   test_dir/
       testingset1/
           images/
   

TODO:

Clean up and upload the training code.

Reference:

[1]. A. Loquercio, A. I. Maqueda, C. R. Del-Blanco, and D. Scaramuzza, “Dronet: Learning to fly by driving,” IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 1088–1095, 2018.

[2]. N. Smolyanskiy, A. Kamenev, J. Smith, and S. Birchfield, “Toward low-flying autonomous mav trail navigation using deep neural networks for environmental awareness,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017, pp. 4241–4247.