A Pyramid CNN for Dense-Leaves Segmentation

Created by Daniel Morris at Michigan State University

Introduction:

Automatic detection and segmentation of overlapping leaves in dense foliage can be a difficult task, particularly for leaves with strong textures and high occlusions. We propose a pyramid convolutional neural network with multi-scale predictions that detects and discriminates leaf boundaries from interior textures. Using these detected boundaries, closed-contour boundaries around individual leaves are estimated with a watershed-based algorithm. The result is an instance segmenter for dense leaves. Promising segmentation results for leaves in dense foliage are obtained and more details are available in the paper.

Citations

If you are using the code/model/data provided here in a publication, please cite our paper:

@INPROCEEDINGS{MorrisPCNN:2018,
  author={Daniel D. Morris},
  booktitle={2018 15th Conference on Computer and Robot Vision (CRV)},
  title={A pyramid CNN for Dense-Leaves Segmentation},
  pages={238-245},
  doi={10.1109/CRV.2018.00041},
  month={May},
  year={2018},}

Changelog

The initial release contains 2 pre-trained models plus code for running the CNN, doing instance segmentation, and for evaluating the results. We anticipate releasing code for training in the near future.

Pretrained model

We provide the pretrained model and training/testing

Installing

0. Install and build MatConvNet: http://www.vlfeat.org/matconvnet/
1. Clone PyramidCNN: https://github.com/dmorris0/PyramidCNN
2. Add the following folders from these repos to the Matlab path:

• <PyramidCNN>/MatlabCode
• <MatConvNet>/matlab
• <MatConvNet>/matlab/mex

Running Network

In Matlab:

>> cd <PyramidCNN>/Scripts
>> runLeafNet

This will download the DenseLeaves dataset from https://www.egr.msu.edu/denseleaves/, load and run the trained network on the test folder, perform instance segmentation, plot results during the process, and output evaluation.

Evaluation

Statistics are output into file: AllStats.txt. A table of results is shown below (corresponding to Table 1 of the paper). Boundary AP indicates average precision in estimating segment boundary pixels. Unlike the paper, boundary pixels at the edge of the image are included. Also the model has been retrained and has slightly different values.

Method	Boundary AP	Segmentation Dice	Precision	Recall
Gradient Mag	0.807	NA	NA	NA
Canny	0.820	NA	NA	NA
PB	0.863	0.599	0.511	0.724
HED	0.849	0.728	0.668	0.800
PN-P1	0.974	0.890	0.883	0.897
PN-P5-S5	0.974	0.929	0.910	0.950
PN-P5-S1	0.991	0.938	0.932	0.944

If something is not working please let me know.