Face segmentation with CNN and CRF

We try different methods to complete face segmentation:

• A CNN Cascade for Landmark Guided Semantic Part Segmentation. Models and more details please refer to Aaron Jackson's website. We add CRF as postprocessing. CRF is implemented by pydensecrf.
• On Face Segmentation, Face Swapping, and Face Perception. Original codes and models.
• Generate convex hull according to landmarks.

Before using all three methods we detect landmarks and crop the image. Instead of using landmarks detection network in A CNN Cascade for Landmark Guided Semantic Part Segmentation we use 2D-FAN to detect landmarks which works very well on large pose images. We also try different methods to crop the image.

Codes

• face_segment_part.py: A CNN Cascade for Landmark Guided Semantic Part Segmentation.
• face_segment_yuval.py: On Face Segmentation, Face Swapping, and Face Perception.
• face_segment_contour.py: Detected landmarks and get convex hull.

Dependencies

Please download caffe(minimum version: 1.0) for face_segment_yuval.py, download caffe-future for face_segment_part.py

Then run the following command in a terminal:

pip install -r requirements.txt

Results

• face_segment_part:

No crf:

Add crf:

No crf:

Add crf:

• face_segment_yuval:

Model face_seg_fcn8s(size:500x500):


Model face_seg_fcn8s_300_no_aug(size:300x300):


Add crf to second model:

• face_segment_contour:

Add crf:

TODO

• Modify probability computation in face_segment_yuval.py
• Add speed test
• Add landmark detection
• Add image cropping
• Contour -> segment
• Unify code style
• Improve CRF results
• Add example images to README
• Compare results of all methods