Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bounding box creation schemes #14

Open
jabooth opened this issue Aug 1, 2015 · 1 comment
Open

Add bounding box creation schemes #14

jabooth opened this issue Aug 1, 2015 · 1 comment

Comments

@jabooth
Copy link
Member

jabooth commented Aug 1, 2015

The code to generate bounding box initialisations should probably live inside menpobench. The two initialisations that interest us from the start are:

  1. Initialise from object detection (to start with from dlib)
  2. Initialise with noise, using the approach of menpofit (post refactor)

For the object detection, we will want to:

  1. Detect bounding boxes using the object detector
  2. Define an measure of 'acceptable performance', based on overlap of gt/detected bounding boxes
  3. If the detection failed, or is below threshold for acceptable performance, a new bounding box will be synthesised
  4. Synthesised bounding boxes are placed by using a learned multivariate gaussian on the acceptable bounding boxes.

@patricksnape @nontas @jalabort Do you agree with this approach? Some help with the following questions would be great:

  1. What do we feel is a good overlap amount? I was thinking that 80% of ground truth is covered by detected?
  2. If there are multiple detections, we just keep one right?
  3. Sorry @jalabort, can you point me once again to the new style noise initialisations in menpofit/refactor?

If we get this sorted I can add lfpw, afw, ibug, and helen datasets to menpobench with detections today. Any other datasets we want for testing?

@jabooth
Copy link
Member Author

jabooth commented Aug 1, 2015

26dc94e implements a first version of this. Usage:

> menpobench bbox dlib ./path/to/images/*.png --synthesize

This creates bounding boxes next to each image found (e.g. for img_001.png, creates img_001_dlib.ljson)

If --synthesize is passed, missing detections are generated from a PCA basis learned on the successful detections. See here for details. Would be great to have another pair of eyes check over that and make sure the approach seems sensible.

See here for where we use these new annotations in loading a dataset in menpobench. Also, note that I am moving to store the datasets on github, e.g menpo/lfpw-train. I think this could be a great way to iteratively fix issues with datasets in an open way.

Reopened as I'd like feedback on the design before we go too much further with it.

@jabooth jabooth closed this as completed Aug 1, 2015
@jabooth jabooth reopened this Aug 1, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant