Learning to Group and Label Fine-Grained Shape Components

This is the code repository for "Learning to Group and Label Fine-Grained Shape Components”

Created by Xiaogang Wang, Bin Zhou, Haiyue Fang, Xiaowu Chen, Qinping Zhao, Kai Xu

Prerequisites:

 windows 7, 
 Visual Studio 2015, 
 OSG 3.0(Open Scene Graph),
 Matlab 2015b.

Dataset

You can download the data here.

0, Flt2Txt: Process models (.flt) into '.txt' files.

cd .../Code/Flt2Txt/Flt2Txt
main_get_txts.cpp
And the output '.txt' files include the following:
points.txt: all vertices of model
faces.txt: all faces of model
groups.txt: all components of model
parts.txt: all semantic parts of model (only for groundtruth model)

1, Txt2Mats: Transform '.txt' file into '.mat' file

cd .../Code/Txt2Mats
Generating_mat_from_Txts.m
And the output .mat file include the following:
model.vertices: all vertices of input model
model.groups: all components of input model (faces of each component)
model.Vertex: all components of input model (vertices of each component)

2, Caffe Network Training: Training Network

2-1, Training Caffe Model

cd .../Code/Caffe_model/Caffe_model_train
motor_multitask_cls.prototxt
motor_multitask_solver.prototxt 

2-2, Training Data Generation

cd .../Code/Caffe_model/Caffe_model_train_test_data_generation
MakeNetInput.m
The output .mat file include the following:
    proposals_200_4kinds:  all part hypotheses of 3D model
    model: all vertices, groups(components faces), grouVertex(components vertices)
    groupPixels_200: The number of elements each component occupies in the voxel space (200 *200*200)
    Edge_pair: The connection relationship between components.
And the output .h5 file include the following:
  data : 30*30*30*3*K (30 is voxel space; 3 is three branch; K is the number of part hypotheses of input model)   
  label: 1*1*2*K (2 represents 'semantic label' and 'confidence score' for each part hypotheses; K is the number of part hypotheses of input model)

3, Caffe Network Test：Test Network

3-1, Caffe Test Model

  cd .../Code/Caffe_model/Caffe_model_test
  Test_CNN_demo.m
  The output .mat models include the following fields:
     column 1: 2048 feature vector by CNN,
     column 2: regression score,
     column 3: class probability distribution

3-2, Caffe Test Data Generation

   1), For .flt model format, step 0->step 1->step 2-2 can be used to generate network test data (.h5 file only have 'data' term).
   2), For .obj model format, 
       cd .../Code/Caffe_model/Caffe_model_test_data_obj
       MakeNetTest_obj.m
       The output .h5 file include the following:
       data : 30*30*30*3*K (30 is voxel space; 3 is three branch; K is the number of part hypotheses of input model)       

4, Composite_inference: Higher-order CRF optimization

cd .../Code/Composite_inference
Batching_adjust_para.m
The output .txt file model is the final labelling result, that assigning a semantic label for each component.

5, Labeling result visualization

cd .../Code/Visualization/osg_renderbytxt
main.cpp
input: labeling result（step 4： .txt file）, original model (.flt or .obj model)
output: image with different color for different labels.

Citation

If you find our paper useful in your research, please cite:

@article{wang_siga18,
title = {Learning to Group and Label Fine-Grained Shape Components},
author = {Xiaogang Wang and Bin Zhou and Haiyue Fang and Xiaowu Chen and Qinping Zhao and Kai Xu},
journal = {ACM Transactions on Graphics (SIGGRAPH Asia 2018)},
volume = {37},
number = {6},
pages = {to appear},
year = {2018}
}