Skip to content

Flat'n'Fold: A Diverse Multi-Modal Dataset for Garment Perception and Manipulation

License

Notifications You must be signed in to change notification settings

lipeng-zhuang521/flat-n-fold

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Flat’n’Fold: A Diverse Multi-Modal Dataset for Garment Perception and Manipulation

Paper Abstract

This repository is published along with a paper Flat’n’Fold: A Diverse Multi-Modal Dataset for Garment Perception and Manipulation. We provided 1212 human and 887 robot demonstrations of flattening and folding 44 garments across 8 categories. Also, we establish two new benchmarks for grasping point prediction and subtask decomposition.

Hardware

Intrinsic parameters and external paramerters (origin is headset) of three cameras, STL files of the grippers are provided in hardware folder.

Dataset

  • Dataset can be download at: https://gla-my.sharepoint.com/:f:/g/personal/2658047z_student_gla_ac_uk/Ekgx_o8q6ZZBtxusMwrP8zoBt2FkZL9vwq3hqe5c1CyHSQ. Some samples of data and data description are also provided.
  • For each data sequences, rgbd images and action information are provided. Pointclouds can be generated through pcreate_gmatch.ipynb in Pointcloud folder. Merged pointclouds of three cameras and action visulization can be gernerated through merge_three_camera.py and pointcloud_convert_match.py in Pointcloud folder.
  • Besides, it's worth noting that the origin of action information of our robot demonstration is base of Baxter. The orgin of action information of our human demonstration is headset. The transformations of transition and rotation are also provided in Pointcloud folder.

Grasping point prediction benchmark

  • We provided a sub-dataset for grasping point task which contains merged pointclouds and their correspoinding grasping point (both position and rotation). Datasets are provided in https://gla-my.sharepoint.com/:f:/g/personal/2658047z_student_gla_ac_uk/Ekgx_o8q6ZZBtxusMwrP8zoBt2FkZL9vwq3hqe5c1CyHSQ .
  • We also provided codes for how to create the sub-dataset from original dataset, including grasping point extraction, pointcloud merging and downsampling, grasping points' position and orientation calculation and so on.
  • Code for two baselines: Pointnet++ and Pointbert are provided in Grasping_point folder, where finetune_rand.py is for 'random split' and finetune_class.py is for 'split by garments'.

Subtask decomposition benchmark

  • Code of data processing, groundtruth extraction and metrics evaluation for one baseline: Universal Visual Decomposer are provided in UVD folder.

Acknowledgments

Citation

if you are using Flat'n'Fold dataset, please consider citing the follwing paper:

@misc{zhuang2024flatnfolddiversemultimodaldataset,
        title={Flat'n'Fold: A Diverse Multi-Modal Dataset for Garment Perception and Manipulation}, 
        author={Lipeng Zhuang and Shiyu Fan and Yingdong Ru and Florent Audonnet and Paul Henderson and Gerardo Aragon-Camarasa},
        year={2024},
        eprint={2409.18297},
        archivePrefix={arXiv},
        primaryClass={cs.RO},
        url={https://arxiv.org/abs/2409.18297}
}

About

Flat'n'Fold: A Diverse Multi-Modal Dataset for Garment Perception and Manipulation

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published