This is the official code for our ICLR2023 paper: "Video Scene Graph Generation from Single-Frame Weak Supervision" openreview link
-
First use the environment.yaml to create the basic env.
-
Then install this repository scene_graph_benchmark.
Download from this repository VinVL.
Download from this repository ActionGenome.
Use lib/extract_bbox_features.py
to extract these features.
- (1) Annotation which keeps objects with confidence greater than 0.2. (here)
- (2) From Annotation (1), only keeps the middle frame for each video. (here)
- (3) From Annotation (2), annotated by the model-free strategy with
$\eta=0.5$ . (here) - You can also assign annotation with different hyperparemeter by
lib/genarate_predicate_pseudo_label.py
.
python train.py --cfg demo.yml
python test.py --cfg demo.yml
Please consider citing this project in your publications if it helps your research.
@inproceedings{chen2023video,
title={Video scene graph generation from single-frame weak supervision},
author={Chen, Siqi and Xiao, Jun and Chen, Long},
booktitle={The Eleventh International Conference on Learning Representations},
year={2023}
}