Importing a video-based datasets

[Jakubelo]

Hi, in the documentation I did not see any information about extractors with videos. I have many videos with annotation files in "CVAT for video" format. So, how can create the project with this videos? What methods and I have to use ( if it's possible can you show me the flow?)?

[zhiltsov-max]

Hi. It is being developed now. We are open for your suggestions about the look and feel of the functionality. Unless implemented, Datumaro expects that a video is split by frames by the user. It can be done with ffmpeg, opencv and CVAT itself. If you have annotated the video in CVAT, probably, you better just export the annotated task as a dataset with images. Once the video is split, you can add images as described in https://openvinotoolkit.github.io/datumaro/docs/user-manual/media_formats/

[Jakubelo]

@okay i understand.

So lets assume I want to do not export tasks in CVAT as dataset, I want to keep annotations in text file (in this case xml) for storage purpose. I can dump annoations from cli from 300 tasks(videos) as xml in "CVAT for images 1.1" and I can slice videos to frames. How can I put that annotations to Datumaro? Can you provide any example, how to inject annoations from xml to Datumaro (in this case probably it will be DatasetItem???).

My sugestion to this implementation is. I have many videos and many annoations dumped in xml for example or in rle(each corrseponding to ech video file). Then, I want to "inject" that video and annot file to something like DatasetItem/whatever and then iterate through each video.

[zhiltsov-max]

I can dump annoations from cli from 300 tasks(videos) as xml in "CVAT for images 1.1" and I can slice videos to frames. How can I put that annotations to Datumaro?

Then, I want to "inject" that video and annot file to something like DatasetItem/whatever and then iterate through each video.

You need to use the following commands:

datum create -o proj/
datum add path -p proj/ -f cvat path/to/dataset/
...

A dataset example is here:
https://github.com/openvinotoolkit/datumaro/tree/develop/tests/assets/cvat_dataset/for_images

A code example is available here:
https://github.com/openvinotoolkit/datumaro/blob/develop/tests/test_cvat_format.py#L38-L68

If you want to merge the tasks, consider using "projects" in CVAT, which allow to join tasks and export them as a whole. Alternatively, look at #16 (comment), if you don't want to use CVAT for this.

[zhiltsov-max]

Related: #129

[zhiltsov-max]

@Jakubelo, were you able to solve the problem?

[Jakubelo]

@zhiltsov-max for the moment I used custom scripts for pulling out annotations and making a dataset, due to lack of time.

Sorry for the late response.

[zhiltsov-max]

@Jakubelo, please check if you have any comments about #129 (comment).