Improper sequence processing of exif images

[tsmock]

Basic information

• Release version: 0.9.2 (I'll have to check and see which commit it was, pip install --upgrade git+https://github.com/mapillary/mapillary_tools)
• System: Linux
• Capture Device: Action Camera (Garmin VIRB Ultra 30)

Steps to reproduce behavior

1. Upload directory of captured images (note: run through exiv2 rename and jpegoptim --preserve, I'll need to check and see if that changed some of the metadata, but it shouldn't have changed the gps time)
2. Wait for uploads to finish processing or notice that most sequences have a suspiciously low number of images

Expected behavior

Non-borked sequences

Actual behavior

Borked sequences

https://www.mapillary.com/app/?pKey=184786507377289&username[]=vorpalblade&dateFrom=2022-09-01

Corresponding data

I'll see if I can upload some sample data Monday. Please note that I still have the pictures, so Mapillary can delete everything uploaded by me between 2022-09-01 and 2022-09-17.

[ptpt]

Could you check (or share) the generated mapillary_image_description.json after processing for investigation?

mapillary_tools process /path/to/images
# will generate /path/to/images/mapillary_image_description.json

What present in this json will be ensured to be uploaded with the specified metadata.

[tsmock]

I'll do that. I just need to get the data (it is at home). If I don't see it in the path/to/images directory, I'll see if I can find it in one of the XDG directories.

[tsmock]

mapillary_image_description.zip

Some filenames of note:

• CopiedPictures/3976941712/1662812226.6850011/100_VIRB/20220912_123231.JPG: 072b742a-13f2-468e-989e-119962bcd286
• CopiedPictures/3976941712/1662812226.6850011/100_VIRB/20220912_123234.JPG: 7894fde9-f31b-4bd5-a1d2-d004acf967e3

[ptpt]

Thanks for reporting this critical bug @tsmock. It should be fixed in the latest main branch.

This is caused by a mistake in the sequence processing (sequence discovering/cutting/duplication): mapillary_tools sorted images in sequences by filenames instead of capture times. It was introduced in v0.8.1 during this refactoring PR.

[ptpt]

The consequence of the bug is that: if your image files are not named by capturing order (e.g. IMG_0001.jpg, IMG_0002.jpg) these images will be group into many short or single-image sequences. In your case, it produced 61793 sequences. After the fix, it produced 602 sequences.