[pixiv] No random tags sorting

[AlttiRi]

I noted that my config: #3265 (comment)

(that uses <div class="tags">["{tags:J", "}"]</div> in the metadata postprocessor)

produces always the different metadata files since the tags are randomly sorted.

For example, https://www.pixiv.net/ajax/illust/63625443 API returns tags with the same order for each requests, so it's seems like a bug.

[mikf]

tags are randomly sorted

Tags get returned in the same order as provided by Pixiv.
(Unless you are on Python 3.4 or 3.5, then they are truly random)

gallery-dl/gallery_dl/extractor/pixiv.py

Line 45 in a24dcbe

work["tags"] = [tag["name"] for tag in work["tags"]]

[AlttiRi]

I use Python 3.9.13.

[AlttiRi]

Here is the problem code

work["tags"] = list(set(tag["translated_name"] or tag["name"] for tag in work["tags"]))

gallery-dl/gallery_dl/extractor/pixiv.py

Lines 40 to 42 in a24dcbe

	work["tags"] = list(set(
	tag["translated_name"] or tag["name"]
	for tag in work["tags"]))

Because of set() does not keep the order. Absolutely.

[AlttiRi]

https://stackoverflow.com/questions/7961363/removing-duplicates-in-lists

It suggests to use OrderedDict.fromkeys.

[AlttiRi]

Yes, it works fine. Checked.

...

  import hashlib
+ from collections import OrderedDict

...

            def transform_tags(work):
-                work["tags"] = list(set(
+                work["tags"] = list(OrderedDict.fromkeys(
                    tag["translated_name"] or tag["name"]
                    for tag in work["tags"]))
...

[mikf]

Because of set() does not keep the order. Absolutely.

I actually was not aware of that and thought sets behaved like dicts in this regard.

Anyway, tag order is now preserved (5a17e15) and I also added an S format specifier to sort lists (42481ae).