[deviantart] support user avatars and banners

[mikf]

From #146 (comment)

[Twi-Hard]

I forgot to mention when this issue was created what I've learned about deviantart avatars..
/avatars-original/ is the biggest (I don't see it in the code)
if that doesn't exist then try /avatars-big/ then /avatars/

Another thing is that users often have several accessible avatars. When they upload an avatar with a different extension then the one with the old extension isn't deleted. So there's often 3 avatars: gif, png and jpg (just replace the extension)

[mikf]

I had tried finding something better than -big a couple of times when implementing this and it always resulted in a 404 so I eventually gave up on that.

Now I've found an avatar that works with -original which is completely different from -big or regular. How should that be handled?

https://a.deviantart.net/avatars-original/a/m/amandajeansart.jpg?15
https://a.deviantart.net/avatars-big/a/m/amandajeansart.jpg?15
https://a.deviantart.net/avatars/a/m/amandajeansart.jpg?15

[Twi-Hard]

They probably had an avatar that was big enough for original resolution then they changed it to something smaller. I've wanted to save all the versions I could but that's probably not best for everybody. There's probably no way to tell what the biggest current url is except trying all of the sizes for each user.

You can probably get the biggest current profile picture by getting all of the sizes then getting the biggest size that matches the smallest size. The smallest resolution should probably always be the right one. If we were to only download one I'd go for the biggest size. It's either go for the biggest size which is probably usually correct or be stuck with the smallest size for everything which kinda sucks.

By the way is this for commenters too? That would be really nice.

I said "probably" a lot :/

[mikf]

then getting the biggest size that matches the smallest size

Downloaded files are just binary blobs for gallery-dl. There is no way to programmatically tell if one size "matches" another. A human (and nowadays probably AI) can do that, but gallery-dl can't even compare pixels.

By the way is this for commenters too? That would be really nice.

Not yet. But that should be doable and won't even cost any extra API requests since comments have the user icon included.

[mikf]

Just found one where different file extensions result in different images.

https://a.deviantart.net/avatars/a/n/anoya.png
https://a.deviantart.net/avatars/a/n/anoya.gif
https://a.deviantart.net/avatars/a/n/anoya.jpg # 404

edit: -big seems to always exist, even if it's the same size as regular

https://a.deviantart.net/avatars-big/a/n/anoya.png # same as /avatar/
https://a.deviantart.net/avatars-big/a/n/anoya.gif # same as /avatar/
https://a.deviantart.net/avatars-big/a/n/anoya.jpg # 404

[Twi-Hard]

I just checked a folder of avatars that start with "an" after excluding the default profile picture and they were all unique images, even ones from the same users.

an-absurd-bird-db9361c21b3178dc.jpg
an-ace-upmysleeve-cf3130ce9b99cc26.gif
an-ace-upmysleeve-fa6ac1a185979ed0.jpg
an-alien-named-jax-8ad0871f2dd63c36.png
an-alien-named-jax-8e346333cecc3393.jpg
an-brony-gaeilge-ee2dc143b7c4a9a1.jpg
an-brony-gaeilge-fb30964bb5c2c6c2.png
an-chan-man-b990c66fb33838c6.png
an-chan-man-bdc3269a27396698.jpg
an-chan123-8f7164de144b926d.jpg
an-chan123-8758621766933e3e.png
an-chan123-e262bf0ec9b129c9.gif
an-eggy-9734684dae98d366.png
an-eldritch-scribe-c40ff49995f8989a.jpg
an-everyday-artist-ee9b9836c7651849.png
an-everyday-artist-efb810a5a1e3a6d2.jpg
an-ggrath-bfd0e1c1d02c7647.jpg
an-ja-ad56432fb5d04b25.gif
an-ja-adb452434b2fb594.png
an-ja-e84e8d66c0b373b1.jpg
an-juu-c16917971ec1cace.png
an-juu-d01643f18a4dcedd.gif
an-nah-ba12e72b80bea764.jpg
an-nah-edb24da01fb18678.gif
an-najmi-82fd4fa03722d0cf.png
an-old-otaku-b16ac6e52a0d4e8f.jpg
an-pseudonym-adf20036370bd3f2.gif
an-pseudonym-ef513d02f7207b84.jpg
an-pseudonymous-a52c83c6e6999b8e.gif
an-pseudonymous-bfa471de8510c693.png

As you can see it's extremely common for users to have multiple unique pics. Most users have multiple even.

Avatars can be extremely big too:

The strings appended to the usernames are phash fingerprints I was using to test stuff by the way.

sizes for the 7,755 unique images in that folder:
under 50x50 - 38 images
50x50 - 6030 images
between 50x50 and 100x100 - 17 images
100x100 - 4472 images
above 100x100 - 985 images

50x50 and 100x100 seem to be the only standard sizes, everything else was pretty random

[DoaJCBlogger]

I downloaded version 1.26.6 and tried downloading the avatar for my account and I got this error. It also doesn't seem to work for other accounts.

[deviantart][debug] Using DeviantartAvatarExtractor for 'https://www.deviantart.com/doajcblogger/avatar'
[deviantart][debug] Using custom API credentials (client-id [removed])
[deviantart][error] An unexpected error occurred: KeyError - 'stats'. Please run gallery-dl again with the --verbose flag, copy its output and report this issue on https://github.com/mikf/gallery-dl/issues .
[deviantart][debug]
Traceback (most recent call last):
File "gallery_dl\job.pyc", line 127, in run
File "gallery_dl\extractor\deviantart.pyc", line 128, in items
File "gallery_dl\extractor\deviantart.pyc", line 239, in prepare
KeyError: 'stats'

[mikf]

This happens when comments are enabled, it seems.

"deviantart": {
    "avatar": {
        "comments": false
    }
}

edit: fixed in 5f9a98c

[mikf]

I tried implementing a formats option for avatars. (39904c9)

[Twi-Hard]

Does it stop checking avatar formats after the first successful one? I was hoping to get all of the formats specified since they are different images usually.
I've been waiting on whether or not the comments avatars will be downloadable before starting to try to download things again so I'm not needing to go back and do the same posts again.

[mikf]

It will try to download all formats. You might have to adjust filename etc to ensure their names don't overlap and formats get skipped that way.