[deviantart] fix crash when handling deleted deviations in status updates

[ClosedPort22]

Example:

"items": [
    {
        "type": "deviation",
        "deviation": {
            "deviationid": "147C8B03-7D34-AE93-9241-FA3C6DBBC655",
            "printid": null,
            "is_deleted": true
        }
    }
],

This deviation isn't actually deleted and can be fetched as usual using the deviation endpoint.

I've been working on several features and bugfixes for DA extractors and this is one of them. Unfortunately they are heavily intertwined with each other, so I'm trying my best to separate them into branches and reduce the overall complexity.

Known issues:

• If there are sta.sh-ed deviations in items and extra is enabled, these deviations will be extracted twice
• metadata and folders doesn't work for deviations in items
• If there are duplicate deviations, the deviation endpoint will be called multiple times

Currently WIP features/bugfixes that will address these issues or affect the implementation of this fix:

• Add metadata to shared deviations in status updates
• Fetch extended metadata (https://www.deviantart.com/developers/http/v1/20210526/deviation_metadata/7824fc14d6fba6acbacca1cf38c24158)
• Extract embedded deviations in status updates
• Add unique option to prevent duplicates (including stash)

[rautamiekka]

If there are sta.sh-ed deviations in items and extra is enabled, these deviations will be extracted twice
If there are duplicate deviations, the deviation endpoint will be called multiple times

We could just add a flag (or >=2, if necessary) to each deviation ID to know which ones are already downloaded/whatever, which is a drop in the ocean in RAM usage. If the flag is missing or equals "not downloaded" (or whatever the precise wording is), skip it.

[ClosedPort22]

If there are sta.sh-ed deviations in items and extra is enabled, these deviations will be extracted twice
If there are duplicate deviations, the deviation endpoint will be called multiple times

We could just add a flag (or >=2, if necessary) to each deviation ID to know which ones are already downloaded/whatever, which is a drop in the ocean in RAM usage. If the flag is missing or equals "not downloaded" (or whatever the precise wording is), skip it.

This has already been fixed, it's just not part of this PR. It's actually more complicated than that because dA's metadata endpoint doesn't like duplicate UUIDs, and I ended up deduplicating them using a dict before further processing.

The first issue was fixed by using seen = set(), similar to other extractors.

[mikf]

Thank you and my condolences for having to deal with DA's inner workings yet again. Why oh why does this site have to be so complicated?

[mikf]

Just a thought, but maybe it'd be better to use a separate _pagination method for status items? Or at least put fixing deleted deviations in its own method.

I admit to having done this myself (plenty of times?) , but it hurts even me seeing ~50 spaces of indentation.

[ClosedPort22]

The implementation here will be different if combined with this branch, which is almost done (I think). I'll try to refactor this section to make it look nicer though.

[mikf]

Why not use self.deviation()?

[ClosedPort22]

Right now (at least on this branch) neither metadata nor folders is supposed to work for shared deviations, and I'd rather not make an exception for "deleted" deviations. This will be fixed by this branch, which uses _call to bypass _metadata and _folders to make API calls more efficient.