Artstation not working in latest version (v1.22.3)

[cabeatty85]

With v1.22.3, whenever I try to download anything from Artstation I get the error "[artstation][error] Unable to download data: JSONDecodeError: Expecting value: line 1 column 1 (char 0)". I get the same error whether I try to download one image or an artist's entire gallery. When I use v1.22.1, it works fine and I have no issues.

Verbose log:

[gallery-dl][debug] Version 1.22.3 - Executable
[gallery-dl][debug] Python 3.7.9 - Windows-10-10.0.19041
[gallery-dl][debug] requests 2.28.0 - urllib3 1.26.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.artstation.com/artwork/Vgm3ZX'
[artstation][debug] Using ArtstationImageExtractor for 'https://www.artstation.com/artwork/Vgm3ZX'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.artstation.com:443
[urllib3.connectionpool][debug] https://www.artstation.com:443 "GET /projects/Vgm3ZX.json HTTP/1.1" 200 None
[artstation][error] Unable to download data: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[artstation][debug]
Traceback (most recent call last):
File "requests\models.pyc", line 972, in json
File "json_init_.pyc", line 348, in loads
File "json\decoder.pyc", line 337, in decode
File "json\decoder.pyc", line 355, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "gallery_dl\job.pyc", line 77, in run
File "gallery_dl\extractor\artstation.pyc", line 32, in items
File "gallery_dl\extractor\artstation.pyc", line 387, in metadata
File "gallery_dl\extractor\artstation.pyc", line 77, in get_project_assets
File "requests\models.pyc", line 976, in json
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[kattjevfel]

Reverting 1bc77ef makes it work again, but that commit was added to fix #2527...

[mikf]

The problem is commit 3a5d5c3 and especially this line here:

gallery-dl/gallery_dl/extractor/common.py

Line 721 in 3a5d5c3

("Accept-Encoding", "gzip, deflate, br"),

The EXE file on the releases page has no brotli decoder, so Artstation is now sending its data in brotli encoding and gallery-dl can't decode it, resulting in the error above.

Workarounds:

• The EXE files from https://github.com/mikf/gallery-dl/actions/workflows/executables.yml do work, for example this one
• Changing the Accept-Encoding header works: -o "headers.Accept-Encoding=gzip, deflate"

[kattjevfel]

The EXE file on the releases page has no brotli decoder, so Artstation is now sending its data in brotli encoding and gallery-dl can't decode it, resulting in the error above.

For what it's worth it doesn't work on Arch Linux either.

katt@main ~ » gallery-dl --ignore-config --verbose https://www.artstation.com/artwork/Vgm3ZX
[gallery-dl][debug] Version 1.22.3
[gallery-dl][debug] Python 3.10.5 - Linux-5.18.7-zen1-1-zen-x86_64-with-glibc2.35
[gallery-dl][debug] requests 2.27.1 - urllib3 1.26.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.artstation.com/artwork/Vgm3ZX'
[artstation][debug] Using ArtstationImageExtractor for 'https://www.artstation.com/artwork/Vgm3ZX'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.artstation.com:443
[urllib3.connectionpool][debug] https://www.artstation.com:443 "GET /projects/Vgm3ZX.json HTTP/1.1" 200 None
[artstation][error] Unable to download data:  JSONDecodeError: [Errno Expecting value] <malformed output>
[artstation][debug] 
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/requests/models.py", line 910, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/gallery_dl/job.py", line 77, in run
    for msg in extractor:
  File "/usr/lib/python3.10/site-packages/gallery_dl/extractor/artstation.py", line 32, in items
    data = self.metadata()
  File "/usr/lib/python3.10/site-packages/gallery_dl/extractor/artstation.py", line 386, in metadata
    self.assets = list(ArtstationExtractor.get_project_assets(
  File "/usr/lib/python3.10/site-packages/gallery_dl/extractor/artstation.py", line 77, in get_project_assets
    data = self.request(url).json()
  File "/usr/lib/python3.10/site-packages/requests/models.py", line 917, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: [Errno Expecting value] <malformed output>

The -o "headers.Accept-Encoding=gzip, deflate" workaround works though.

[mikf]

Arch Linux does not have a python-brotli package, does it?
I guess you can do pip install brotli to add the necessary brotli support.
Maybe it also needs pacman -S brotli, not sure.

I really should not have included 3a5d5c3 in the last release ...

[kattjevfel]

python-brotli is in the repos and with it installed all is well again, should I just add it as a optdepend for the PKGBUILD? (I'm the maintainer for the g-dl AUR package)
Also would be nice if gallery-dl would output a somewhat nicer and clearer message when it lacks this package.

[mikf]

OK, brotli support is now truly optional, as it should have been in the first place. (de20cad)

should I just add it as a optdepend for the PKGBUILD?

Yeah, that would be nice.
It is technically already covered by gallery-dl -> requests -> urllib3 -> brotli, but the same is true for pysocks and that is already an optional dependency.

[kattjevfel]

python-brotli is actually just a optdepend for python-urllib3, so it isn't automatically pulled, same for python-pysocks. I'll add it to optdepends for gallery-dl and gallery-dl-git.