Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bilibili download support #3029

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

gaoxingliang
Copy link
Contributor

If you want to download 1080 or upper, it requires a login,
export the cookie in firefox by installing a plugin named: cookies.txt
and export the cookie in domain bilibili.com as txt.
run the command:

you-get -c /Users/edward/Downloads/cookies-bilibili-com.txt -l -k --debug https://www.bilibili.com/bangumi/play/ep742483

then you can download an eposides.
Note this has a breaking change for normal bv xxx videos.

@tuta-gh
Copy link

tuta-gh commented Oct 21, 2024

(.conda) (base) etrade@etrade-cn:~/softing/you-get$ you-get 'https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2' -c cookies.sqlite --debug [DEBUG] get_content: https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2 [DEBUG] HTTP Error with code431 [DEBUG] HTTP Error with code431 [DEBUG] HTTP Error with code431 you-get: version 0.4.1730, a tiny downloader that scrapes the web. you-get: Namespace(version=False, help=False, info=False, url=False, json=False, no_merge=False, no_caption=False, postfix=False, prefix=None, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies='cookies.sqlite', timeout=600, debug=True, input_file=None, password=None, playlist=False, first=None, last=None, size=None, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, m3u8=False, URL=['https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2']) Traceback (most recent call last): File "/home/etrade/softing/you-get/.conda/bin/you-get", line 8, in <module> sys.exit(main()) ^^^^^^ File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/__main__.py", line 92, in main main(**kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1883, in main script_main(any_download, any_download_playlist, **kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1772, in script_main download_main( File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1386, in download_main download(url, **kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1874, in any_download m.download(url, **kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/extractor.py", line 48, in download_by_url self.prepare(**kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/extractors/bilibili.py", line 214, in prepare initial_state = json.loads(initial_state_text) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/etrade/softing/you-get/.conda/lib/python3.12/json/__init__.py", line 339, in loads raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not NoneType (.conda) (base) etrade@etrade-cn:~/softing/you-get$ you-get 'https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2' -c cookies.sqlite --debug [DEBUG] get_content: https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2 [DEBUG] HTTP Error with code431 [DEBUG] HTTP Error with code431 [DEBUG] HTTP Error with code431 you-get: version 0.4.1730, a tiny downloader that scrapes the web. you-get: Namespace(version=False, help=False, info=False, url=False, json=False, no_merge=False, no_caption=False, postfix=False, prefix=None, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies='cookies.sqlite', timeout=600, debug=True, input_file=None, password=None, playlist=False, first=None, last=None, size=None, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, m3u8=False, URL=['https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2']) Traceback (most recent call last): File "/home/etrade/softing/you-get/.conda/bin/you-get", line 8, in <module> sys.exit(main()) ^^^^^^ File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/__main__.py", line 92, in main main(**kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1883, in main script_main(any_download, any_download_playlist, **kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1772, in script_main download_main( File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1386, in download_main download(url, **kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/common.py", line 1874, in any_download m.download(url, **kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/extractor.py", line 48, in download_by_url self.prepare(**kwargs) File "/home/etrade/softing/you-get/.conda/lib/python3.12/site-packages/you_get/extractors/bilibili.py", line 214, in prepare initial_state = json.loads(initial_state_text) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/etrade/softing/you-get/.conda/lib/python3.12/json/__init__.py", line 339, in loads raise TypeError(f'the JSON object must be str, bytes or bytearray, ' TypeError: the JSON object must be str, bytes or bytearray, not NoneType
今天ubuntu上失败了

@gaoxingliang
Copy link
Contributor Author

gaoxingliang commented Oct 23, 2024

@tuta-gh you need to checkout my branch. - https://github.com/gaoxingliang/you-get/tree/develop
and I test the link you provided, it's working on my mac (not ubuntu. I guess it;s os independent)

@tuta-gh
Copy link

tuta-gh commented Oct 26, 2024

@tuta-gh you need to checkout my branch. - https://github.com/gaoxingliang/you-get/tree/develop and I test the link you provided, it's working on my mac (not ubuntu. I guess it;s os independent)
I have cloned this git. It still can't work..
(base) etrade@etrade-cn:/workspace/github/you-get-dev$ ./you-get 'https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2' -c cookies.sqlite --debug
[DEBUG] get_content: https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2
[DEBUG] HTTP Error with code431
[DEBUG] HTTP Error with code431
[DEBUG] HTTP Error with code431
you-get: version 0.4.1730-dev, a tiny downloader that scrapes the web.
you-get: Namespace(version=False, help=False, info=False, url=False, json=False, no_merge=False, no_caption=False, postfix=False, prefix=None, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies='cookies.sqlite', timeout=600, debug=True, input_file=None, password=None, playlist=False, first=None, last=None, size=None, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, m3u8=False, URL=['https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2'])
Traceback (most recent call last):
File "/home/etrade/workspace/github/you-get-dev/./you-get", line 11, in
you_get.main(repo_path=_filepath)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/main.py", line 92, in main
main(**kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1883, in main
script_main(any_download, any_download_playlist, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1772, in script_main
download_main(
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1386, in download_main
download(url, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1874, in any_download
m.download(url, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/extractor.py", line 48, in download_by_url
self.prepare(**kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/extractors/bilibili.py", line 216, in prepare
initial_state = json.loads(initial_state_text)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/etrade/miniconda3/lib/python3.12/json/init.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType
(base) etrade@etrade-cn:
/workspace/github/you-get-dev$
Note:
I change 0.4.1730 to 0.4.1730-dev to avoid use the shell's build but not github's build

2 similar comments
@tuta-gh
Copy link

tuta-gh commented Oct 26, 2024

@tuta-gh you need to checkout my branch. - https://github.com/gaoxingliang/you-get/tree/develop and I test the link you provided, it's working on my mac (not ubuntu. I guess it;s os independent)
I have cloned this git. It still can't work..
(base) etrade@etrade-cn:/workspace/github/you-get-dev$ ./you-get 'https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2' -c cookies.sqlite --debug
[DEBUG] get_content: https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2
[DEBUG] HTTP Error with code431
[DEBUG] HTTP Error with code431
[DEBUG] HTTP Error with code431
you-get: version 0.4.1730-dev, a tiny downloader that scrapes the web.
you-get: Namespace(version=False, help=False, info=False, url=False, json=False, no_merge=False, no_caption=False, postfix=False, prefix=None, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies='cookies.sqlite', timeout=600, debug=True, input_file=None, password=None, playlist=False, first=None, last=None, size=None, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, m3u8=False, URL=['https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2'])
Traceback (most recent call last):
File "/home/etrade/workspace/github/you-get-dev/./you-get", line 11, in
you_get.main(repo_path=_filepath)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/main.py", line 92, in main
main(**kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1883, in main
script_main(any_download, any_download_playlist, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1772, in script_main
download_main(
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1386, in download_main
download(url, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1874, in any_download
m.download(url, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/extractor.py", line 48, in download_by_url
self.prepare(**kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/extractors/bilibili.py", line 216, in prepare
initial_state = json.loads(initial_state_text)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/etrade/miniconda3/lib/python3.12/json/init.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType
(base) etrade@etrade-cn:
/workspace/github/you-get-dev$
Note:
I change 0.4.1730 to 0.4.1730-dev to avoid use the shell's build but not github's build

@tuta-gh
Copy link

tuta-gh commented Oct 26, 2024

@tuta-gh you need to checkout my branch. - https://github.com/gaoxingliang/you-get/tree/develop and I test the link you provided, it's working on my mac (not ubuntu. I guess it;s os independent)
I have cloned this git. It still can't work..
(base) etrade@etrade-cn:/workspace/github/you-get-dev$ ./you-get 'https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2' -c cookies.sqlite --debug
[DEBUG] get_content: https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2
[DEBUG] HTTP Error with code431
[DEBUG] HTTP Error with code431
[DEBUG] HTTP Error with code431
you-get: version 0.4.1730-dev, a tiny downloader that scrapes the web.
you-get: Namespace(version=False, help=False, info=False, url=False, json=False, no_merge=False, no_caption=False, postfix=False, prefix=None, force=False, skip_existing_file_size_check=False, format=None, output_filename=None, output_dir='.', player=None, cookies='cookies.sqlite', timeout=600, debug=True, input_file=None, password=None, playlist=False, first=None, last=None, size=None, auto_rename=False, insecure=False, http_proxy=None, extractor_proxy=None, no_proxy=False, socks_proxy=None, stream=None, itag=None, m3u8=False, URL=['https://www.bilibili.com/video/BV1KV411i7Ph?t=299.0&p=2'])
Traceback (most recent call last):
File "/home/etrade/workspace/github/you-get-dev/./you-get", line 11, in
you_get.main(repo_path=_filepath)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/main.py", line 92, in main
main(**kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1883, in main
script_main(any_download, any_download_playlist, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1772, in script_main
download_main(
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1386, in download_main
download(url, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/common.py", line 1874, in any_download
m.download(url, **kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/extractor.py", line 48, in download_by_url
self.prepare(**kwargs)
File "/home/etrade/workspace/github/you-get-dev/src/you_get/extractors/bilibili.py", line 216, in prepare
initial_state = json.loads(initial_state_text)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/etrade/miniconda3/lib/python3.12/json/init.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneType
(base) etrade@etrade-cn:
/workspace/github/you-get-dev$
Note:
I change 0.4.1730 to 0.4.1730-dev to avoid use the shell's build but not github's build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants