Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to decode n-parameter: download likely to be throttled #31509

Closed
6 tasks done
paviad opened this issue Feb 1, 2023 · 25 comments
Closed
6 tasks done

Unable to decode n-parameter: download likely to be throttled #31509

paviad opened this issue Feb 1, 2023 · 25 comments
Labels
broken-IE problem with existing site extraction fixed

Comments

@paviad
Copy link

paviad commented Feb 1, 2023

Checklist

  • I'm reporting a broken site support issue
  • I've verified that I'm running youtube-dl version 2021.12.17
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar bug reports including closed ones
  • I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.youtube.com/watch?v=03Gug8rMTSI']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.9 (CPython) - Linux-5.10.102.1-microsoft-standard-WSL2-x86_64-with
[debug] exe versions: ffmpeg 5.1.2, ffprobe 5.1.2
[debug] Proxy map: {}
[youtube] 03Gug8rMTSI: Downloading webpage
[youtube] 03Gug8rMTSI: Downloading player dac945fd
WARNING: [youtube] Unable to decode n-parameter: download likely to be throttled (Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/youtube.py", line 1524, in _n_descramble
    raise ExtractorError('Unhandled exception in decode')
youtube_dl.utils.ExtractorError: Unhandled exception in decode; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
)
[debug] Default format spec: bestvideo+bestaudio/best
WARNING: Requested formats are incompatible for merge and will be merged into mkv.

Description

I was trying to download a simple short you tube video, nothing fancy. The tool issued the above output and requested that I submit a bug report, and as the good netizen I am, I have just submitted.

Regards

@ranelpadon
Copy link

Encountered the same issue also. Actually, yesterday it was working fine. Today, downloading YouTube/Facebook videos has issues for some reason. Reddit still works fine though.

@ranelpadon
Copy link

Looks like they already have idea of the issue/fix as per the recent discussion here: #30976 (comment).

@noembryo
Copy link

noembryo commented Feb 2, 2023

Re-installed from main now, and the time it takes to download the info of the video streams is x30 💢
I use the multi-threading example from #30641 to check.

@ranelpadon
Copy link

Nice, so the fix indeed worked. Thanks!

@noembryo
Copy link

noembryo commented Feb 3, 2023

If nothing more changes, it will work (slower) for single threaded downloads but for me that I use it in a multi threaded app, its unusable.. 😞

@dirkf
Copy link
Contributor

dirkf commented Feb 4, 2023

Nothing significant has changed in the mini JS interpreter. Perhaps the n-sig code being sent by YT is now more demanding.

@noembryo
Copy link

noembryo commented Feb 4, 2023

Yes, maybe it is this.
But, testing the same script using the current yt-dlp needs ~7sec, where yt-dl needs ~210sec.

@rautamiekka
Copy link
Contributor

Do you use the same Python 3 version for both ?

@noembryo
Copy link

noembryo commented Feb 4, 2023

Nope. The reason I have to use yt-dl and not yt-dlp, is that my app uses Python 2.
So the real test for me is Python 2 with yt-dl vs Python 3 with yt-dlp.
But even if I use Python 3 for both, I get yt-dlp ~7sec vs yt-dl ~60sec.

@dirkf
Copy link
Contributor

dirkf commented Feb 4, 2023

An order of magnitude difference, let alone x30, is quite surprising.

A simple test of yt-dl git master against yt-dlp 2022.11.22 (ie not fixed) has this:

Interpreter test (test/test_jsinterp.py): exercises supported JS syntax.

Py2.7/yt-dl    Py3.9/yt-dl    Py3.9/yt-dlp
0.53           0.58           0.48

In the yt-dlp case there are slightly fewer test cases.

Signature test (test/test_youtube_signature.py): verifies n-sig and URL signature processing including n-sig processing for all known player JS.

Py2.7/yt-dl    Py3.9/yt-dl    Py3.9/yt-dlp
44.1           28.1           25.4

In the yt-dlp case there is one fewer test case (27 vs 28).

Probably my Python3.3 and lower shim for ChainMap, which is used to store variable bindings, is slower than the built-in.

@noembryo
Copy link

noembryo commented Feb 4, 2023

I'm testing the speed with the multi-threading script from #30641 in a AMD Ryzen 7 5700G 3.8GHz system using Python 2.7 and 3.8.
Can you check it too?

Edit: using yt-dlp 2023.1.6

@rautamiekka
Copy link
Contributor

Nope. The reason I have to use yt-dl and not yt-dlp, is that my app uses Python 2. So the real test for me is Python 2 with yt-dl vs Python 3 with yt-dlp. But even if I use Python 3 for both, I get yt-dlp ~7sec vs yt-dl ~60sec.

Py3 is mostly way faster anyway, especially as of 3.11, and ytdlp upgraded to many faster replacements, so 7 vs 60 makes a lotta sense.

@noembryo
Copy link

noembryo commented Feb 4, 2023

The answer to which version of Python is the fastest, is not that simple.
I can do many comparisons between them and most of the time P2 is faster.
So, for me the answer is "that depends"..
Googling about it gave me a general 1.19x speedier P3, but even this does not explains the ~9x difference I see here.
And I'm not talking about 3.11. I do the tests with 3.7 & 3.8.

@ranelpadon
Copy link

BTW, I'm using Py3.11, the fix removed the throttling. Interestingly, the yt-dlp (2023.1.6) is still a bit faster than youtube-dl (384f632e commit):

  • youtube-dl (2.60s)
λ youtube-dl --format mp4 https://www.youtube.com/watch\?v\=mTQorpZnSIk

[youtube] mTQorpZnSIk: Downloading webpage
[download] Destination: Avatar - The Way of Water _ #1 for 7 Weeks-mTQorpZnSIk.mp4
[download] 100% of 3.31MiB in 00:00
  • yt-dlp (1.55s)
λ yt-dlp --format mp4 https://www.youtube.com/watch\?v\=mTQorpZnSIk

[youtube] Extracting URL: https://www.youtube.com/watch?v=mTQorpZnSIk
[youtube] mTQorpZnSIk: Downloading webpage
[youtube] mTQorpZnSIk: Downloading android player API JSON
[info] mTQorpZnSIk: Downloading 1 format(s): 22
[download] Destination: Avatar: The Way of Water | #1 for 7 Weeks [mTQorpZnSIk].mp4
[download] 100% of    3.31MiB in 00:00:00 at 8.24MiB/s

@noembryo
Copy link

noembryo commented Feb 4, 2023

As I wrote already, my testing is done with the #30641 script..
Can you give it a try and report your findings?

@dirkf dirkf mentioned this issue Feb 7, 2023
@dirkf dirkf added broken-IE problem with existing site extraction fixed labels Feb 7, 2023
@pukkandan
Copy link
Contributor

The reason yt-dlp appears faster was already discussed #30641 (comment)

@noembryo
Copy link

@pukkandan Yes, OK. but the problem remains.
Maybe yt-dl should use a similar strategy?
Anyway, I'm almost done porting my app to Python 3..

alxlive pushed a commit to alxlive/youtube-dl that referenced this issue Feb 27, 2023
Fix for yt-dlp/yt_dlp#6131
Closes ytdl-org#31509
@jenskutilek
Copy link

I still get a similar error with the current GitHub version (fa7f0ef):

youtube-dl --verbose -f 22 HG9kCMU24x8                                                                                         1
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-f', '22', 'HG9kCMU24x8']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.11.0 (CPython) - macOS-13.4-arm64-arm-64bit
[debug] exe versions: ffmpeg N-108460-g5c94b6694a, ffprobe N-108460-g5c94b6694a
[debug] Proxy map: {}
[youtube] HG9kCMU24x8: Downloading webpage
[youtube] HG9kCMU24x8: Downloading player b7910ca8
WARNING: [youtube] Unable to decode n-parameter: download likely to be throttled (Unable to extract Initial JS player n function alias (Usa[0]); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last):
  File "/Users/kuti/Library/Python/3.11/lib/python/site-packages/youtube_dl/extractor/youtube.py", line 1668, in _n_descramble
    self._player_cache[player_id] = self._extract_n_function(video_id, player_url)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../Library/Python/3.11/lib/python/site-packages/youtube_dl/extractor/youtube.py", line 1640, in _extract_n_function
    funcname = self._extract_n_function_name(jscode)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../Library/Python/3.11/lib/python/site-packages/youtube_dl/extractor/youtube.py", line 1623, in _extract_n_function_name
    real_nfunc = self._search_regex(
                 ^^^^^^^^^^^^^^^^^^^
  File "/Users/.../Library/Python/3.11/lib/python/site-packages/youtube_dl/extractor/common.py", line 1013, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract Initial JS player n function alias (Usa[0]); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
)
[debug] Invoking downloader on 'https://rr5---sn-i5h7lnl6.googlevideo.com/videoplayback?expire=1687526406&ei=pUeVZNGMPImSgQeU-5SoAg&ip=2.205.60.224&id=o-ACbhyt6XInuSC6_ugZ-OcYRN9cj-Fou47e8XnlHcjWSq&itag=22&source=youtube&requiressl=yes&mh=bU&mm=31%2C29&mn=sn-i5h7lnl6%2Csn-i5heen7z&ms=au%2Crdu&mv=m&mvi=5&pl=21&initcwndbps=1060000&bui=AZ6lgGGg2ku3kUg-xvewhJ0Xi_fNmMSLCTki5ZZUKRgBHePr2nbIb-ZVYmM1x9Je8rwqxa_p5R6lUQh2WKH33SO1wfcmq6WE&spc=qEK7B3Ucg15SW3qVlneKssToSvkcdSHwqMdvsA8Ycg&vprv=1&svpuc=1&mime=video%2Fmp4&ns=nrirAu3an4N3odiiiF9p5ncN&cnr=14&ratebypass=yes&dur=29568.801&lmt=1686871739115562&mt=1687504378&fvip=1&fexp=24007246%2C51000014%2C51000022&c=WEB&txp=7218224&n=nQ1yjep762mMQ1f&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cbui%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Ccnr%2Cratebypass%2Cdur%2Clmt&sig=AOq0QJ8wRQIhAMWvExbpyxsHx2Fl8UiPDwizHLwvSo9liwIwgyqNJC1aAiBtPOdv_lL-5mFfjnDcb0ytKrNmJJlSmc9C0JDq-hPD8A%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhANMAPs4M7_I7AIBqtomOH8y6QCfW92aGuUQENnkoLC4xAiEAqWczwsxOkbTzfFx9QNhevPc73di6RC0CVAOfZNg4xDc%3D'
[download] Resuming download at byte 1089231
[download] Destination: TypeLab 2023 Europe Day 2-HG9kCMU24x8.mp4
[download]   0.1% of 3.13GiB at 77.66KiB/s ETA 11:44:52

@dirkf
Copy link
Contributor

dirkf commented Jun 23, 2023

You're running the old code somehow. There's no OpenSSL diagnostic on the Python version line.

Make sure your pip command includes --force-reinstall and is run at the same privilege as the original installation, or follow "none of the suggested update methods work for me".

I always find it a weird journey of discovery trying to get stuff running on macOS, worrying that some funky Apple security stuff will block my progress; logs may show what's happening.

What should happen:

$ python -m youtube_dl -v -F  HG9kCMU24x8
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'HG9kCMU24x8']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 58fc5bde4
[debug] Python 2.7.18 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial - OpenSSL 1.1.1t  7 Feb 2023 - glibc 2.15
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[youtube] HG9kCMU24x8: Downloading webpage
[youtube] HG9kCMU24x8: Downloading player b7910ca8
[debug] [youtube] Decrypted nsig AxT9rrznZbDk4B3 => w_CGHX-e5hj9lQ
[debug] [youtube] Decrypted nsig gCqFS-lmLWRjUIE => oopF5idGnyCQzQ
[info] Available formats for HG9kCMU24x8:
format code  extension  resolution note
251          webm       audio only tiny   85k , webm_dash container, opus  (48000Hz), 299.83MiB
140          m4a        audio only tiny  129k , m4a_dash container, mp4a.40.2 (44100Hz), 456.37MiB
160          mp4        256x144    144p   53k , mp4_dash container, avc1.4d400c, 30fps, video only, 187.83MiB
243          webm       640x360    360p  185k , webm_dash container, vp9, 30fps, video only, 653.54MiB
134          mp4        640x360    360p  220k , mp4_dash container, avc1.4d401e, 30fps, video only, 777.63MiB
136          mp4        1280x720   720p  781k , mp4_dash container, avc1.64001f, 30fps, video only, 2.69GiB
18           mp4        640x360    360p  315k , avc1.42001E, 30fps, mp4a.40.2 (44100Hz), 1.08GiB
22           mp4        1280x720   720p  910k , avc1.64001F, 30fps, mp4a.40.2 (44100Hz) (best)
$

@jenskutilek
Copy link

D'oh, I have two versions of Python installed, 3.10 and 3.11. pip calls the Python 3.10 pip, while the youtube_dl command finds an older version installed in Python 3.11 first in the path.

After uninstalling the old version, everything works. Thanks for your help!

@dirkf
Copy link
Contributor

dirkf commented Jun 23, 2023

python3.11 -m pip ... or python3.10 -m pip ... rather than just pip ...

@JBlocklove
Copy link

Just tried reinstalling from the instructions here: #30871 (comment) and I'm running into this issue. I had run into this issue a couple of months ago and doing this fixed it but now the problem is back and installing from the master branch here doesn't seem to solve it for me.

@dirkf
Copy link
Contributor

dirkf commented Jul 6, 2023

Best to review the instructions in #31530 rather than issue necromancy (yes, @jenskutilek too!). Also, --force-reinstall, but there is now a nightly build release that you can use to avoid that.

@Tectract
Copy link

Oh nooooo, this bug is back from the dead. My Youtube-DL worked just fine until today... Now I'm getting:

WARNING: [youtube] Unable to decode n-parameter

@dirkf
Copy link
Contributor

dirkf commented Jul 15, 2024

#32842

@ytdl-org ytdl-org locked as resolved and limited conversation to collaborators Jul 15, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
broken-IE problem with existing site extraction fixed
Projects
None yet
Development

No branches or pull requests

9 participants