[YouTube] Unable to extract nsig jsi ... #32842
Comments
|
Thanks for the report. YT have now discovered that JS This should be fairly easy to fix, though maybe there's some more nonsense that I haven't yet noticed . |
11 tasks
dirkf
changed the title
|
Indeed, now it's been fixed twice: $ python -m youtube_dl -vF zPHM0q0xgFg
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-vF', u'zPHM0q0xgFg']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 0b2907665
[debug] Python 2.7.15 (CPython i686 32bit) - Linux-6.1.0-22-686-pae-i686-with-debian-12.6 - OpenSSL 1.1.1a 20 Nov 2018 - glibc 2.1.3
[debug] exe versions: ffmpeg 5.1.5-0, ffprobe 5.1.5-0
[debug] Proxy map: {}
[youtube] zPHM0q0xgFg: Downloading webpage
[debug] [youtube] Decrypted nsig M2_bGJLKmHc2Zr7Bh => rV2Hqx1WU9cpuw
[debug] [youtube] Decrypted nsig 7EPTM2rsj4Z306LCZ => cYnQHsXJW70IRA
[info] Available formats for zPHM0q0xgFg:
format code extension resolution note
249 webm audio only audio_quality_low 49k , webm_dash container, opus (48000Hz), 2.10MiB
250 webm audio only audio_quality_low 64k , webm_dash container, opus (48000Hz), 2.75MiB
251 webm audio only audio_quality_medium 127k , webm_dash container, opus (48000Hz), 5.42MiB
140 m4a audio only audio_quality_medium 129k , m4a_dash container, mp4a.40.2 (44100Hz), 5.51MiB
160 mp4 256x144 144p 13k , mp4_dash container, avc1.4d400c, 30fps, video only, 573.67KiB
394 mp4 256x144 144p 18k , mp4_dash container, av01.0.00M.08, 30fps, video only, 823.95KiB
278 webm 256x144 144p 22k , webm_dash container, vp9, 30fps, video only, 985.53KiB
133 mp4 426x240 240p 20k , mp4_dash container, avc1.4d4015, 30fps, video only, 886.64KiB
395 mp4 426x240 240p 27k , mp4_dash container, av01.0.00M.08, 30fps, video only, 1.19MiB
242 webm 426x240 240p 32k , webm_dash container, vp9, 30fps, video only, 1.40MiB
134 mp4 640x360 360p 30k , mp4_dash container, avc1.4d401e, 30fps, video only, 1.28MiB
396 mp4 640x360 360p 46k , mp4_dash container, av01.0.01M.08, 30fps, video only, 1.96MiB
243 webm 640x360 360p 57k , webm_dash container, vp9, 30fps, video only, 2.46MiB
135 mp4 854x480 480p 40k , mp4_dash container, avc1.4d401f, 30fps, video only, 1.74MiB
397 mp4 854x480 480p 59k , mp4_dash container, av01.0.04M.08, 30fps, video only, 2.55MiB
244 webm 854x480 480p 78k , webm_dash container, vp9, 30fps, video only, 3.33MiB
136 mp4 1280x720 720p 85k , mp4_dash container, avc1.4d401f, 30fps, video only, 3.63MiB
398 mp4 1280x720 720p60 117k , mp4_dash container, av01.0.08M.08, 60fps, video only, 5.01MiB
298 mp4 1280x720 720p60 121k , mp4_dash container, avc1.4d4020, 60fps, video only, 5.16MiB
247 webm 1280x720 720p 129k , webm_dash container, vp9, 30fps, video only, 5.49MiB
302 webm 1280x720 720p60 167k , webm_dash container, vp9, 60fps, video only, 7.15MiB
399 mp4 1920x1080 1080p60 184k , mp4_dash container, av01.0.09M.08, 60fps, video only, 7.84MiB
299 mp4 1920x1080 1080p60 192k , mp4_dash container, avc1.64002a, 60fps, video only, 8.20MiB
303 webm 1920x1080 1080p60 245k , webm_dash container, vp9, 60fps, video only, 10.46MiB
18 mp4 640x360 360p 160k , avc1.42001E, 30fps, mp4a.40.2 (44100Hz), 6.81MiB (best)
$ And yt-dlp/yt-dlp#10397 (comment) too. I'll combine that in the upcoming yt-dl PR. |
|
I have identically looking error. Full, verbose dump here. PS.
To devs: would you mind to add small note about nightly builds for youtube-dl, possibly in I was thinking about writing separate issue about it, and use easy to remember number like 32844, that is coming soon but maybe better to put it in #30839. Or maybe both. |
This comment was marked as outdated.
This comment was marked as outdated.
|
I will always provide logs for both normal and nightly in the youtube-specific issues I post, both of which from a freshly pulled and compiled instance of youtube-dl and youtube-dl-nightly, and since i use nightly to download all the stuff that I watch, I am quite quick at noticing errors that happen in both at least the same day they happen. ^^ Can't wait for this to be fixed/released, luckily i got a backlog of videos XD |
dirkf
added
broken-IE
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Since I suspect this might cause you problems, please note that applying the 2nd commit without the 3rd commit will cause yt-dlp (and presumably also YouTube-dl) to incorrectly cache the new player, meaning nsig extraction will fail until you run --rm-cache-dir |
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as off-topic.
This comment was marked as off-topic.
A problem I have previously encountered and also while debugging the fix that generated the log above. I just nullify the result of loading the bad cached code. |
patched functionsdef _extract_n_function_name(self, jscode):
# func_name, idx = self._search_regex(
# r'\.get\("n"\)\)&&\(b=(?P<nfunc>[a-zA-Z_$][\w$]*)(?:\[(?P<idx>\d+)\])?\([\w$]+\)',
# jscode, 'Initial JS player n function name', group=('nfunc', 'idx'))
# if not idx:
# return func_name
func_name, idx = self._search_regex(r'\.get\(b\)\)&&\(c=(?P<nfunc>[a-zA-Z_$][\w$]*)(?:\[(?P<idx>\d+)\])?\([\w$]+\)', jscode, 'Initial JS player n function name', group=('nfunc', 'idx'))
if not idx:
return func_name
return self._parse_json(self._search_regex(
r'var {0}\s*=\s*(\[.+?\])\s*[,;]'.format(re.escape(func_name)), jscode,
'Initial JS player n function list ({0}.{1})'.format(func_name, idx)),
func_name, transform_source=js_to_json)[int(idx)]def _extract_n_function_code(self, video_id, player_url):
player_id = self._extract_player_info(player_url)
func_code = self.cache.load('youtube-nsig', player_id)
jscode = func_code or self._load_player(video_id, player_url)
jsi = JSInterpreter(jscode)
if func_code:
return jsi, player_id, func_code
func_name = self._extract_n_function_name(jscode)
# For redundancy
# func_code = self._search_regex(
# r'''(?xs)%s\s*=\s*function\s*\((?P<var>[\w$]+)\)\s*
# # NB: The end of the regex is intentionally kept strict
# {(?P<code>.+?}\s*return\ [\w$]+.join\(""\))};''' % func_name,
# jscode, 'nsig function', group=('var', 'code'), default=None)
func_code = self._search_regex(r'''(?xs)%s\s*=\s*function\s*\((?P<var>[\w$]+)\)\s*{(?P<code>.+?}\s*return\ Array\.prototype\.join\.call\([\w$]+,\s*""\))};''' % func_name, jscode, 'nsig function', group=('var', 'code'), default=None)
var_name = ''
match_var = re.search(r'var (\w+)=String\.prototype\.split\.call', func_code[1])
if match_var:
var_name = match_var.group(1)
temp_func_code = func_code[1].replace('String.prototype.split.call(a,"")', func_code[0] + '.split(\'\')')
temp_func_code = temp_func_code.replace('Array.prototype.join.call(' + var_name + ',"")', var_name + '.join(\'\')')
func_code = (func_code[0], temp_func_code)
if func_code:
func_code = ([func_code[0]], func_code[1])
else:
self.write_debug('Extracting nsig function with jsinterp')
func_code = jsi.extract_function_code(func_name)
self.cache.store('youtube-nsig', player_id, func_code)
return jsi, player_id, func_code
i can't commit now but if somebody can, please do it, this two function is located in youtube.py patch with two functions for better code optimizationdef _extract_n_function_name(self, jscode):
func_name, idx = self._search_regex(
r'\.get\(b\)\)&&\(c=(?P<nfunc>[a-zA-Z_$][\w$]*)(?:\[(?P<idx>\d+)\])?\([\w$]+\)',
jscode, 'Initial JS player n function name', group=('nfunc', 'idx'))
if not idx:
return func_name
return self._parse_json(self._search_regex(
r'var {0}\s*=\s*(\[.+?\])\s*[,;]'.format(re.escape(func_name)), jscode,
'Initial JS player n function list ({0}.{1})'.format(func_name, idx)),
func_name, transform_source=js_to_json)[int(idx)]
def _extract_var_name(func_code):
match_var = re.search(r'var (\w+)=String\.prototype\.split\.call', func_code)
if match_var:
return match_var.group(1)
return None
def _replace_func_code(func_code, var_input, var_name):
temp_func_code = func_code.replace('String.prototype.split.call(' + var_input + ',"")', func_code.split('=')[0] + '.split(\'\')')
if var_name:
temp_func_code = temp_func_code.replace(f'Array.prototype.join.call({var_name},"")', f'{var_name}.join(\'\')')
return temp_func_code
def _extract_n_function_code(self, video_id, player_url):
player_id = self._extract_player_info(player_url)
func_code = self.cache.load('youtube-nsig', player_id)
jscode = func_code or self._load_player(video_id, player_url)
jsi = JSInterpreter(jscode)
if func_code:
return jsi, player_id, func_code
func_name = self._extract_n_function_name(jscode)
func_code = self._search_regex(
r'''(?xs)%s\s=\sfunction\s\((?P<var>[\w$]+)\)\s{(?P<code>.+?}\sreturn\ Array\.prototype\.join\.call\([\w$]+,\s""\))};''' % func_name,
jscode, 'nsig function', group=('var', 'code'), default=None)
if func_code:
var_name = _extract_var_name(func_code[1])
temp_func_code = _replace_func_code(func_code[1], func_code[0], var_name)
func_code = (func_code[0], temp_func_code)
else:
self.write_debug('Extracting nsig function with jsinterp')
func_code = jsi.extract_function_code(func_name)
self.cache.store('youtube-nsig', player_id, func_code)
return jsi, player_id, func_code |
6 tasks
|
@Duster98 thanks, but as you can see from #32842 (comment) a fix is already in progress. Your approach looks like it solves the immediate problem and changes only one file but doesn't allow for future |
Tried to test it but I'm unsure about lay two lines and their indentation. Would you mind to to attach or put in paste a diff/patch (`diff -rup youtube.py~ youtube.py > youtube.py.diff`)? Ed. Managed to run the 'first', 'unoptimized' version and it works. Diff here. If that's your will, and devs agree can request a pull. |
|
@tansy this is the file: youtube.py If you can kindly test it so I can understand if at least temporarily the problem is solved |
This comment was marked as outdated.
This comment was marked as outdated.
tansy
added a commit
to tansy/youtube-dl
that referenced
this issue
Jul 10, 2024
thanks to @Duster98 #issuecomment-2220376175
This comment was marked as outdated.
This comment was marked as outdated.
3 tasks
This comment was marked as outdated.
This comment was marked as outdated.
11 tasks
11 tasks
This comment was marked as resolved.
This comment was marked as resolved.
|
Just came back to say, it works for me with both normal and nightly now, and consider this resolved for me, but I do compile it myself manually, so I can't tell for the things other people use. ^^ |
This comment was marked as resolved.
This comment was marked as resolved.
This was referenced Jul 11, 2024
|
To reiterate the solution, the problem is fixed in the master code and builds of 2024--7-11 or later. You can install a fixed release from the nightly builds at ytdl-org/ytdl-nightly. If you are already running a nightly release, and depending on how you installed it, you may be able to update using If not:
Or if you are using Homebrew, you can install from the latest master code at that time using something like |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
6 tasks
Any chance you could tag a new release in this repo? Homebrew plans to disable |
|
There have been releases since last year. |
|
I was occurred the mentioned error during installation by dnf package manager. But it's resolved by installing with make file. |
|
After upgrading to the latest nightly build, this issue was initially resolved but has since returned. I've followed the instructions in the rest of this thread:
both of these options return the same error. Note I am using a cookies.txt file, I've tried recreating that with a new session, but that also hasn't resolved the issue. I've also tried this on a few youtube videos, all public, all return the same issue. I've included the verbose debug output below, let me know if there is any other info I can provide. |
|
Thanks for the report. I can't initially repro here but I am getting player |
|
A new circumlocution for This patch addresses the problem: def _extract_n_function_name(self, jscode):
func_name, idx = self._search_regex(
# new: (b=String.fromCharCode(110),c=a.get(b))&&c=nfunc[idx](c)
+ # or: (b="nn"[+a.D],c=a.get(b))&&(c=nfunc[idx](c),
# old: .get("n"))&&(b=nfunc[idx](b)
# older: .get("n"))&&(b=nfunc(b)
r'''(?x)
- (?:\(\s*(?P<b>[a-z])\s*=\s*String\s*\.\s*fromCharCode\s*\(\s*110\s*\)\s*,(?P<c>[a-z])\s*=\s*[a-z]\s*)?
- \.\s*get\s*\(\s*(?(b)(?P=b)|"n")(?:\s*\)){2}\s*&&\s*\(\s*(?(c)(?P=c)|b)\s*=\s*
+ (?:\(\s*(?P<b>[a-z])\s*=\s*(?:
+ String\s*\.\s*fromCharCode\s*\(\s*110\s*\)|
+ "n+"\[\s*\+?s*[\w$.]+\s*]
+ )\s*,(?P<c>[a-z])\s*=\s*[a-z]\s*)?
+ \.\s*get\s*\(\s*(?(b)(?P=b)|"n{1,2}")(?:\s*\)){2}\s*&&\s*\(\s*(?(c)(?P=c)|b)\s*=\s*
(?P<nfunc>[a-zA-Z_$][\w$]*)(?:\s*\[(?P<idx>\d+)\])?\s*\(\s*[\w$]+\s*\)
''', jscode, 'Initial JS player n function name', group=('nfunc', 'idx'))
if not idx: |
|
Also fixed in yt-dlp/yt-dlp#10542. |
This was referenced Jul 24, 2024
|
I am only here because I am providing Logs, I saw the backlog and just wanted to make sure Data is being delivered. ^^ Edit: by the way it is still broken for player 1f8742dc |
This comment was marked as duplicate.
This comment was marked as duplicate.
|
Player |
3 tasks
|
I'm using e1b3fa2 and still seeing this issue: Perhaps yet another player needs this? |
|
As per @derekatkins, I'm seeing the same error for player 20dfca59: This is a source build against latest master branch, with following commit: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Checklist
Verbose log
Description
New Error Message on Youtube, yay, time to provide the Devs with the Logs!
Unable to extract nsig jsi, player_id, func_codefunction code (caused by RegexNotFoundError('Unable to extract \x1b[0;34mInitial JS player n function name\x1b[0m;
The text was updated successfully, but these errors were encountered: