scripts/lb-wrapper: Try yt-dlp with "--format best --format-sort 'tbr~1000'" [testing shows need for URL validation, clearly communicated, for users who accidentally request the impossible after clicking "Download to IIAB" > "Start"]

[holta]

@deldesir @EMG70 please consider testing this alternative alongside:

• PR Sort formats [720p codec, viewing bitrate ~2 Mbit/s] #60
• PR scripts/lb-wrapper: Try "--format-sort='tbr~1000'" [i.e. target 1000 kbit/s codecs] #62

Building on:

• PR Pin max format [Set "best" <= 720p with VP9 codec for now: later consider 30FPS instead of 60FPS and WebM to tighten up further?] #48
• PR Use correct option [lb --write-thumbnail, and also fix logging explanation to match recent changes] #54

[holta]

@deldesir @EMG70 please consider testing this alternative alongside:

Start by testing these same 4 URL's ?

• scripts/lb-wrapper: Try "--format-sort='tbr~1000'" [i.e. target 1000 kbit/s codecs] #62 (comment)

[holta]

@EMG70 explanation is emerging at:

• "Tweaking Video Quality"
  https://github.com/iiab/calibre-web/wiki#tweaking-video-quality

(If this explanation contains shortcomings, please shout with your suggestions!)

[EMG70]

Latest test

[EMG70]

[EMG70]

[holta]

MKV Results may not yet be 100% ideal, but perhaps good enough for now?

• Overall bit rate 737 kbit/s

• Width 1920

• Height 1080

Binary Search Algorithm in 100 Seconds
https://www.youtube.com/watch?v=MFhxShGxHWc

[EMG70]

4 Things all great listeners know-Ted Ed

[EMG70]

Successful download of mp4 video but server crushed after refreshing.(403 Forbidden)

[EMG70]

[EMG70]

502 Bad Gateway upon entering http://ip/books

[EMG70]

[EMG70]

[EMG70]

[EMG70]

pastebinit -b sprunge.us//var/log/xklb.log
http://sprunge.us/BT3ZHs?en

pastebinit -b sprunge.us/var/log/calibre-web.log
http://sprunge.us/ohha35?en

[holta]

pastebinit -b sprunge.us//var/log/xklb.log
http://sprunge.us/BT3ZHs?en

@deldesir can you help us understand why Line 86 is very different from the others?

86 2023-12-09 13:49:03 - Running command: lb tubeadd /library/downloads/calibre-web/survey.db https://yt-dlp --format "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]" https://www.youtube.com/watch?v=sOE8Slo3Pqw" --verbose && lb dl /library/downloads/calibre-web/survey.db --prefix /library/downloads/calibre-web --write-thumbnail -f best -S 'tbr~1000' --video https://yt-dlp --format "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]" https://www.youtube.com/watch?v=sOE8Slo3Pqw" --verbose

[EMG70]

pastebinit -b sprunge.us//var/log/xklb.log
http://sprunge.us/BT3ZHs?en

@deldesir can you help us understand why Line 86 is very different from the others?

86 2023-12-09 13:49:03 - Running command: lb tubeadd /library/downloads/calibre-web/survey.db https://yt-dlp --format "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]" https://www.youtube.com/watch?v=sOE8Slo3Pqw" --verbose && lb dl /library/downloads/calibre-web/survey.db --prefix /library/downloads/calibre-web --write-thumbnail -f best -S 'tbr~1000' --video https://yt-dlp --format "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]" https://www.youtube.com/watch?v=sOE8Slo3Pqw" --verbose

CLARIFICATION****
Could Line 86 be a result of my error?I copied the following url from my Mp4 downloads yt-dlp -f “bestvideo[ext=mp4]+bestaudio[ext=m4a]” "https://www.youtube.com/watch?v=1xhlg9WkPQg."
I initially forgot to strip the yt-dlp prefix in calibre Download.This download however failed.I then noticed the error and stripped it correctly as and downloaded successfully as mp4 on Calibre.Apologies.

I thought the link was corrupted so tried another one yt-dlp --format "bv*[ext=mp4]+ba[ext=m4a]/b[ext=mp4]" https://www.youtube.com/watch?v=sOE8Slo3Pqw"
I realized this was 3.5hr video and immediately cancelled in Tasks.
Apologies for sending you on a wild chase but result was successful mp4 download 2min video though system later crushed after playing the video and could not refresh .

[holta]

Thanks @EMG70 for clarifying:

@deldesir can we learn anything from this?

(ASIDE: Certainly we need much stronger URL validation — when people click on "Download to IIAB" — so the front-end makes 100% clear telling them exactly what the back-end is and is not capable of, i.e. based on the URL they request!)

[holta]

@EMG70 advised me to go ahead and merge this after his extensive testing today:

Huge Thanks to him for his very hard work, confirming this PR is generally very solid i.e. an earnest real step forward helping schools/parents/kids getting the very manageable/tolerable codecs they need in offline and low-income communities!