Youtube live playlist request is extremely slow #5045

Closed
opened 2026-02-21 00:31:01 -05:00 by deekerman · 6 comments
Owner

Originally created by @jhb50 on GitHub (Jun 29, 2015).

This "Live now-Animals" playlist link obtained from https://www.youtube.com/channel/UC4R8DWoMoI7CAwX8_LjQHig returns 50 youtube live video streams.
"C:\Program Files\Serviio\lib\youtube-dl.exe " --max-downloads 50 -j https://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyNXhloQ
The problem is it takes 400 seconds!
A direct request to youtube with https://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyNXhloQ is instananeous and selected videos play immediately.
Why should youtube-dl take any longer

Originally created by @jhb50 on GitHub (Jun 29, 2015). This "Live now-Animals" playlist link obtained from https://www.youtube.com/channel/UC4R8DWoMoI7CAwX8_LjQHig returns 50 youtube live video streams. "C:\Program Files\Serviio\lib\youtube-dl.exe " --max-downloads 50 -j https://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyNXhloQ The problem is it takes 400 seconds! A direct request to youtube with https://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyNXhloQ is instananeous and selected videos play immediately. Why should youtube-dl take any longer
Author
Owner

@dstftw commented on GitHub (Jun 29, 2015):

Post the full output of youtube-dl when run with --verbose option.
What do you expect from playlist extraction at all? It extracts all metadata for every video in playlist therefore it can't be fast.

@dstftw commented on GitHub (Jun 29, 2015): Post the **full** output of youtube-dl when run [with `--verbose` option](https://github.com/rg3/youtube-dl/blob/master/README.md#bugs). What do you expect from playlist extraction at all? It extracts **all** metadata for **every** video in playlist therefore it can't be fast.
Author
Owner

@jhb50 commented on GitHub (Jun 29, 2015):

@ sergeyM - you responded "There is no such option --max-downloads in youtube-dl."
Note that youtube-dl --help shows option " --max-downloads NUMBER Abort after downloading NUMBER files" and it works correctly for me.

@ dstfw - Yes I am looking for "all metadata for every video in playlist " which I then parse in my Serviio YouTube plugin to select the feed I want to play. Youtube-dl does return all the metadata successfully with the --verbose messages below.

The issue is that a "normal" playlist (the second one below with 50 celine songs) returns 1,252KB of metadata in 36 seconds, (less than a second for each video) but the "live" playlist (the first one below of 50 Live animals) returns 993KB of metadata in 400 seconds (10 times as long!). The same problem occurs if I give YouTube-dl a list of the 50 live video-ids rather than the playlist itself, so the problem is in the time youtube-dl is taking (~8 seconds) to extract each live video, where-as with non-live video-ids or using the browser directly, access to this metadata is instantaneous.

I would appreciate if you could investigate the reason for the ~8 second delay.

Thanks

C:\Users\John>"C:\Program Files\Serviio\lib\youtube-dl.exe " --verbose --max-dow
nloads 50 -j https://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyN
XhloQ > D:liveout.txt
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'--max-downloads', u'50', u'-j', u'ht
tps://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyNXhloQ']
[debug] Encodings: locale cp1252, fs mbcs, out None, pref cp1252
[debug] youtube-dl version 2015.06.25
[debug] Python version 2.7.8 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-67331-g547fce9, ffprobe N-67331-g547fce9, rtmpdum
p 2.4
[debug] Proxy map: {}

C:\Users\John> "C:\Program Files\Serviio\lib\youtube-dl.exe " --verbose --max-do
wnloads 50 -j https://www.youtube.com/playlist?list=PLVNmwycFozUGB6b6fKcldWyfmZR
4RZIJj > D:celineout.txt
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'--max-downloads', u'50', u'-j', u'ht
tps://www.youtube.com/playlist?list=PLVNmwycFozUGB6b6fKcldWyfmZR4RZIJj']
[debug] Encodings: locale cp1252, fs mbcs, out None, pref cp1252
[debug] youtube-dl version 2015.06.25
[debug] Python version 2.7.8 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-67331-g547fce9, ffprobe N-67331-g547fce9, rtmpdum
p 2.4
[debug] Proxy map: {}

@jhb50 commented on GitHub (Jun 29, 2015): @ sergeyM - you responded "There is no such option --max-downloads in youtube-dl." Note that youtube-dl --help shows option " --max-downloads NUMBER Abort after downloading NUMBER files" and it works correctly for me. @ dstfw - Yes I am looking for "all metadata for every video in playlist " which I then parse in my Serviio YouTube plugin to select the feed I want to play. Youtube-dl does return all the metadata successfully with the --verbose messages below. The issue is that a "normal" playlist (the second one below with 50 celine songs) returns 1,252KB of metadata in 36 seconds, (less than a second for each video) but the "live" playlist (the first one below of 50 Live animals) returns 993KB of metadata in 400 seconds (10 times as long!). The same problem occurs if I give YouTube-dl a list of the 50 live video-ids rather than the playlist itself, so the problem is in the time youtube-dl is taking (~8 seconds) to extract each live video, where-as with non-live video-ids or using the browser directly, access to this metadata is instantaneous. I would appreciate if you could investigate the reason for the ~8 second delay. Thanks C:\Users\John>"C:\Program Files\Serviio\lib\youtube-dl.exe " --verbose --max-dow nloads 50 -j https://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyN XhloQ > D:liveout.txt [debug] System config: [] [debug] User config: [] [debug] Command-line args: [u'--verbose', u'--max-downloads', u'50', u'-j', u'ht tps://www.youtube.com/playlist?list=PLIFqWCuxNyohg709Oqz-EvNoudyNXhloQ'] [debug] Encodings: locale cp1252, fs mbcs, out None, pref cp1252 [debug] youtube-dl version 2015.06.25 [debug] Python version 2.7.8 - Windows-7-6.1.7601-SP1 [debug] exe versions: ffmpeg N-67331-g547fce9, ffprobe N-67331-g547fce9, rtmpdum p 2.4 [debug] Proxy map: {} C:\Users\John> "C:\Program Files\Serviio\lib\youtube-dl.exe " --verbose --max-do wnloads 50 -j https://www.youtube.com/playlist?list=PLVNmwycFozUGB6b6fKcldWyfmZR 4RZIJj > D:celineout.txt [debug] System config: [] [debug] User config: [] [debug] Command-line args: [u'--verbose', u'--max-downloads', u'50', u'-j', u'ht tps://www.youtube.com/playlist?list=PLVNmwycFozUGB6b6fKcldWyfmZR4RZIJj'] [debug] Encodings: locale cp1252, fs mbcs, out None, pref cp1252 [debug] youtube-dl version 2015.06.25 [debug] Python version 2.7.8 - Windows-7-6.1.7601-SP1 [debug] exe versions: ffmpeg N-67331-g547fce9, ffprobe N-67331-g547fce9, rtmpdum p 2.4 [debug] Proxy map: {}
Author
Owner

@dstftw commented on GitHub (Jun 29, 2015):

DASH manifests for lives are rather heavy (1-2 Mb each comparing to 15 Kb for regular videos). You may want to skip downloading DASH manifests with --youtube-skip-dash-manifest.

@dstftw commented on GitHub (Jun 29, 2015): DASH manifests for lives are rather heavy (1-2 Mb each comparing to 15 Kb for regular videos). You may want to skip downloading DASH manifests with `--youtube-skip-dash-manifest`.
Author
Owner

@dstftw commented on GitHub (Jun 29, 2015):

After f0714c9 it should be even faster.

@dstftw commented on GitHub (Jun 29, 2015): After f0714c9 it should be even faster.
Author
Owner

@jhb50 commented on GitHub (Jun 29, 2015):

Thanks. skip-dash reduced time to ~240 seconds. Awaiting update with f0714c9 !
Appreciate your fast response.

@jhb50 commented on GitHub (Jun 29, 2015): Thanks. skip-dash reduced time to ~240 seconds. Awaiting update with f0714c9 ! Appreciate your fast response.
Author
Owner

@jhb50 commented on GitHub (Jun 30, 2015):

Final results
Original time ~ 400 seconds
with Skip dash ~ 240 seconds
with Improved regex ~ 285 seconds
with Skip dash + improved regex ~ 93 seconds
Outstanding! Thanks guys!

@jhb50 commented on GitHub (Jun 30, 2015): Final results Original time ~ 400 seconds with Skip dash ~ 240 seconds with Improved regex ~ 285 seconds with Skip dash + improved regex ~ 93 seconds Outstanding! Thanks guys!
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/youtube-dl#5045
No description provided.