youtube-dl dumps wrong URL for YouTube videos #1951

Closed
opened 2026-02-21 00:10:24 -05:00 by deekerman · 6 comments
Owner

Originally created by @hdclark on GitHub (Mar 3, 2014).

Description

youtube-dl will properly download YouTube videos, but the [-g] option spits out URLs to different videos. Wget, cURL, and mplayer download the wrong video. Behaviour does not vary when changing user-agent in any four (or any combination of the four) programs.

Most youtube links will result in the same video being downloaded (https://www.youtube.com/watch?v=5mPDhmP6ztQ). Behaviour happens across several machines (32 and 64 bit) at different addresses.

Minimal test program

!/bin/sh

BAREURL="$( youtube-dl -g '$1')"
wget "$BAREURL"
exit

Machine and Version info

youtube-dl version: 2014.02.25
OS: Arch Linux
uname: Linux XXX 3.12.8-1-ARCH #1 SMP PREEMPT Thu Jan 16 09:46:30 CET 2014 i686 GNU/Linux
glibc version: glibc
Python version: 3.3.3
Country: Canada

Originally created by @hdclark on GitHub (Mar 3, 2014). # Description youtube-dl will properly download YouTube videos, but the [-g] option spits out URLs to different videos. Wget, cURL, and mplayer download the wrong video. Behaviour does not vary when changing user-agent in any four (or any combination of the four) programs. Most youtube links will result in the same video being downloaded (https://www.youtube.com/watch?v=5mPDhmP6ztQ). Behaviour happens across several machines (32 and 64 bit) at different addresses. # Minimal test program > # !/bin/sh > > BAREURL="$( youtube-dl -g '$1')" > wget "$BAREURL" > exit # Machine and Version info youtube-dl version: 2014.02.25 OS: Arch Linux uname: Linux XXX 3.12.8-1-ARCH #1 SMP PREEMPT Thu Jan 16 09:46:30 CET 2014 i686 GNU/Linux glibc version: glibc Python version: 3.3.3 Country: Canada
Author
Owner

@jaimeMF commented on GitHub (Mar 3, 2014):

How are you calling the script?

@jaimeMF commented on GitHub (Mar 3, 2014): How are you calling the script?
Author
Owner

@phihag commented on GitHub (Mar 3, 2014):

The script works fine for me from the US.

Can you define wrong? Is it another video? Is it random data?

Does the problem occur when you use youtube-dl's built-in downloader, i.e. do not pass in -g? What output do you get if you pass in the -v option instead of -g?

Can you reproduce the problem with http://www.youtube.com/watch?v=BaW_jenozKc ?

@phihag commented on GitHub (Mar 3, 2014): The script works fine for me from the US. Can you define `wrong`? Is it another video? Is it random data? Does the problem occur when you use youtube-dl's built-in downloader, i.e. do not pass in `-g`? What output do you get if you pass in the `-v` option instead of `-g`? Can you reproduce the problem with http://www.youtube.com/watch?v=BaW_jenozKc ?
Author
Owner

@jaimeMF commented on GitHub (Mar 3, 2014):

I have added set -x to the start of the script and this is the output

./script.sh
++ youtube-dl -g '$1'
[debug] System config: []
[debug] User config: ['--keep-video', '--write-info-json', '--write-thumbnail', '--verbose']
[debug] Command-line args: ['-g', '$1']
[debug] Encodings: locale 'UTF-8', fs 'utf-8', out None, pref: 'UTF-8'
[debug] youtube-dl version 2014.02.28
[debug] Python version 2.7.6 - Darwin-13.1.0-x86_64-i386-64bit
[debug] Proxy map: {}
+ BAREURL='blah'
+ wget 'blah'

it downloads https://www.youtube.com/watch?v=J0-rgeU1E4w

My /bin/sh version is:

/bin/sh --version
GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13)
Copyright (C) 2007 Free Software Foundation, Inc.

This script works fine:

#!/bin/sh
BAREURL="$( youtube-dl -g "$1")"
wget "$BAREURL"
exit

@phihag Maybe we should disable the automic search by default and let it fail.

@jaimeMF commented on GitHub (Mar 3, 2014): I have added `set -x` to the start of the script and this is the output ``` ./script.sh ++ youtube-dl -g '$1' [debug] System config: [] [debug] User config: ['--keep-video', '--write-info-json', '--write-thumbnail', '--verbose'] [debug] Command-line args: ['-g', '$1'] [debug] Encodings: locale 'UTF-8', fs 'utf-8', out None, pref: 'UTF-8' [debug] youtube-dl version 2014.02.28 [debug] Python version 2.7.6 - Darwin-13.1.0-x86_64-i386-64bit [debug] Proxy map: {} + BAREURL='blah' + wget 'blah' ``` it downloads https://www.youtube.com/watch?v=J0-rgeU1E4w My /bin/sh version is: ``` /bin/sh --version GNU bash, version 3.2.51(1)-release (x86_64-apple-darwin13) Copyright (C) 2007 Free Software Foundation, Inc. ``` This script works fine: ``` bash #!/bin/sh BAREURL="$( youtube-dl -g "$1")" wget "$BAREURL" exit ``` @phihag Maybe we should disable the automic search by default and let it fail.
Author
Owner

@hdclark commented on GitHub (Mar 3, 2014):

Calling the aforementioned script 'test.sh' and performing

./test.sh 'https://www.youtube.com/watch?v=yrBrH5PZtcc'

will result in downloading the YouTube video https://www.youtube.com/watch?v=5mPDhmP6ztQ. The URL youtube-dl spits out is:

http://r2---sn-j5o5gpx-t8ge.googlevideo.com/videoplayback?ipbits=0&mv=m&source=youtube&gcr=ca&sparams=gcr%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cexpire&ip=WWW.XXX.YYY.ZZZ&expire=1393891572&fexp=935627%2C909708%2C944900%2C904316%2C916624%2C902529%2C937417%2C937416%2C913434%2C936910%2C936913%2C902907&ms=au&mt=1393868125&ratebypass=yes&itag=22&sver=3&id=274fab81e535138c&upn=H0rTMY6w6HQ&key=yt5&signature=F4AF73DC612B7DFAB91AC90ECDDEFDFFCFB50325.B479CBFC47AEE9CF0862A1B1A9897EEE2B3A3ACC

The same behaviour happens on the test file at http://www.youtube.com/watch?v=BaW_jenozKc : youtube-dl spits out a googlevideo URL that points to another video (the same damn "TrapDoor" video at https://www.youtube.com/watch?v=5mPDhmP6ztQ).

In comparison, performing

youtube-dl 'https://www.youtube.com/watch?v=yrBrH5PZtcc'

will download the correct video (https://www.youtube.com/watch?v=yrBrH5PZtcc - "Opeth- Demon of the Fall-yrBrH5PZtcc.mp4").

Motivated by an earlier youtube-dl issue, I've tried removing ~/.config/youtube-dl/. It did not change the behaviour. The strace is non-distinct...

@hdclark commented on GitHub (Mar 3, 2014): Calling the aforementioned script 'test.sh' and performing > ./test.sh 'https://www.youtube.com/watch?v=yrBrH5PZtcc' will result in downloading the YouTube video https://www.youtube.com/watch?v=5mPDhmP6ztQ. The URL youtube-dl spits out is: > http://r2---sn-j5o5gpx-t8ge.googlevideo.com/videoplayback?ipbits=0&mv=m&source=youtube&gcr=ca&sparams=gcr%2Cid%2Cip%2Cipbits%2Citag%2Cratebypass%2Csource%2Cupn%2Cexpire&ip=WWW.XXX.YYY.ZZZ&expire=1393891572&fexp=935627%2C909708%2C944900%2C904316%2C916624%2C902529%2C937417%2C937416%2C913434%2C936910%2C936913%2C902907&ms=au&mt=1393868125&ratebypass=yes&itag=22&sver=3&id=274fab81e535138c&upn=H0rTMY6w6HQ&key=yt5&signature=F4AF73DC612B7DFAB91AC90ECDDEFDFFCFB50325.B479CBFC47AEE9CF0862A1B1A9897EEE2B3A3ACC The same behaviour happens on the test file at http://www.youtube.com/watch?v=BaW_jenozKc : youtube-dl spits out a googlevideo URL that points to another video (the same damn "TrapDoor" video at https://www.youtube.com/watch?v=5mPDhmP6ztQ). In comparison, performing > youtube-dl 'https://www.youtube.com/watch?v=yrBrH5PZtcc' will download the correct video (https://www.youtube.com/watch?v=yrBrH5PZtcc - "Opeth- Demon of the Fall-yrBrH5PZtcc.mp4"). Motivated by an earlier youtube-dl issue, I've tried removing ~/.config/youtube-dl/. It did not change the behaviour. The strace is non-distinct...
Author
Owner

@hdclark commented on GitHub (Mar 3, 2014):

Forgot to mention that this is intermittent.

Here is the verbose info you requested. Nothing fishy happening when youtube-dl handles the download itself.

[me@mymachine ~]$ youtube-dl -v 'http://www.youtube.com/watch?v=BaW_jenozKc'
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=BaW_jenozKc']
[debug] Encodings: locale 'UTF-8', fs 'utf-8', out 'UTF-8', pref: 'UTF-8'
[debug] youtube-dl version 2014.02.26
[debug] Python version 3.3.3 - Linux-3.12.7-1-ARCH-x86_64-with-arch-Arch-Linux
[debug] Proxy map: {}
[youtube] Setting language
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading video info webpage
[youtube] BaW_jenozKc: Extracting video information
[download] Destination: youtube-dl test video /SOME UNICODE CHARS HERE THAT GITHUB DISLIKES/ BaW_jenozKc.mp4
[download] 100% of 1.53MiB in 00:00

Hmmm.... maybe there is something strange with my machine. (Or maybe YouTube is playing with me?) Forgoing the script seems to work at the moment:

[me@mymachine ~]$ youtube-dl -g 'http://www.youtube.com/watch?v=BaW_jenozKc'
https://r3---sn-j5o5gpx-t8ge.googlevideo.com/videoplayback?expire=1393889820&pcm2fr=yes&source=youtube&key=yt5&signature=AE9ED0C1E31CBE2BB691936C82F7B39C90C9B7BA.52CA633C6D3CADC71376BA55DE754396BB18D88A&ip=WWW.XXX.YYY.ZZZ&mv=m&ms=au&ratebypass=yes&id=05a5bf8de9e8cca7&sparams=id%2Cip%2Cipbits%2Citag%2Cpcm2fr%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&fexp=935637%2C919121%2C909207%2C916623%2C937417%2C937416%2C913434%2C936910%2C936913%2C902907&upn=yZIfgu23iJQ&mt=1393869116&itag=22&requiressl=yes&ipbits=0&sver=3
[me@mymachine ~]$ wget 'https://r3---sn-j5o5gpx-t8ge.googlevideo.com/videoplayback?expire=1393889820&pcm2fr=yes&source=youtube&key=yt5&signature=AE9ED0C1E31CBE2BB691936C82F7B39C90C9B7BA.52CA633C6D3CADC71376BA55DE754396BB18D88A&ip=WWW.XXX.YYY.ZZZ&mv=m&ms=au&ratebypass=yes&id=05a5bf8de9e8cca7&sparams=id%2Cip%2Cipbits%2Citag%2Cpcm2fr%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&fexp=935637%2C919121%2C909207%2C916623%2C937417%2C937416%2C913434%2C936910%2C936913%2C902907&upn=yZIfgu23iJQ&mt=1393869116&itag=22&requiressl=yes&ipbits=0&sver=3' -O file.mp4

seems to correctly download the test file. And the first example from my last post appears to work now too:

./test.sh 'https://www.youtube.com/watch?v=yrBrH5PZtcc'

@hdclark commented on GitHub (Mar 3, 2014): Forgot to mention that this is intermittent. Here is the verbose info you requested. Nothing fishy happening when youtube-dl handles the download itself. > [me@mymachine ~]$ youtube-dl -v 'http://www.youtube.com/watch?v=BaW_jenozKc' > [debug] System config: [] > [debug] User config: [] > [debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=BaW_jenozKc'] > [debug] Encodings: locale 'UTF-8', fs 'utf-8', out 'UTF-8', pref: 'UTF-8' > [debug] youtube-dl version 2014.02.26 > [debug] Python version 3.3.3 - Linux-3.12.7-1-ARCH-x86_64-with-arch-Arch-Linux > [debug] Proxy map: {} > [youtube] Setting language > [youtube] BaW_jenozKc: Downloading webpage > [youtube] BaW_jenozKc: Downloading video info webpage > [youtube] BaW_jenozKc: Extracting video information > [download] Destination: youtube-dl test video /SOME UNICODE CHARS HERE THAT GITHUB DISLIKES/ BaW_jenozKc.mp4 > [download] 100% of 1.53MiB in 00:00 Hmmm.... maybe there is something strange with my machine. (Or maybe YouTube is playing with me?) Forgoing the script seems to work at the moment: > [me@mymachine ~]$ youtube-dl -g 'http://www.youtube.com/watch?v=BaW_jenozKc' > https://r3---sn-j5o5gpx-t8ge.googlevideo.com/videoplayback?expire=1393889820&pcm2fr=yes&source=youtube&key=yt5&signature=AE9ED0C1E31CBE2BB691936C82F7B39C90C9B7BA.52CA633C6D3CADC71376BA55DE754396BB18D88A&ip=WWW.XXX.YYY.ZZZ&mv=m&ms=au&ratebypass=yes&id=05a5bf8de9e8cca7&sparams=id%2Cip%2Cipbits%2Citag%2Cpcm2fr%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&fexp=935637%2C919121%2C909207%2C916623%2C937417%2C937416%2C913434%2C936910%2C936913%2C902907&upn=yZIfgu23iJQ&mt=1393869116&itag=22&requiressl=yes&ipbits=0&sver=3 > [me@mymachine ~]$ wget 'https://r3---sn-j5o5gpx-t8ge.googlevideo.com/videoplayback?expire=1393889820&pcm2fr=yes&source=youtube&key=yt5&signature=AE9ED0C1E31CBE2BB691936C82F7B39C90C9B7BA.52CA633C6D3CADC71376BA55DE754396BB18D88A&ip=WWW.XXX.YYY.ZZZ&mv=m&ms=au&ratebypass=yes&id=05a5bf8de9e8cca7&sparams=id%2Cip%2Cipbits%2Citag%2Cpcm2fr%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&fexp=935637%2C919121%2C909207%2C916623%2C937417%2C937416%2C913434%2C936910%2C936913%2C902907&upn=yZIfgu23iJQ&mt=1393869116&itag=22&requiressl=yes&ipbits=0&sver=3' -O file.mp4 seems to correctly download the test file. And the first example from my last post appears to work now too: > ./test.sh 'https://www.youtube.com/watch?v=yrBrH5PZtcc'
Author
Owner

@hdclark commented on GitHub (Mar 3, 2014):

I'm not sure what the issue is, but I seem to be the only one having it. I'll close for now and open again when (if) I can get a better handle on it.

@hdclark commented on GitHub (Mar 3, 2014): I'm not sure what the issue is, but I seem to be the only one having it. I'll close for now and open again when (if) I can get a better handle on it.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/youtube-dl-ytdl-org#1951
No description provided.