starred/youtube-dl

Fork 0

mirror of https://github.com/ytdl-org/youtube-dl.git synced 2026-03-02 23:07:04 -05:00

Youtube-dl unnecessarily peeking at stdin #4068

New issue

Closed

opened 2026-02-21 00:07:14 -05:00 by deekerman · 3 comments

deekerman commented

2026-02-21 00:07:14 -05:00

Owner

Originally created by @hdclark on GitHub (Feb 13, 2015).

Youtube-dl appears to be peeking at stdin for no reason.

Steps to recreate:

Step 1. Create a file with one url per line.

echo 'https://www.youtube.com/watch?v=x4WFTpU8yHk'  > /tmp/playlist
echo 'https://www.youtube.com/watch?v=HsoICLXZpao' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=rYmeUDnwIAo' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=zGZKui5SRyo' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=TVBRG4EN2Bs' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=C-Dmu5AofZg' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=kIGmrBObUxQ' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=kazwE_4Vv84' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=UQLyLr3cxLw' >> /tmp/playlist
echo 'https://www.youtube.com/watch?v=dC2I-PDXbKc' >> /tmp/playlist

Step 2. Loop over the file with a shell loop.

while read aurl ; do
    youtube-dl --extract-audio "${aurl}"
done < /tmp/playlist

Output on my machine

[hal@MACHINE temporarydir]$ while read aurl ; do youtube-dl --extract-audio "${aurl}"; done < /tmp/playlist
[youtube] x4WFTpU8yHk: Downloading webpage
[youtube] x4WFTpU8yHk: Extracting video information
[youtube] x4WFTpU8yHk: Downloading js player en_US-vflZvrDTQ
[youtube] x4WFTpU8yHk: Downloading DASH manifest
[download] Destination: Iron Maiden - Gates of Tomorrow (lyrics)-x4WFTpU8yHk.m4a
[download] 100% of 9.49MiB in 00:24
[ffmpeg] Correcting container in "Iron Maiden - Gates of Tomorrow (lyrics)-x4WFTpU8yHk.m4a"
[youtube] Post-process file Iron Maiden - Gates of Tomorrow (lyrics)-x4WFTpU8yHk.m4a exists, skipping
WARNING: The url doesn't specify the protocol, trying with http
[generic] watch?v=HsoICLXZpao: Requesting header
WARNING: Could not send HEAD request to http://://www.youtube.com/watch?v=HsoICLXZpao: <urlopen error [Errno -2] Name or service not known>
[generic] watch?v=HsoICLXZpao: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno -2] Name or service not known> (caused by URLError(gaierror(-2, 'Name or service not known'),))
[youtube] rYmeUDnwIAo: Downloading webpage
[youtube] rYmeUDnwIAo: Extracting video information
[youtube] rYmeUDnwIAo: Downloading DASH manifest
[download] Destination: Iron Maiden - Face in the Sand (lyrics)-rYmeUDnwIAo.m4a
[download] 100% of 11.89MiB in 00:16
[ffmpeg] Correcting container in "Iron Maiden - Face in the Sand (lyrics)-rYmeUDnwIAo.m4a"
[youtube] Post-process file Iron Maiden - Face in the Sand (lyrics)-rYmeUDnwIAo.m4a exists, skipping
[generic] watch?v=zGZKui5SRyo: Requesting header
WARNING: Could not send HEAD request to s://www.youtube.com/watch?v=zGZKui5SRyo: <urlopen error unknown url type: s>
[generic] watch?v=zGZKui5SRyo: Downloading webpage
ERROR: Unable to download webpage: <urlopen error unknown url type: s> (caused by URLError('unknown url type: s',))
[youtube] TVBRG4EN2Bs: Downloading webpage
[youtube] TVBRG4EN2Bs: Extracting video information
[youtube] TVBRG4EN2Bs: Downloading js player en_US-vflZvrDTQ
[youtube] TVBRG4EN2Bs: Downloading DASH manifest
[download] Destination: Iron Maiden - Dance of Death (lyrics)-TVBRG4EN2Bs.m4a
[download] 100% of 15.71MiB in 00:17
[ffmpeg] Correcting container in "Iron Maiden - Dance of Death (lyrics)-TVBRG4EN2Bs.m4a"
[youtube] Post-process file Iron Maiden - Dance of Death (lyrics)-TVBRG4EN2Bs.m4a exists, skipping
WARNING: The url doesn't specify the protocol, trying with http
[generic] watch?v=C-Dmu5AofZg: Requesting header
WARNING: Could not send HEAD request to http://://www.youtube.com/watch?v=C-Dmu5AofZg: <urlopen error [Errno -2] Name or service not known>
[generic] watch?v=C-Dmu5AofZg: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno -2] Name or service not known> (caused by URLError(gaierror(-2, 'Name or service not known'),))
[youtube] kIGmrBObUxQ: Downloading webpage
[youtube] kIGmrBObUxQ: Extracting video information
[youtube] kIGmrBObUxQ: Downloading DASH manifest
[download] Destination: Iron Maiden - Rainmaker (lyrics)-kIGmrBObUxQ.m4a
[download] 100% of 6.96MiB in 00:12
[ffmpeg] Correcting container in "Iron Maiden - Rainmaker (lyrics)-kIGmrBObUxQ.m4a"
[youtube] Post-process file Iron Maiden - Rainmaker (lyrics)-kIGmrBObUxQ.m4a exists, skipping
[generic] watch?v=kazwE_4Vv84: Requesting header
WARNING: Could not send HEAD request to s://www.youtube.com/watch?v=kazwE_4Vv84: <urlopen error unknown url type: s>
[generic] watch?v=kazwE_4Vv84: Downloading webpage
ERROR: Unable to download webpage: <urlopen error unknown url type: s> (caused by URLError('unknown url type: s',))
[youtube] UQLyLr3cxLw: Downloading webpage
[youtube] UQLyLr3cxLw: Extracting video information
[youtube] UQLyLr3cxLw: Downloading DASH manifest
[download] Destination: Iron Maiden - Journeyman (lyrics)-UQLyLr3cxLw.m4a
[download] 100% of 12.99MiB in 00:12
[ffmpeg] Correcting container in "Iron Maiden - Journeyman (lyrics)-UQLyLr3cxLw.m4a"
[youtube] Post-process file Iron Maiden - Journeyman (lyrics)-UQLyLr3cxLw.m4a exists, skipping
WARNING: The url doesn't specify the protocol, trying with http
[youtube] dC2I-PDXbKc: Downloading webpage
[youtube] dC2I-PDXbKc: Extracting video information
[youtube] dC2I-PDXbKc: Downloading DASH manifest
[download] Destination: Iron Maiden - Wildest Dreams (lyrics)-dC2I-PDXbKc.m4a
[download] 100% of 7.08MiB in 00:04
[ffmpeg] Correcting container in "Iron Maiden - Wildest Dreams (lyrics)-dC2I-PDXbKc.m4a"
[youtube] Post-process file Iron Maiden - Wildest Dreams (lyrics)-dC2I-PDXbKc.m4a exists, skipping

Issue

stdin is being read for no reason, consuming input before shell's read can read it.

Workarounds

Use another file descriptor. Avoid using stdin.

while read -u 9 aurl ; do
    youtube-dl --extract-audio "${aurl}"
done 9< /tmp/playlist

Pass all urls to youtube-dl at once. This is not always possible.

Possibly-relevant information

Happens when using urxvt and xterm. Identical results happen with each.
Issue occurs also when dumping json.
System info:

[hal@MACHINE ~]$ uname -a
Linux luderro 3.18.4-1-ARCH #1 SMP PREEMPT Tue Jan 27 20:45:02 CET 2015 x86_64 GNU/Linux

[hal@MACHINE ~]$ fribidi --version
fribidi (GNU FriBidi) 0.19.6
interface version 3,
Unicode Character Database version 6.2.0,
Configure options --enable-debug --enable-malloc --with-glib.

Copyright (C) 2004  Sharif FarsiWeb, Inc.
Copyright (C) 2001, 2002, 2004, 2005  Behdad Esfahbod
Copyright (C) 1999, 2000  Dov Grobgeld
GNU FriBidi comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU FriBidi under
the terms of the GNU Lesser General Public License.
For more information about these matters, see the file named COPYING.

Written by Behdad Esfahbod and Dov Grobgeld.

Suspected cause

urxvt is not bi-directional text safe. xterm is. Using --bidi-workaround makes dumping json fail. Is Fribidi or some other library being used peeking at stdin?

Wishlist fix

Separate the youtube-dl into a core library and a command line wrapper. Having a proper api would obviate issues resulting from driving youtube-dl through a terminal emulator. :)

Originally created by @hdclark on GitHub (Feb 13, 2015). # Youtube-dl appears to be peeking at stdin for no reason. ### Steps to recreate: Step 1. Create a file with one url per line. ``` bash echo 'https://www.youtube.com/watch?v=x4WFTpU8yHk' > /tmp/playlist echo 'https://www.youtube.com/watch?v=HsoICLXZpao' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=rYmeUDnwIAo' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=zGZKui5SRyo' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=TVBRG4EN2Bs' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=C-Dmu5AofZg' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=kIGmrBObUxQ' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=kazwE_4Vv84' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=UQLyLr3cxLw' >> /tmp/playlist echo 'https://www.youtube.com/watch?v=dC2I-PDXbKc' >> /tmp/playlist ``` Step 2. Loop over the file with a shell loop. ``` bash while read aurl ; do youtube-dl --extract-audio "${aurl}" done < /tmp/playlist ``` ### Output on my machine ``` bash [hal@MACHINE temporarydir]$ while read aurl ; do youtube-dl --extract-audio "${aurl}"; done < /tmp/playlist [youtube] x4WFTpU8yHk: Downloading webpage [youtube] x4WFTpU8yHk: Extracting video information [youtube] x4WFTpU8yHk: Downloading js player en_US-vflZvrDTQ [youtube] x4WFTpU8yHk: Downloading DASH manifest [download] Destination: Iron Maiden - Gates of Tomorrow (lyrics)-x4WFTpU8yHk.m4a [download] 100% of 9.49MiB in 00:24 [ffmpeg] Correcting container in "Iron Maiden - Gates of Tomorrow (lyrics)-x4WFTpU8yHk.m4a" [youtube] Post-process file Iron Maiden - Gates of Tomorrow (lyrics)-x4WFTpU8yHk.m4a exists, skipping WARNING: The url doesn't specify the protocol, trying with http [generic] watch?v=HsoICLXZpao: Requesting header WARNING: Could not send HEAD request to http://://www.youtube.com/watch?v=HsoICLXZpao: <urlopen error [Errno -2] Name or service not known> [generic] watch?v=HsoICLXZpao: Downloading webpage ERROR: Unable to download webpage: <urlopen error [Errno -2] Name or service not known> (caused by URLError(gaierror(-2, 'Name or service not known'),)) [youtube] rYmeUDnwIAo: Downloading webpage [youtube] rYmeUDnwIAo: Extracting video information [youtube] rYmeUDnwIAo: Downloading DASH manifest [download] Destination: Iron Maiden - Face in the Sand (lyrics)-rYmeUDnwIAo.m4a [download] 100% of 11.89MiB in 00:16 [ffmpeg] Correcting container in "Iron Maiden - Face in the Sand (lyrics)-rYmeUDnwIAo.m4a" [youtube] Post-process file Iron Maiden - Face in the Sand (lyrics)-rYmeUDnwIAo.m4a exists, skipping [generic] watch?v=zGZKui5SRyo: Requesting header WARNING: Could not send HEAD request to s://www.youtube.com/watch?v=zGZKui5SRyo: <urlopen error unknown url type: s> [generic] watch?v=zGZKui5SRyo: Downloading webpage ERROR: Unable to download webpage: <urlopen error unknown url type: s> (caused by URLError('unknown url type: s',)) [youtube] TVBRG4EN2Bs: Downloading webpage [youtube] TVBRG4EN2Bs: Extracting video information [youtube] TVBRG4EN2Bs: Downloading js player en_US-vflZvrDTQ [youtube] TVBRG4EN2Bs: Downloading DASH manifest [download] Destination: Iron Maiden - Dance of Death (lyrics)-TVBRG4EN2Bs.m4a [download] 100% of 15.71MiB in 00:17 [ffmpeg] Correcting container in "Iron Maiden - Dance of Death (lyrics)-TVBRG4EN2Bs.m4a" [youtube] Post-process file Iron Maiden - Dance of Death (lyrics)-TVBRG4EN2Bs.m4a exists, skipping WARNING: The url doesn't specify the protocol, trying with http [generic] watch?v=C-Dmu5AofZg: Requesting header WARNING: Could not send HEAD request to http://://www.youtube.com/watch?v=C-Dmu5AofZg: <urlopen error [Errno -2] Name or service not known> [generic] watch?v=C-Dmu5AofZg: Downloading webpage ERROR: Unable to download webpage: <urlopen error [Errno -2] Name or service not known> (caused by URLError(gaierror(-2, 'Name or service not known'),)) [youtube] kIGmrBObUxQ: Downloading webpage [youtube] kIGmrBObUxQ: Extracting video information [youtube] kIGmrBObUxQ: Downloading DASH manifest [download] Destination: Iron Maiden - Rainmaker (lyrics)-kIGmrBObUxQ.m4a [download] 100% of 6.96MiB in 00:12 [ffmpeg] Correcting container in "Iron Maiden - Rainmaker (lyrics)-kIGmrBObUxQ.m4a" [youtube] Post-process file Iron Maiden - Rainmaker (lyrics)-kIGmrBObUxQ.m4a exists, skipping [generic] watch?v=kazwE_4Vv84: Requesting header WARNING: Could not send HEAD request to s://www.youtube.com/watch?v=kazwE_4Vv84: <urlopen error unknown url type: s> [generic] watch?v=kazwE_4Vv84: Downloading webpage ERROR: Unable to download webpage: <urlopen error unknown url type: s> (caused by URLError('unknown url type: s',)) [youtube] UQLyLr3cxLw: Downloading webpage [youtube] UQLyLr3cxLw: Extracting video information [youtube] UQLyLr3cxLw: Downloading DASH manifest [download] Destination: Iron Maiden - Journeyman (lyrics)-UQLyLr3cxLw.m4a [download] 100% of 12.99MiB in 00:12 [ffmpeg] Correcting container in "Iron Maiden - Journeyman (lyrics)-UQLyLr3cxLw.m4a" [youtube] Post-process file Iron Maiden - Journeyman (lyrics)-UQLyLr3cxLw.m4a exists, skipping WARNING: The url doesn't specify the protocol, trying with http [youtube] dC2I-PDXbKc: Downloading webpage [youtube] dC2I-PDXbKc: Extracting video information [youtube] dC2I-PDXbKc: Downloading DASH manifest [download] Destination: Iron Maiden - Wildest Dreams (lyrics)-dC2I-PDXbKc.m4a [download] 100% of 7.08MiB in 00:04 [ffmpeg] Correcting container in "Iron Maiden - Wildest Dreams (lyrics)-dC2I-PDXbKc.m4a" [youtube] Post-process file Iron Maiden - Wildest Dreams (lyrics)-dC2I-PDXbKc.m4a exists, skipping ``` ### Issue `stdin` is being read for no reason, consuming input before shell's `read` can read it. ### Workarounds - Use another file descriptor. Avoid using stdin. ``` bash while read -u 9 aurl ; do youtube-dl --extract-audio "${aurl}" done 9< /tmp/playlist ``` - Pass all urls to youtube-dl at once. This is not always possible. ### Possibly-relevant information - Happens when using `urxvt` and `xterm`. Identical results happen with each. - Issue occurs also when dumping json. - System info: ``` bash [hal@MACHINE ~]$ uname -a Linux luderro 3.18.4-1-ARCH #1 SMP PREEMPT Tue Jan 27 20:45:02 CET 2015 x86_64 GNU/Linux [hal@MACHINE ~]$ fribidi --version fribidi (GNU FriBidi) 0.19.6 interface version 3, Unicode Character Database version 6.2.0, Configure options --enable-debug --enable-malloc --with-glib. Copyright (C) 2004 Sharif FarsiWeb, Inc. Copyright (C) 2001, 2002, 2004, 2005 Behdad Esfahbod Copyright (C) 1999, 2000 Dov Grobgeld GNU FriBidi comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of GNU FriBidi under the terms of the GNU Lesser General Public License. For more information about these matters, see the file named COPYING. Written by Behdad Esfahbod and Dov Grobgeld. ``` ### Suspected cause `urxvt` is not bi-directional text safe. `xterm` is. Using `--bidi-workaround` makes dumping json fail. Is Fribidi or some other library being used peeking at stdin? ### Wishlist fix Separate the `youtube-dl` into a core library and a command line wrapper. Having a proper api would obviate issues resulting from driving `youtube-dl` through a terminal emulator. :)

deekerman closed this issue

2026-02-21 00:07:15 -05:00

deekerman commented

2026-02-21 00:07:16 -05:00

Author

Owner

@jaimeMF commented on GitHub (Feb 13, 2015):

Thanks for the report! It should work in the next version, the problem was when running ffmpeg for extracting the audio.
Note that for this usercase, you could just use --batch-file <filename>.

Separate the youtube-dl into a core library and a command line wrapper. Having a proper api would obviate issues resulting from driving youtube-dl through a terminal emulator. :)

Maybe I misunderstood this part, but you can just import the youtube_dl module from a python script. If that's not what you wanted, you'll have to open a new issue.

@jaimeMF commented on GitHub (Feb 13, 2015): Thanks for the report! It should work in the next version, the problem was when running ffmpeg for extracting the audio. Note that for this usercase, you could just use `--batch-file <filename>`. > Separate the youtube-dl into a core library and a command line wrapper. Having a proper api would obviate issues resulting from driving youtube-dl through a terminal emulator. :) Maybe I misunderstood this part, but you can just import the `youtube_dl` module from a python script. If that's not what you wanted, you'll have to open a new issue.

deekerman commented

2026-02-21 00:07:17 -05:00

Author

Owner

@hdclark commented on GitHub (Feb 13, 2015):

Great, thank you.

My final comment was tongue-in-cheek wishing for a C api. However, I was unaware the internal api was stable enough to call from Python. Thanks for the tip.

@hdclark commented on GitHub (Feb 13, 2015): Great, thank you. My final comment was tongue-in-cheek wishing for a C api. However, I was unaware the internal api was stable enough to call from Python. Thanks for the tip.

deekerman commented

2026-02-21 00:07:17 -05:00

Author

Owner

@jaimeMF commented on GitHub (Feb 14, 2015):

You can always call python from C.

Most of the changes are for adding/improving websites supports, so there aren't many breaking changes.

@jaimeMF commented on GitHub (Feb 14, 2015): You can always call python from C. Most of the changes are for adding/improving websites supports, so there aren't many breaking changes.

deekerman referenced this issue

2026-02-21 03:17:46 -05:00

Possibility of changing the project license #8720

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/youtube-dl#4068

No description provided.

Rows
Columns