starred/youtube-dl-ytdl-org

Fork 0

mirror of https://github.com/ytdl-org/youtube-dl.git synced 2026-03-02 19:17:00 -05:00

HLS vulnerability in both FFMpeg and NativeHlsFD #6657

New issue

Closed

opened 2026-02-21 03:21:28 -05:00 by deekerman · 3 comments

deekerman commented

2026-02-21 03:21:28 -05:00

Owner

Originally created by @yan12125 on GitHub (Jan 13, 2016).

A recent post points out that FFMpeg can cause contents of arbitrary files (for example /etc/passwd) being accessible on the Internet via a malicious input file. [1][2] The reaction of Arch Linux developers is disabling affected components before they are fixed. [3] In this commit, concat: protocol and HLS support are disabled. The former one is not used in the mainline codebase, just in some pull requests (#2844). The latter one is more serious. I've just updated my copy to the latest official Arch binary. Downloading an YouTube live stream gives:

$ youtube-dl -v "https://www.youtube.com/watch?v=clO8XxFkrj4"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=clO8XxFkrj4']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.01.09
[debug] Git HEAD: 40cf7fc
[debug] Python version 3.5.1 - Linux-4.3.3-2-ARCH-x86_64-with-arch-Arch-Linux
[debug] exe versions: ffmpeg 2.8.4, ffprobe 2.8.4, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] clO8XxFkrj4: Downloading webpage
[youtube] clO8XxFkrj4: Downloading video info webpage
[youtube] clO8XxFkrj4: Extracting video information
[youtube] clO8XxFkrj4: Downloading formats manifest
[youtube] clO8XxFkrj4: Downloading DASH manifest
[debug] Invoking downloader on 'https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8'
[download] Destination: 中天電視直播HD頻道(總統大選直播區如下) │Taiwan CTITV News HD Live-clO8XxFkrj4.mp4
[debug] ffmpeg command line: ffmpeg -y -headers 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)
Accept-Language: en-us,en;q=0.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
' -i https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8 -f mp4 -c copy -bsf:a aac_adtstoasc 'file:中天電視直播HD頻道(總統大選直播區如下) │Taiwan CTITV News HD Live-clO8XxFkrj4.mp4.part'
ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.3.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab --disable-demuxer=hls --disable-protocol='concat,hls'
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8: Invalid data found when processing input


ERROR: ffmpeg exited with code 1
  File "/usr/bin/youtube-dl", line 9, in <module>
    load_entry_point('youtube-dl==2016.1.9', 'console_scripts', 'youtube-dl')()
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/__init__.py", line 410, in main
    _real_main(argv)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/__init__.py", line 400, in _real_main
    retcode = ydl.download(all_urls)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1677, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 676, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 722, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1347, in process_video_result
    self.process_info(new_info)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1609, in process_info
    success = dl(filename, info_dict)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1551, in dl
    return fd.download(name, info)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/downloader/common.py", line 342, in download
    return self.real_download(filename, info_dict)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/downloader/hls.py", line 63, in real_download
    self.report_error('%s exited with code %d' % (ffpp.basename, retval))
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/downloader/common.py", line 155, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 540, in report_error
    self.trouble(error_message, tb)
  File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 502, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

Running the command directly does not give more information:

$ ffmpeg -y -headers 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)
Accept-Language: en-us,en;q=0.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
' -i https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8 -f mp4 -c copy -bsf:a aac_adtstoasc 'file:中天電視直播HD頻道(總統大選直播區如下) │Taiwan CTITV News HD Live-clO8XxFkrj4.mp4.part'
ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers
  built with gcc 5.3.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab --disable-demuxer=hls --disable-protocol='concat,hls'
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[https @ 0x55f0db7dfac0] No trailing CRLF found in HTTP header.
https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8: Invalid data found when processing input

Before the problem fixed or Arch developers decided to have a different workaround, the only way is suggesting --hls-native-native. However, NativeHlsFD is also vulnerable. With the following evil.m3u8:

#EXTM3U
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.0
file:///etc/passwd
#EXT-X-ENDLIST

youtube-dl gives undesired results:

$ youtube-dl -v --hls-prefer-native http://localhost/yen/test/evil.m3u8
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', '--hls-prefer-native', 'http://localhost/yen/test/evil.m3u8']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.01.09
[debug] Git HEAD: 40cf7fc
[debug] Python version 3.5.1 - Linux-4.3.3-2-ARCH-x86_64-with-arch-Arch-Linux
[debug] exe versions: ffmpeg 2.8.4, ffprobe 2.8.4, rtmpdump 2.4
[debug] Proxy map: {}
[generic] evil: Requesting header
WARNING: Falling back on generic information extractor.
[generic] evil: Downloading webpage
WARNING: URL could be a direct video link, returning it as such.
[debug] Invoking downloader on 'http://localhost/yen/test/evil.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 1
[download] Destination: evil-evil.m3u8
[download] 100% of 2.25KiB in 00:00

$ head -n 3 evil-evil.m3u8 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/bin/false
daemon:x:2:2:daemon:/sbin:/bin/false

All servers using youtube-dl are affected, whether they use FFMpeg or NativeHlsFD. I guess only http:, https: and data: protocols are necessary for YoutubeDL.urlopen()? We should check URLs before passing them into urllib2.

[1] https://news.ycombinator.com/item?id=10893301
[2] http://habrahabr.ru/company/mailru/blog/274855/ (The original post, in Russian)
[3] https://projects.archlinux.org/svntogit/packages.git/commit/trunk/PKGBUILD?h=packages/ffmpeg&id=ef0b4890e18a52e976274d02a09738f73a07f4d2

Originally created by @yan12125 on GitHub (Jan 13, 2016). A recent post points out that FFMpeg can cause contents of arbitrary files (for example `/etc/passwd`) being accessible on the Internet via a malicious input file. [1][2] The reaction of Arch Linux developers is disabling affected components before they are fixed. [3] In this commit, `concat:` protocol and HLS support are disabled. The former one is not used in the mainline codebase, just in some pull requests (#2844). The latter one is more serious. I've just updated my copy to the latest official Arch binary. Downloading an YouTube live stream gives: ``` $ youtube-dl -v "https://www.youtube.com/watch?v=clO8XxFkrj4" [debug] System config: [] [debug] User config: [] [debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=clO8XxFkrj4'] [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2016.01.09 [debug] Git HEAD: 40cf7fc [debug] Python version 3.5.1 - Linux-4.3.3-2-ARCH-x86_64-with-arch-Arch-Linux [debug] exe versions: ffmpeg 2.8.4, ffprobe 2.8.4, rtmpdump 2.4 [debug] Proxy map: {} [youtube] clO8XxFkrj4: Downloading webpage [youtube] clO8XxFkrj4: Downloading video info webpage [youtube] clO8XxFkrj4: Extracting video information [youtube] clO8XxFkrj4: Downloading formats manifest [youtube] clO8XxFkrj4: Downloading DASH manifest [debug] Invoking downloader on 'https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8' [download] Destination: 中天電視直播HD頻道(總統大選直播區如下) │Taiwan CTITV News HD Live-clO8XxFkrj4.mp4 [debug] ffmpeg command line: ffmpeg -y -headers 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome) Accept-Language: en-us,en;q=0.5 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 ' -i https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8 -f mp4 -c copy -bsf:a aac_adtstoasc 'file:中天電視直播HD頻道(總統大選直播區如下) │Taiwan CTITV News HD Live-clO8XxFkrj4.mp4.part' ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers built with gcc 5.3.0 (GCC) configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab --disable-demuxer=hls --disable-protocol='concat,hls' libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100 https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8: Invalid data found when processing input ERROR: ffmpeg exited with code 1 File "/usr/bin/youtube-dl", line 9, in <module> load_entry_point('youtube-dl==2016.1.9', 'console_scripts', 'youtube-dl')() File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/__init__.py", line 410, in main _real_main(argv) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/__init__.py", line 400, in _real_main retcode = ydl.download(all_urls) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1677, in download url, force_generic_extractor=self.params.get('force_generic_extractor', False)) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 676, in extract_info return self.process_ie_result(ie_result, download, extra_info) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 722, in process_ie_result return self.process_video_result(ie_result, download=download) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1347, in process_video_result self.process_info(new_info) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1609, in process_info success = dl(filename, info_dict) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 1551, in dl return fd.download(name, info) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/downloader/common.py", line 342, in download return self.real_download(filename, info_dict) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/downloader/hls.py", line 63, in real_download self.report_error('%s exited with code %d' % (ffpp.basename, retval)) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/downloader/common.py", line 155, in report_error self.ydl.report_error(*args, **kargs) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 540, in report_error self.trouble(error_message, tb) File "/home/yen/Executables/Multimedia/youtube-dl/youtube_dl/YoutubeDL.py", line 502, in trouble tb_data = traceback.format_list(traceback.extract_stack()) ``` Running the command directly does not give more information: ``` $ ffmpeg -y -headers 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome) Accept-Language: en-us,en;q=0.5 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 ' -i https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8 -f mp4 -c copy -bsf:a aac_adtstoasc 'file:中天電視直播HD頻道(總統大選直播區如下) │Taiwan CTITV News HD Live-clO8XxFkrj4.mp4.part' ffmpeg version 2.8.4 Copyright (c) 2000-2015 the FFmpeg developers built with gcc 5.3.0 (GCC) configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-avresample --enable-fontconfig --enable-gnutls --enable-gpl --enable-ladspa --enable-libass --enable-libbluray --enable-libdcadec --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-shared --enable-version3 --enable-x11grab --disable-demuxer=hls --disable-protocol='concat,hls' libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100 [https @ 0x55f0db7dfac0] No trailing CRLF found in HTTP header. https://manifest.googlevideo.com/api/manifest/hls_playlist/id/clO8XxFkrj4.2/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/Y2xPOFh4RmtyajQuMg.95/hls_chunk_host/r5---sn-5njj-u2xl.googlevideo.com/playlist_type/LIVE/pmbypass/yes/gcr/tw/mm/32/mn/sn-5njj-u2xl/ms/lv/mv/m/pl/16/dover/3/fexp/9416126,9420452,9422596,9423459,9423662,9427015/upn/96epxf2PvxI/sver/3/mt/1452716770/ip/140.112.230.216/ipbits/0/expire/1452738400/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,playlist_type,pmbypass,gcr,mm,mn,ms,mv,pl/signature/483695472960B09D033F3C6FA6C5E2BB5769C122.93B057124DC884D539E8ED21C5A6CBE825B69EBB/key/dg_yt0/playlist/index.m3u8: Invalid data found when processing input ``` Before the problem fixed or Arch developers decided to have a different workaround, the only way is suggesting `--hls-native-native`. However, `NativeHlsFD` is also vulnerable. With the following `evil.m3u8`: ``` #EXTM3U #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:10.0 file:///etc/passwd #EXT-X-ENDLIST ``` youtube-dl gives undesired results: ``` $ youtube-dl -v --hls-prefer-native http://localhost/yen/test/evil.m3u8 [debug] System config: [] [debug] User config: [] [debug] Command-line args: ['-v', '--hls-prefer-native', 'http://localhost/yen/test/evil.m3u8'] [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2016.01.09 [debug] Git HEAD: 40cf7fc [debug] Python version 3.5.1 - Linux-4.3.3-2-ARCH-x86_64-with-arch-Arch-Linux [debug] exe versions: ffmpeg 2.8.4, ffprobe 2.8.4, rtmpdump 2.4 [debug] Proxy map: {} [generic] evil: Requesting header WARNING: Falling back on generic information extractor. [generic] evil: Downloading webpage WARNING: URL could be a direct video link, returning it as such. [debug] Invoking downloader on 'http://localhost/yen/test/evil.m3u8' [hlsnative] Downloading m3u8 manifest [hlsnative] Total fragments: 1 [download] Destination: evil-evil.m3u8 [download] 100% of 2.25KiB in 00:00 $ head -n 3 evil-evil.m3u8 root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/bin/false daemon:x:2:2:daemon:/sbin:/bin/false ``` All servers using youtube-dl are affected, whether they use FFMpeg or `NativeHlsFD`. I guess only `http:`, `https:` and `data:` protocols are necessary for `YoutubeDL.urlopen()`? We should check URLs before passing them into urllib2. [1] https://news.ycombinator.com/item?id=10893301 [2] http://habrahabr.ru/company/mailru/blog/274855/ (The original post, in Russian) [3] https://projects.archlinux.org/svntogit/packages.git/commit/trunk/PKGBUILD?h=packages/ffmpeg&id=ef0b4890e18a52e976274d02a09738f73a07f4d2

deekerman

2026-02-21 03:21:28 -05:00

closed this issue
added the
bug
label

deekerman commented

2026-02-21 03:21:31 -05:00

Author

Owner

@jaimeMF commented on GitHub (Jan 13, 2016):

I've suggested a patch for disabling the 'file:' protocol in #8228. Note that actually all you need to do to get access to a sensitive file is to run youtube-dl file:///etc/passwd, you don't need to craft a m3u8 file and serve it online.

Unfortunately there doesn't seem to be a way to disable protocols on ffmpeg at runtime, which would be the simple solution.

@jaimeMF commented on GitHub (Jan 13, 2016): I've suggested a patch for disabling the 'file:' protocol in #8228. Note that actually all you need to do to get access to a sensitive file is to run `youtube-dl file:///etc/passwd`, you don't need to craft a m3u8 file and serve it online. Unfortunately there doesn't seem to be a way to disable protocols on ffmpeg at runtime, which would be the simple solution.

deekerman commented

2026-02-21 03:21:31 -05:00

Author

Owner

@Kagami commented on GitHub (Jan 15, 2016):

Fixed in ffmpeg git master (and few more recent commits).

@Kagami commented on GitHub (Jan 15, 2016): [Fixed in ffmpeg git master](https://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=7145e80b4f78cff5ed5fee04d4c4d53daaa0e077;hp=92465a2347d959cbd9864b017a39b2a4ab9313ff) (and few more recent commits).

deekerman commented

2026-02-21 03:21:31 -05:00

Author

Owner

@yan12125 commented on GitHub (Jan 15, 2016):

Thanks for the useful information @Kagami ! Now ffmpeg is fixed both in git-master and Arch Linux's package, and I guess other distributions will fix their packages soon. If there are more HLS vulnerability discovered, this issue can be re-opened.

@yan12125 commented on GitHub (Jan 15, 2016): Thanks for the useful information @Kagami ! Now ffmpeg is fixed both in git-master and Arch Linux's package, and I guess other distributions will fix their packages soon. If there are more HLS vulnerability discovered, this issue can be re-opened.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/youtube-dl-ytdl-org#6657

No description provided.

Rows
Columns