starred/youtube-dl-ytdl-org

Fork 0

mirror of https://github.com/ytdl-org/youtube-dl.git synced 2026-03-02 19:17:00 -05:00

Problem with local m3u/m3u8 files #17915

New issue

Closed

opened 2026-02-21 09:43:26 -05:00 by deekerman · 6 comments

deekerman commented

2026-02-21 09:43:26 -05:00

Owner

Originally created by @atom-smasher on GitHub (Sep 12, 2019).

Checklist

I've verified that I'm running youtube-dl version 2019.09.12.1
I've checked that all provided URLs are alive and playable in a browser
I've checked that all URLs and arguments with special characters are properly quoted or escaped
I've searched the bugtracker for similar bug reports including closed ones
I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--ignore-config', u'-a', u'./640x360-test.m3u8']
[debug] Batch file urls: [u'https://walterebert.com/_/playground/videos/ts/640x3600.ts', u'https://walterebert.com/_/playground/videos/ts/640x3601.ts', u'https://walterebert.com/_/playground/videos/ts/640x3602.ts', u'https://walterebert.com/_/playground/videos/ts/640x3603.ts', u'https://walterebert.com/_/playground/videos/ts/640x3604.ts', u'https://walterebert.com/_/playground/videos/ts/640x3605.ts']
WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
[debug] Encodings: locale ANSI_X3.4-1968, fs ANSI_X3.4-1968, out ANSI_X3.4-1968, pref ANSI_X3.4-1968
[debug] youtube-dl version 2019.09.12.1
[debug] Python version 2.7.16 (CPython) - Linux-5.0.0-13-generic-x86_64-with-Ubuntu-19.04-disco
[debug] exe versions: ffmpeg 4.1.1-1, ffprobe 4.1.1-1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] 640x3600: Requesting header
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3600.ts'
[download] Destination: 640x3600-640x3600.ts
[download] 100% of 993.61KiB in 00:02
[generic] 640x3601: Requesting header
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3601.ts'
[download] Destination: 640x3601-640x3601.ts
[download] 100% of 803.59KiB in 00:02
[generic] 640x3602: Requesting header
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3602.ts'
[download] Destination: 640x3602-640x3602.ts
[download] 100% of 1.14MiB in 00:02
[generic] 640x3603: Requesting header
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3603.ts'
[download] Destination: 640x3603-640x3603.ts
[download] 100% of 900.53KiB in 00:02
[generic] 640x3604: Requesting header
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3604.ts'
[download] Destination: 640x3604-640x3604.ts
[download] 100% of 475.51KiB in 00:02
[generic] 640x3605: Requesting header
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3605.ts'
[download] Destination: 640x3605-640x3605.ts
[download] 100% of 116.95KiB in 00:01

After that finishes, I have these files:

ls -tr1
640x3600-640x3600.ts
640x3601-640x3601.ts
640x3602-640x3602.ts
640x3603-640x3603.ts
640x3604-640x3604.ts
640x3605-640x3605.ts
640x360-test.m3u8

Description

When using a remote m3u8 file, youtube-dl performs as expected. All individual files are downloaded and properly concatenated.

This works as expected:

youtube-dl https://walterebert.com/_/playground/videos/ts/640x360.m3u8

and outputs a single mp4 file, 640x360-640x360.mp4.

When using a local copy of an m3u8 file, in this case edited to include FQDNs, youtube-dl treats it as a normal batch-file and just downloads the transport-stream "pieces".

youtube-dl -v --ignore-config -a ./640x360-test.m3u8

Expected behaviour would be for youtube-dl to recognise that the batch-file is actually an m3u/m3u8 file and behave accordingly. Alternatively, there should be a command-line option to "force" a local file to be interpreted as an m3u/m3u8 file.

Originally created by @atom-smasher on GitHub (Sep 12, 2019). ## Checklist - [x] I've verified that I'm running youtube-dl version **2019.09.12.1** - [x] I've checked that all provided URLs are alive and playable in a browser - [x] I've checked that all URLs and arguments with special characters are properly quoted or escaped - [x] I've searched the bugtracker for similar bug reports including closed ones - [x] I've read bugs section in FAQ ## Verbose log ``` [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'-v', u'--ignore-config', u'-a', u'./640x360-test.m3u8'] [debug] Batch file urls: [u'https://walterebert.com/_/playground/videos/ts/640x3600.ts', u'https://walterebert.com/_/playground/videos/ts/640x3601.ts', u'https://walterebert.com/_/playground/videos/ts/640x3602.ts', u'https://walterebert.com/_/playground/videos/ts/640x3603.ts', u'https://walterebert.com/_/playground/videos/ts/640x3604.ts', u'https://walterebert.com/_/playground/videos/ts/640x3605.ts'] WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this. [debug] Encodings: locale ANSI_X3.4-1968, fs ANSI_X3.4-1968, out ANSI_X3.4-1968, pref ANSI_X3.4-1968 [debug] youtube-dl version 2019.09.12.1 [debug] Python version 2.7.16 (CPython) - Linux-5.0.0-13-generic-x86_64-with-Ubuntu-19.04-disco [debug] exe versions: ffmpeg 4.1.1-1, ffprobe 4.1.1-1, rtmpdump 2.4 [debug] Proxy map: {} [generic] 640x3600: Requesting header [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3600.ts' [download] Destination: 640x3600-640x3600.ts [download] 100% of 993.61KiB in 00:02 [generic] 640x3601: Requesting header [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3601.ts' [download] Destination: 640x3601-640x3601.ts [download] 100% of 803.59KiB in 00:02 [generic] 640x3602: Requesting header [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3602.ts' [download] Destination: 640x3602-640x3602.ts [download] 100% of 1.14MiB in 00:02 [generic] 640x3603: Requesting header [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3603.ts' [download] Destination: 640x3603-640x3603.ts [download] 100% of 900.53KiB in 00:02 [generic] 640x3604: Requesting header [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3604.ts' [download] Destination: 640x3604-640x3604.ts [download] 100% of 475.51KiB in 00:02 [generic] 640x3605: Requesting header [debug] Default format spec: bestvideo+bestaudio/best [debug] Invoking downloader on u'https://walterebert.com/_/playground/videos/ts/640x3605.ts' [download] Destination: 640x3605-640x3605.ts [download] 100% of 116.95KiB in 00:01 ``` After that finishes, I have these files: ``` ls -tr1 640x3600-640x3600.ts 640x3601-640x3601.ts 640x3602-640x3602.ts 640x3603-640x3603.ts 640x3604-640x3604.ts 640x3605-640x3605.ts 640x360-test.m3u8 ``` ## Description When using a remote m3u8 file, youtube-dl performs as expected. All individual files are downloaded and properly concatenated. This works as expected: `youtube-dl https://walterebert.com/_/playground/videos/ts/640x360.m3u8` and outputs a single mp4 file, `640x360-640x360.mp4`. When using a local copy of an m3u8 file, in this case edited to include FQDNs, youtube-dl treats it as a normal batch-file and just downloads the transport-stream "pieces". `youtube-dl -v --ignore-config -a ./640x360-test.m3u8` Expected behaviour would be for youtube-dl to recognise that the batch-file is actually an m3u/m3u8 file and behave accordingly. Alternatively, there should be a command-line option to "force" a local file to be interpreted as an m3u/m3u8 file.

deekerman closed this issue

2026-02-21 09:43:30 -05:00

deekerman commented

2026-02-21 09:43:32 -05:00

Author

Owner

@atom-smasher commented on GitHub (Sep 12, 2019):

Here's a copy of the m3u file used above, edited to include FQDNs.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:9.189889,
https://walterebert.com/_/playground/videos/ts/640x3600.ts
#EXTINF:8.916667,
https://walterebert.com/_/playground/videos/ts/640x3601.ts
#EXTINF:9.333333,
https://walterebert.com/_/playground/videos/ts/640x3602.ts
#EXTINF:10.666667,
https://walterebert.com/_/playground/videos/ts/640x3603.ts
#EXTINF:9.000000,
https://walterebert.com/_/playground/videos/ts/640x3604.ts
#EXTINF:5.083333,
https://walterebert.com/_/playground/videos/ts/640x3605.ts
#EXT-X-ENDLIST

@atom-smasher commented on GitHub (Sep 12, 2019): Here's a copy of the m3u file used above, edited to include FQDNs. ``` #EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:11 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:9.189889, https://walterebert.com/_/playground/videos/ts/640x3600.ts #EXTINF:8.916667, https://walterebert.com/_/playground/videos/ts/640x3601.ts #EXTINF:9.333333, https://walterebert.com/_/playground/videos/ts/640x3602.ts #EXTINF:10.666667, https://walterebert.com/_/playground/videos/ts/640x3603.ts #EXTINF:9.000000, https://walterebert.com/_/playground/videos/ts/640x3604.ts #EXTINF:5.083333, https://walterebert.com/_/playground/videos/ts/640x3605.ts #EXT-X-ENDLIST ```

deekerman commented

2026-02-21 09:43:32 -05:00

Author

Owner

@atom-smasher commented on GitHub (Sep 12, 2019):

As a workaround, I tried handling the m3u file as a local URL.

youtube-dl --ignore-config file:///tmp/test-m3u8/640x360-test.m3u8

That failed, apparently as a result of a previous security fix, #8228

WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
[generic] 640x360-test: Requesting header
WARNING: Could not send HEAD request to file:///tmp/test-m3u8/640x360-test.m3u8: <urlopen error file:// scheme is explicitly disabled in youtube-dl for security reasons>
[generic] 640x360-test: Downloading webpage
ERROR: Unable to download webpage: <urlopen error file:// scheme is explicitly disabled in youtube-dl for security reasons> (caused by URLError(u'file:// scheme is explicitly disabled in youtube-dl for security reasons',))

While that security fix makes sense, and a command-line option alone would be a poor way to over-ride it, I'm wondering if a config-file option and a command-line option together may be a reasonable way to over-ride that security fix.

Ultimately, the fix for this problem should be some combination of (a) a command-line option to force local files to be interpreted as m3u/m3u8 files, (b) automatic recognition of m3u/m3u8 files by parsing, (c) automatic recognition of m3u/m3u8 files by filename extension.

@atom-smasher commented on GitHub (Sep 12, 2019): As a workaround, I tried handling the m3u file as a local URL. `youtube-dl --ignore-config file:///tmp/test-m3u8/640x360-test.m3u8` That failed, apparently as a result of a previous security fix, #8228 ``` WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this. [generic] 640x360-test: Requesting header WARNING: Could not send HEAD request to file:///tmp/test-m3u8/640x360-test.m3u8: <urlopen error file:// scheme is explicitly disabled in youtube-dl for security reasons> [generic] 640x360-test: Downloading webpage ERROR: Unable to download webpage: <urlopen error file:// scheme is explicitly disabled in youtube-dl for security reasons> (caused by URLError(u'file:// scheme is explicitly disabled in youtube-dl for security reasons',)) ``` While that security fix makes sense, and a command-line option alone would be a poor way to over-ride it, I'm wondering if a config-file option and a command-line option together may be a reasonable way to over-ride that security fix. Ultimately, the fix for this problem should be some combination of (a) a command-line option to force local files to be interpreted as m3u/m3u8 files, (b) automatic recognition of m3u/m3u8 files by parsing, (c) automatic recognition of m3u/m3u8 files by filename extension.

deekerman commented

2026-02-21 09:43:32 -05:00

Author

Owner

@atom-smasher commented on GitHub (Sep 14, 2019):

Python is not a strong point of mine, but it looks like this is where remote URLs are parsed to see if they're M3U files - github.com/ytdl-org/youtube-dl@7cb51b5daf/youtube_dl/extractor/generic.py (L2349)

Should be fairly easy to apply the same test to local files...?

@atom-smasher commented on GitHub (Sep 14, 2019): Python is not a strong point of mine, but it looks like this is where remote URLs are parsed to see if they're M3U files - https://github.com/ytdl-org/youtube-dl/blob/7cb51b5daf07a6a627a9084394636c570194cc4e/youtube_dl/extractor/generic.py#L2349 Should be fairly easy to apply the same test to local files...?

deekerman commented

2026-02-21 09:43:32 -05:00

Author

Owner

@dstftw commented on GitHub (Sep 15, 2019):

Batch file has nothing to do with m3u8 and not supposed to be used with -a. What you can do is setup local webserver where to put m3u8 with all contents.

@dstftw commented on GitHub (Sep 15, 2019): Batch file has nothing to do with m3u8 and not supposed to be used with `-a`. What you can do is setup local webserver where to put m3u8 with all contents.

deekerman commented

2026-02-21 09:43:32 -05:00

Author

Owner

@atom-smasher commented on GitHub (Sep 15, 2019):

Setting up a local web-server is not always a viable option.

Parsing a local file (automatically and/or with a command-line option) should be available.

Is there any reason why a local file could not be an M3U file???

@atom-smasher commented on GitHub (Sep 15, 2019): Setting up a local web-server is not always a viable option. Parsing a local file (automatically and/or with a command-line option) should be available. Is there any reason why a local file could not be an M3U file???

deekerman commented

2026-02-21 09:43:32 -05:00

Author

Owner

@Zabeus commented on GitHub (Feb 12, 2021):

@atom-smasher I was able to work around this by using a temporary file sharing service like tmpfiles or file.io.

@Zabeus commented on GitHub (Feb 12, 2021): @atom-smasher I was able to work around this by using a temporary file sharing service like tmpfiles or file.io.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/youtube-dl-ytdl-org#17915

No description provided.

Rows
Columns