RTLXL.nl stopped working #3318

Closed
opened 2026-02-21 01:15:42 -05:00 by deekerman · 12 comments
Owner

Originally created by @MatrixView on GitHub (Nov 5, 2014).

youtube-dl http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400

[rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading JSON metadata
[rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading f4m manifest
ERROR: Unable to download f4m manifest: HTTP Error 404: File or directory not found. (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Be sure
to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update.

I'm running the latest YT-DL version on Win7. Something probably changed on the RTLxl.nl site?
Thanx in advance for looking into this issue!

Originally created by @MatrixView on GitHub (Nov 5, 2014). ## youtube-dl http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400 [rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading JSON metadata [rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading f4m manifest ERROR: Unable to download f4m manifest: HTTP Error 404: File or directory not found. (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update. I'm running the latest YT-DL version on Win7. Something probably changed on the RTLxl.nl site? Thanx in advance for looking into this issue!
Author
Owner

@iggyvolz commented on GitHub (Nov 5, 2014):

Be sure to call youtube-dl with the --verbose flag and include its complete output.

@iggyvolz commented on GitHub (Nov 5, 2014): > Be sure to call youtube-dl with the --verbose flag and include its complete output.
Author
Owner

@MatrixView commented on GitHub (Nov 5, 2014):

youtube-dl --verbose http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2014.11.04
[debug] Python version 2.7.8 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-67388-, ffprobe N-67388-, rtmpdump 2.4-47-gc26c541
[debug] Proxy map: {}
[rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading JSON metadata
[rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading f4m manifest
ERROR: Unable to download f4m manifest: HTTP Error 404: File or directory not found. (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Be sure
to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update.
File "youtube_dl\extractor\common.pyo", line 232, in _request_webpage
File "youtube_dl\YoutubeDL.pyo", line 1289, in urlopen
File "urllib2.pyo", line 410, in open
File "urllib2.pyo", line 523, in http_response
File "urllib2.pyo", line 448, in error
File "urllib2.pyo", line 382, in _call_chain
File "urllib2.pyo", line 531, in http_error_default

@MatrixView commented on GitHub (Nov 5, 2014): ## youtube-dl --verbose http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400 [debug] System config: [] [debug] User config: [] [debug] Command-line args: ['--verbose', 'http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400'] [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252 [debug] youtube-dl version 2014.11.04 [debug] Python version 2.7.8 - Windows-7-6.1.7601-SP1 [debug] exe versions: ffmpeg N-67388-, ffprobe N-67388-, rtmpdump 2.4-47-gc26c541 [debug] Proxy map: {} [rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading JSON metadata [rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading f4m manifest ERROR: Unable to download f4m manifest: HTTP Error 404: File or directory not found. (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update. File "youtube_dl\extractor\common.pyo", line 232, in _request_webpage File "youtube_dl\YoutubeDL.pyo", line 1289, in urlopen File "urllib2.pyo", line 410, in open File "urllib2.pyo", line 523, in http_response File "urllib2.pyo", line 448, in error File "urllib2.pyo", line 382, in _call_chain File "urllib2.pyo", line 531, in http_error_default
Author
Owner

@iggyvolz commented on GitHub (Nov 5, 2014):

Strangely enough, that command wouldn't work at all on my mac's Bash shell - I believe it was picking up ! as an event of some sort. Running export E="!" then replacing the ! with $E in the command gave me the same error:

$ export E="!"
$ youtube-dl --verbose "http://www.rtlxl.nl/#$E/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400']
[debug] Encodings: locale UTF-8, fs utf-8, out None, pref UTF-8
[debug] youtube-dl version 2014.11.02.1
[debug] Python version 2.7.8 - Darwin-13.4.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 2.4.2, ffprobe 2.4.2
[debug] Proxy map: {}
[rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading JSON metadata
[rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading f4m manifest
ERROR: Unable to download f4m manifest: HTTP Error 404: File or directory not found. (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 231, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1286, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

I'll look more into this later, but it seems to be that the extractor API has changed.

@iggyvolz commented on GitHub (Nov 5, 2014): Strangely enough, that command wouldn't work at all on my mac's Bash shell - I believe it was picking up ! as an event of some sort. Running `export E="!"` then replacing the ! with $E in the command gave me the same error: ``` bash $ export E="!" $ youtube-dl --verbose "http://www.rtlxl.nl/#$E/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400" [debug] System config: [] [debug] User config: [] [debug] Command-line args: ['--verbose', 'http://www.rtlxl.nl/#!/rtl-late-night-301978/14291f72-b3c2-33f3-8fa5-01cf48b31400'] [debug] Encodings: locale UTF-8, fs utf-8, out None, pref UTF-8 [debug] youtube-dl version 2014.11.02.1 [debug] Python version 2.7.8 - Darwin-13.4.0-x86_64-i386-64bit [debug] exe versions: ffmpeg 2.4.2, ffprobe 2.4.2 [debug] Proxy map: {} [rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading JSON metadata [rtlxl.nl] 14291f72-b3c2-33f3-8fa5-01cf48b31400: Downloading f4m manifest ERROR: Unable to download f4m manifest: HTTP Error 404: File or directory not found. (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update. File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 231, in _request_webpage return self._downloader.urlopen(url_or_request) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1286, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 410, in open response = meth(req, response) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 523, in http_response 'http', request, response, code, msg, hdrs) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 448, in error return self._call_chain(*args) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 382, in _call_chain result = func(*args) File "/usr/local/Cellar/python/2.7.8_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 531, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) ``` I'll look more into this later, but it seems to be that the extractor API has changed.
Author
Owner

@naglis commented on GitHub (Nov 18, 2014):

I have changed the rtlxl.nl extractor to use m3u8 streams instead of f4m, so this issue should be fixed in the next release of youtube-dl.

@iggyvolz, wow that's quite a trick you've pulled with the variable substitution! However, you could've just used single quotes instead of double quotes, as single quotes inhibit all interpretation of the string inside the quotes.

@naglis commented on GitHub (Nov 18, 2014): I have changed the rtlxl.nl extractor to use m3u8 streams instead of f4m, so this issue should be fixed in the next release of youtube-dl. @iggyvolz, wow that's quite a trick you've pulled with the variable substitution! However, you could've just used single quotes instead of double quotes, as single quotes [inhibit all interpretation](http://www.gnu.org/software/bash/manual/html_node/Quoting.html#Quoting) of the string inside the quotes.
Author
Owner

@Reino17 commented on GitHub (Nov 19, 2014):

I see you've chosen to go for the 'adaptive-json' (DRM variant). Why? (d=pc/ in line 33 of rtlnl.py isnt necessary btw)
I hardly know any python, but wouldn't the following work? (with the emphasis on replace())

info = self._download_json(
    'http://www.rtl.nl/system/s4m/vfd/version=2/uuid=%s/fmt=flash/' % uuid,
uuid)
...
videopath = material['videopath'].replace('f4m','m3u8')
m3u8_url = 'http://manifest.us.rtl.nl' + videopath

formats = self._extract_m3u8_formats(m3u8_url, uuid, ext='mp4')

video_urlpart = videopath.split('/flash/')[1][:-4]

Simply replace 'f4m' with 'm3u8' in "videopath" of the 'flash-json', and you have the 'non-DRM-manifest-playlist'.

@Reino17 commented on GitHub (Nov 19, 2014): I see you've chosen to go for the 'adaptive-json' (DRM variant). Why? (`d=pc/` in [line 33 of rtlnl.py](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/rtlnl.py#L33) isnt necessary btw) I hardly know any python, but wouldn't the following work? (with the emphasis on `replace()`) ``` python info = self._download_json( 'http://www.rtl.nl/system/s4m/vfd/version=2/uuid=%s/fmt=flash/' % uuid, uuid) ... videopath = material['videopath'].replace('f4m','m3u8') m3u8_url = 'http://manifest.us.rtl.nl' + videopath formats = self._extract_m3u8_formats(m3u8_url, uuid, ext='mp4') video_urlpart = videopath.split('/flash/')[1][:-4] ``` Simply replace 'f4m' with 'm3u8' in "videopath" of the 'flash-json', and you have the 'non-DRM-manifest-playlist'.
Author
Owner

@naglis commented on GitHub (Nov 19, 2014):

My motivation was to make it more robust, so I have tried to mimic the behavior of the flash player in the browser. If you can confirm that the method of simply replacing the f4m with m3u8 works ubiquitously (as I don't use rtlxl.nl personally) and/or you have issues with the encrypted HLS streams - I don't have anything against changing it so that it would work better for you and others 😉

@naglis commented on GitHub (Nov 19, 2014): My motivation was to make it more robust, so I have tried to mimic the behavior of the flash player in the browser. If you can confirm that the method of simply replacing the `f4m` with `m3u8` works ubiquitously (as I don't use rtlxl.nl personally) and/or you have issues with the encrypted HLS streams - I don't have anything against changing it so that it would work better for you and others :wink:
Author
Owner

@Reino17 commented on GitHub (Nov 20, 2014):

I have just tested numerous videos and I can confirm this method works.

@Reino17 commented on GitHub (Nov 20, 2014): I have just tested numerous videos and I can confirm this method works.
Author
Owner

@Reino17 commented on GitHub (Nov 22, 2014):

Hi naglis, are you still going to make that change? If not, then at least change line 47 accordingly:

video_urlpart = videopath.split('/adaptive/')[1][:-5]
@Reino17 commented on GitHub (Nov 22, 2014): Hi naglis, are you still going to make that change? If not, then at least change [line 47](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/rtlnl.py#L47) accordingly: ``` python video_urlpart = videopath.split('/adaptive/')[1][:-5] ```
Author
Owner

@naglis commented on GitHub (Nov 22, 2014):

Hi @corone17,
I've made the change, it should be included in the next release of youtube-dl. Thanks again for your help.

BTW, this got a little mixed with the original issue, feel free to open new issues for any problems/suggestions you might have in the future ;)

@naglis commented on GitHub (Nov 22, 2014): Hi @corone17, I've made the change, it should be included in the next release of youtube-dl. Thanks again for your help. BTW, this got a little mixed with the original issue, feel free to open new issues for any problems/suggestions you might have in the future ;)
Author
Owner

@Reino17 commented on GitHub (Nov 23, 2014):

Sorry to yet again comment here, but this still concerns this issue. Like I pointed out above, line 47 is still incorrect.

From the JSON:

"videopath":"/rtlxl/v166/network/pc/flash/components/actueel/rtlnieuws/307089/2014w45/e9ee31
e4-e18b-3bca-bca4-dd9aa0da6976.ssm/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976.f4m",

Then because videopath = material['videopath'].replace('.f4m', '.m3u8') videopath becomes:

/rtlxl/v166/network/pc/flash/components/actueel/rtlnieuws/307089/2014w45/e9ee31e4-e18b-3bca-
bca4-dd9aa0da6976.ssm/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976.m3u8

The extension is now 4 characters instead of 3 and therefore after

 video_urlpart = videopath.split('/flash/')[1][:-4]
PG_URL_TEMPLATE = 'http://pg.us.rtl.nl/rtlxl/network/%s/progressive/%s.mp4'
formats.extend([
{
'url': PG_URL_TEMPLATE % ('a2m', video_urlpart),
'format_id': 'pg-sd',
},
{
'url': PG_URL_TEMPLATE % ('a3m', video_urlpart),
'format_id': 'pg-hd',
'quality': 0,
}
])

format pg-hd is

http://pg.us.rtl.nl/rtlxl/network/a3m/progressive/components/actueel/rtlnieuws/307089/2014w4
5/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976.ssm/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976..mp4

and has a period too many. Line 47 therefore has to be:

video_urlpart = videopath.split('/adaptive/')[1][:-5]
@Reino17 commented on GitHub (Nov 23, 2014): Sorry to yet again comment here, but this still concerns this issue. Like I pointed out above, line 47 is still incorrect. From the JSON: ``` "videopath":"/rtlxl/v166/network/pc/flash/components/actueel/rtlnieuws/307089/2014w45/e9ee31 e4-e18b-3bca-bca4-dd9aa0da6976.ssm/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976.f4m", ``` Then because `videopath = material['videopath'].replace('.f4m', '.m3u8')` videopath becomes: ``` /rtlxl/v166/network/pc/flash/components/actueel/rtlnieuws/307089/2014w45/e9ee31e4-e18b-3bca- bca4-dd9aa0da6976.ssm/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976.m3u8 ``` The extension is now 4 characters instead of 3 and therefore after ``` python video_urlpart = videopath.split('/flash/')[1][:-4] PG_URL_TEMPLATE = 'http://pg.us.rtl.nl/rtlxl/network/%s/progressive/%s.mp4' formats.extend([ { 'url': PG_URL_TEMPLATE % ('a2m', video_urlpart), 'format_id': 'pg-sd', }, { 'url': PG_URL_TEMPLATE % ('a3m', video_urlpart), 'format_id': 'pg-hd', 'quality': 0, } ]) ``` format `pg-hd` is ``` http://pg.us.rtl.nl/rtlxl/network/a3m/progressive/components/actueel/rtlnieuws/307089/2014w4 5/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976.ssm/e9ee31e4-e18b-3bca-bca4-dd9aa0da6976..mp4 ``` and has a period too many. [Line 47](https://github.com/rg3/youtube-dl/blob/master/youtube_dl/extractor/rtlnl.py#L47) therefore has to be: ``` python video_urlpart = videopath.split('/adaptive/')[1][:-5] ```
Author
Owner

@naglis commented on GitHub (Nov 23, 2014):

That was pretty sloppy of me. Thanks! Should be fixed in the next version of youtube-dl.

@naglis commented on GitHub (Nov 23, 2014): That was pretty sloppy of me. Thanks! Should be fixed in the next version of youtube-dl.
Author
Owner

@Reino17 commented on GitHub (Nov 25, 2014):

Working fine now. Thanks, naglis.

@Reino17 commented on GitHub (Nov 25, 2014): Working fine now. Thanks, naglis.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/youtube-dl-ytdl-org#3318
No description provided.