Proxy only when needed (Global feature)

deekerman commented

2026-02-21 04:26:35 -05:00

Owner

Originally created by @spookyahell on GitHub (Aug 29, 2016).

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2016.08.28. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

I've verified and I assure that I'm running youtube-dl 2016.08.28

Before submitting an issue make sure you have:

At least skimmed through README and most notably FAQ and BUGS sections
Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

Bug report (encountered problems with youtube-dl)
Site support request (request for adding support for a new site)
Feature request (request for a new functionality)
Question
Other

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/

Description of your issue, suggested solution and other information

It would be nice to be able to set a proxy that only gets active when a regular request fails.
So, shall any request be getting an error 403 then it would repeat the request with proxy instead of outputting error imediatly. Option could be something like --proxy-on-occasion and -proxoc and then the HTTP-Proxy Link. This would need to be global for the entire application. By default 403 should result in a re-request of the "Forbidden" object. And then there could be a comma seperated list option like this --proxoc-error-types and a valid value would be 403,404,503

Originally created by @spookyahell on GitHub (Aug 29, 2016). ## Please follow the guide below - You will be asked some questions and requested to provide some information, please read them **carefully** and answer honestly - Put an `x` into all the boxes [ ] relevant to your _issue_ (like that [x]) - Use _Preview_ tab to see how your issue will actually look like --- ### Make sure you are using the _latest_ version: run `youtube-dl --version` and ensure your version is _2016.08.28_. If it's not read [this FAQ entry](https://github.com/rg3/youtube-dl/blob/master/README.md#how-do-i-update-youtube-dl) and update. Issues with outdated version will be rejected. - [x] I've **verified** and **I assure** that I'm running youtube-dl **2016.08.28** ### Before submitting an _issue_ make sure you have: - [x] At least skimmed through [README](https://github.com/rg3/youtube-dl/blob/master/README.md) and **most notably** [FAQ](https://github.com/rg3/youtube-dl#faq) and [BUGS](https://github.com/rg3/youtube-dl#bugs) sections - [x] [Searched](https://github.com/rg3/youtube-dl/search?type=Issues) the bugtracker for similar issues including closed ones ### What is the purpose of your _issue_? - [ ] Bug report (encountered problems with youtube-dl) - [ ] Site support request (request for adding support for a new site) - [x] Feature request (request for a new functionality) - [ ] Question - [ ] Other --- ### If the purpose of this _issue_ is a _site support request_ please provide all kinds of example URLs support for which should be included (replace following example URLs by **yours**): - Single video: http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/ --- ### Description of your _issue_, suggested solution and other information It would be nice to be able to set a proxy that only gets active when a regular request fails. So, shall any request be getting an error 403 then it would repeat the request with proxy instead of outputting error imediatly. Option could be something like `--proxy-on-occasion` and `-proxoc` and then the HTTP-Proxy Link. This would need to be global for the entire application. By default 403 should result in a re-request of the "Forbidden" object. And then there could be a comma seperated list option like this `--proxoc-error-types` and a valid value would be `403,404,503`

deekerman closed this issue

2026-02-21 04:26:37 -05:00

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@yan12125 commented on GitHub (Aug 30, 2016):

Not quite understand the motivation. Why not just add --proxy if you know a video is geo-restricted?

@yan12125 commented on GitHub (Aug 30, 2016): Not quite understand the motivation. Why not just add `--proxy` if you know a video is geo-restricted?

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@spookyahell commented on GitHub (Aug 30, 2016):

Because the proxy itself currently doesn't support all URLs but frankly it needs a proxy. But theres a proxy already setup in the network, but it also doesn't support them all, so it's like fail standard then use this proxy, it's complicated.

@spookyahell commented on GitHub (Aug 30, 2016): Because the proxy itself currently doesn't support all URLs but frankly it needs a proxy. But theres a proxy already setup in the network, but it also doesn't support them all, so it's like fail standard then use this proxy, it's complicated.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@yan12125 commented on GitHub (Aug 30, 2016):

Sounds like --geo-verification-proxy or a Proxy Auto Config (PAC) script is what you need. The latter is requested in #8278, and I'm working on it recently. There are several requests involved in http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/. Can you elaborate what kind of URLs should go to the first proxy and what should go to the second?

@yan12125 commented on GitHub (Aug 30, 2016): Sounds like `--geo-verification-proxy` or a Proxy Auto Config (PAC) script is what you need. The latter is requested in #8278, and I'm working on it recently. There are several requests involved in http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/. Can you elaborate what kind of URLs should go to the first proxy and what should go to the second?

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@spookyahell commented on GitHub (Aug 30, 2016):

Well, urls with link.theplatform need to go reguarly and ipad-streaming.cbs.com need to go via the proxy specified wherever.

@spookyahell commented on GitHub (Aug 30, 2016): Well, urls with link.theplatform need to go reguarly and ipad-streaming.cbs.com need to go via the proxy specified wherever.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@yan12125 commented on GitHub (Aug 30, 2016):

OK PAC scripts should fit your need. Please move to #8278 for further discussion.

@yan12125 commented on GitHub (Aug 30, 2016): OK PAC scripts should fit your need. Please move to #8278 for further discussion.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@dstftw commented on GitHub (Aug 30, 2016):

PAC script won't help when same host serves geo restricted videos along with non geo restricted.

@dstftw commented on GitHub (Aug 30, 2016): PAC script won't help when same host serves geo restricted videos along with non geo restricted.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@yan12125 commented on GitHub (Aug 30, 2016):

FindProxyForURL, the main function in PAC scripts, accepts two arguments url and host. As long as the restriction is determined by URL, PAC scripts can handle it.

@yan12125 commented on GitHub (Aug 30, 2016): `FindProxyForURL`, the main function in PAC scripts, accepts two arguments `url` and `host`. As long as the restriction is determined by URL, PAC scripts can handle it.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@dstftw commented on GitHub (Aug 30, 2016):

Most of geo restrictions I've seen have no special pattern for geo restricted videos thus managing rules per URL is not feasible. It's much easier to simply pass --proxy in this case rather than bother writing a rule for exact URL that will only be used once.

@dstftw commented on GitHub (Aug 30, 2016): Most of geo restrictions I've seen have no special pattern for geo restricted videos thus managing rules per URL is not feasible. It's much easier to simply pass `--proxy` in this case rather than bother writing a rule for exact URL that will only be used once.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@yan12125 commented on GitHub (Aug 31, 2016):

Did you mean some videos on link.theplatform.com are geo-restricted while others not? My idea is that all requests to link.theplatform.com go through a proxy, no matter it's geo-restricted or not. A browser plugin Unblock-Youku uses PAC scripts to bypass geo-restrictions on several Chinese websites. If it still sounds bad, please go ahead and reopen this issue.

Note that for http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/, the key request http://link.theplatform.com/s/dJ5BDC/media/guid/2198311517/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3?mbr=true&manifest=m3u&format=SMIL always returns HTTP 200 OK, so the proposed approach does not work. A possible solution may be adding a new Exception class for geo-restriction errors, just like #10308 for authentication.

@yan12125 commented on GitHub (Aug 31, 2016): Did you mean some videos on link.theplatform.com are geo-restricted while others not? My idea is that all requests to link.theplatform.com go through a proxy, no matter it's geo-restricted or not. A browser plugin [Unblock-Youku](https://github.com/uku/Unblock-Youku) uses PAC scripts to bypass geo-restrictions on several Chinese websites. If it still sounds bad, please go ahead and reopen this issue. Note that for http://www.cbs.com/shows/big_bang_theory/video/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3/the-big-bang-theory-the-sales-call-sublimation/, the key request http://link.theplatform.com/s/dJ5BDC/media/guid/2198311517/B4432DE4-B3E6-E36F-48A3-1956A2D37BE3?mbr=true&manifest=m3u&format=SMIL always returns HTTP 200 OK, so the proposed approach does not work. A possible solution may be adding a new Exception class for geo-restriction errors, just like #10308 for authentication.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@dstftw commented on GitHub (Aug 31, 2016):

I wasn't concretely about theplatform. What I was trying to say is that if say there is an API URL http://foo.bar/video/<id> where we put video id to fetch video data and this URL succeeds for non-restricted videos and fails with 403 for restricted videos then there is no way to figure out beforehand whether access to this URL needs to be proxied or not as there is not special pattern in URL that allows to distinguish them.

@dstftw commented on GitHub (Aug 31, 2016): I wasn't concretely about theplatform. What I was trying to say is that if say there is an API URL `http://foo.bar/video/<id>` where we put video id to fetch video data and this URL succeeds for non-restricted videos and fails with 403 for restricted videos then there is no way to figure out beforehand whether access to this URL needs to be proxied or not as there is not special pattern in URL that allows to distinguish them.

deekerman commented

2026-02-21 04:26:40 -05:00

Author

Owner

@yan12125 commented on GitHub (Aug 31, 2016):

Maybe I wasn't clear. My proposal is: proxy all such API requests, no matter the corresponding video is geo-restricted or not. That is, assume an API request always requires proxy. What PAC scripts do is distinguishing URLs that may require proxy.

@yan12125 commented on GitHub (Aug 31, 2016): Maybe I wasn't clear. My proposal is: proxy all such API requests, no matter the corresponding video is geo-restricted or not. That is, assume an API request always requires proxy. What PAC scripts do is distinguishing URLs that _may_ require proxy.

Rows
Columns

Proxy only when needed (Global feature) #8591

Please follow the guide below

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.08.28. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

Before submitting an issue make sure you have:

What is the purpose of your issue?

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Description of your issue, suggested solution and other information

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2016.08.28. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.