starred/Prowlarr

Fork 0

mirror of https://github.com/Prowlarr/Prowlarr.git synced 2026-03-02 22:57:22 -05:00

Customizable Up check for Indexer Proxies #1082

New issue

Open

opened 2026-02-20 10:07:07 -05:00 by deekerman · 8 comments

deekerman commented

2026-02-20 10:07:07 -05:00

Owner

Originally created by @mtalexan on GitHub (Sep 12, 2024).

Is there an existing issue for this?

I have searched the existing open and closed issues

All Indexer Proxies are currently tested to see if they're working by trying to reach https://prowlarr.servarr.com/v1/ping, but that assumes:

The proxy has a path to clearnet
The proxy doesn't run afoul of the servarr.com abuse prevention detection

Not all proxies (e.g. I2P) expect to have stable clearnet access, so currently they're largely unusable with Prowlarr.

The servarr.com site has pretty strong abuse prevention filters which make it highly likely connections will be blocked by it that wouldn't be blocked by most other sites. For example, most VPN connections to it are blocked but are accepted by most of the indexers. Tor exit nodes also seem to be geoblocked so they have inconsistent results.

Describe the solution you'd like

An advanced option in the Indexer Proxy configuration that allows specifying a custom endpoint instead of the default https://prowlarr.servarr.com/v1/ping.

For non-clearnet proxies, this would allow configuring an endpoint within the closed network.

It would also allow the servarr.com abuse prevention mechanisms to be independently managed relative to the indexer proxy support, instead of like now where servarr.com abuse prevention directly controls the unrelated prowlarr functionality.

Describe alternatives you've considered

Alternative hardcoded endpoints available from a drop down that have different levels of abuse prevention filtering.
This doesn't solve the clearnet access issue, but it would make it clear if your proxy might be blocked when accessing an indexer due to proxy reputation or other abuse prevention mechanisms.

Anything else?

None

Originally created by @mtalexan on GitHub (Sep 12, 2024). ### Is there an existing issue for this? - [X] I have searched the existing open and closed issues ### Is your feature request related to a problem? Please describe All Indexer Proxies are currently tested to see if they're working by trying to reach https://prowlarr.servarr.com/v1/ping, but that assumes: 1. The proxy has a path to clearnet 2. The proxy doesn't run afoul of the servarr.com abuse prevention detection Not all proxies (e.g. I2P) expect to have stable clearnet access, so currently they're largely unusable with Prowlarr. The servarr.com site has pretty strong abuse prevention filters which make it highly likely connections will be blocked by it that wouldn't be blocked by most other sites. For example, most VPN connections to it are blocked but are accepted by most of the indexers. Tor exit nodes also seem to be geoblocked so they have inconsistent results. ### Describe the solution you'd like An advanced option in the Indexer Proxy configuration that allows specifying a custom endpoint instead of the default https://prowlarr.servarr.com/v1/ping. For non-clearnet proxies, this would allow configuring an endpoint within the closed network. It would also allow the servarr.com abuse prevention mechanisms to be independently managed relative to the indexer proxy support, instead of like now where servarr.com abuse prevention directly controls the unrelated prowlarr functionality. ### Describe alternatives you've considered Alternative hardcoded endpoints available from a drop down that have different levels of abuse prevention filtering. This doesn't solve the clearnet access issue, but it would make it clear if your proxy might be blocked when accessing an indexer due to proxy reputation or other abuse prevention mechanisms. ### Anything else? None

deekerman added the

Type: Feature Request

Status: Help Wanted

Priority: Low

Status: Needs Triage

labels

2026-02-20 10:07:07 -05:00

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@mynameisbogdan commented on GitHub (Sep 13, 2024):

You can provide a suggestion as an alternative that we can retry on if the Servarr ping fails, but we don't adding a way to disable or allow customizable addresses because it simply defeats the point of checking if the proxy connection is green or not.

@mynameisbogdan commented on GitHub (Sep 13, 2024): You can provide a suggestion as an alternative that we can retry on if the Servarr ping fails, but we don't adding a way to disable or allow customizable addresses because it simply defeats the point of checking if the proxy connection is green or not.

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@mtalexan commented on GitHub (Sep 13, 2024):

but we don't adding a way to disable or allow customizable addresses because it simply defeats the point of checking if the proxy connection is green or not.

The purpose is to determine if the proxy is working properly, but in many cases th e current endpoint will give a false negative.

Consider the postman indexer on the i2p network. It exists on that network only, and is accessed by entering the network via a proxy and never leaving. In fact, i2p is not designed to route traffic to exit nodes at all. The only valid health check endpoint for this proxy would be one within the i2p network.

Similarly, if I had an indexer at a tor onion addresses, I'd only need a tor proxy where exit nodes are irrelevant. The valid health check on it could only be at an onion address.

Those are just two examples of Non-Clearnet proxies. In practice there could any number, which in most cases would by design not have clearnet access to reach the current hardcoded endpoint.

Additionally, the current endpoint is on a server which actively blocks connections using what seems to be standard aggressive anti-abuse heuristics that are notorious for a very very high false block rate. That's reasonable for a website that's regularly abused, but not for a simple proxy health check endpoint. Basically, if the current endpoint isn't producing the same results as using something like a ping of google.com (it's not), it's not working solely as a health check.
Instead, the current endpoint is checking both health, and reputation scores. The nature of what the indexers are, and what they enable, gives they themselves relatively poor reputation scores and consequently they often allow connections from endpoints with similar scores. The current health check endpoint mandates that the proxy have a high reputation however, by not replying to low reputation proxy exit points, and will mark the proxy as down if it doesn't have the higher reputation than what's needed.
In fact, standard anti-abuse heuristics will lower reputation of most VPNs just for being a VPN. And VPNs that support P2P traffic are almost always even lower reputation because they are private and anonymous (which abusive users also want). Since indexers exist solely to facilitate P2P traffic, the current implementation means a proxy thru the same VPN used for the P2P traffic will fail the "health check" almost 100% of the time.

It could be useful to be aware of your proxy's reputation since it could affect your access to some indexers, but mandating a very high bar as the only option isn't reasonable, which is why I proposed a way for users to opt-in to an alternative measure. By making it an advanced setting, the users who make use of it will presumably be aware of the repercussions.

@mtalexan commented on GitHub (Sep 13, 2024): > but we don't adding a way to disable or allow customizable addresses because it simply defeats the point of checking if the proxy connection is green or not. The purpose is to determine if the proxy is working properly, but in many cases th e current endpoint will give a false negative. Consider the postman indexer on the i2p network. It exists on that network only, and is accessed by entering the network via a proxy and never leaving. In fact, i2p is not designed to route traffic to exit nodes at all. The only valid health check endpoint for this proxy would be one within the i2p network. Similarly, if I had an indexer at a tor onion addresses, I'd only need a tor proxy where exit nodes are irrelevant. The valid health check on it could only be at an onion address. Those are just two examples of Non-Clearnet proxies. In practice there could any number, which in most cases would by design not have clearnet access to reach the current hardcoded endpoint. Additionally, the current endpoint is on a server which actively blocks connections using what seems to be standard aggressive anti-abuse heuristics that are notorious for a very very high false block rate. That's reasonable for a website that's regularly abused, but not for a simple proxy health check endpoint. Basically, if the current endpoint isn't producing the same results as using something like a ping of google.com (it's not), it's not working solely as a health check. Instead, the current endpoint is checking both health, and reputation scores. The nature of what the indexers are, and what they enable, gives they themselves relatively poor reputation scores and consequently they often allow connections from endpoints with similar scores. The current health check endpoint mandates that the proxy have a high reputation however, by not replying to low reputation proxy exit points, and will mark the proxy as down if it doesn't have the higher reputation than what's needed. In fact, standard anti-abuse heuristics will lower reputation of most VPNs just for being a VPN. And VPNs that support P2P traffic are almost always even lower reputation because they are private and anonymous (which abusive users also want). Since indexers exist solely to facilitate P2P traffic, the current implementation means a proxy thru the same VPN used for the P2P traffic will fail the "health check" almost 100% of the time. It could be useful to be aware of your proxy's reputation since it could affect your access to some indexers, but mandating a very high bar as the only option isn't reasonable, which is why I proposed a way for users to opt-in to an alternative measure. By making it an advanced setting, the users who make use of it will presumably be aware of the repercussions.

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@mynameisbogdan commented on GitHub (Sep 13, 2024):

You wrote some stuff, but I don't see a proper solution to all of this since the indexer proxy has nothing to do with P2P traffic, just basic HTTP.

Feel free to propose a viable solution, maybe even a pull request if you're interested in making it work for you.

@mynameisbogdan commented on GitHub (Sep 13, 2024): You wrote some stuff, but I don't see a proper solution to all of this since the indexer proxy has nothing to do with P2P traffic, just basic HTTP. Feel free to propose a viable solution, maybe even a pull request if you're interested in making it work for you.

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@mtalexan commented on GitHub (Sep 13, 2024):

You can provide a suggestion as an alternative that we can retry on if the Servarr ping fails, [...]

Ignoring the need for a non-clearnet endpoints for every possible non-clearnet proxy, 1.1.1.1 or 4.4.4.4 are industry standard choices. It's a simple ping being routed over the proxy, and those are the Cloudflare and Google DNS servers respectively that intentionally allow ping replies and do no filtering.

I'm guessing the current endpoint was picked because it's used from the global proxy setting where access to the servarr server is necessary for indexer updates and update checks. It makes sense for that use, but makes no sense to try and use an unrelated restricted access server as the ping endpoint of an intentionally unrestrictive indexer proxy.

@mtalexan commented on GitHub (Sep 13, 2024): > You can provide a suggestion as an alternative that we can retry on if the Servarr ping fails, [...] Ignoring the need for a non-clearnet endpoints for every possible non-clearnet proxy, 1.1.1.1 or 4.4.4.4 are industry standard choices. It's a simple ping being routed over the proxy, and those are the Cloudflare and Google DNS servers respectively that intentionally allow ping replies and do no filtering. I'm guessing the current endpoint was picked because it's used from the global proxy setting where access to the servarr server is necessary for indexer updates and update checks. It makes sense for that use, but makes no sense to try and use an unrelated restricted access server as the ping endpoint of an intentionally unrestrictive indexer proxy.

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@mtalexan commented on GitHub (Sep 13, 2024):

You wrote some stuff, but I don't see a proper solution to all of this since the indexer proxy has nothing to do with P2P traffic, just basic HTTP.

Here's the short:

add support for I2P proxies required to reach the I2P indexer already in the list (postman)
Allow indexer proxies to function as designed
- ISPs block access to many of the indexers over HTTPS as well, a VPN is necessary to reach any/most of them.
- The current healthcheck blocks most VPNs.

@mtalexan commented on GitHub (Sep 13, 2024): > You wrote some stuff, but I don't see a proper solution to all of this since the indexer proxy has nothing to do with P2P traffic, just basic HTTP. Here's the short: - add support for I2P proxies required to reach the I2P indexer already in the list (postman) - Allow indexer proxies to function as designed - ISPs block access to many of the indexers over HTTPS as well, a VPN is necessary to reach any/most of them. - The current healthcheck blocks most VPNs.

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@OdinVex commented on GitHub (Dec 2, 2024):

I'd rather disable all of the health-checks entirely. Try-and-if-fail-notify as per usual. For now I'm intercepting DNS to rewrite to a stub proxy that bulls a fake ping.

@OdinVex commented on GitHub (Dec 2, 2024): I'd rather disable all of the health-checks entirely. Try-and-if-fail-notify as per usual. For now I'm intercepting DNS to rewrite to a stub proxy that bulls a fake ping.

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@bakerboy448 commented on GitHub (Dec 2, 2024):

I'd rather disable all of the health-checks entirely. Try-and-if-fail-notify as per usual. For now I'm intercepting DNS to rewrite to a stub proxy that bulls a fake ping.

No plans to allow disabling all health checks and arguably off topic to this FR

@bakerboy448 commented on GitHub (Dec 2, 2024): > I'd rather disable all of the health-checks entirely. Try-and-if-fail-notify as per usual. For now I'm intercepting DNS to rewrite to a stub proxy that bulls a fake ping. No plans to allow disabling all health checks and arguably off topic to this FR

deekerman commented

2026-02-20 10:07:09 -05:00

Author

Owner

@OdinVex commented on GitHub (Dec 2, 2024):

I'd rather disable all of the health-checks entirely. Try-and-if-fail-notify as per usual. For now I'm intercepting DNS to rewrite to a stub proxy that bulls a fake ping.

No plans to allow disabling all health checks and arguably off topic to this FR

I misread the issue as a general health-check, my bad.

@OdinVex commented on GitHub (Dec 2, 2024): > > I'd rather disable all of the health-checks entirely. Try-and-if-fail-notify as per usual. For now I'm intercepting DNS to rewrite to a stub proxy that bulls a fake ping. > > No plans to allow disabling all health checks and arguably off topic to this FR I misread the issue as a general health-check, my bad.