I plan to create a PR for missing and hardcoded session settings #5257

Closed
opened 2026-02-21 17:45:12 -05:00 by deekerman · 15 comments
Owner

Originally created by @Falcosc on GitHub (Feb 24, 2017).

qBittorrent version and Operating System:

all versions

If on linux, libtorrent and Qt version:

all os

What is the problem:

There are settings missing or there are hardcoded and does not match to my usecase

What is the expected behavior:

I could configure any libtorrent setting to match my usecase

Steps to reproduce:

try to configure a libtorrent setting which is not implemented at qbittorrent session or which got set by qbittorrent session with an unwanted value.

Extra info(if any):

I would could create a PR for the full list of my needed settings which I need to change to match my usecase.
Currently I have them hardcoded in the session class.
I don't want to create a UI for all these settings, there are just to many of them.

There are already some config keys which are not configure able over webui, but I guess they are at least configureable over QT UI.
Would it be ok for you if I just implement settings keys for the config file without implementing these at WebUI and without QTUI?
Because I don't spend any time into this if you don't want to have this kind of config key PR. Because it's much more easy to merge my hardcoded settingslist into session class instead of merge a config key implementation into every change.

I guess there are other issues which just need a new config key. But I don't find any of them. Feel free to link them to this and I would try to add them as well. Or just give me keyword for issue search to find them.

Originally created by @Falcosc on GitHub (Feb 24, 2017). ### qBittorrent version and Operating System: all versions ### If on linux, libtorrent and Qt version: all os ### What is the problem: There are settings missing or there are hardcoded and does not match to my usecase ### What is the expected behavior: I could configure any libtorrent setting to match my usecase ### Steps to reproduce: try to configure a libtorrent setting which is not implemented at qbittorrent session or which got set by qbittorrent session with an unwanted value. ### Extra info(if any): I would could create a PR for the full list of my needed settings which I need to change to match my usecase. Currently I have them hardcoded in the session class. I don't want to create a UI for all these settings, there are just to many of them. There are already some config keys which are not configure able over webui, but I guess they are at least configureable over QT UI. Would it be ok for you if I just implement settings keys for the config file without implementing these at WebUI and without QTUI? Because I don't spend any time into this if you don't want to have this kind of config key PR. Because it's much more easy to merge my hardcoded settingslist into session class instead of merge a config key implementation into every change. I guess there are other issues which just need a new config key. But I don't find any of them. Feel free to link them to this and I would try to add them as well. Or just give me keyword for issue search to find them.
Author
Owner

@Chocobo1 commented on GitHub (Feb 24, 2017):

For a start, do you mind listing out what is hardcoded and what is missing in the GUI?
I would very much like to have more tuning knobs in the Options-Advanced section.

@Chocobo1 commented on GitHub (Feb 24, 2017): For a start, do you mind listing out what is hardcoded and what is missing in the GUI? I would very much like to have more tuning knobs in the Options-Advanced section.
Author
Owner

@Falcosc commented on GitHub (Feb 24, 2017):

Of course it is, but you should know what happens if you change these values. Just a input field on UI would could be even bad.

 pack.set_int(libt::settings_pack::urlseed_pipeline_size, 1);
    pack.set_int(libt::settings_pack::urlseed_max_request_bytes, 1024 * 1024 * 1024);
        // 160mb cache
        pack.set_int(libt::settings_pack::cache_size, 10240);
        pack.set_bool(libt::settings_pack::use_read_cache, true);
        pack.set_int(libt::settings_pack::suggest_mode, libt::settings_pack::suggest_read_cache);
        // of 500 ms, and a send rate of 10 MB/s, the upper
        // limit should be 5 MB
        pack.set_int(libt::settings_pack::send_buffer_watermark, 5 * 1024 * 1024);
        // put 1.5 seconds worth of data in the send buffer
        // this gives the disk I/O more heads-up on disk
        // reads, and can maximize throughput
        pack.set_int(libt::settings_pack::send_buffer_watermark_factor, 150);
        // always stuff at least 1 MiB down each peer
        // pipe, to quickly ramp up send rates
        pack.set_int(libt::settings_pack::send_buffer_low_watermark, 1 * 1024 * 1024);

And this is a setting which is missing ad webui:
Bittorrent\AnnounceToAllTrackers

But my main problem is that I haven't enoght time for a webui settings implementation. Qt UI implementation would be even worse because I don't use Qt.

But I could add config keys.

@Falcosc commented on GitHub (Feb 24, 2017): Of course it is, but you should know what happens if you change these values. Just a input field on UI would could be even bad. ``` pack.set_int(libt::settings_pack::urlseed_pipeline_size, 1); pack.set_int(libt::settings_pack::urlseed_max_request_bytes, 1024 * 1024 * 1024); // 160mb cache pack.set_int(libt::settings_pack::cache_size, 10240); pack.set_bool(libt::settings_pack::use_read_cache, true); pack.set_int(libt::settings_pack::suggest_mode, libt::settings_pack::suggest_read_cache); // of 500 ms, and a send rate of 10 MB/s, the upper // limit should be 5 MB pack.set_int(libt::settings_pack::send_buffer_watermark, 5 * 1024 * 1024); // put 1.5 seconds worth of data in the send buffer // this gives the disk I/O more heads-up on disk // reads, and can maximize throughput pack.set_int(libt::settings_pack::send_buffer_watermark_factor, 150); // always stuff at least 1 MiB down each peer // pipe, to quickly ramp up send rates pack.set_int(libt::settings_pack::send_buffer_low_watermark, 1 * 1024 * 1024); ``` And this is a setting which is missing ad webui: `Bittorrent\AnnounceToAllTrackers` But my main problem is that I haven't enoght time for a webui settings implementation. Qt UI implementation would be even worse because I don't use Qt. But I could add config keys.
Author
Owner

@Seeker2 commented on GitHub (Feb 24, 2017):

Also needs to be a:
announce_to_all_tiers
mixed_mode_algorithm
...and whatever it is that could allocate file space when creating a torrent without forcing it to 0-fill, since that can be painfully slow on 20+ GB torrents.

@Seeker2 commented on GitHub (Feb 24, 2017): Also needs to be a: announce_to_all_tiers mixed_mode_algorithm ...and whatever it is that could allocate file space when creating a torrent without forcing it to 0-fill, since that can be painfully slow on 20+ GB torrents.
Author
Owner

@Chocobo1 commented on GitHub (Feb 25, 2017):

And this is a setting which is missing ad webui:

Currently nobody is working on webUI.

But my main problem is that I haven't enoght time for a webui settings implementation

After adding the config keys, at least you will be able to change the settings via ini files.

But I could add config keys.

Speaking of this, I opened PR #3235 a long time ago... I can revive it and do it there or you want to open a new PR?

I'm curious about send_buffer_watermark*, does it really uses more memory? How many seeding jobs do you have? any speed difference?

@Chocobo1 commented on GitHub (Feb 25, 2017): >And this is a setting which is missing ad webui: Currently nobody is working on webUI. >But my main problem is that I haven't enoght time for a webui settings implementation After adding the config keys, at least you will be able to change the settings via ini files. >But I could add config keys. Speaking of this, I opened PR #3235 a long time ago... I can revive it and do it there or you want to open a new PR? I'm curious about `send_buffer_watermark*`, does it really uses more memory? How many seeding jobs do you have? any speed difference?
Author
Owner

@Chocobo1 commented on GitHub (Feb 25, 2017):

Also needs to be a:
announce_to_all_tiers
mixed_mode_algorithm

I can add those in PR #3235.

...and whatever it is that could allocate file space when creating a torrent without forcing it to 0-fill

Uncheck Options -> Downloads -> Pre-allocate disk space for all files, then libtorrent will use sparse allocation.
https://en.wikipedia.org/wiki/Sparse_file

@Chocobo1 commented on GitHub (Feb 25, 2017): > Also needs to be a: > announce_to_all_tiers > mixed_mode_algorithm I can add those in PR #3235. >...and whatever it is that could allocate file space when creating a torrent without forcing it to 0-fill Uncheck `Options -> Downloads -> Pre-allocate disk space for all files`, then libtorrent will use sparse allocation. https://en.wikipedia.org/wiki/Sparse_file
Author
Owner

@Falcosc commented on GitHub (Feb 25, 2017):

send_buffer_watermark does only work if you have high upload rates. You set the limit with send_buffer_watermark and send_buffer_watermark_factor does calculate the acutall buffer. If you have slow peers you won't allocate more memory than the default values.
But on high rates you could run into your harddrive limitations if your max send_buffer_watermark is to small. The factor multiplier should be cover the access time of new data and an overloaded harddrive could take more then 1s time to access a new data.

An easy way to improve the performance it to follow this: http://www.rasterbar.com/products/libtorrent/tuning.html

But it's sometimes a little bit hard to follow because you have to search in qbittorent code which settings are available and which you have to compile into sourcecode. But I don't care about this problem because I haven't a good idea to deal with it.

Now something about this ticket.
I want to create a PR to help you. And I will start with settings which are used for high speed peer transfer (less peers with very high rates) But my PR would only include configuration keys for settings files. Would you like such PR because sometimes it sounds the team don't want to pull changes which are not accessable over UI
Thats the reason why I ask before I do something :)

@Falcosc commented on GitHub (Feb 25, 2017): send_buffer_watermark does only work if you have high upload rates. You set the limit with send_buffer_watermark and send_buffer_watermark_factor does calculate the acutall buffer. If you have slow peers you won't allocate more memory than the default values. But on high rates you could run into your harddrive limitations if your max send_buffer_watermark is to small. The factor multiplier should be cover the access time of new data and an overloaded harddrive could take more then 1s time to access a new data. An easy way to improve the performance it to follow this: http://www.rasterbar.com/products/libtorrent/tuning.html But it's sometimes a little bit hard to follow because you have to search in qbittorent code which settings are available and which you have to compile into sourcecode. But I don't care about this problem because I haven't a good idea to deal with it. Now something about this ticket. I want to create a PR to help you. And I will start with settings which are used for high speed peer transfer (less peers with very high rates) But my PR would only include configuration keys for settings files. Would you like such PR because sometimes it sounds the team don't want to pull changes which are not accessable over UI Thats the reason why I ask before I do something :)
Author
Owner

@Seeker2 commented on GitHub (Feb 25, 2017):

"Uncheck Options -> Downloads -> Pre-allocate disk space for all files, then libtorrent will use sparse allocation."

No, I don't mean sparse files, because this happens:
http://forum.deluge-torrent.org/viewtopic.php?f=12&t=36959
What I mean was mentioned here:
https://github.com/xlgjjff/libtorrent/issues/576

@Seeker2 commented on GitHub (Feb 25, 2017): "Uncheck Options -> Downloads -> Pre-allocate disk space for all files, then libtorrent will use sparse allocation." No, I don't mean sparse files, because this happens: http://forum.deluge-torrent.org/viewtopic.php?f=12&t=36959 What I mean was mentioned here: https://github.com/xlgjjff/libtorrent/issues/576
Author
Owner

@Chocobo1 commented on GitHub (Feb 25, 2017):

Would you like such PR because sometimes it sounds the team don't want to pull changes which are not accessable over UI

I have this mindset too... At least you need 1 place for a particular setting in GUI.
If only having config keys, then nobody will ever notice that.

No, I don't mean sparse files, because this happens:

Weird, it's marked as 'fixed' in the end.
Ever tried running with Administrator privilege? If it still doesn't work, you should open a ticket at libtorrent.

@Chocobo1 commented on GitHub (Feb 25, 2017): >Would you like such PR because sometimes it sounds the team don't want to pull changes which are not accessable over UI I have this mindset too... At least you need 1 place for a particular setting in GUI. If only having config keys, then nobody will ever notice that. >No, I don't mean sparse files, because this happens: Weird, it's marked as 'fixed' in the end. Ever tried running with Administrator privilege? If it still doesn't work, you should open a ticket at libtorrent.
Author
Owner

@Falcosc commented on GitHub (Feb 25, 2017):

Thats bad, good that I did ask before I did spend time. But I still want to help.

Should I close this ticket or could we share the work? I create and test the new keys and somebody else take care of the Qt UI?

I would document the new keys at a Wiki page as well. I would place a hint that we need somebody who implement it at UI. The main Idea of the wiki page is to have a mapping between UI settings and the according config keys and a mapping to libtorrent setting including a usefull short documentation about how these settings should be used and a link to the detailed libtorrent documentation. If you want to read more about my idea, take a look at https://github.com/qbittorrent/qBittorrent/issues/6426

@Falcosc commented on GitHub (Feb 25, 2017): Thats bad, good that I did ask before I did spend time. But I still want to help. Should I close this ticket or could we share the work? I create and test the new keys and somebody else take care of the Qt UI? I would document the new keys at a Wiki page as well. I would place a hint that we need somebody who implement it at UI. The main Idea of the wiki page is to have a mapping between UI settings and the according config keys and a mapping to libtorrent setting including a usefull short documentation about how these settings should be used and a link to the detailed libtorrent documentation. If you want to read more about my idea, take a look at https://github.com/qbittorrent/qBittorrent/issues/6426
Author
Owner

@Seeker2 commented on GitHub (Feb 26, 2017):

Chocobo1, what was fixed was that Deluge could be run with sparse files disabled -- but that it would do full allocation of files including zero-filling them, which could be a big delay on starting 20+ GB torrents.

qBitTorrent likewise can avoid using sparse files but also does so by full allocation and zero-filling -- worse, it only starts doing that after it receives its first downloaded 16-64 KB chunk (and needs a place to put it) rather than when the torrent is first started. So it even wastes the delay between starting a torrent and peers/seeds sending data that it could be creating the "blank" files. Reference: https://github.com/qbittorrent/qBittorrent/issues/6195

In my sparse file NTFS tests, I've seen qBT create over 30,000 file fragments from a 14 file ~2.5 GB torrent. ~2400 per file! If I wasn't doing it on a ramdrive that's capable of 1 GB/sec even with tiny file fragments, it would've been very slow.

Sparse file support doesn't exist under certain drive formats (FAT32, exFAT, and probably others), so those at least avoid some of the sparse file file fragmentation problems.

Other file systems lack write-in-place, so zero-filling newly-created torrent files is a double waste. Especially on SSDs.

@Seeker2 commented on GitHub (Feb 26, 2017): Chocobo1, what was fixed was that Deluge could be run with sparse files disabled -- but that it would do full allocation of files including zero-filling them, which could be a big delay on starting 20+ GB torrents. qBitTorrent likewise can avoid using sparse files but also does so by full allocation and zero-filling -- worse, it only starts doing that after it receives its first downloaded 16-64 KB chunk (and needs a place to put it) rather than when the torrent is first started. So it even wastes the delay between starting a torrent and peers/seeds sending data that it could be creating the "blank" files. Reference: https://github.com/qbittorrent/qBittorrent/issues/6195 In my sparse file NTFS tests, I've seen qBT create over 30,000 file fragments from a 14 file ~2.5 GB torrent. ~2400 per file! If I wasn't doing it on a ramdrive that's capable of 1 GB/sec even with tiny file fragments, it would've been very slow. Sparse file support doesn't exist under certain drive formats (FAT32, exFAT, and probably others), so those at least avoid some of the sparse file file fragmentation problems. Other file systems lack write-in-place, so zero-filling newly-created torrent files is a double waste. Especially on SSDs.
Author
Owner

@Chocobo1 commented on GitHub (Mar 1, 2017):

I create and test the new keys and somebody else take care of the Qt UI?

Sure, I can handle this GUI part.
Before you start working, I agree with all keys you mentioned except urlseed_*, seems that is not suitable for everyone.

I would document the new keys at a Wiki page as well.

FYI, we have this page already: https://github.com/qbittorrent/qBittorrent/wiki/Explanation-of-Options-in-qBittorrent#Advanced
But I think nobody ever read it.

@Chocobo1 commented on GitHub (Mar 1, 2017): >I create and test the new keys and somebody else take care of the Qt UI? Sure, I can handle this GUI part. Before you start working, I agree with all keys you mentioned except `urlseed_*`, seems that is not suitable for everyone. >I would document the new keys at a Wiki page as well. FYI, we have this page already: https://github.com/qbittorrent/qBittorrent/wiki/Explanation-of-Options-in-qBittorrent#Advanced But I think nobody ever read it.
Author
Owner

@Chocobo1 commented on GitHub (Mar 1, 2017):

Chocobo1, what was fixed was that Deluge could be run with sparse files disabled

The fix is in libtorrent, so in theory, qbt should be able to utilize it.

qBitTorrent likewise can avoid using sparse files but also does so by full allocation and zero-filling -- worse, it only starts doing that after it receives its first downloaded 16-64 KB chunk (and needs a place to put it) rather than when the torrent is first started

I noticed this as well, please open a new ticket for this.

@Chocobo1 commented on GitHub (Mar 1, 2017): >Chocobo1, what was fixed was that Deluge could be run with sparse files disabled The fix is in libtorrent, so in theory, qbt should be able to utilize it. >qBitTorrent likewise can avoid using sparse files but also does so by full allocation and zero-filling -- worse, it only starts doing that after it receives its first downloaded 16-64 KB chunk (and needs a place to put it) rather than when the torrent is first started I noticed this as well, please open a new ticket for this.
Author
Owner

@Falcosc commented on GitHub (Mar 1, 2017):

Ok, I will start with the others. But I guess urlseed is also very importand if you want to use very restrictive servers for url seed.

urlseed_pipeline_size is quite importand, if you have set non 1 you could get disconnected or banned from the server.

And urlseed_max_request_bytes is importand as well, because if your requests are to small you get disconnected as well. (this is now less importand after libtorrent changed the default from 1MiB to 16MiB) But if you have a realy fast connection you would get blocked with 16MiB requests as well because I guess they just block ips based on requests per minute.

@Falcosc commented on GitHub (Mar 1, 2017): Ok, I will start with the others. But I guess urlseed is also very importand if you want to use very restrictive servers for url seed. urlseed_pipeline_size is quite importand, if you have set non 1 you could get disconnected or banned from the server. And urlseed_max_request_bytes is importand as well, because if your requests are to small you get disconnected as well. (this is now less importand after libtorrent changed the default from 1MiB to 16MiB) But if you have a realy fast connection you would get blocked with 16MiB requests as well because I guess they just block ips based on requests per minute.
Author
Owner

@Seeker2 commented on GitHub (Mar 3, 2017):

"I noticed this as well, please open a new ticket for this."

The ticket already exists: https://github.com/qbittorrent/qBittorrent/issues/6195
Probably links back into this: http://www.rasterbar.com/products/libtorrent/reference-Storage.html#storage_mode_t mentioned in detail: http://forum.deluge-torrent.org/viewtopic.php?t=27725

This was done as a side-effect of this: https://github.com/qbittorrent/qBittorrent/issues/1072

@Seeker2 commented on GitHub (Mar 3, 2017): "I noticed this as well, please open a new ticket for this." The ticket already exists: https://github.com/qbittorrent/qBittorrent/issues/6195 Probably links back into this: http://www.rasterbar.com/products/libtorrent/reference-Storage.html#storage_mode_t mentioned in detail: http://forum.deluge-torrent.org/viewtopic.php?t=27725 This was done as a side-effect of this: https://github.com/qbittorrent/qBittorrent/issues/1072
Author
Owner

@Chocobo1 commented on GitHub (Sep 28, 2017):

Closing as done in #3235.

@Chocobo1 commented on GitHub (Sep 28, 2017): Closing as done in #3235.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/qBittorrent#5257
No description provided.