mirror of
https://github.com/qbittorrent/qBittorrent.git
synced 2026-03-02 22:57:32 -05:00
Preview File to use piece_deadline instead of sequential_download #163
Labels
No labels
Accessibility
AppImage
Bounty
Build system
CI
Can't reproduce
Code cleanup
Confirmed bug
Confirmed bug
Core
Crash
Data loss
Discussion
Docker
Documentation
Duplicate
Feature
Feature request
Feature request
Feature request
Filters
Flatpak
GUI
Has workaround
I2P
Invalid
Libtorrent
Look and feel
Meta
NSIS
Network
Not an issue
OS: *BSD
OS: Linux
OS: Windows
OS: macOS
PPA
Performance
Project management
Proxy/VPN
Qt bugs
Qt6 compat
RSS
Search engine
Security
Temp folder
Themes
Translations
Triggers
Waiting diagnosis
Waiting info
Waiting upstream
Waiting web implementation
Watched folders
WebAPI
WebUI
autoCloseOldIssue
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/qBittorrent#163
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @slacka on GitHub (Oct 23, 2012).
Instead of using sequential_download to enable streaming, I would like to see qBittorrent use the piece_deadline feature to create a streaming buffer. This is discussed here:
http://code.google.com/p/libtorrent/issues/detail?id=376&sort=-id
The main benefit of using piece deadlines is that the fast peers will be used first, to download those pieces. the sequential download feature will make any peer request happen in sequential order, including the very slow peers. the order in which pieces are requested is not necessarily the order they complete.
The other benefit of piece deadline is that any spare bandwidth, not required to request the deadline pieces, will be used for normal rarest-first downloading.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
DISCLAIMER: There are trustworthiness and solvency issues with BountySource. qBittorrent is not affiliated with them. Use them at your own risk.
@PacoBell commented on GitHub (May 14, 2013):
I too would like to see deadline implemented, even though no dev has apparently replied in 7 months. It's a much more worthwhile idea than sequential piece selection. Let's try to be respectful citizens on the network. Sequential is pure greed and harms everyone in the long run.
@sledgehammer999 commented on GitHub (Aug 10, 2014):
@slacka I was looking at the libtorrent API about the piece_deadline thing. It seems that it also needs a deadline in milliseconds. What would be an ok value to set in your opinion? Any thoughts on this?
NOTE: The smaller the deadline the harder libtorrent will try to fullfill it.
@arvidn commented on GitHub (Aug 12, 2014):
the deadline is used for two things.
If you know the bit rate of the file you're streaming, you can take advantage of this and schedule the pieces to be downloaded at optimal times (probably with some safety margin). If you don't, you can just always pass in a short deadline, but increasing with decreasing priority.
@sledgehammer999 commented on GitHub (Aug 13, 2014):
@arvidn
So basically, if I understand it correctly, the deadline time doesn't really matter. It only matters if there are pieces with different deadlines in order to prioritize amongst them.
Let's say that a set one deadline at 1s. Let's say that one second passes and the piece wasn't downloaded yet.(due to slow peers). Does that deadline clear automatically or does it persist regardless of the time passed?
I assume that the deadline isn't cleared if its time has passed. So I just have to set a deadline(eg 1ms) for every piece in the current file. Then wait for it to complete. Then set the deadlines for the pieces of the next file etc
@arvidn commented on GitHub (Aug 13, 2014):
your understanding is correct. if a deadline is not met, that piece is still considered time-critical, still prioritized by its deadline.
it's probably a good idea to keep a certain amount of pieces as time-critical ahead of the playback point. (ideally, a certain number of seconds, but then you need to know the bitrate).
Here's an example of how to stream from a torrent over http.
https://github.com/arvidn/libtorrent-webui/blob/master/src/file_downloader.cpp
@sledgehammer999 commented on GitHub (Aug 14, 2014):
@arvidn
I think I have misunderstood what sequential_download does. Correct me if I am wrong but this is how I understand it now. Let's say file A consists of pieces 10-20. Sequential download will download the 10th, then the 11th then the 12th etc. It will not download 15th piece first(because there are fast peers for that) and then the 10th piece, right?
So if I want to use deadlines I should set for piece 10 eg 10ms deadline, then for piece 11 10ms+5ms, then for 12 10ms+5ms+5ms etc until number 20.
Can you tell me from where to start looking? ie what is the entry point in this file?
@arvidn commented on GitHub (Aug 14, 2014):
sequential_download is a property per-torrent, not per file. If you set sequential_download on a torrent, its pieces will be requested in order, not necessarily completed in order.
yes, you can use deadlines that way, to download a file sequentially. It will try harder to complete pieces in-order, by having more aggressive time-out behavior, and picking fast peers for urgent requests.
file_downloader.cpp is essentially an HTTP handler (that fits into the wrapper around mongoose the libtorrent-webui has). file_downloader::handle_http is the entry point when a request is coming in from the http client.
@zeule commented on GitHub (Apr 15, 2016):
@LordNyriox: yes, this one is simple. However, to implement it nicely (you're right) a possibility to apply the deadlines to individual files will be good. To do that, one needs to map file indices onto pieces indices. This is already a part of PR #4936. Once it gets merged, this one can be implemented.
Perhaps we can also add a context menu action like "Try to complete in ..." where the user can enter a running time for a file thus defining the pieces deadlines (provided that average bitrate value is stable enough)?
@zeule commented on GitHub (Apr 16, 2016):
@LordNyriox: if I correctly understand the comment by @arvidn the best result can be achieved if (suppose for simplicity that we have a single file torrent)
piece_deadline == (piece_index + 1) * piece_average_running_time. Since it is more natural for a user to specify running time for the whole file but not per piece, I propose that action.@zeule commented on GitHub (Apr 16, 2016):
@LordNyriox: quoting the libtorrent documentation "deadline is the number of milliseconds until this piece should be completed". How can one compute the ideal deadlines without knowing the running time for a file?
@zeule commented on GitHub (Apr 25, 2016):
@LordNyriox: I have 5 feature branches now (not even all of them are submitted as PRs yet); all of them I need to merge into my own integration branch that I use. The sixth one will be too much for a hobby project, as to me. As soon as any of the submitted PRs got merged, will implement this one (provided nobody does this quicker).
@Seeker2 commented on GitHub (Jun 29, 2017):
"Let's say that a set one deadline at 1s. Let's say that one second passes and the piece wasn't downloaded yet.(due to slow peers)."
Piece size for videos is typically 256 KB or larger. Uploading that in a second requires >2 mbit/sec UL bw. That isn't slow...that is unrealistic expectations.
Even 16 KB chunks are unlikely to complete in 1 second (16 KB/sec).
"It seems that it also needs a deadline in milliseconds." ... "So if I want to use deadlines I should set for piece 10 eg 10ms deadline, then for piece 11 10ms+5ms, then for 12 10ms+5ms+5ms etc until number 20."
To get a single piece in 10 milliseconds would require a download speed that is 100x faster than the piece size divided by 1 second.
So a 1 MB piece size needs a 100 MB/sec speed to do that in 10 ms. 1 gbit/sec minimum!
If piece #11 and #12 are to start downloading before piece #10 finishes and/or have to finish in 5 ms instead of 10 ms, then even faster speeds are needed.
This should be proof enough that piece deadlines need to be in seconds rather than milliseconds!
Pieces tend to be downloaded from a single peer or seed for each piece, so if there's a hash fail you know who's likely to blame for it.
That means while you may gain the benefits of aggregate speed from many peers+seeds at the TORRENT level, you are still stuck at individual peer/seed speed at the PIECE level. And for many public torrents, that's very slow!
With a "standard" ADSL service with 100 KiloBYTES/second usable upload speed (~1 mbit/sec UL bw) using 4 upload slots per torrent -- each peer of the 4 it uploads to gets 25 KB/sec. A 1 MB piece size is going to take about 41 seconds at that rate, assuming absolutely no other delays and only 4 peers connected to that ADSL connection. This alone is almost 3 times larger than a 15 seconds minimum buffer size for video playback.
At 2 or 4 MB piece size, the delay will be far worse...better measured in minutes instead of seconds!
Even if a single piece appears to offer a buffer that's over 15 seconds... You don't want to be 1 piece away from an incomplete piece when playing the video, so for safety the buffer needs at least 2-10 continuous pieces past the playback point.
The next piece probably needs to be started before the previous piece finishes DLing, but that often can't be done by the same peer/seed because of protocol restrictions and also risks a multi-piece stall if that peer/seed disconnects or slows its upload.
The only way to achieve speeds necessary to do torrent video streaming is either to not use pure Seq. DLing, to have extremely fast peers/seeds, and/or have more than 1 peer/seed giving each piece.
@arvidn commented on GitHub (Jun 29, 2017):
@Seeker2
The main purpose of the time duration passed to
set_piece_deadline()is to delay starting the download, if it's far enough out in the future. Setting a short timeout essentially only determines the priority order of pieces, it doesn't require the actual download rate to keep up.From the documentation:
Seeker2:
I don't believe this is true in general. libtorrent has a heuristic where fast peers will have whole pieces requested from them, and peers that are suspected of sending corrupt data (in parole mode) may also have whole pieces allocated to them.
In general, it is preferred to request bocks from the same piece from as many peers as possible, to concentrate the download rate to as few pieces as possible. This shortens the delay from downloading a block until it can be uploaded again (because the piece is complete).
In streaming mode (i.e. with
set_piece_deadline()) it is still the case that multiple peers may contribute to the same piece.I think the buffer is best defined in time units (some number of seconds). But you're right that the quantization caused by large pieces need to be taken into account, and that there should probably always be at least 2 pieces in the buffer ahead of the playback point.
The difficulty of defining the buffer in terms of seconds is that you may need to parse part of the video container header to estimate the bitrate (uTorrent does this for avi, mp4 and mkv iirc). Another issue is that mp4 may have as much as 1 MiB of header at the front of the file before any video data, so the bitrate cannot necessarily be assumed to be evenly distributed across the file.
@ntzb commented on GitHub (Jul 20, 2021):
is this planned to be added strictly to the "Preview" feature, or will this be able to be set as a property via WEB UI as well?
@lucasmbrown commented on GitHub (Apr 29, 2023):
It looks like there's now a small bounty on this: https://app.bountysource.com/issues/1262120-preview-file-to-use-piece_deadline-instead-of-sequential_download
@MilesBHuff commented on GitHub (Apr 2, 2024):
That link appears to be broken.
@xavier2k6 commented on GitHub (May 23, 2025):
ANNOUNCEMENT!
For anybody coming across this "Feature Request" & would like/love to see a potential implementation in the future!
Here are some options available to you:
Please select/click the 👍 &/or ❤
reactionsin the original/opening post of this ticket.Please feel free (If you have the "skillset") to create a "Pull Request" implementing what's being requested in this ticket.
(new/existing contributors/developers are always welcome)
DO:
DO NOT:
(These will be disregarded/hidden as "spam/abuse/off-topic" etc. as they don't provide anything constructive.)