mirror of
https://github.com/qbittorrent/qBittorrent.git
synced 2026-03-02 22:57:32 -05:00
Crash when checking torrent with piece size of 256.0 MiB #15877
Labels
No labels
Accessibility
AppImage
Bounty
Build system
CI
Can't reproduce
Code cleanup
Confirmed bug
Confirmed bug
Core
Crash
Data loss
Discussion
Docker
Documentation
Duplicate
Feature
Feature request
Feature request
Feature request
Filters
Flatpak
GUI
Has workaround
I2P
Invalid
Libtorrent
Look and feel
Meta
NSIS
Network
Not an issue
OS: *BSD
OS: Linux
OS: Windows
OS: macOS
PPA
Performance
Project management
Proxy/VPN
Qt bugs
Qt6 compat
RSS
Search engine
Security
Temp folder
Themes
Translations
Triggers
Waiting diagnosis
Waiting info
Waiting upstream
Waiting web implementation
Watched folders
WebAPI
WebUI
autoCloseOldIssue
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/qBittorrent#15877
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @YouMiNope on GitHub (Jun 30, 2024).
qBittorrent & operating system versions
qBittorrent: 4.6.5 x64
Operating system: Windows 10 22H2
What is the problem?
I have a torrent that contains 3813 files with a total size of 279.43GiB, when checking it consumes all my 16Gb RAM and then crash. I assume there is some algorithm flaw in the checking codes that need to be fixed.
Steps to reproduce
Additional context
My RAM useage:
qbittorrent:
Log(s) & preferences file(s)
No response
@stalkerok commented on GitHub (Jun 30, 2024):
This is a consequence of incorrect client configuration, also you didn't specify libtorrent version, I suspect it's 2.0.
Also
@YouMiNope commented on GitHub (Jun 30, 2024):
here are my supplementary informations:
crash report
qBittorrent version: v4.6.5 (64-bit)
Libtorrent version: 1.2.19.0
Qt version: 6.4.3
Boost version: 1.85.0
OpenSSL version: 1.1.1w
zlib version: 1.3.1
OS version: Windows 10 Version 22H2 10.0.19045 x86_64
Caught signal: SIGABRT
Log
@glassez commented on GitHub (Jun 30, 2024):
It looks like your qBittorrent installation is broken, so it can't produce a useful stack trace.
@YouMiNope commented on GitHub (Jun 30, 2024):
I think it's just some random trace due to out of memory, I tried to reproduce but the crash reporter just crashed itself, and my PC almost crash too.
Im curious if there are some qbittorent settings that can get rid of that. How do I get the right configuration?
@xavier2k6 commented on GitHub (Jun 30, 2024):
@YouMiNope any crashdumps in
%LOCALAPPDATA%\CrashDumps?also, run
sfc /scannowin an elevated command prompt & that should highlight/take care of any corrupt OS files etc.@xavier2k6 commented on GitHub (Jun 30, 2024):
Related to #19745
@YouMiNope commented on GitHub (Jun 30, 2024):
I found two crashdumps titled qbittorrent.exe that created yesterday when my PC crashed.
CrashDumps.zip
the
sfc /scannowsays Windows Resource Protection did not find any integrity violations.@xavier2k6 commented on GitHub (Jun 30, 2024):
Dump gives above but no symbols/real info to go on.
You have qBittorrent installed here, does it have an associated
qbittorrent.pdbfile in this folder too?@YouMiNope commented on GitHub (Jun 30, 2024):
sorry I forgot to mention that I reinstalled my qbittorrent with latest v4.6.5 while the crashdumps were created when I was using v4.6.2, I don't know if the v4.6.5 .pdb will help. that file is too big to upload in github, if you want it I can email it to you.
by the way I tested libtorrent 2.0 built and it works very well.
@xavier2k6 commented on GitHub (Jun 30, 2024):
No need to upload
pdpfile, just ensure it is next toqbittorrent.exewhere qbittorrent is installed.The issue seems to be only with libtorrent 1.2.x version, It would be most helpful & highly appreciated if you could try to produce a valid stack trace (using libtorrent 1.2.x version) from instructions/method in https://github.com/qbittorrent/qBittorrent/issues/20869#issuecomment-2168749126
@YouMiNope commented on GitHub (Jun 30, 2024):
Seems like I captured a crash. the log file produced by debugdiag as follows:
qbittorrent__PID__16324__Date__06_30_2024__Time_06_09_49PM__611__Log.txt
hope that will help
@scomnoob commented on GitHub (Jun 30, 2024):
cant confirm that bug
5.0 beta1 on win11
RAM usage is ok on 400gb torrent while checking
@stalkerok commented on GitHub (Jun 30, 2024):
@YouMiNope Can you share the torrent file or hash (if it's not a private torrent)?
@xavier2k6 commented on GitHub (Jun 30, 2024):
@scomnoob This type of crash only seems to manifest itself when using libtorrent 1.2.x - you are using 2.0.x (I am open to correction)
Stack trace of
0XC0000409appears to be what was captured in https://github.com/qbittorrent/qBittorrent/issues/19745#issuecomment-1776695563When this is produced, it also gives
GetLastError returns 0x000005AFwhich is "ERROR_COMMITMENT_LIMIT" (The paging file is too small for this operation to complete.) ref.: Win32 Error Codes@arvidn Can you look at https://github.com/qbittorrent/qBittorrent/issues/19745#issuecomment-1776695563 & also other exceptions/stack traces in log from https://github.com/qbittorrent/qBittorrent/issues/21011#issuecomment-2198510072
@YouMiNope commented on GitHub (Jun 30, 2024):
@stalkerok here is the torrent file I used
@stalkerok commented on GitHub (Jun 30, 2024):
Piece size 256mb
Related https://github.com/qbittorrent/qBittorrent/pull/19535
@xavier2k6 commented on GitHub (Jun 30, 2024):
torrent file from https://github.com/qbittorrent/qBittorrent/issues/19745#issuecomment-1774087133 has same Piece size.
@2peer commented on GitHub (Jul 4, 2024):
What is the expected memory complexity of the check operation?
Because I regularly see memory spikes of additional <torrent_data_size/3> to <torrent_data_size/2> during the check operation in qBittorrent. Definitely not something I am used to from other clients.
qBittorrent v4.6.5
libtorrent-rasterbar 2.0.10
Fedora 40 (64 bit)
@glassez commented on GitHub (Jul 4, 2024):
Perhaps @arvidn could comment on it.
Note that libtorrent 2.0 uses memory mapped files, so that memory can be consumed also for performing I/O operations.
@2peer commented on GitHub (Jul 7, 2024):
From my understanding that would show up in VIRT mem in top, but not in RES/SHR & it shouldn't generally result in memory starving other processes. Just checked a 7.8GB torrent and the RES/SHR shot up from somewhere around 300MB to 5.3GB. When the check was finished it went back down. So something is amiss.
Thinking about the specifics of my setup (assuming this is not the most generally observed behavior), could a file access through a symlink be a part of the issue?
By the way, the torrent creation memory requirements are also horrendous on my system, but I can use other tools for that. The check is kind of unavoidable.
@xavier2k6 commented on GitHub (Aug 27, 2024):
I've tested & confirm this crash with torrents containing (piece size of 256.0 MiB) only with libtorrent 1.2 based builds.
As you can see from the statistics window below, the cache buffer grows & grows - it will use all available memory/RAM & use what storage space is available on a drive expanding the paging file......when there's no space left...it will eventually lead to a crash.
@stalkerok commented on GitHub (Aug 27, 2024):
When I tested this PR, creating torrents with 512MB and 1GB piece size also crashed in lt2.0, so I think it is also prone to this issue but with larger piece size.
@glassez commented on GitHub (Sep 24, 2024):
Has anyone investigated this issue in more detail? Are there any other conditions besides piece size: the full size of the torrent (and as a result the number of pieces), the size of the files in the torrent?
As I understand it, it doesn't crash immediately after checking is started, right? Then how would it behave if checking is paused at some point before it crashed, and then resumed?
@xavier2k6 commented on GitHub (Sep 24, 2024):
@glassez I've downloaded 3 files & created 3 torrents via qBittorrent v5 (LT2.0) with 256.0 MiB piece size:
Loaded them into qBittorrent 4.6.7 (LT1.2)
gimp-2.10.38-setup.exe= 328.5 MiB (2 x 256.0 MiB)Hybridqt-everywhere-opensource-src-5.15.15.zip= 1.04GiB (5 x 256.0 MiB)V1cache/total buffer size(statistics window) hits ~1GiB & will not free or appears to be held for a long period of time, left it for an hour!, any subsequent re-check will just keep increasing cache size by ~1GiB+ & no doubt will eventually lead to crash due to being out of space/memory (have to close application to regain memory being used)ubunutu-24.04-desktop-amd64.iso= 5.69GiB (23 x 256.0 MiB)Hybrid@glassez commented on GitHub (Sep 24, 2024):
Doesn't the 1. have such symptoms, but in proportion to its size?
@glassez commented on GitHub (Sep 24, 2024):
How does it behave when created using 128 MiB piece size?
@xavier2k6 commented on GitHub (Sep 24, 2024):
No, it doesn't even seem to make a blip on cache
@stalkerok commented on GitHub (Sep 24, 2024):
Could this setting be related?
https://www.libtorrent.org/reference-Settings.html#checking_mem_usage
In libtorrent, the default is 256MiB.
@glassez commented on GitHub (Sep 24, 2024):
That's 256 blocks of 16 KiB each, so it's only 4 MiB.
You could test different values anyway.
@glassez commented on GitHub (Sep 24, 2024):
I would rather sin against disk read buffers.
@xavier2k6 commented on GitHub (Sep 24, 2024):
Created another torrent:
744.0 MiB (3x 256.0 MiB) - recheck & cache/buffer hits 744.0 MiB & drops to 256.0 MiB which isn't freed.
@glassez commented on GitHub (Sep 24, 2024):
The bad news is that this is a problem with libtorrent 1.2, not libtorrent 2.0. This reduces the chances that I will deal with it.
@xavier2k6 commented on GitHub (Sep 24, 2024):
The problem though is that if certain users make torrents with the likes of
py3createtorrentor other programs, we will more than likely see the same type issues with libtorrent 2.x as we are now with libtorrent 1.2, we know libtorrent 1.2 can only handle torrents/pieces of 128MiB & libtorrent 2.0 can only handle torrents/pieces of 256MiB - there's nothing stopping a user from creating torrents with 512/1024 etc.@xavier2k6 commented on GitHub (Sep 24, 2024):
Libtorrent 2.0 seems to reject torrents with piece size of 512MiB/1GiB from even being loaded/logs show invalid or missing piece etc. however libtorrent 1.2 creates similar error in the logs but allows these torrents to be loaded in to session & they remain stuck on "checking resume data"
libtorrent 1.2 should flat out reject any torrent above 128MiB & not allow it to be loaded in to session.
@glassez commented on GitHub (Sep 24, 2024):
Good news. For some unknown reason, I still decided to make a "recon mission" to this problem. Fortunately, the cause of the problem was not too deep, so I seem to have detected it. I need to figure out some more details, but I'll deal with that later.
BTW, is there a related Issue in libtorrent repo? If not, could someone create it?
@xavier2k6 commented on GitHub (Sep 24, 2024):
I'll create one in the morning if nobody else does it in mean time.
@xavier2k6 commented on GitHub (Sep 28, 2024):
@glassez Upstream ticket has been filed, see: https://github.com/arvidn/libtorrent/issues/7735
@Pryodon commented on GitHub (Oct 16, 2024):
I am having the same issue with the same files. I will share with you some torrents that cause this crash.
files.zip
@Pryodon commented on GitHub (Oct 16, 2024):
In order to successfully check those torrents that I just linked in my previous post, I had to create a page file of 300 GB!!! That is ridiculous to have to create a page file that huge. There is no legitimate reason that a person should have a page file that big! The issue is that there is a memory leak in qbit when checking these large files! It reads the entire file into memory and if there is a subsequent very large torrent file (such as the ones I just submitted) then it tries to also read that entire file into memory while the previous file is also still in memory and it doesn't matter how much memory you have reserved in your page file, the system will always run out of memory!
To successfully check those huge torrents, I had to check one of them (which takes an eternity). Then I have to completely shutdown qbit and wait for my computer to release all of that memory that was being used, then start qbit up again and then manually check the next file. If I attempt to check multiple files at once, it will crash guaranteed because it will always run out of page file memory no matter how big the page file is!
Download those torrents and see for yourself!
@xavier2k6 commented on GitHub (Oct 16, 2024):
@Pryodon I've downloaded those torrent files & indeed they were created using a piece size of 256.0MiB - the author of libtorrent has now ensured that these types of torrents will be rejected from even loading.
qBittorrent will include this block in it's next release & will also limit the creation of torrents to a MAX piece size of 128MiB
Your option now is to remove those torrents/find an alternative source for the file which do not exceed 128MiB piece size or re-create these torrents keeping to the supported piece size.
@Remzi1993 commented on GitHub (Oct 29, 2024):
What I don't understand is: Why is qBittorrent providing both 1.2 and 2.0 libtorrent versions? Why not the latest stable version?
@qBittUser commented on GitHub (Nov 4, 2024):
@Remzi1993 https://github.com/qbittorrent/qBittorrent/issues/21011#issuecomment-2445580061
Both branches are still developed, they work differently and have different settings to tweak.
So if there's any kind of issues with one, then the issue might not be producable with the other and it's giving one extra easy way to figure out regressions and what exactly still needs improvement.
Everyone doesn't need the latest version features.
Many users still get faster more stable speed and more efficient computer resource usage with v1.2+.
Latest version from v2_0 branch has annoying flaws that still need fixing before it should be recommend to everyone.
There's no guarantee that in the future, the v2.1+ releases will immediately be good enough to replace v2.0+ & v1.2+.
@luzpaz commented on GitHub (Mar 18, 2025):
Looks like we're waiting on upstream
@i990049 commented on GitHub (Apr 18, 2025):
private torrent made by tixati 3.3.3
piece 256MiB
qbtorrent5.0.5 x64 libtorrent 1.2.x
Win10
Can't not open torrent
@stalkerok commented on GitHub (Apr 18, 2025):
Libtorrent 1.2.x does not support 256mb, for such torrents use 2.0.x