mirror of
https://github.com/qbittorrent/qBittorrent.git
synced 2026-03-02 22:57:32 -05:00
Memory leak with memory mapped file IO #15155
Labels
No labels
Accessibility
AppImage
Bounty
Build system
CI
Can't reproduce
Code cleanup
Confirmed bug
Confirmed bug
Core
Crash
Data loss
Discussion
Docker
Documentation
Duplicate
Feature
Feature request
Feature request
Feature request
Filters
Flatpak
GUI
Has workaround
I2P
Invalid
Libtorrent
Look and feel
Meta
NSIS
Network
Not an issue
OS: *BSD
OS: Linux
OS: Windows
OS: macOS
PPA
Performance
Project management
Proxy/VPN
Qt bugs
Qt6 compat
RSS
Search engine
Security
Temp folder
Themes
Translations
Triggers
Waiting diagnosis
Waiting info
Waiting upstream
Waiting web implementation
Watched folders
WebAPI
WebUI
autoCloseOldIssue
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/qBittorrent#15155
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Lastique on GitHub (Nov 9, 2023).
qBittorrent & operating system versions
qBittorrent: 4.5.5 x86-64, 4.4.1 x86-64
libtorrent-rasterbar: 2.0.9, 2.0.5
Operating System: Kubuntu 22.04
KDE Plasma Version: 5.24.7
KDE Frameworks Version: 5.92.0
Qt Version: 5.15.3
Kernel Version: 6.2.0-1016-lowlatency (64-bit)
Graphics Platform: X11
Processors: 16 × 12th Gen Intel® Core™ i7-12700K
Memory: 31.1 GiB of RAM
What is the problem?
When qBittorrent 4.5.5 is started with "Disk IO type" option in the Advanced tab set to "Default" or "Memory mapped files", qBittorrent starts leaking memory as it is seeding torrents. This also reproduces with qBittorrent 4.4.1, although there was no "Disk IO type" option in that version (I'm assuming, it used memory mapped files internally by default).
In my case, I have 37 torrents, all seeding. A few of the torrents have some files excluded from downloading, but all torrents are at 100% (i.e. everything selected is fully downloaded). The total size of the torrents is about 280 GiB. Upon starting qBittorrent, it starts seeding the torrents at 40-90 Mbps, and its resident memory consumption (as shown in
htopas "RES") starts constantly raising. In a few minutes it reaches around 10 GB and continues on.Setting "Disk IO type" to "POSIX-compliant" fixes the memory leak. With this setting and no other changes, resident memory for qbittorrent process is around 200MB and not climbing further. Torrent seeding bitrate is about the same as with the memory mapped IO.
Steps to reproduce
qbittorrentprocess memory consumption inhtop.Additional context
qBittorrent 4.5.5 is installed from this PPA (package version is
1:4.5.5.99~202308312232-7899-7d7097b02~ubuntu22.04.1). It comes with libtorrent-rasterbar2.0.9.git20230830.3a44a5a78e-1ppa1~22.04. I installed this version in attempt to work around the problem.qBittorrent 4.4.1 is the version that is available in Kubuntu 22.04 repository (package version is
4.4.1-2). It comes with libtorrent-rasterbar2.0.5-5. This is the version I originally installed and with which I first had the problem.In the Advanced options, "Physical memory (RAM) usage limit" is set to 512 MiB (which is the default). This limit is definitely exceeded with memory mapped IO.
Log(s) & preferences file(s)
qBittorrent.conf.gz
@Lastique commented on GitHub (Nov 10, 2023):
There is no 4.6.0 package in the qbittorrent-stable PPA. I've installed the
4.6.0~202307200732-8070-2c08dc9da~ubuntu22.04.1package from qbittorrent-unstable PPA. The problem reproduces with it.If you mean "File pool size" in the Advanced settings, it was set to 256 before. Setting it to 100 does not seem to have any effect on the memory leak.
@Lastique commented on GitHub (Nov 10, 2023):
Here's my config file with which the problem reproduces.
qBittorrent.conf.gz
@luzpaz commented on GitHub (Nov 19, 2023):
@Lastique can you use one of the other non-PPA 4.6.0 installs to check this on ?
@Lastique commented on GitHub (Nov 19, 2023):
Sorry, but no. Due to various issues (this memory leak, low download/upload speeds, .part files, file selection issues), I switched to a different torrent client.
Also, in general I avoid using software from non-deb packages.
@vincejv commented on GitHub (Dec 3, 2023):
@Lastique
I switched to a different torrent client.can you post here this alternative client to also help other people encountering this mountain of issues with QBT? I'm also having several issues with QBT recently and piling up, like crashing due to OOM kills due to memory leaks and file name encoding issues and it's taking several mos for QBT/Libtorrent team to fix it, hoping for a quick and easy solution.
Is it still based on libtorrent or is it transmission?
@Lastique commented on GitHub (Dec 3, 2023):
It's KTorrent. It's not without issues either, but those are not as critical as the ones I listed here.
It uses its own implementation of BitTorrent protocol.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
@luzpaz this is not a bug. This is how libtorrent 2.x works unfortunately.
There are lots of duplicates of the issues here: https://github.com/qbittorrent/qBittorrent/issues?q=is%3Aissue+memory+ram
I think we may want to create a single meta-issue for that and close others as duplicates, if you don't mind.
The root cause is infamous https://github.com/arvidn/libtorrent/issues/6667
@vincejv commented on GitHub (Dec 3, 2023):
QBT 5.0 is going to drop anytime soon, perhaps Q1-Q2 2024, I hope libtorrent can catch up🙏🙏 (but as per the lt's head dev, it's going to take some time reworking the code 😵😵)
As we've known in the past QBT's dev are pretty strong in their stand on upgrading dependencies unless there's a mob throwing pitch forks, last stunt they pulled was quietly dropping support for Win 8.1 due to QT, hope it won't be the same scenario again the past year wherein they forced Libtorrent v2.
Should keep the complaints coming so they don't pull off the same move again.
Citing: https://github.com/qbittorrent/qBittorrent/issues/17545 - Go back to using libtorrent 1.2.x until qBittorrent 5.x
@HanabishiRecca commented on GitHub (Dec 3, 2023):
Yeah. You can kinda track the progress in https://github.com/arvidn/libtorrent/pull/7013
@Lastique commented on GitHub (Dec 3, 2023):
I know I'm not a qBitTorrent user at this point, but that does not sound like a valid excuse to me. If the memory leak is by design of libtorrent 2.x then that design is broken and needs to be fixed. You cannot just dismiss the problem as "not a bug", as it clearly affects many users and definitely not the expected behavior. Read: it is a bug.
Also I'd like to point out that qBitTorrent, as it is distributed in Kubuntu and PPAs (I'm not sure how "official" they are), only comes with libtorrent 2.x. This is the way people will use qBitTorrent, unless they are willing to go the extra mile of building it themselves or using some other build that uses libtorrent 1.x. And even before that, they must first find out that libtorrent 2.x is the cause of their problems, which isn't obvious at all to a new user.
If your (qBitTorrent maintainers) response is that libtorrent 2.x is broken beyond repair and qBitTorrent should only ever be used with libtorrent 1.x then please make that abundantly clear everywhere - on your website, on this GitHub project front page, etc. Also make sure the distro packages and PPAs are built against libtorrent 1.x. Make a configure-time check that prohibits the use of libtorrent 2.x as a build dependency.
As it stands, as long as qBitTorrent + libtorrent 2.x remains a valid combination, I consider problems coming from it to be valid bugs.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
It is not a memory leak. This is how memory-mapped files work both in Linux and Windows. Check the issue I linked or use
htop/free/whatever yourself to see how much memory is actually used in your system.It may be a problem, but it is not a bug. A bug is when something works not as intended. But current behavior is totally by design.
So it is better not to complain about things you don't understand and read something like https://www.linuxatemyram.com/
@Lastique commented on GitHub (Dec 3, 2023):
As I said in the problem description, I was using
htopto monitor memory consumption. And that memory consumption is very real as it causes OOM handler to kill processes at some point.Exhausting system memory uncontrollably certainly isn't the intended behavior of a torrent client. Again, if that's the behavior by design then the design is broken.
Thanks, I do understand a few things, including about memory accounting in Linux. I also understand it when I see that the system runs out of RAM.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
Well, then you have not paid enough attention to the actual used metric.
That's an another question. Yes, this thing happening is a bug. But unfortunately this is a bug of the kernel itself. This is not supposed to happen.
Nor qBittorrent or libtorrent control how memory is allocated or deallocated for memory-mapped files. This is 100% the kernel job. And turns out it does not do it well unfortunately.
So yeah, that's why this idea got a pushback. Again, read the discussion in https://github.com/arvidn/libtorrent/issues/6667
@vincejv commented on GitHub (Dec 3, 2023):
I've read the thread, this PR is mostly arvidn addressing issues with FUSE or Network file system performance, not necessarily the RAM or OOM issues. Has someone actually reported the "kernel bug" if it is indeed "an issue" with the kernel?
I'd stick with libtorrentv1 imo, even after that PR.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
They are memory-mapped I/O exclusive problems. Switching form it fixes that problems automatically.
I don't think so. I personally not proficient enough to dive into the kernel source and find why it misbehaves.
But it is not really worth investigating for us, because:
So yeah, we definitely want to switch from it entirely. Which means we effectively can only wait. Or help Arvid to implement that new I/O engine, if you can.
@glassez commented on GitHub (Dec 3, 2023):
Is there any problem with the kernel? Doesn't it behave the way it's supposed to?
The problem is not that "memory mapped files" feature does not work correctly, but that it is used incorrectly. Apparently, @arvidn just overestimated its capabilities, considering that it was some kind of "magic box" that would do all the work itself. But "memory mapped files" is not some kind of high-level tool. It just does its job the way the operating system is supposed to behave, i.e. it just consumes all available resources.
If @arvidn had implemented memory consumption control (as it was before libtorrent 2.0), "memory mapped files" would not have caused any problems.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
That's arguable indeed. But I'm fairly sure VFS cache should not cause OOM.
That's exactly how it is advertised though.
True. But again, it is not supposed to be able to cause OOM. Cached FS pages are simply evicted when the actual working set is required by some app.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
But reading again about the kernel tunables, there is actually some interesting phrasing.
https://docs.kernel.org/admin-guide/sysctl/vm.html#vfs-cache-pressure
Which actually drives me to a conclusion that OOM happens when application produces large enough amounts of I/O to exceed the kernel's "fair" reclaim rate. And that's why it is hard to reproduce the problem intentionally.
@Lastique commented on GitHub (Dec 3, 2023):
Memory mapped files are not VFS cache though. If a process maps a file (or its region), the pages backing that mapping become "locked" in the sense that the kernel can no longer reuse those pages for other purposes. Either those pages need to be in the physical RAM or swapped out. This is unlike the pages that are used by the kernel for regular file IO caching.
If a process continuously maps files without unmapping, this will eventually exhaust both physical RAM and swap and cause OOM. This is entirely expected outcome and is not a bug in the kernel. I'm not sure what else was expected to happen.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
Not really. Again, if you run
free -wh, memory-mapped files go into "cache" column, not into "used". Despite being reported as resident memory of the process, they do not decrease amount of "available" memory in the system.According to my testing with qBt+LT2.0, it is fine to seed large amounts of data, significantly larger than RAM size and not using swap at all. Its behavior is inconsistent though, OOM may not happen for weeks or even months of uptime, despite physical RAM being 100% full all the time. Opening new apps does not cause OOM either, "cache" memory is simply being reclaimed as intended.
Even when 1 single file is larger than the entire RAM, it does not cause OOM. You can easily test it by forcing such torrent recheck. So there is definitely inconsistency in that. I'm not into the details though.
Btw, some people noticed that reducing
File pool sizevalue in advanced options helps.I think reducing the amount of mapped files at a time simply reduces probability of hitting certain I/O threshold, as unmapping reclaims the memory immediately.
@HanabishiRecca commented on GitHub (Dec 3, 2023):
Forgot to mention, limiting the working set of the process mitigates the issue. Client reaching artifical limit does not cause the kernel to commit OOM on it.
Windows users seem to be happy with in-built qBittorrent option for that. Unfortunately, it does not work on Linux and you need to use external tools for that.
@vincejv commented on GitHub (Dec 3, 2023):
Are you referring to this option in QBT? If so, might as well change this option to "applied if libtorrent >= 2.0 and Windows only"
@glassez commented on GitHub (Dec 3, 2023):
It cannot exhaust the swap, since the swap in this case is the mapped files themselves, and not a regular limited-size swap file.
@glassez commented on GitHub (Dec 3, 2023):
Limiting the working set is just a workaround. It cannot fully replace the lack of a managed "cache" in libtorrent 2.0.
It was supposed to run on Linux.
@qbittorrent/bug-handlers
Can someone confirm that it doesn't work? Then we would need to remove this option on Linux in the same way as we previously did on macOS.
@HanabishiRecca commented on GitHub (Dec 4, 2023):
Yes, as I said, its performance is suboptimal.
Umm, that's what the source tells
github.com/qbittorrent/qBittorrent@4ba8eaf4b4/src/app/application.cpp (L1167)And your own conversations
And I tested it before, it does not work.
@glassez commented on GitHub (Dec 4, 2023):
@Chocobo1, ping.
@luzpaz commented on GitHub (Mar 16, 2025):
soft bump
@e-dervieux commented on GitHub (Sep 18, 2025):
Not an expert on memory-mapped I/O, but I would tend to agree with Lastique's point of view: it is not an expected behaviour that a torrent client is slowly eating up more and more RAM while uploading. I just noticed this issue, upon startup QBT RAM usage is about 80MB but it slowly grows with upload to reach several GB. I do hope the OS does something about it at some point (I have 64GB RAM so waiting for QBT to put some pressure on available RAM would take quite some time).
Thus I would very much be inclined to see this as a bug... and in all cases, whether you call it a bug or a (broken) design does not really matter imho. This is very much not a desired behaviour for a torrent client and thus ought to be fixed.
Anyway, is there a known workaround for this? I saw something about limiting RAM usage in the advanced settings somewhere but could not find it in the settings (v5.1.2, Win11), am I blind?
Note: I do not have the skills to try to understand and fix this, but I'd be glad to help with further experimentation / diagnosis / debug log report / etc. if need be, do not hesitate to ping me.
@HanabishiRecca commented on GitHub (Sep 18, 2025):
Settings>Advanced>Disk IO typesetSimple pread/pwrite.Alternatively, use libtorrent 1.x build (installer without
lt20in its name).