&gt;750 torrents are not remembered when qTorrent restarts #2047

Author

Owner

@specul8 commented on GitHub (Jan 16, 2015):

Version 3.1.11

From: Chris Hirst [mailto:notifications@github.com]
Sent: 16 January, 2015 10:48 PM
To: qbittorrent/qBittorrent
Cc: specul8
Subject: Re: [qBittorrent] >750 torrents are not remembered when qTorrent restarts (#2424)

What version??

—
Reply to this email directly or view it on GitHub https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70243110 . https://github.com/notifications/beacon/AKED06waOYKNmgQubWPyutKSuDkM-9OVks5niPIIgaJpZM4DTBRE.gif

@specul8 commented on GitHub (Jan 16, 2015): Version 3.1.11 From: Chris Hirst [mailto:notifications@github.com] Sent: 16 January, 2015 10:48 PM To: qbittorrent/qBittorrent Cc: specul8 Subject: Re: [qBittorrent] >750 torrents are not remembered when qTorrent restarts (#2424) What version?? — Reply to this email directly or view it on GitHub https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70243110 . https://github.com/notifications/beacon/AKED06waOYKNmgQubWPyutKSuDkM-9OVks5niPIIgaJpZM4DTBRE.gif

deekerman commented

Author

Owner

@sledgehammer999 commented on GitHub (Jan 17, 2015):

Which os are you using?
When you do File->Exit do you wait for qbt to disappear from the process list?

@sledgehammer999 commented on GitHub (Jan 17, 2015): Which os are you using? When you do File->Exit do you wait for qbt to disappear from the **process list**?

deekerman commented

Author

Owner

@specul8 commented on GitHub (Jan 17, 2015):

Windows 8 64-bit
Yes, I have waited for the process to finish before - In fact, I made sure
I did it before logging the issue - I wanted to validate that the issue
exists when using the program normally (It does).

HTH - Brad

On 17 January 2015 at 20:42, sledgehammer999 notifications@github.com
wrote:

Which os are you using?
When you do File->Exit do you wait for qbt to disappear from the process
list?

—
Reply to this email directly or view it on GitHub
https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70360895
.

@specul8 commented on GitHub (Jan 17, 2015): Windows 8 64-bit Yes, I have waited for the process to finish before - In fact, I made sure I did it before logging the issue - I wanted to validate that the issue exists when using the program normally (It does). HTH - Brad On 17 January 2015 at 20:42, sledgehammer999 notifications@github.com wrote: > Which os are you using? > When you do File->Exit do you wait for qbt to disappear from the _process > list_? > > — > Reply to this email directly or view it on GitHub > https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70360895 > .

deekerman commented

Author

Owner

@chrishirst commented on GitHub (Jan 17, 2015):

A few questions.

Do the numbers in the '#' column change?

Have you adjusted the queue position of any of these tasks?

What column is the task list sorted by?

Are these all 'download' tasks?

@chrishirst commented on GitHub (Jan 17, 2015): A few questions. Do the numbers in the '#' column change? Have you adjusted the queue position of any of these tasks? What column is the task list sorted by? Are these all 'download' tasks?

deekerman commented

Author

Owner

@specul8 commented on GitHub (Jan 17, 2015):

Yes, the numbers assigned to each download change in addition to me losing the torrents I've added after the 750th one.

Yes, I reorder them every time I start the program.

When I shut down, it is sorted by number. When I start up, it is sorted by date added.

Yes, all download tasks - the seeds seem to be fine.

Regards
Brad Saide
M +61 (0) 458 237 230
http://kb4sp.wordpress.com

-----Original Message-----
From: "Chris Hirst" notifications@github.com
Sent: ‎18/‎01/‎2015 12:50 AM
To: "qbittorrent/qBittorrent" qBittorrent@noreply.github.com
Cc: "specul8" brad.saide@gmail.com
Subject: Re: [qBittorrent] >750 torrents are not remembered when qTorrentrestarts (#2424)

A few questions.
Do the numbers in the '#' column change?
Have you adjusted the queue position of any of these tasks?
What column is the task list sorted by?
Are these all 'download' tasks?
—
Reply to this email directly or view it on GitHub.=

@specul8 commented on GitHub (Jan 17, 2015): Yes, the numbers assigned to each download change in addition to me losing the torrents I've added after the 750th one. Yes, I reorder them every time I start the program. When I shut down, it is sorted by number. When I start up, it is sorted by date added. Yes, all download tasks - the seeds seem to be fine. Regards Brad Saide M +61 (0) 458 237 230 http://kb4sp.wordpress.com -----Original Message----- From: "Chris Hirst" notifications@github.com Sent: ‎18/‎01/‎2015 12:50 AM To: "qbittorrent/qBittorrent" qBittorrent@noreply.github.com Cc: "specul8" brad.saide@gmail.com Subject: Re: [qBittorrent] >750 torrents are not remembered when qTorrentrestarts (#2424) A few questions. Do the numbers in the '#' column change? Have you adjusted the queue position of any of these tasks? What column is the task list sorted by? Are these all 'download' tasks? — Reply to this email directly or view it on GitHub.=

deekerman commented

Author

Owner

@chrishirst commented on GitHub (Jan 17, 2015):

Yes, all download tasks - the seeds seem to be fine.
Seeds do not have a "queue" order

When I shut down, it is sorted by number. When I start up, it is sorted by date added.

So; the actual problem is simply that the column selected for the task list sort is not being saved rather the queue numbering is changing?

@chrishirst commented on GitHub (Jan 17, 2015): > Yes, all download tasks - the seeds seem to be fine. > Seeds do not have a "queue" order > > When I shut down, it is sorted by number. When I start up, it is sorted by date added. So; the actual problem is simply that the column selected for the task list sort is not being saved rather the queue numbering is changing?

deekerman commented

Author

Owner

@sledgehammer999 commented on GitHub (Jan 17, 2015):

Out of curiosity and in the hopes of avoiding extra time debugging this can you also try with the latest alpha?
Link: http://builds.shiki.hu/qbittorrent_3.2.0alpha_20141229_ca2dc325_setup.exe

@sledgehammer999 commented on GitHub (Jan 17, 2015): Out of curiosity and in the hopes of avoiding extra time debugging this can you also try with the latest alpha? Link: http://builds.shiki.hu/qbittorrent_3.2.0alpha_20141229_ca2dc325_setup.exe

deekerman commented

Author

Owner

@specul8 commented on GitHub (Jan 18, 2015):

No - The problem is:
Queued torrents above about 780 are disappearing when qbt is cleanly shut
down and restarted (this is the reason I reported the error).
There is also an issue where the queue ordering is changing on QBT restart
as well

It's the loss of queued torrents that is the concern - the other stuff I'm
OK to deal with, as much of a pain in the butt as it is...

On 18 January 2015 at 09:37, Chris Hirst notifications@github.com wrote:

Yes, all download tasks - the seeds seem to be fine.
Seeds do not have a "queue" order

When I shut down, it is sorted by number. When I start up, it is sorted by
date added.

So; the actual problem is simply that the column selected for the task
list sort is not being saved rather the queue numbering is changing?

—
Reply to this email directly or view it on GitHub
https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70387566
.

@specul8 commented on GitHub (Jan 18, 2015): No - The problem is: Queued torrents above about 780 are disappearing when qbt is cleanly shut down and restarted (this is the reason I reported the error). There is also an issue where the queue ordering is changing on QBT restart as well It's the loss of queued torrents that is the concern - the other stuff I'm OK to deal with, as much of a pain in the butt as it is... On 18 January 2015 at 09:37, Chris Hirst notifications@github.com wrote: > Yes, all download tasks - the seeds seem to be fine. > Seeds do not have a "queue" order > > When I shut down, it is sorted by number. When I start up, it is sorted by > date added. > > So; the actual problem is simply that the column selected for the task > list sort is not being saved rather the queue numbering is changing? > > — > Reply to this email directly or view it on GitHub > https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70387566 > .

deekerman commented

Author

Owner

@specul8 commented on GitHub (Jan 18, 2015):

I will try with the latest alpha and let you know - it may take a couple of
days though.

On 18 January 2015 at 10:13, sledgehammer999 notifications@github.com
wrote:

Out of curiosity and in the hopes of avoiding extra time debugging this
can you also try with the latest alpha?
Link:
http://builds.shiki.hu/qbittorrent_3.2.0alpha_20141229_ca2dc325_setup.exe

—
Reply to this email directly or view it on GitHub
https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70388774
.

@specul8 commented on GitHub (Jan 18, 2015): I will try with the latest alpha and let you know - it may take a couple of days though. On 18 January 2015 at 10:13, sledgehammer999 notifications@github.com wrote: > Out of curiosity and in the hopes of avoiding extra time debugging this > can you also try with the latest alpha? > Link: > http://builds.shiki.hu/qbittorrent_3.2.0alpha_20141229_ca2dc325_setup.exe > > — > Reply to this email directly or view it on GitHub > https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70388774 > .

deekerman commented

Author

Owner

@sledgehammer999 commented on GitHub (Jan 18, 2015):

So 2 different problems:

The queue numbers get screwed up
It remembers ~750-780 torrents and it totally forgets the rest. Until you test the last alpha build here are some info, that may give you an idea on what is happening.
qbt saves a copy of each .torrent along with its .fastresume file inside a folder named BT_backup.
Normally in that folder the number of .torrent files should be exactly the same as the ones you have in the transfer list. Unless some of those torrents are magnet links that haven't downloaded their metadata files yet.

@sledgehammer999 commented on GitHub (Jan 18, 2015): So 2 different problems: 1. The queue numbers get screwed up 2. It remembers ~750-780 torrents and it totally forgets the rest. Until you test the last alpha build here are some info, that may give you an idea on what is happening. qbt saves a copy of each .torrent along with its .fastresume file inside a folder named BT_backup. Normally in that folder the number of .torrent files should be exactly the same as the ones you have in the transfer list. Unless some of those torrents are magnet links that haven't downloaded their metadata files yet.

deekerman commented

Author

Owner

@specul8 commented on GitHub (Jan 18, 2015):

Yep, 2 different problems.

Regarding the lost torrents, I have tested it, and it looks like this is the root cause: Unless some of those torrents are magnet links that haven't downloaded their metadata files yet.

I use a blend of magnet and torrent links – upon testing, it is only the magnet links that disappear. When you add torrents the magnet metadata file does not download before the torrent is marked as queued and the downloading stops. So really the enhancement should be something like

When a torrent is added either:

    The Magnet URL is logged or recorded and “re-added” to the queue on restart (seems like the user experience would be better for this one, but maybe more complex to implement?)

OR

    QBT automatically forces the download of metadata files for all added torrents before checking to see if they should be paused in a queue (probably easier to implement this one, but may require another “state” like “Downloading Metadata File” as opposed to “downloading” to make it more user intuitive)

Thanks guys – I look forward to seeing this enhancement come through!

Brad

From: sledgehammer999 [mailto:notifications@github.com]
Sent: 19 January, 2015 12:35 AM
To: qbittorrent/qBittorrent
Cc: specul8
Subject: Re: [qBittorrent] >750 torrents are not remembered when qTorrent restarts (#2424)

So 2 different problems:

The queue numbers get screwed up
It remembers ~750-780 torrents and it totally forgets the rest. Until you test the last alpha build here are some info, that may give you an idea on what is happening.
qbt saves a copy of each .torrent along with its .fastresume file inside a folder named BT_backup.
Normally in that folder the number of .torrent files should be exactly the same as the ones you have in the transfer list. Unless some of those torrents are magnet links that haven't downloaded their metadata files yet.

—
Reply to this email directly or view it on GitHub https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70408491 . https://github.com/notifications/beacon/AKED07fjtMBsf0xc63plNnyzCpX6VRAOks5ni64UgaJpZM4DTBRE.gif

@specul8 commented on GitHub (Jan 18, 2015): Yep, 2 different problems. Regarding the lost torrents, I have tested it, and it looks like this is the root cause: Unless some of those torrents are magnet links that haven't downloaded their metadata files yet. I use a blend of magnet and torrent links – upon testing, it is only the magnet links that disappear. When you add torrents the magnet metadata file does not download before the torrent is marked as queued and the downloading stops. So really the enhancement should be something like When a torrent is added either: - The Magnet URL is logged or recorded and “re-added” to the queue on restart (seems like the user experience would be better for this one, but maybe more complex to implement?) OR - QBT automatically forces the download of metadata files for all added torrents before checking to see if they should be paused in a queue (probably easier to implement this one, but may require another “state” like “Downloading Metadata File” as opposed to “downloading” to make it more user intuitive) Thanks guys – I look forward to seeing this enhancement come through! Brad From: sledgehammer999 [mailto:notifications@github.com] Sent: 19 January, 2015 12:35 AM To: qbittorrent/qBittorrent Cc: specul8 Subject: Re: [qBittorrent] >750 torrents are not remembered when qTorrent restarts (#2424) So 2 different problems: 1. The queue numbers get screwed up 2. It remembers ~750-780 torrents and it totally forgets the rest. Until you test the last alpha build here are some info, that may give you an idea on what is happening. qbt saves a copy of each .torrent along with its .fastresume file inside a folder named BT_backup. Normally in that folder the number of .torrent files should be exactly the same as the ones you have in the transfer list. Unless some of those torrents are magnet links that haven't downloaded their metadata files yet. — Reply to this email directly or view it on GitHub https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70408491 . https://github.com/notifications/beacon/AKED07fjtMBsf0xc63plNnyzCpX6VRAOks5ni64UgaJpZM4DTBRE.gif

deekerman commented

Author

Owner

@chrishirst commented on GitHub (Jan 18, 2015):

Actually it is more like one and the same root cause

If the metadata has not downloaded, so fast resume files have NOT been created, qBT / libtorrent can ONLY provide a queue number for the task that DO HAVE a resume metadata file in BT_backup and therefore the rest do not even exist.

The solution:

Do NOT restart the client until the metadata has loaded for ALL tasks.

but may require another “state” like “Downloading Metadata File” as opposed to “downloading” to make it more user intuitive)

It already has and does. You have to make the column wide enough to see it.

@chrishirst commented on GitHub (Jan 18, 2015): Actually it is more like one and the same root cause If the metadata has not downloaded, so fast resume files have NOT been created, qBT / libtorrent can ONLY provide a queue number for the task that DO HAVE a resume metadata file in BT_backup and therefore the rest do not even exist. The solution: Do NOT restart the client until the metadata has loaded for ALL tasks. > but may require another “state” like “Downloading Metadata File” as opposed to “downloading” to make it more user intuitive) It already has and does. You have to make the column wide enough to see it.

deekerman commented

Author

Owner

@specul8 commented on GitHub (Jan 18, 2015):

Just to be clear - It does not download for days - in fact, it does not
download at all until the torrent reaches the active queue. It really seems
like unexpected behaviour... as a user, I would imagine the product just
managed all of those housekeeping tasks automatically and in the
background. If you test the scenario I have presented, you will see what I
mean - from a user perspective:

I add 30 torrents to my download queue on Monday
- There are 500 items to download before them, so they enter a Queued
  state immediately
- At this time, the Magnet torrents have not downloaded the metadata
  files but the normal torrents are fine
On Wednesday I cleanly shut down QBT and restart it
- Some of the 30 torrents disappear AND
- The download order is all screwed up.

Looking at it from a user perspective, there seems to be an "unexpected
behaviour" issue (deleting "random" torrents) and the ordering information
I have entered is lost. Is that a fair summary of the problem?

On 19 January 2015 at 04:26, Chris Hirst notifications@github.com wrote:

Actually it is more like one and the same root cause

If the metadata has not downloaded, so fast resume files have NOT been
created, qBT / libtorrent can ONLY provide a queue number for the task that
DO HAVE a resume metadata file in BT_backup and therefore the rest do not
even exist.

The solution:

Do NOT restart the client until the metadata has loaded for ALL tasks.

—
Reply to this email directly or view it on GitHub
https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70416821
.

@specul8 commented on GitHub (Jan 18, 2015): Just to be clear - It does not download for days - in fact, it does not download at all until the torrent reaches the active queue. It really seems like unexpected behaviour... as a user, I would imagine the product just managed all of those housekeeping tasks automatically and in the background. If you test the scenario I have presented, you will see what I mean - from a user perspective: - I add 30 torrents to my download queue on Monday - There are 500 items to download before them, so they enter a Queued state immediately - At this time, the Magnet torrents have not downloaded the metadata files but the normal torrents are fine - On Wednesday I cleanly shut down QBT and restart it - Some of the 30 torrents disappear AND - The download order is all screwed up. Looking at it from a user perspective, there seems to be an "unexpected behaviour" issue (deleting "random" torrents) and the ordering information I have entered is lost. Is that a fair summary of the problem? On 19 January 2015 at 04:26, Chris Hirst notifications@github.com wrote: > Actually it is more like one and the same root cause > > If the metadata has not downloaded, so fast resume files have NOT been > created, qBT / libtorrent can ONLY provide a queue number for the task that > DO HAVE a resume metadata file in BT_backup and therefore the rest do not > even exist. > > The solution: > > Do NOT restart the client until the metadata has loaded for ALL tasks. > > — > Reply to this email directly or view it on GitHub > https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70416821 > .

deekerman commented

Author

Owner

@sledgehammer999 commented on GitHub (Jan 18, 2015):

as a user, I would imagine the product just managed all of those housekeeping tasks automatically and in the background.

Yes, it's probably a bug. magnet links are saved qbittorrent-resume.ini. So either there is a bug somewhere or that file isn't written correctly.
Does the log say anything funny? (view->execution log)

@sledgehammer999 commented on GitHub (Jan 18, 2015): > as a user, I would imagine the product just managed all of those housekeeping tasks automatically and in the background. Yes, it's probably a bug. magnet links are saved qbittorrent-resume.ini. So either there is a bug somewhere or that file isn't written correctly. Does the log say anything funny? (view->execution log)

deekerman commented

Author

Owner

@sledgehammer999 commented on GitHub (Jan 18, 2015):

Is that a fair summary of the problem?

Yes.

@sledgehammer999 commented on GitHub (Jan 18, 2015): > Is that a fair summary of the problem? Yes.

deekerman commented

Author

Owner

@specul8 commented on GitHub (Jan 18, 2015):

Thanks Sledgehammer - I'll check on this tonight. (Australia time).

On 19 January 2015 at 09:52, sledgehammer999 notifications@github.com
wrote:

as a user, I would imagine the product just managed all of those
housekeeping tasks automatically and in the background.

Yes, it's probably a bug. magnet links are saved qbittorrent-resume.ini.
So either there is a bug somewhere or that file isn't written correctly.
Does the log say anything funny? (view->execution log)

—
Reply to this email directly or view it on GitHub
https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70430542
.

@specul8 commented on GitHub (Jan 18, 2015): Thanks Sledgehammer - I'll check on this tonight. (Australia time). On 19 January 2015 at 09:52, sledgehammer999 notifications@github.com wrote: > as a user, I would imagine the product just managed all of those > housekeeping tasks automatically and in the background. > > Yes, it's probably a bug. magnet links are saved qbittorrent-resume.ini. > So either there is a bug somewhere or that file isn't written correctly. > Does the log say anything funny? (view->execution log) > > — > Reply to this email directly or view it on GitHub > https://github.com/qbittorrent/qBittorrent/issues/2424#issuecomment-70430542 > .

deekerman commented

Author

Owner

@specul8 commented on GitHub (Feb 8, 2015):

I have been running some tests on this over the last few days, and it seems like:

The issues only happens when you shut down the computer and restart it, and
It only triggers when there are a large number of queued torrents (including some that do not yet have their metadata downloaded)

Shutting down qBittorrent normally takes about 3-4 minutes for me with the 750-odd torrents in it, however when I reboot the computer it shuts down in about 1 minute (and qBittorrent does not come up in the "waiting to close" app list) which makes me think that during a reboot cycle it does not shut down cleanly.

Hopefully this helps narrow down the problem. Shutting down qBittorrent cleanly (waiting until it had disappeared from the Task manager) and restarting it works fine, even with magnet links that have not downloaded the metadata files yet.

@specul8 commented on GitHub (Feb 8, 2015): I have been running some tests on this over the last few days, and it seems like: - The issues only happens when you shut down the computer and restart it, and - It only triggers when there are a large number of queued torrents (including some that do not yet have their metadata downloaded) Shutting down qBittorrent normally takes about 3-4 minutes for me with the 750-odd torrents in it, however when I reboot the computer it shuts down in about 1 minute (and qBittorrent does not come up in the "waiting to close" app list) which makes me think that during a reboot cycle it does not shut down cleanly. Hopefully this helps narrow down the problem. Shutting down qBittorrent cleanly (waiting until it had disappeared from the Task manager) and restarting it works fine, even with magnet links that have not downloaded the metadata files yet.

deekerman commented

Author

Owner

@sledgehammer999 commented on GitHub (Feb 8, 2015):

Yes this helps. And it should get fixed when #1984 gets merged. Short story: after Windows XP, if your app doesn't notify the OS that it needs time to cleanly shutdown, the OS will just kill it. This is what happens here. qbt takes too much time shutting down and it doesn't inform the OS, so Windows kills it.
So for the time being, exit qbt first, wait for it to disappear from the task manager and then issue a reboot.

@sledgehammer999 commented on GitHub (Feb 8, 2015): Yes this helps. And it should get fixed when #1984 gets merged. Short story: after Windows XP, if your app doesn't notify the OS that it needs time to cleanly shutdown, the OS will just kill it. This is what happens here. qbt takes too much time shutting down and it doesn't inform the OS, so Windows kills it. So for the time being, exit qbt first, wait for it to disappear from the task manager and then issue a reboot.

deekerman commented