Filter directory filling up with old filter files

deekerman commented

2026-03-04 02:51:20 -05:00

Owner

Originally created by @gabrielwhite on GitHub (Sep 8, 2021).

Originally assigned to: @EugeneOne1 on GitHub.

Issue Details

Version of AdGuard Home server:
- v0.107.0-a.155+b77ceaaf
How did you install AdGuard Home:
- docker
CPU architecture:
- ARM64
Operating system and version:
- Armbian 21.08.1 Buster with Linux 5.10.60-sunxi64

Expected Behavior

That old filter files are deleted automatically

Actual Behavior

I got a "disk full" warning from my OrangePi One Plus and found that the filters directory had 22GB of files in it across more than 100 files in dockers/adguardhome/data/work/data/filters

I have 2 filters enabled, which total around 60MB.

I deleted all the files in the directory, reinstalled the filters. This issue is likely accumulated over time earlier than the current version as there were many files dating back at least weeks.

Subsequently when I check for updates one of the filters generates an error shown in a red toast:

Error: control/filtering/refresh | <html> ,<head><title>502 Bad Gateway</title></head>, ,<body> ,<center><h1>502 Bad Gateway</h1></center>, ,<hr />,<center>openresty</center>, </body>, </html> | 502

This error doesn't show on the other adguard server I use with the same configuration, but running v0.106.3.

When I look in the filter directory I see a new file created without a "txt" extension.

-rw-r--r-- 1 root users  63M Sep  8 09:55 1631105697.txt
-rw-r--r-- 1 root users 746K Sep  8 09:56 1631105792.txt
-rw-r--r-- 1 root users  63M Sep  8 10:01 2463411087
-rw-r--r-- 1 root users  63M Sep  8 10:39 2963478244

Originally created by @gabrielwhite on GitHub (Sep 8, 2021). Originally assigned to: @EugeneOne1 on GitHub. ### Issue Details * **Version of AdGuard Home server:** * v0.107.0-a.155+b77ceaaf * **How did you install AdGuard Home:** * docker * **CPU architecture:** * ARM64 * **Operating system and version:** * Armbian 21.08.1 Buster with Linux 5.10.60-sunxi64 ### Expected Behavior That old filter files are deleted automatically ### Actual Behavior I got a "disk full" warning from my OrangePi One Plus and found that the filters directory had 22GB of files in it across more than 100 files in dockers/adguardhome/data/work/data/filters I have 2 filters enabled, which total around 60MB. I deleted all the files in the directory, reinstalled the filters. This issue is likely accumulated over time earlier than the current version as there were many files dating back at least weeks. Subsequently when I check for updates one of the filters generates an error shown in a red toast: `Error: control/filtering/refresh | <html> ,<head><title>502 Bad Gateway</title></head>, ,<body> ,<center><h1>502 Bad Gateway</h1></center>, ,<hr />,<center>openresty</center>, </body>, </html> | 502` This error doesn't show on the other adguard server I use with the same configuration, but running v0.106.3. When I look in the filter directory I see a new file created without a "txt" extension. ``` -rw-r--r-- 1 root users 63M Sep 8 09:55 1631105697.txt -rw-r--r-- 1 root users 746K Sep 8 09:56 1631105792.txt -rw-r--r-- 1 root users 63M Sep 8 10:01 2463411087 -rw-r--r-- 1 root users 63M Sep 8 10:39 2963478244 ```

deekerman

2026-03-04 02:51:20 -05:00

closed this issue
added the
bug

P3: Medium
labels

deekerman commented

2026-03-04 02:51:32 -05:00

Author

Owner

@ainar-g commented on GitHub (Sep 9, 2021):

Hello and thank you for your report. We can't seem to reproduce this issue on our machines. There are filter backup files, but they all must have a .old extension. If you stop AGH, delete the files, and restart it, do they reappear?

@ainar-g commented on GitHub (Sep 9, 2021): Hello and thank you for your report. We can't seem to reproduce this issue on our machines. There are filter backup files, but they all must have a `.old` extension. If you stop AGH, delete the files, and restart it, do they reappear?

deekerman commented

2026-03-04 02:51:32 -05:00

Author

Owner

@gabrielwhite commented on GitHub (Sep 9, 2021):

Here's the state of the directory one day later:

total 753M
drwxr-S--- 2 root users  12K Sep  9 14:31 .
drwxr-S--- 3 root users 4.0K Sep  9 14:28 ..
-rw-r--r-- 1 root users  63M Sep  8 10:42 1558400762
-rw-r--r-- 1 root users  63M Sep  9 14:31 1571306289
-rw-r--r-- 1 root users  63M Sep  9 13:01 1631105697.txt
-rw-r--r-- 1 root users 747K Sep  9 13:23 1631105792.txt
-rw-r--r-- 1 root users  63M Sep  9 10:45 1642326027
-rw-r--r-- 1 root users  63M Sep  8 10:39 2049054227
-rw-r--r-- 1 root users  63M Sep  8 10:01 2463411087
-rw-r--r-- 1 root users  63M Sep  9 10:44 2726644904
-rw-r--r-- 1 root users  63M Sep  8 10:39 2963478244
-rw-r--r-- 1 root users  63M Sep  9 10:23 3197215412
-rw-r--r-- 1 root users  63M Sep  9 12:58 4045172626
-rw-r--r-- 1 root users  63M Sep  9 11:44 4060345806
-rw-r--r-- 1 root users  63M Sep  9 11:38 4093212594

These are the two blocklists I have installed, and it seems it's the first file that's causing problems somehow.

https://blocklistproject.github.io/Lists/porn.txt
https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt

FWIW I'm now running v0.107.0-a.157+80548233

@gabrielwhite commented on GitHub (Sep 9, 2021): Here's the state of the directory one day later: ``` total 753M drwxr-S--- 2 root users 12K Sep 9 14:31 . drwxr-S--- 3 root users 4.0K Sep 9 14:28 .. -rw-r--r-- 1 root users 63M Sep 8 10:42 1558400762 -rw-r--r-- 1 root users 63M Sep 9 14:31 1571306289 -rw-r--r-- 1 root users 63M Sep 9 13:01 1631105697.txt -rw-r--r-- 1 root users 747K Sep 9 13:23 1631105792.txt -rw-r--r-- 1 root users 63M Sep 9 10:45 1642326027 -rw-r--r-- 1 root users 63M Sep 8 10:39 2049054227 -rw-r--r-- 1 root users 63M Sep 8 10:01 2463411087 -rw-r--r-- 1 root users 63M Sep 9 10:44 2726644904 -rw-r--r-- 1 root users 63M Sep 8 10:39 2963478244 -rw-r--r-- 1 root users 63M Sep 9 10:23 3197215412 -rw-r--r-- 1 root users 63M Sep 9 12:58 4045172626 -rw-r--r-- 1 root users 63M Sep 9 11:44 4060345806 -rw-r--r-- 1 root users 63M Sep 9 11:38 4093212594 ``` These are the two blocklists I have installed, and it seems it's the first file that's causing problems somehow. https://blocklistproject.github.io/Lists/porn.txt https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt FWIW I'm now running v0.107.0-a.157+80548233

deekerman commented

2026-03-04 02:51:32 -05:00

Author

Owner

@gabrielwhite commented on GitHub (Sep 14, 2021):

As of now the issue has partly resolved itself. The directory is no longer filling up with temporary files.

The problem is now limited to when I trigger a manual update only.

I get this error in the Web UI:

Error: control/filtering/refresh | <html> ,<head><title>504 Gateway Time-out</title></head>, ,<body> ,<center><h1>504 Gateway Time-out</h1></center>, ,<hr />,<center>openresty</center>, </body>, </html> | 504

And a temporary file is created and not deleted in the filter folder:

total 127M
drwxr-S--- 2 root users  12K Sep 14 12:18 .
drwxr-S--- 3 root users 4.0K Sep 13 18:15 ..
-rw-r--r-- 1 root users  63M Sep 13 17:39 1631105697.txt
-rw-r--r-- 1 root users 748K Sep 13 17:48 1631105792.txt
-rw-r--r-- 1 root users  63M Sep 14 12:18 3960263269

I'm now running v0.107.0-a.161+fac574d3

@gabrielwhite commented on GitHub (Sep 14, 2021): As of now the issue has partly resolved itself. The directory is no longer filling up with temporary files. The problem is now limited to when I trigger a manual update only. I get this error in the Web UI: `Error: control/filtering/refresh | <html> ,<head><title>504 Gateway Time-out</title></head>, ,<body> ,<center><h1>504 Gateway Time-out</h1></center>, ,<hr />,<center>openresty</center>, </body>, </html> | 504` And a temporary file is created and not deleted in the filter folder: ``` total 127M drwxr-S--- 2 root users 12K Sep 14 12:18 . drwxr-S--- 3 root users 4.0K Sep 13 18:15 .. -rw-r--r-- 1 root users 63M Sep 13 17:39 1631105697.txt -rw-r--r-- 1 root users 748K Sep 13 17:48 1631105792.txt -rw-r--r-- 1 root users 63M Sep 14 12:18 3960263269 ``` I'm now running v0.107.0-a.161+fac574d3

deekerman commented

2026-03-04 02:51:33 -05:00

Author

Owner

@ainar-g commented on GitHub (Sep 14, 2021):

Thanks for the info, we seem to have found the root cause of the issue and are working on fixing it.

@ainar-g commented on GitHub (Sep 14, 2021): Thanks for the info, we seem to have found the root cause of the issue and are working on fixing it.

deekerman commented

2026-03-04 02:51:33 -05:00

Author

Owner

@EugeneOne1 commented on GitHub (Sep 15, 2021):

@gabrielwhite, we've just pushed the latest build to the edge channel. It should fix some logic problems related to the issue. Could you please check, if it works with your filter lists?

@EugeneOne1 commented on GitHub (Sep 15, 2021): @gabrielwhite, we've just pushed the latest build to the edge channel. It should fix some logic problems related to the issue. Could you please check, if it works with your filter lists?

deekerman commented

2026-03-04 02:51:33 -05:00

Author

Owner

@gabrielwhite commented on GitHub (Sep 15, 2021):

Upon upgrading, I ran into the same issue again. Additionally it seemed that Adguard crashed and restarted after triggering a manual update. I'm running v0.107.0-a.166+176a344a.

Once it started crashing I thought it may have been a memory issue, so I increased the memory limitation on the docker to 320MB (it had been capped at 256MB), and this stopped the issue.

I'd previously moved to using the edge version because it ran without crashing in 256MB (0.106 wouldn't run with less memory). Adguard had been causing crashes on the OrangePi because it was chewing up too much memory.

@gabrielwhite commented on GitHub (Sep 15, 2021): Upon upgrading, I ran into the same issue again. Additionally it seemed that Adguard crashed and restarted after triggering a manual update. I'm running v0.107.0-a.166+176a344a. Once it started crashing I thought it may have been a memory issue, so I increased the memory limitation on the docker to 320MB (it had been capped at 256MB), and this stopped the issue. I'd previously moved to using the edge version because it ran without crashing in 256MB (0.106 wouldn't run with less memory). Adguard had been causing crashes on the OrangePi because it was chewing up too much memory.

deekerman commented

2026-03-04 02:51:33 -05:00

Author

Owner

@EugeneOne1 commented on GitHub (Sep 15, 2021):

Well, could you please help us to troubleshoot this issue.

We need you to configure Home to collect verbose log into a file. Note that in case of Docker's container the path to log file points to the container's filesystem, so the simplest way is a file in working or configuration directory which are mounted.

Then trigger the manual filter update and post the file here or send it to devteam@adguard.com with the issue number.

@EugeneOne1 commented on GitHub (Sep 15, 2021): Well, could you please help us to troubleshoot this issue. We need you to [configure](https://github.com/AdguardTeam/AdGuardHome/wiki/FAQ#verboselog) Home to collect verbose log into a file. Note that in case of Docker's container the path to log _file_ points to the container's filesystem, so the simplest way is a file in working or configuration directory which are mounted. Then trigger the manual filter update and post the file here or send it to devteam@adguard.com with the issue number.

deekerman commented

2026-03-04 02:51:33 -05:00

Author

Owner

@leo15dev commented on GitHub (Sep 15, 2021):

Hello and thank you for your report. We can't seem to reproduce this issue on our machines. There are filter backup files, but they all must have a .old extension. If you stop AGH, delete the files, and restart it, do they reappear?

@ainar-g It seems to when you take the action Delete in the DNS blocklists page to delete the blocklists, the system will automatically rename the deleted blocklist file with .old extension (e.g. 1631743246.txt --> 1631743246.txt.old ). And when you added the same filter again, the system will create new file (1631743247.txt) not use the backup one (1631743246.txt.old).

-rw-r--r-- 1 root root 65683147 Sep 16 08:30 1631743246.txt.old
-rw-r--r-- 1 root root 65683147 Sep 16 08:36 1631743247.txt

Since the user do the such things which most often means that they don't need those blocklist, if the user want to temporary pause it, they should temporary not enable the blocklist to instead of deleting it. If it is possible not to rename the deleted blocklist but delete the blocklist file when you take the action Delete in the DNS blocklists page? It will save the disk space, especially for those who install Adguard Home in the router. Thank you.

BTW I used the https://blocklistproject.github.io/Lists/porn.txt filter for the example.

@leo15dev commented on GitHub (Sep 15, 2021): > > > Hello and thank you for your report. We can't seem to reproduce this issue on our machines. There are filter backup files, but they all must have a `.old` extension. If you stop AGH, delete the files, and restart it, do they reappear? @ainar-g It seems to when you take the action `Delete` in the DNS blocklists page to delete the blocklists, the system will automatically rename the deleted blocklist file with `.old` extension (e.g. 1631743246.txt --> 1631743246.txt.old ). And when you added the same filter again, the system will create new file (1631743247.txt) not use the backup one (1631743246.txt.old). ``` -rw-r--r-- 1 root root 65683147 Sep 16 08:30 1631743246.txt.old -rw-r--r-- 1 root root 65683147 Sep 16 08:36 1631743247.txt ``` Since the user do the such things which most often means that they don't need those blocklist, if the user want to temporary pause it, they should temporary not enable the blocklist to instead of deleting it. If it is possible not to rename the deleted blocklist but delete the blocklist file when you take the action `Delete` in the DNS blocklists page? It will save the disk space, especially for those who install Adguard Home in the router. Thank you. BTW I used the https://blocklistproject.github.io/Lists/porn.txt filter for the example.

deekerman commented

2026-03-04 02:51:33 -05:00

Author

Owner

@EugeneOne1 commented on GitHub (Sep 17, 2021):

@gabrielwhite, we've received your logs, thanks! We'll inspect them as soon as possible.

@EugeneOne1 commented on GitHub (Sep 17, 2021): @gabrielwhite, we've received your logs, thanks! We'll inspect them as soon as possible.

deekerman commented

2026-03-04 02:51:33 -05:00

Author

Owner

@EugeneOne1 commented on GitHub (Sep 17, 2021):

@gabrielwhite, we've reproduced the issue and got same results. I'm afraid the amount of RAM allocated for the container is the main cause of these crashes and unremoved temporary files. We've performed a couple of memory tests and concluded that 512 should be enough to handle updating of lists with around 3 million rules. However, you may try to decrease the limitation to some sensible minimum, because it's the only parameter for now which directly affects AdGuard Home stability with regards to updating filters.

We'll close the issue for now, since we cannot do anything about unrecoverable crashes for the moment. We'll consider further optimizations.

@EugeneOne1 commented on GitHub (Sep 17, 2021): @gabrielwhite, we've reproduced the issue and got same results. I'm afraid the amount of RAM allocated for the container is the main cause of these crashes and unremoved temporary files. We've performed a couple of memory tests and concluded that 512 should be enough to handle updating of lists with around 3 million rules. However, you may try to decrease the limitation to some sensible minimum, because it's the only parameter for now which directly affects AdGuard Home stability with regards to updating filters. We'll close the issue for now, since we cannot do anything about unrecoverable crashes for the moment. We'll consider further optimizations.

Rows
Columns

Filter directory filling up with old filter files #3022

Issue Details

Expected Behavior

Actual Behavior