Speed up query log filtering #2893

New issue

Closed

opened 2026-03-04 02:35:56 -05:00 by deekerman · 1 comment

deekerman commented

2026-03-04 02:35:56 -05:00

Owner

Originally created by @Dubz on GitHub (Jul 11, 2021).

Running AdGuard via docker on a RasPi that's running Hypriot.

Querying DNS logs seems to take quite some time. As in, at least a few minutes, just to choose a preset filter. There's currently about half a million logs it's pulling, and only a few that are flagged as malicious/adult. Those are what I'm trying to pull essentially.
I think there should be some level of indexing for categories like this, since there is a much lower number of blocks overall. Mind you that I've only been running this for about a week or two.

Perhaps maybe increase log retention rules so blocks could be held longer than accepted? For instance, limit accepted domains to three days, while blocks could be thirty days. User customizable so we can decide based on our situation.

*I know different/better hardware would help some, but I don't think this should take that long to pull. I know I've had issues with querying PiHole on my gaming rig, experiencing similar results. I know that's a different platform entirely, but the overall performance is pretty similar with searching (so I'd imagine AdGuard would be slow as well).

Originally created by @Dubz on GitHub (Jul 11, 2021). Running AdGuard via docker on a RasPi that's running Hypriot. Querying DNS logs seems to take quite some time. As in, at least a few minutes, just to choose a preset filter. There's currently about half a million logs it's pulling, and only a few that are flagged as malicious/adult. Those are what I'm trying to pull essentially. I think there should be some level of indexing for categories like this, since there is a much lower number of blocks overall. Mind you that I've only been running this for about a week or two. Perhaps maybe increase log retention rules so blocks could be held longer than accepted? For instance, limit accepted domains to three days, while blocks could be thirty days. User customizable so we can decide based on our situation. *I know different/better hardware would help some, but I don't think this should take that long to pull. I know I've had issues with querying PiHole on my gaming rig, experiencing similar results. I know that's a different platform entirely, but the overall performance is pretty similar with searching (so I'd imagine AdGuard would be slow as well).

deekerman

2026-03-04 02:35:56 -05:00

closed this issue
added the
duplicate
label

deekerman commented

2026-03-04 02:36:08 -05:00

Author

Owner

@ainar-g commented on GitHub (Jul 12, 2021):

This is mostly a duplicate of #2290. The current implementation of the query log has major disadvantages, one of which is performance whenever a lot of data has to be scanned, and we won't be able to do anything about that until we change the storage engine behind it. I'll merge this issue into that one, if you don't mind.

@ainar-g commented on GitHub (Jul 12, 2021): This is mostly a duplicate of #2290. The current implementation of the query log has major disadvantages, one of which is performance whenever a lot of data has to be scanned, and we won't be able to do anything about that until we change the storage engine behind it. I'll merge this issue into that one, if you don't mind.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/AdGuardHome#2893

No description provided.

Rows
Columns