Memory only databases option (stats , queries etc) #4582

Open
opened 2026-03-04 05:17:05 -05:00 by deekerman · 10 comments
Owner

Originally created by @finBitorg on GitHub (Jul 10, 2023).

Prerequisites

  • I have checked the Wiki and Discussions and found no answer

  • I have searched other issues and found no duplicates

  • I want to request a feature or enhancement and not ask a question

The problem

Adguard is writing a ton of queries to disk and this is often causes wear on SDCard with Raspberry pi.

Proposed solution

please add a configuration setting to keep the query cache and query stats "in memory only" since you already have a max cache size option anyway this should limit memory use to whatever user wants. I am totally ok with loosing the stats on occasional reboot as opposed to having the Sdcards fail

Alternatives considered and additional information

trying to set overlay FS on raspberrypi but this will not allow for adguard updates and uses more memory then an option to just store query data and cache in memory with a specified size limit

Originally created by @finBitorg on GitHub (Jul 10, 2023). ### Prerequisites - [X] I have checked the [Wiki](https://github.com/AdguardTeam/AdGuardHome/wiki) and [Discussions](https://github.com/AdguardTeam/AdGuardHome/discussions) and found no answer - [X] I have searched other issues and found no duplicates - [X] I want to request a feature or enhancement and not ask a question ### The problem Adguard is writing a ton of queries to disk and this is often causes wear on SDCard with Raspberry pi. ### Proposed solution please add a configuration setting to keep the query cache and query stats "in memory only" since you already have a max cache size option anyway this should limit memory use to whatever user wants. I am totally ok with loosing the stats on occasional reboot as opposed to having the Sdcards fail ### Alternatives considered and additional information trying to set overlay FS on raspberrypi but this will not allow for adguard updates and uses more memory then an option to just store query data and cache in memory with a specified size limit
Author
Owner

@ainar-g commented on GitHub (Jul 10, 2023):

For query logs, there is already the querylog.file_enabled option. I think we can add something similar for stats as well.

@ainar-g commented on GitHub (Jul 10, 2023): For query logs, there is already the `querylog.file_enabled` option. I think we can add something similar for stats as well.
Author
Owner

@finBitorg commented on GitHub (Jul 10, 2023):

That would be great; also if possible that be exposed as a checkbox in Web UI setting section.

@finBitorg commented on GitHub (Jul 10, 2023): That would be great; also if possible that be exposed as a checkbox in Web UI setting section.
Author
Owner

@nunu6689 commented on GitHub (Jan 17, 2024):

Any update on this?
AGH killed my 2 sdcard. I give up on historical stats. A few days stats and queries while running is enough. I disabled stats for now.

@nunu6689 commented on GitHub (Jan 17, 2024): Any update on this? AGH killed my 2 sdcard. I give up on historical stats. A few days stats and queries while running is enough. I disabled stats for now.
Author
Owner

@schzhn commented on GitHub (Feb 15, 2024):

Since v0.108.0-b.53, you can specify tmpfs directory to store query log and statistics using the new querylog.dir_path and statistics.dir_path properties in the configuration file.

@schzhn commented on GitHub (Feb 15, 2024): Since [v0.108.0-b.53](https://github.com/AdguardTeam/AdGuardHome/releases/tag/v0.108.0-b.53), you can specify [tmpfs](https://en.wikipedia.org/wiki/Tmpfs) directory to store query log and statistics using the new `querylog.dir_path` and `statistics.dir_path` properties in the [configuration file](https://github.com/AdguardTeam/AdGuardHome/wiki/Configuration#configuration-file).
Author
Owner

@nunu6689 commented on GitHub (Feb 26, 2024):

Since v0.108.0-b.53, you can specify tmpfs directory to store query log and statistics using the new querylog.dir_path and statistics.dir_path properties in the configuration file.

That's good.
If I specify a directory path would AGH still store the stats in ram as well? That would double the ram usage then. For small ram like a router it does matter.
Since the query logs has in memory only option with query_logs.file_enabled why not make it for the stats also?

@nunu6689 commented on GitHub (Feb 26, 2024): > Since [v0.108.0-b.53](https://github.com/AdguardTeam/AdGuardHome/releases/tag/v0.108.0-b.53), you can specify [tmpfs](https://en.wikipedia.org/wiki/Tmpfs) directory to store query log and statistics using the new `querylog.dir_path` and `statistics.dir_path` properties in the [configuration file](https://github.com/AdguardTeam/AdGuardHome/wiki/Configuration#configuration-file). That's good. If I specify a directory path would AGH still store the stats in ram as well? That would double the ram usage then. For small ram like a router it does matter. Since the query logs has in memory only option with query_logs.file_enabled why not make it for the stats also?
Author
Owner

@ElTopo commented on GitHub (Mar 18, 2024):

Any update on this? AGH killed my 2 sdcard. I give up on historical stats. A few days stats and queries while running is enough. I disabled stats for now.

Although this feature request (i.e., writing constantly changes to tmpfs) helps your SD card a lot, you should consider using overlayroot of Raspberry Pi OS (sudo raspi-config / Performance Options / Overlay File System), after you configured your Pi properly and enabled overlayroot, all changes (were to your SD card) will be written in RAM (and discarded when you reboot the system), your SD card will be fully read-only and no bytes will be written to it, and your Pi will survive even when it loses power. It should work with other processes that write too much to your SD card.

@ElTopo commented on GitHub (Mar 18, 2024): > Any update on this? AGH killed my 2 sdcard. I give up on historical stats. A few days stats and queries while running is enough. I disabled stats for now. Although this feature request (i.e., writing constantly changes to tmpfs) helps your SD card a lot, you should consider using overlayroot of Raspberry Pi OS (sudo raspi-config / Performance Options / Overlay File System), after you configured your Pi properly and enabled overlayroot, all changes (were to your SD card) will be written in RAM (and discarded when you reboot the system), your SD card will be fully read-only and no bytes will be written to it, and your Pi will survive even when it loses power. It should work with other processes that write too much to your SD card.
Author
Owner

@ainar-g commented on GitHub (Mar 20, 2024):

@ElTopo, as @schzhn mentioned, v0.107.46 includes the ability to change statistics and querylog directories without changing the entire workdir, which should allow using e.g. tmpfs for that.

Any real in-memory statistics storage would have to wait either until v0.108.0 and #2290 or etcd-io/bbolt#227.

@ainar-g commented on GitHub (Mar 20, 2024): @ElTopo, as @schzhn mentioned, v0.107.46 includes the ability to change statistics and querylog directories without changing the entire workdir, which should allow using e.g. tmpfs for that. Any real in-memory statistics storage would have to wait either until v0.108.0 and #2290 or etcd-io/bbolt#227.
Author
Owner

@tipuraneo commented on GitHub (Mar 21, 2024):

I updated to v0.107.46 but could not find the option – neither in the gui nor in https://github.com/AdguardTeam/AdGuardHome/wiki/Configuration
Where is the option located?

@tipuraneo commented on GitHub (Mar 21, 2024): I updated to v0.107.46 but could not find the option – neither in the gui nor in https://github.com/AdguardTeam/AdGuardHome/wiki/Configuration Where is the option located?
Author
Owner

@jwgn commented on GitHub (Mar 21, 2024):

The option is dir_path and it is mentioned in the documentation.

For example, this is my current configuration:

querylog:
  dir_path: ""
  ignored: []
  interval: 168h
  size_memory: 1000
  enabled: true
  file_enabled: false
statistics:
  dir_path: /tmp
  ignored: []
  interval: 720h
  enabled: true

In my case, /tmp is tmpfs. You can also use /dev/shm if your /tmp is on disk.

@jwgn commented on GitHub (Mar 21, 2024): The option is `dir_path` and it is mentioned in the documentation. For example, this is my current configuration: ``` querylog: dir_path: "" ignored: [] interval: 168h size_memory: 1000 enabled: true file_enabled: false statistics: dir_path: /tmp ignored: [] interval: 720h enabled: true ``` In my case, `/tmp` is `tmpfs`. You can also use `/dev/shm` if your `/tmp` is on disk.
Author
Owner

@ElTopo commented on GitHub (Mar 22, 2024):

I found the easiest way to write log/db in memory is:

sudo systemctl stop AdGuardHome.service
cd AdGuardHome
rm -rf data
ln -sf /dev/shm data
sudo systemctl start AdGuardHome.service

/dev/shm is typically created using 50% of memory. you can create your own tmpfs in /etc/fstab if you want more, for example
tmpfs /mnt/mycache tmpfs nodev,nosuid,relatime,size=80%,mode=0777 0 0
then use /mnt/mycache as data directory.

@ElTopo commented on GitHub (Mar 22, 2024): I found the easiest way to write log/db in memory is: ``` sudo systemctl stop AdGuardHome.service cd AdGuardHome rm -rf data ln -sf /dev/shm data sudo systemctl start AdGuardHome.service ``` /dev/shm is typically created using 50% of memory. you can create your own tmpfs in /etc/fstab if you want more, for example `tmpfs /mnt/mycache tmpfs nodev,nosuid,relatime,size=80%,mode=0777 0 0` then use /mnt/mycache as data directory.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/AdGuardHome#4582
No description provided.