[Enhancement]: Previous daily logs should be gzip-ed or compressed to save space #2401

Open
opened 2026-02-20 02:17:05 -05:00 by deekerman · 3 comments
Owner

Originally created by @ZLoth on GitHub (Sep 2, 2024).

Type of Enhancement

Server Backend

Describe the Feature/Enhancement

In the directory /metadata/logs/daily, previous daily logs should be compressed to save space on server.

Why would this be helpful?

Space savings

Future Implementation (Screenshot)

Screenshot shows existing files as well as an example compressed text file.

image

Audiobookshelf Server Version

v2.12.3

Current Implementation (Screenshot)

See above screenshot.

Originally created by @ZLoth on GitHub (Sep 2, 2024). ### Type of Enhancement Server Backend ### Describe the Feature/Enhancement In the directory /metadata/logs/daily, previous daily logs should be compressed to save space on server. ### Why would this be helpful? Space savings ### Future Implementation (Screenshot) Screenshot shows existing files as well as an example compressed text file. ![image](https://github.com/user-attachments/assets/a6b77ea3-daf4-4f6c-a875-f00e0949f2f9) ### Audiobookshelf Server Version v2.12.3 ### Current Implementation (Screenshot) See above screenshot.
Author
Owner

@nichwall commented on GitHub (Nov 11, 2024):

How big is your log directory and what is the breakdown?

Are daily log files deleted after 7 days and most of the space is from scan logs?

https://github.com/advplyr/audiobookshelf/discussions/3602

@nichwall commented on GitHub (Nov 11, 2024): How big is your log directory and what is the breakdown? Are daily log files deleted after 7 days and most of the space is from scan logs? https://github.com/advplyr/audiobookshelf/discussions/3602
Author
Owner

@ZLoth commented on GitHub (Nov 11, 2024):

@nichwall : Does this help? It's less of a concern for me with terabytes of disk space, but may be a concern on smaller setups.

root@vaultron[/mnt/pool/app-config/audiobookshelf/logs]# ls -la daily
total 1601
drwxrwx--- 2 apps root      11 Nov 11 00:00 .
drwxrwx--- 4 apps root       5 Sep  2 12:11 ..
-rwxrwx--- 1 apps root  889976 Nov  3 23:46 2024-11-03.txt
-rwxrwx--- 1 apps root  567116 Nov  4 15:41 2024-11-04.txt
-rwxrwx--- 1 apps root  778560 Nov  5 20:24 2024-11-05.txt
-rwxrwx--- 1 apps root 1086197 Nov  6 23:47 2024-11-06.txt
-rwxrwx--- 1 apps root  593850 Nov  7 21:22 2024-11-07.txt
-rwxrwx--- 1 apps root  832502 Nov  8 22:05 2024-11-08.txt
-rwxrwx--- 1 apps root 1808385 Nov  9 23:00 2024-11-09.txt
-rwxrwx--- 1 apps root 1857764 Nov 10 23:00 2024-11-10.txt
-rwxrwx--- 1 apps root  515547 Nov 11 16:30 2024-11-11.txt
root@vaultron[/mnt/pool/app-config/audiobookshelf/logs]# ls -la scans
total 993
drwxrwx--- 2 apps root      43 Nov 11 03:00 .
drwxrwx--- 4 apps root       5 Sep  2 12:11 ..
-rwxrwx--- 1 apps root  259153 Aug 10 21:48 2024-08-10_157e6026-3bba-488a-927a-020e0521d890.txt
-rwxrwx--- 1 apps root  128150 Aug 23 06:54 2024-08-23_1f6a4a27-12b0-4680-b165-17451fdbe009.txt
-rwxrwx--- 1 apps root  671540 Sep  2 07:39 2024-09-02_1365c8d6-2c0d-4917-820f-a3cc60236e59.txt
-rwxrwx--- 1 apps root  183263 Sep 15 22:08 2024-09-15_0a9ce32f-7e17-4e96-9cd6-a91c692f3430.txt
-rwxrwx--- 1 apps root 1043898 Sep 15 20:35 2024-09-15_a756654e-f2d2-47ee-b1f2-127a75de38d6.txt
-rwxrwx--- 1 apps root    3477 Sep 28 03:00 2024-09-28_0a94c4f8-187e-48d8-81a9-5a5afd205946.txt
-rwxrwx--- 1 apps root    2232 Oct  8 03:00 2024-10-08_2cd63786-0278-46d7-b49f-a76dfccd7c98.txt
-rwxrwx--- 1 apps root    2233 Oct  9 03:00 2024-10-09_498097dc-b968-44ee-abb0-47d239392956.txt
-rwxrwx--- 1 apps root    2232 Oct 10 03:00 2024-10-10_50de0cf5-e622-4c86-af0f-49a218ed3941.txt
-rwxrwx--- 1 apps root    2232 Oct 11 03:00 2024-10-11_2ab3156a-8061-43b2-9704-9bcb683664cb.txt
-rwxrwx--- 1 apps root    2232 Oct 12 03:00 2024-10-12_49b29d4d-701c-4a27-9cf3-bdb819576c36.txt
-rwxrwx--- 1 apps root    2232 Oct 13 03:00 2024-10-13_2a2839fb-03bc-47e0-86f6-4c619eed27dd.txt
-rwxrwx--- 1 apps root    2232 Oct 14 03:00 2024-10-14_0186f870-7320-47ae-a2c1-05223e243d19.txt
-rwxrwx--- 1 apps root    2232 Oct 15 03:00 2024-10-15_0d80c5c8-6d32-40fd-81f6-591f6a87c517.txt
-rwxrwx--- 1 apps root    2232 Oct 16 03:00 2024-10-16_77fd96de-2773-45dc-90ce-292664045c38.txt
-rwxrwx--- 1 apps root    2232 Oct 17 03:00 2024-10-17_c753a87c-de79-4371-9be1-c43421acdbfe.txt
-rwxrwx--- 1 apps root    2232 Oct 18 03:00 2024-10-18_c4a18e3c-b1c8-46e0-a934-f31082ffe241.txt
-rwxrwx--- 1 apps root    2232 Oct 19 03:00 2024-10-19_937765e4-0582-45a4-8f6a-da60f4b0d69b.txt
-rwxrwx--- 1 apps root    2232 Oct 20 03:00 2024-10-20_b2f94a1f-092f-42e4-aa2d-a6c83eeae73c.txt
-rwxrwx--- 1 apps root    2232 Oct 21 03:00 2024-10-21_ce7f21d2-3242-4d62-ad8a-a791a3a8e123.txt
-rwxrwx--- 1 apps root    2232 Oct 22 03:00 2024-10-22_6e302213-b9d6-482f-a5aa-201fb674c6c2.txt
-rwxrwx--- 1 apps root    2232 Oct 23 03:00 2024-10-23_c018892e-938f-49a9-8f29-0133f1ecf6fd.txt
-rwxrwx--- 1 apps root    2232 Oct 24 03:00 2024-10-24_26750b2d-3237-4c59-9b78-c113c10f70ee.txt
-rwxrwx--- 1 apps root    2232 Oct 25 03:00 2024-10-25_d98e805d-9ebc-4a99-a5cd-1dbeedbf6034.txt
-rwxrwx--- 1 apps root    2232 Oct 26 03:00 2024-10-26_94dccf2a-c219-46f3-bafb-3af6aa1d468f.txt
-rwxrwx--- 1 apps root    2232 Oct 27 03:00 2024-10-27_01689b76-24f8-4d8d-8c4e-5f34d32a5185.txt
-rwxrwx--- 1 apps root    2232 Oct 28 03:00 2024-10-28_e912ddd7-2053-4023-8cb3-0c58c8851821.txt
-rwxrwx--- 1 apps root    2232 Oct 29 03:00 2024-10-29_448ba6be-2c23-497d-a63e-e6d8e96f1fca.txt
-rwxrwx--- 1 apps root    2232 Oct 30 03:00 2024-10-30_846b334a-21b5-4d9a-886e-2d7b1543442c.txt
-rwxrwx--- 1 apps root    2233 Oct 31 03:00 2024-10-31_b3d29a2a-9dd7-497b-b90e-9bde57f65060.txt
-rwxrwx--- 1 apps root    2232 Nov  1 03:00 2024-11-01_df4eb5ab-be47-44e5-9b8b-e875a6db8865.txt
-rwxrwx--- 1 apps root    2233 Nov  2 03:00 2024-11-02_ad897e56-2682-4596-8bf3-6339faeea691.txt
-rwxrwx--- 1 apps root    2232 Nov  3 03:00 2024-11-03_8ee5fb0b-fd97-4e24-9e36-8df52c5da9df.txt
-rwxrwx--- 1 apps root    2232 Nov  4 03:00 2024-11-04_49405cdc-b3aa-4d96-8bed-b74e34601599.txt
-rwxrwx--- 1 apps root    2232 Nov  5 03:00 2024-11-05_9c7dcc5e-2385-4ef7-9b60-d964163b410c.txt
-rwxrwx--- 1 apps root    2233 Nov  6 03:00 2024-11-06_1e131b66-52f4-4412-a1af-7d2a4e93bcd5.txt
-rwxrwx--- 1 apps root    2232 Nov  7 03:00 2024-11-07_b06994a1-608c-4b61-8526-75da7001c6dd.txt
-rwxrwx--- 1 apps root    2232 Nov  8 03:00 2024-11-08_856ceb07-54ab-4577-a8d6-2226db31183e.txt
-rwxrwx--- 1 apps root    2232 Nov  9 03:00 2024-11-09_9cd3e14d-e40c-4505-b662-6e35b8a74b42.txt
-rwxrwx--- 1 apps root    3962 Nov 10 03:00 2024-11-10_d85ea2e7-c90d-4bda-a387-ff8d23dd2992.txt
-rwxrwx--- 1 apps root    2504 Nov 11 03:00 2024-11-11_fc0af0fe-1910-440d-b739-d8970c963c30.txt
root@vaultron[/mnt/pool/app-config/audiobookshelf/logs]#
@ZLoth commented on GitHub (Nov 11, 2024): @nichwall : Does this help? It's less of a concern for me with terabytes of disk space, but may be a concern on smaller setups. ``` root@vaultron[/mnt/pool/app-config/audiobookshelf/logs]# ls -la daily total 1601 drwxrwx--- 2 apps root 11 Nov 11 00:00 . drwxrwx--- 4 apps root 5 Sep 2 12:11 .. -rwxrwx--- 1 apps root 889976 Nov 3 23:46 2024-11-03.txt -rwxrwx--- 1 apps root 567116 Nov 4 15:41 2024-11-04.txt -rwxrwx--- 1 apps root 778560 Nov 5 20:24 2024-11-05.txt -rwxrwx--- 1 apps root 1086197 Nov 6 23:47 2024-11-06.txt -rwxrwx--- 1 apps root 593850 Nov 7 21:22 2024-11-07.txt -rwxrwx--- 1 apps root 832502 Nov 8 22:05 2024-11-08.txt -rwxrwx--- 1 apps root 1808385 Nov 9 23:00 2024-11-09.txt -rwxrwx--- 1 apps root 1857764 Nov 10 23:00 2024-11-10.txt -rwxrwx--- 1 apps root 515547 Nov 11 16:30 2024-11-11.txt root@vaultron[/mnt/pool/app-config/audiobookshelf/logs]# ls -la scans total 993 drwxrwx--- 2 apps root 43 Nov 11 03:00 . drwxrwx--- 4 apps root 5 Sep 2 12:11 .. -rwxrwx--- 1 apps root 259153 Aug 10 21:48 2024-08-10_157e6026-3bba-488a-927a-020e0521d890.txt -rwxrwx--- 1 apps root 128150 Aug 23 06:54 2024-08-23_1f6a4a27-12b0-4680-b165-17451fdbe009.txt -rwxrwx--- 1 apps root 671540 Sep 2 07:39 2024-09-02_1365c8d6-2c0d-4917-820f-a3cc60236e59.txt -rwxrwx--- 1 apps root 183263 Sep 15 22:08 2024-09-15_0a9ce32f-7e17-4e96-9cd6-a91c692f3430.txt -rwxrwx--- 1 apps root 1043898 Sep 15 20:35 2024-09-15_a756654e-f2d2-47ee-b1f2-127a75de38d6.txt -rwxrwx--- 1 apps root 3477 Sep 28 03:00 2024-09-28_0a94c4f8-187e-48d8-81a9-5a5afd205946.txt -rwxrwx--- 1 apps root 2232 Oct 8 03:00 2024-10-08_2cd63786-0278-46d7-b49f-a76dfccd7c98.txt -rwxrwx--- 1 apps root 2233 Oct 9 03:00 2024-10-09_498097dc-b968-44ee-abb0-47d239392956.txt -rwxrwx--- 1 apps root 2232 Oct 10 03:00 2024-10-10_50de0cf5-e622-4c86-af0f-49a218ed3941.txt -rwxrwx--- 1 apps root 2232 Oct 11 03:00 2024-10-11_2ab3156a-8061-43b2-9704-9bcb683664cb.txt -rwxrwx--- 1 apps root 2232 Oct 12 03:00 2024-10-12_49b29d4d-701c-4a27-9cf3-bdb819576c36.txt -rwxrwx--- 1 apps root 2232 Oct 13 03:00 2024-10-13_2a2839fb-03bc-47e0-86f6-4c619eed27dd.txt -rwxrwx--- 1 apps root 2232 Oct 14 03:00 2024-10-14_0186f870-7320-47ae-a2c1-05223e243d19.txt -rwxrwx--- 1 apps root 2232 Oct 15 03:00 2024-10-15_0d80c5c8-6d32-40fd-81f6-591f6a87c517.txt -rwxrwx--- 1 apps root 2232 Oct 16 03:00 2024-10-16_77fd96de-2773-45dc-90ce-292664045c38.txt -rwxrwx--- 1 apps root 2232 Oct 17 03:00 2024-10-17_c753a87c-de79-4371-9be1-c43421acdbfe.txt -rwxrwx--- 1 apps root 2232 Oct 18 03:00 2024-10-18_c4a18e3c-b1c8-46e0-a934-f31082ffe241.txt -rwxrwx--- 1 apps root 2232 Oct 19 03:00 2024-10-19_937765e4-0582-45a4-8f6a-da60f4b0d69b.txt -rwxrwx--- 1 apps root 2232 Oct 20 03:00 2024-10-20_b2f94a1f-092f-42e4-aa2d-a6c83eeae73c.txt -rwxrwx--- 1 apps root 2232 Oct 21 03:00 2024-10-21_ce7f21d2-3242-4d62-ad8a-a791a3a8e123.txt -rwxrwx--- 1 apps root 2232 Oct 22 03:00 2024-10-22_6e302213-b9d6-482f-a5aa-201fb674c6c2.txt -rwxrwx--- 1 apps root 2232 Oct 23 03:00 2024-10-23_c018892e-938f-49a9-8f29-0133f1ecf6fd.txt -rwxrwx--- 1 apps root 2232 Oct 24 03:00 2024-10-24_26750b2d-3237-4c59-9b78-c113c10f70ee.txt -rwxrwx--- 1 apps root 2232 Oct 25 03:00 2024-10-25_d98e805d-9ebc-4a99-a5cd-1dbeedbf6034.txt -rwxrwx--- 1 apps root 2232 Oct 26 03:00 2024-10-26_94dccf2a-c219-46f3-bafb-3af6aa1d468f.txt -rwxrwx--- 1 apps root 2232 Oct 27 03:00 2024-10-27_01689b76-24f8-4d8d-8c4e-5f34d32a5185.txt -rwxrwx--- 1 apps root 2232 Oct 28 03:00 2024-10-28_e912ddd7-2053-4023-8cb3-0c58c8851821.txt -rwxrwx--- 1 apps root 2232 Oct 29 03:00 2024-10-29_448ba6be-2c23-497d-a63e-e6d8e96f1fca.txt -rwxrwx--- 1 apps root 2232 Oct 30 03:00 2024-10-30_846b334a-21b5-4d9a-886e-2d7b1543442c.txt -rwxrwx--- 1 apps root 2233 Oct 31 03:00 2024-10-31_b3d29a2a-9dd7-497b-b90e-9bde57f65060.txt -rwxrwx--- 1 apps root 2232 Nov 1 03:00 2024-11-01_df4eb5ab-be47-44e5-9b8b-e875a6db8865.txt -rwxrwx--- 1 apps root 2233 Nov 2 03:00 2024-11-02_ad897e56-2682-4596-8bf3-6339faeea691.txt -rwxrwx--- 1 apps root 2232 Nov 3 03:00 2024-11-03_8ee5fb0b-fd97-4e24-9e36-8df52c5da9df.txt -rwxrwx--- 1 apps root 2232 Nov 4 03:00 2024-11-04_49405cdc-b3aa-4d96-8bed-b74e34601599.txt -rwxrwx--- 1 apps root 2232 Nov 5 03:00 2024-11-05_9c7dcc5e-2385-4ef7-9b60-d964163b410c.txt -rwxrwx--- 1 apps root 2233 Nov 6 03:00 2024-11-06_1e131b66-52f4-4412-a1af-7d2a4e93bcd5.txt -rwxrwx--- 1 apps root 2232 Nov 7 03:00 2024-11-07_b06994a1-608c-4b61-8526-75da7001c6dd.txt -rwxrwx--- 1 apps root 2232 Nov 8 03:00 2024-11-08_856ceb07-54ab-4577-a8d6-2226db31183e.txt -rwxrwx--- 1 apps root 2232 Nov 9 03:00 2024-11-09_9cd3e14d-e40c-4505-b662-6e35b8a74b42.txt -rwxrwx--- 1 apps root 3962 Nov 10 03:00 2024-11-10_d85ea2e7-c90d-4bda-a387-ff8d23dd2992.txt -rwxrwx--- 1 apps root 2504 Nov 11 03:00 2024-11-11_fc0af0fe-1910-440d-b739-d8970c963c30.txt root@vaultron[/mnt/pool/app-config/audiobookshelf/logs]# ```
Author
Owner

@nichwall commented on GitHub (Nov 13, 2024):

Thanks, yes that is useful. Looks like the daily logs are being removed automatically (as expected). In the discussion, it was mentioned we could delete old scan logs.

Since those are smaller than daily logs, maybe we can keep the last 21 days of scan logs? I think that's enough time to catch issues but don't take up too much space (about 2kB each during what appears to be daily scans for your server).

@nichwall commented on GitHub (Nov 13, 2024): Thanks, yes that is useful. Looks like the daily logs are being removed automatically (as expected). In the discussion, it was mentioned we could delete old scan logs. Since those are smaller than daily logs, maybe we can keep the last 21 days of scan logs? I think that's enough time to catch issues but don't take up too much space (about 2kB each during what appears to be daily scans for your server).
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/audiobookshelf#2401
No description provided.