ETag comparison #5239

New issue

Open

opened 2026-03-04 05:57:51 -05:00 by deekerman · 0 comments

deekerman commented

2026-03-04 05:57:51 -05:00

Owner

Originally created by @GrumpyGiuseppe on GitHub (Jun 1, 2024).

Prerequisites

I have checked the Wiki and Discussions and found no answer
I have searched other issues and found no duplicates
I want to request a feature or enhancement and not ask a question

The problem

High bandwidth usage if big filter lists get updated frequently.

Proposed solution

Compare the ETag of the local filter file with the one on the server before downloading.

Alternatives considered and additional information

I update filters with curl now because it also uses way less RAM and IO.
Example (thx to ChatGPT):

#!/bin/bash

# URL of the file to download
URL="https://example.txt"

# Local file to save the downloaded content
LOCAL_FILE="/mnt/example.txt"

# File to store the ETag
ETAG_FILE="/mnt/etagexample.txt"

# Function to download the file
download_file() {
    echo "Downloading file..."
    curl -o "$LOCAL_FILE" -D - "$URL" | grep -i etag > "$ETAG_FILE"
}

# Check if the ETag file exists
if [ -f "$ETAG_FILE" ]; then
    # Get the saved ETag
    SAVED_ETAG=$(cat "$ETAG_FILE" | awk -F'"' '/etag/ {print $2}')

    # Make a HEAD request to get the current ETag from the server
    CURRENT_ETAG=$(curl -I "$URL" 2>/dev/null | grep -i etag | awk -F'"' '/etag/ {print $2}')

    # Compare the ETags
    if [ "$SAVED_ETAG" != "$CURRENT_ETAG" ]; then
        echo "ETag has changed. Downloading new file..."
        download_file
    else
        echo "ETag has not changed. No download needed."
    fi
else
    echo "No previous ETag found. Downloading file..."
    download_file
fi

Originally created by @GrumpyGiuseppe on GitHub (Jun 1, 2024). ### Prerequisites - [X] I have checked the [Wiki](https://github.com/AdguardTeam/AdGuardHome/wiki) and [Discussions](https://github.com/AdguardTeam/AdGuardHome/discussions) and found no answer - [X] I have searched other issues and found no duplicates - [X] I want to request a feature or enhancement and not ask a question ### The problem High bandwidth usage if big filter lists get updated frequently. ### Proposed solution Compare the ETag of the local filter file with the one on the server before downloading. ### Alternatives considered and additional information I update filters with curl now because it also uses way less RAM and IO. Example (thx to ChatGPT): ``` #!/bin/bash # URL of the file to download URL="https://example.txt" # Local file to save the downloaded content LOCAL_FILE="/mnt/example.txt" # File to store the ETag ETAG_FILE="/mnt/etagexample.txt" # Function to download the file download_file() { echo "Downloading file..." curl -o "$LOCAL_FILE" -D - "$URL" | grep -i etag > "$ETAG_FILE" } # Check if the ETag file exists if [ -f "$ETAG_FILE" ]; then # Get the saved ETag SAVED_ETAG=$(cat "$ETAG_FILE" | awk -F'"' '/etag/ {print $2}') # Make a HEAD request to get the current ETag from the server CURRENT_ETAG=$(curl -I "$URL" 2>/dev/null | grep -i etag | awk -F'"' '/etag/ {print $2}') # Compare the ETags if [ "$SAVED_ETAG" != "$CURRENT_ETAG" ]; then echo "ETag has changed. Downloading new file..." download_file else echo "ETag has not changed. No download needed." fi else echo "No previous ETag found. Downloading file..." download_file fi ```

deekerman added the

enhancement

performance

labels

2026-03-04 05:57:51 -05:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/AdGuardHome#5239

No description provided.

Rows
Columns