Duplicate files, remainder is 0, but files are still being uploaded #2789

Closed
opened 2026-02-20 01:17:29 -05:00 by deekerman · 3 comments
Owner

Originally created by @navennn on GitHub (May 15, 2024).

The bug

Hi
I'm having a difficult time understanding the 'duplicates', and how backup upload works.
I've some 8200 photos and videos on my android phone that I want to backup up with Immich.
Initially, "Total" (assets in backup tab), said 8190 (approx. around this much).

Today, finally I've reached 0 on the remainder (quite poor connection to server, usually 100-2000kbps, upload took like a week).
But files are still being uploaded. I assume these are the 'duplicates'?
AFAIK, I do not have any duplicates, probably not even one.

I've read https://github.com/immich-app/immich/issues/9365 and https://github.com/immich-app/immich/issues/7738 which did not actually explain anything. I'm quite a bit confused about this.
8197−7224 = 973, not 1063 (count of duplicates) - see screenshots at the bottom of the post

Could I view the duplicates somehow?
Are duplicates being uploaded as well?

This was this morning (the first time the counter hit 0 remaining)
Screenshot_2024-05-15-14-27-16-26_d8aea95ef633eb6b7eb1cae13c34b40e

This is now
Screenshot_2024-05-15-16-36-25-48_d8aea95ef633eb6b7eb1cae13c34b40e

Duplicates in settings
Screenshot_2024-05-15-16-36-18-63_d8aea95ef633eb6b7eb1cae13c34b40e

Thank you!

The OS that Immich Server is running on

Debian 11.9

Version of Immich Server

v1.105.1

Version of Immich Mobile App

v1.105.0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:84882e87b54734154586e5f8abd4dce69fe7311315e2fc6d67c29614c8de2672
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/media/immich/
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=xxxx

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. Do fresh install
2. Wait for all files to upload, wait for it to finish
3. Open backup tab
4. Open settings  -> advanced
...

Relevant log output

No response

Additional information

Phone is OnePlus 9, Android 14.
App installed from google play store.

Originally created by @navennn on GitHub (May 15, 2024). ### The bug Hi I'm having a difficult time understanding the 'duplicates', and how backup upload works. I've some 8200 photos and videos on my android phone that I want to backup up with Immich. Initially, "Total" (assets in backup tab), said 8190 (approx. around this much). Today, finally I've reached 0 on the remainder (quite poor connection to server, usually 100-2000kbps, upload took like a week). But files are still being uploaded. I assume these are the 'duplicates'? AFAIK, I do not have any duplicates, probably not even one. I've read https://github.com/immich-app/immich/issues/9365 and https://github.com/immich-app/immich/issues/7738 which did not actually explain anything. I'm quite a bit confused about this. 8197−7224 = 973, not 1063 (count of duplicates) - see screenshots at the bottom of the post Could I view the duplicates somehow? Are duplicates being uploaded as well? This was this morning (the first time the counter hit 0 remaining) ![Screenshot_2024-05-15-14-27-16-26_d8aea95ef633eb6b7eb1cae13c34b40e](https://github.com/immich-app/immich/assets/26810830/5621481d-c374-484a-85f0-93427255ce6a) This is now ![Screenshot_2024-05-15-16-36-25-48_d8aea95ef633eb6b7eb1cae13c34b40e](https://github.com/immich-app/immich/assets/26810830/f8acfda0-19c2-4386-baa2-1af7b0a2a9a4) Duplicates in settings ![Screenshot_2024-05-15-16-36-18-63_d8aea95ef633eb6b7eb1cae13c34b40e](https://github.com/immich-app/immich/assets/26810830/3f46d58b-f824-4818-96e0-3a1fda587d17) Thank you! ### The OS that Immich Server is running on Debian 11.9 ### Version of Immich Server v1.105.1 ### Version of Immich Mobile App v1.105.0 ### Platform with the issue - [X] Server - [ ] Web - [X] Mobile ### Your docker-compose.yml content ```YAML # # WARNING: Make sure to use the docker-compose.yml of the current release: # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. # name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: ['start.sh', 'immich'] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding # file: hwaccel.transcoding.yml # service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding command: ['start.sh', 'microservices'] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration # file: hwaccel.ml.yml # service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:84882e87b54734154586e5f8abd4dce69fe7311315e2fc6d67c29614c8de2672 restart: always database: container_name: immich_postgres image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - ${DB_DATA_LOCATION}:/var/lib/postgresql/data restart: always volumes: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/media/immich/ # The location where your database files are stored DB_DATA_LOCATION=./postgres # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secret for postgres. You should change it to a random password DB_PASSWORD=xxxx # The values below this line do not need to be changed ################################################################################### DB_USERNAME=postgres DB_DATABASE_NAME=immich ``` ### Reproduction steps ```bash 1. Do fresh install 2. Wait for all files to upload, wait for it to finish 3. Open backup tab 4. Open settings -> advanced ... ``` ### Relevant log output _No response_ ### Additional information Phone is OnePlus 9, Android 14. App installed from google play store.
Author
Owner

@danieldietzler commented on GitHub (May 20, 2024):

This seems to be the typical backup state issue the app currently has

@danieldietzler commented on GitHub (May 20, 2024): This seems to be the typical backup state issue the app currently has
Author
Owner

@frejaya commented on GitHub (May 30, 2024):

This seems to be the typical backup state issue the app currently has

@danieldietzler

Is there another issue number where we can track the resolution?

@frejaya commented on GitHub (May 30, 2024): > This seems to be the typical backup state issue the app currently has @danieldietzler Is there another issue number where we can track the resolution?
Author
Owner

@danieldietzler commented on GitHub (May 31, 2024):

@frejaya #2567

@danieldietzler commented on GitHub (May 31, 2024): @frejaya #2567
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/immich#2789
No description provided.