deekerman commented

Owner

Originally created by @malanxi2022 on GitHub (Nov 21, 2023).

The bug

Import some photo with a duplicate prompt, but cannot be found in the library folder and assets table
How do I determine whether it is a duplicate?
Why are some photos in the local table, but not in the library? I tried to import the photos in the local table again, but it also prompted a duplicate.

The OS that Immich Server is running on

docker desktop

Version of Immich Server

v1.88.1

Version of Immich Mobile App

v1.88.1

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

no changes

Your .env content

no changes

Reproduction steps

1. Import the photo 
2. duplicate

Additional information

No response

Originally created by @malanxi2022 on GitHub (Nov 21, 2023). ### The bug Import some photo with a duplicate prompt, but cannot be found in the library folder and assets table How do I determine whether it is a duplicate? Why are some photos in the local table, but not in the library? I tried to import the photos in the local table again, but it also prompted a duplicate. ### The OS that Immich Server is running on docker desktop ### Version of Immich Server v1.88.1 ### Version of Immich Mobile App v1.88.1 ### Platform with the issue - [X] Server - [X] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML no changes ``` ### Your .env content ```Shell no changes ``` ### Reproduction steps ```bash 1. Import the photo 2. duplicate ``` ### Additional information _No response_

deekerman closed this issue

2026-02-20 00:13:18 -05:00

deekerman commented

2026-02-20 00:13:18 -05:00

Author

Owner

@alextran1502 commented on GitHub (Nov 21, 2023):

how are you importing the photos?

@alextran1502 commented on GitHub (Nov 21, 2023): how are you importing the photos?

deekerman commented

2026-02-20 00:13:19 -05:00

Author

Owner

@malanxi2022 commented on GitHub (Nov 22, 2023):

how are you importing the photos?你是如何导入照片的？

Upload photos through the UploadButton on the web page

@malanxi2022 commented on GitHub (Nov 22, 2023): > how are you importing the photos?你是如何导入照片的？ Upload photos through the UploadButton on the web page

deekerman commented

2026-02-20 00:13:19 -05:00

Author

Owner

@tedstriker commented on GitHub (Dec 13, 2023):

got the same behaviour with version 1.90.2

It might be, that this photo has been in the library previously. If this was the case, it has been deleted before I tried to upload it again.
The way I used to upload it, was by dropping it on an opened album.

Is there a way to search for orphan database entries?

@tedstriker commented on GitHub (Dec 13, 2023): got the same behaviour with version 1.90.2 It _might_ be, that this photo has been in the library previously. If this was the case, it has been deleted before I tried to upload it again. The way I used to upload it, was by dropping it on an opened album. Is there a way to search for orphan database entries?

deekerman commented

2026-02-20 00:13:19 -05:00

Author

Owner

@mgreystone commented on GitHub (Dec 20, 2023):

I've got the same issue, i think. I uploaded a small number of photos through the web app. They are no where to be found. When i re-upload them, they are marked as duplicates. When i upload them to an album, it says they have been added to the album, but i cannot find them there either.

Curiously, though... one of the photos does appear as the album's thumbnail. So they are in the db somewhere. Anyway to force-upload and ignore the duplicate?

@mgreystone commented on GitHub (Dec 20, 2023): I've got the same issue, i think. I uploaded a small number of photos through the web app. They are no where to be found. When i re-upload them, they are marked as duplicates. When i upload them to an album, it says they have been added to the album, but i cannot find them there either. Curiously, though... one of the photos _does_ appear as the album's thumbnail. So they are in the db somewhere. Anyway to force-upload and ignore the duplicate?

deekerman commented

2026-02-20 00:13:20 -05:00

Author

Owner

@tedstriker commented on GitHub (Feb 20, 2024):

In Version 1.94.1 this error is still existing.
I removed a broken tile from the photo library, because it pointed to a folder that did not exist.
After that, I tried to re-upload it via the website and it gets denied with of a duplicate error message.
I found, there's a "repair" section in the administration section of immich now.
Also the problematic image is listed there, but unfortunately the documentation doesn't mention this new section yet and it's not intuitive to use.
But I think we can have hope...

@tedstriker commented on GitHub (Feb 20, 2024): In Version 1.94.1 this error is still existing. I removed a broken tile from the photo library, because it pointed to a folder that did not exist. After that, I tried to re-upload it via the website and it gets denied with of a duplicate error message. I found, there's a "repair" section in the administration section of immich now. Also the problematic image is listed there, but unfortunately the [documentation](https://immich.app/docs/administration/user-management) doesn't mention this new section yet and it's not intuitive to use. But I think we can have hope...

deekerman commented

2026-02-20 00:13:20 -05:00

Author

Owner

@stephen304 commented on GitHub (Feb 20, 2024):

I'm also experiencing some "duplicates" when uploading through the web uploader, but I dragged and dropped from a single folder with ~2000 uniquely named files and for me it's a first time import of these files.

It's made harder to diagnose since the duplicate entries in the upload list go away as soon as it's detected as a duplicate, and there's no way to review which files were detected as duplicates unless you were screen recording the upload or remembering the filename as soon as you see the duplicate counter increment.

It would be great if after a batch upload finishes with errors or duplicates, if there was a persistent notification with a button to open a modal with a scrollable list of all the filenames that had issues. Then I would be able to inspect the exact files, check to see if they already exist somehow, or retry just those files.

@stephen304 commented on GitHub (Feb 20, 2024): I'm also experiencing some "duplicates" when uploading through the web uploader, but I dragged and dropped from a single folder with ~2000 uniquely named files and for me it's a first time import of these files. It's made harder to diagnose since the duplicate entries in the upload list go away as soon as it's detected as a duplicate, and there's no way to review which files were detected as duplicates unless you were screen recording the upload or remembering the filename as soon as you see the duplicate counter increment. It would be great if after a batch upload finishes with errors or duplicates, if there was a persistent notification with a button to open a modal with a scrollable list of all the filenames that had issues. Then I would be able to inspect the exact files, check to see if they already exist somehow, or retry just those files.

deekerman commented

2026-02-20 00:13:20 -05:00

Author

Owner

@jee-r commented on GitHub (Feb 28, 2024):

Is there any update on this ?
I'm also stuck on this issue, have you found any workaround other than starting from scratch ?

@jee-r commented on GitHub (Feb 28, 2024): Is there any update on this ? I'm also stuck on this issue, have you found any workaround other than starting from scratch ?

deekerman commented

2026-02-20 00:13:20 -05:00

Author

Owner

@MrColumbo commented on GitHub (Apr 6, 2024):

I am not sure if it is same on my side. It started actually with a problem with the reverse proxy where i got trouble to backup a large - 4.5GB video.
I might have fixed the reverse proxy issue but when I try to upload the video via the browser I get a skipped dupliacate message.

On postgres i see this

06.04.2024
13:55:44
2024-04-06 11:55:44.346 UTC [2167] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
06.04.2024
13:55:44
2024-04-06 11:55:44.346 UTC [2167] DETAIL:  Key ("ownerId", "libraryId", checksum)=(0ebe999a-0c1c-4331-92bd-4d6690b62629, 52db6a1c-261a-4eed-81c1-1af5c4bace01, \x8ead89ed7867fb940b99ee55816e26e69f7731cd) already exists.
06.04.2024
13:55:44
2024-04-06 11:55:44.346 UTC [2167] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, $16, DEFAULT, $17, $18, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "isVisible"

Since I know the filename i did a search but i am not able to find the asset / video.

Oh - i should mention that i started to try uploading via the webintrface since the client does not even show that video anymore.

Basically when I look in the library and look into the local folder the app pretends that this particular video doet not exist anymore and there is nothing left to backup.

The repair button does not show anything ....

i am on 101 - mobile app and server

after a while i got this message on the fix page

504 Gateway Time-out

openresty </html> - 504 he@https://mydomain/_app/immutable/chunks/fetch-client.CUHQ0TcQ.js:1:2911 de@https://mydomain/_app/immutable/chunks/fetch-client.CUHQ0TcQ.js:1:2835

@MrColumbo commented on GitHub (Apr 6, 2024): I am not sure if it is same on my side. It started actually with a problem with the reverse proxy where i got trouble to backup a large - 4.5GB video. I might have fixed the reverse proxy issue but when I try to upload the video via the browser I get a skipped dupliacate message. On postgres i see this 06.04.2024 13:55:44 2024-04-06 11:55:44.346 UTC [2167] ERROR: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum" 06.04.2024 13:55:44 2024-04-06 11:55:44.346 UTC [2167] DETAIL: Key ("ownerId", "libraryId", checksum)=(0ebe999a-0c1c-4331-92bd-4d6690b62629, 52db6a1c-261a-4eed-81c1-1af5c4bace01, \x8ead89ed7867fb940b99ee55816e26e69f7731cd) already exists. 06.04.2024 13:55:44 2024-04-06 11:55:44.346 UTC [2167] STATEMENT: INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, $16, DEFAULT, $17, $18, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "isVisible" Since I know the filename i did a search but i am not able to find the asset / video. Oh - i should mention that i started to try uploading via the webintrface since the client does not even show that video anymore. Basically when I look in the library and look into the local folder the app pretends that this particular video doet not exist anymore and there is nothing left to backup. The repair button does not show anything .... i am on 101 - mobile app and server --- after a while i got this message on the fix page <html> <head><title>504 Gateway Time-out</title></head> <body> <center><h1>504 Gateway Time-out</h1></center> <hr><center>openresty</center> </body> </html> - 504 he@https://mydomain/_app/immutable/chunks/fetch-client.CUHQ0TcQ.js:1:2911 de@https://mydomain/_app/immutable/chunks/fetch-client.CUHQ0TcQ.js:1:2835

deekerman commented

2026-02-20 00:13:21 -05:00

Author

Owner

@alextran1502 commented on GitHub (Apr 6, 2024):

Please try double check in the trash/archive

@alextran1502 commented on GitHub (Apr 6, 2024): Please try double check in the trash/archive

deekerman commented

2026-02-20 00:13:21 -05:00

Author

Owner

@MrColumbo commented on GitHub (Apr 6, 2024):

nothing in neither archive nor deleted ...

@MrColumbo commented on GitHub (Apr 6, 2024): nothing in neither archive nor deleted ...

deekerman commented

2026-02-20 00:13:21 -05:00

Author

Owner

@MrColumbo commented on GitHub (Apr 6, 2024):

i just found the file in the library ....so the file actually exists and one or the other way was successfully uploaded

@MrColumbo commented on GitHub (Apr 6, 2024): i just found the file in the library ....so the file actually exists and one or the other way was successfully uploaded

deekerman commented

2026-02-20 00:13:21 -05:00

Author

Owner

@MrColumbo commented on GitHub (Apr 7, 2024):

i fooled myself - the video is shown but with a duration of 0 seconds. All the other videos are showing with the right duration and the one i missed is about 35 minutes but since it showed 0 seconds i thought it is not the one i am looking for

@MrColumbo commented on GitHub (Apr 7, 2024): i fooled myself - the video is shown but with a duration of 0 seconds. All the other videos are showing with the right duration and the one i missed is about 35 minutes but since it showed 0 seconds i thought it is not the one i am looking for

deekerman commented

2026-02-20 00:13:22 -05:00

Author

Owner

@ap-andersson commented on GitHub (Jul 2, 2024):

Please try double check in the trash/archive

This was my problem. I had "deleted" the images. When I created a new album and tried to re-upload them in that album they already existed (in trash) and was not visible anywhere. I emptied trash and tried again and it worked.

@ap-andersson commented on GitHub (Jul 2, 2024): > Please try double check in the trash/archive This was my problem. I had "deleted" the images. When I created a new album and tried to re-upload them in that album they already existed (in trash) and was not visible anywhere. I emptied trash and tried again and it worked.

deekerman commented

2026-02-20 00:13:22 -05:00

Author

Owner

@hakong commented on GitHub (Mar 29, 2025):

I'm having this issue. Trying to upload - but it's refusing, saying it's a duplicate.

Trash is empty
Photos is not in the archive
Searching for the filename: 0 results.
Searching for the date the picture is taken: 0 results

[immich@container-2 ~]$ podman exec -it systemd-immich-server immich upload /Takeout/missing-metadata-originals/PXL_20231213_180400860.jpg
Crawling for assets...
Hashing files           | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets
Checking for duplicates | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets
Found 0 new files and 1 duplicate
All assets were already uploaded, nothing to do.

[immich@container-2 ~]$ podman exec -it systemd-immich-server immich upload --skip-hash /Takeout/missing-metadata-originals/PXL_20231213_180400860.jpg
Crawling for assets...
Skipping hash check, assuming all files are new
Uploading assets | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 0% | ETA: NFs | 753.7 kB/753.7 kB
Successfully uploaded 0 new assets (0 B)
Skipped 1 duplicate asset (753.7 kB)
[immich@container-2 ~]$

[immich@container-2 ~]$ curl -s -L -X POST https://im.domain.com/api/search/metadata -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'x-api-key: xxxxxx' -d '{"originalFileName": "PXL_20231213_180400860.jpg"}'
{"albums":{"total":0,"count":0,"items":[],"facets":[]},"assets":{"total":0,"count":0,"items":[],"facets":[],"nextPage":null}}

[immich@container-2 ~]$ exif /net/172.21.21.1/datapool/immich/Takeout/missing-metadata-originals/PXL_20231213_180400860.jpg | grep Date
Date and Time       |2023:12:13 18:04:00
Date and Time (Origi|2023:12:13 18:04:00
Date and Time (Digit|2023:12:13 18:04:00
GPS Date            |2023:12:13

@hakong commented on GitHub (Mar 29, 2025): I'm having this issue. Trying to upload - but it's refusing, saying it's a duplicate. - Trash is empty - Photos is not in the archive - Searching for the filename: 0 results. - Searching for the date the picture is taken: 0 results ``` [immich@container-2 ~]$ podman exec -it systemd-immich-server immich upload /Takeout/missing-metadata-originals/PXL_20231213_180400860.jpg Crawling for assets... Hashing files | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets Checking for duplicates | ████████████████████████████████████████ | 100% | ETA: 0s | 1/1 assets Found 0 new files and 1 duplicate All assets were already uploaded, nothing to do. [immich@container-2 ~]$ podman exec -it systemd-immich-server immich upload --skip-hash /Takeout/missing-metadata-originals/PXL_20231213_180400860.jpg Crawling for assets... Skipping hash check, assuming all files are new Uploading assets | ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 0% | ETA: NFs | 753.7 kB/753.7 kB Successfully uploaded 0 new assets (0 B) Skipped 1 duplicate asset (753.7 kB) [immich@container-2 ~]$ ``` ``` [immich@container-2 ~]$ curl -s -L -X POST https://im.domain.com/api/search/metadata -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'x-api-key: xxxxxx' -d '{"originalFileName": "PXL_20231213_180400860.jpg"}' {"albums":{"total":0,"count":0,"items":[],"facets":[]},"assets":{"total":0,"count":0,"items":[],"facets":[],"nextPage":null}} ``` ``` [immich@container-2 ~]$ exif /net/172.21.21.1/datapool/immich/Takeout/missing-metadata-originals/PXL_20231213_180400860.jpg | grep Date Date and Time |2023:12:13 18:04:00 Date and Time (Origi|2023:12:13 18:04:00 Date and Time (Digit|2023:12:13 18:04:00 GPS Date |2023:12:13 ``` ![Image](https://github.com/user-attachments/assets/bc20e97e-99cb-4fce-8c68-d139fe259985) ![Image](https://github.com/user-attachments/assets/077c1f1c-e52a-49f5-89e4-cfbfb1a1569d) ![Image](https://github.com/user-attachments/assets/b07fe1ab-ce5e-4c36-9d28-8adc466ead13)

deekerman commented

2026-02-20 00:13:22 -05:00

Author

Owner

@hakong commented on GitHub (Mar 29, 2025):

Should I open a new bug report? Or should this one be reopened?

@hakong commented on GitHub (Mar 29, 2025): Should I open a new bug report? Or should this one be reopened?

deekerman referenced this issue

2026-02-20 00:20:34 -05:00

Video Larger than 22GB fails to upload #1926

deekerman referenced this issue

2026-02-20 01:17:16 -05:00