Read-only external library metadata edits silently fail #3241

Author

Owner

@kvalev commented on GitHub (Sep 1, 2024):

What seems to be a bit weird is that when you add new tags, those will be saved, even tho xmp writing fails. However changing the date or the location straight up doesnt work. It will be nice if the behavior is consistent.

@kvalev commented on GitHub (Sep 1, 2024): What seems to be a bit weird is that when you add new tags, those will be saved, even tho xmp writing fails. However changing the date or the location straight up doesnt work. It will be nice if the behavior is consistent.

deekerman commented

Author

Owner

@pyorot commented on GitHub (Sep 1, 2024):

What seems to be a bit weird is that when you add new tags, those will be saved, even tho xmp writing fails. However changing the date or the location straight up doesnt work. It will be nice if the behavior is consistent.

Does this mean the tags go in the database? 🤔

@pyorot commented on GitHub (Sep 1, 2024): > What seems to be a bit weird is that when you add new tags, those will be saved, even tho xmp writing fails. However changing the date or the location straight up doesnt work. It will be nice if the behavior is consistent. Does this mean the tags go in the database? 🤔

deekerman commented

Author

Owner

@bo0tzz commented on GitHub (Sep 2, 2024):

The reason that happens is that extracting date or location from the file metadata overwrites what is in the database because an asset can only have one date or location, which doesn't apply to tags.

@bo0tzz commented on GitHub (Sep 2, 2024): The reason that happens is that extracting date or location from the file metadata overwrites what is in the database because an asset can only have one date or location, which doesn't apply to tags.

deekerman commented

Author

Owner

@jrasm91 commented on GitHub (Sep 2, 2024):

It should apply to tags as well. Tags are written to xmp and replaced on a subsequent metadata extraction.

@jrasm91 commented on GitHub (Sep 2, 2024): It should apply to tags as well. Tags are written to xmp and replaced on a subsequent metadata extraction.

deekerman commented

Author

Owner

@pypeaday commented on GitHub (Sep 19, 2024):

FWIW I am having this issue with tags on my RO external library

[Nest] 7  - 09/19/2024, 10:19:28 AM   DEBUG [Microservices:LibraryService] Asset already exists in database and on disk, will not import: /mnt/nextcloud/Twitter/rwq17d.jpg

[Nest] 7  - 09/19/2024, 10:19:57 AM    WARN [Microservices:MetadataRepository] Error writing exif data (/mnt/nextcloud/Twitter/20240917_194444.jpg.xmp): Error: Error creating file: /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp - /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp

[Nest] 7  - 09/19/2024, 10:19:57 AM    WARN [Microservices:Error: File not found - /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp

    at ReadTask._ExifToolTask_parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:76:71)

    at ReadTask.parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:47:167)

    at ReadTask._Task_resolve (/usr/src/app/node_modules/batch-cluster/dist/Task.js:146:40)] Error reading exif data (/mnt/nextcloud/Twitter/20240917_194444.jpg.xmp): Error: File not found - /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp

[Nest] 7  - 09/19/2024, 10:19:57 AM   DEBUG [Microservices:MetadataService] Asset ea468aeb-6745-4ff5-ba2c-bb87973658e0 date time is undefined

[Nest] 7  - 09/19/2024, 10:19:57 AM    WARN [Microservices:MetadataService] Asset ea468aeb-6745-4ff5-ba2c-bb87973658e0 has no valid date (undefined), falling back to asset.fileCreatedAt

@pypeaday commented on GitHub (Sep 19, 2024): FWIW I am having this issue with tags on my RO external library ``` [Nest] 7 - 09/19/2024, 10:19:28 AM DEBUG [Microservices:LibraryService] Asset already exists in database and on disk, will not import: /mnt/nextcloud/Twitter/rwq17d.jpg [Nest] 7 - 09/19/2024, 10:19:57 AM WARN [Microservices:MetadataRepository] Error writing exif data (/mnt/nextcloud/Twitter/20240917_194444.jpg.xmp): Error: Error creating file: /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp - /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp [Nest] 7 - 09/19/2024, 10:19:57 AM WARN [Microservices:Error: File not found - /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp at ReadTask._ExifToolTask_parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:76:71) at ReadTask.parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:47:167) at ReadTask._Task_resolve (/usr/src/app/node_modules/batch-cluster/dist/Task.js:146:40)] Error reading exif data (/mnt/nextcloud/Twitter/20240917_194444.jpg.xmp): Error: File not found - /mnt/nextcloud/Twitter/20240917_194444.jpg.xmp [Nest] 7 - 09/19/2024, 10:19:57 AM DEBUG [Microservices:MetadataService] Asset ea468aeb-6745-4ff5-ba2c-bb87973658e0 date time is undefined [Nest] 7 - 09/19/2024, 10:19:57 AM WARN [Microservices:MetadataService] Asset ea468aeb-6745-4ff5-ba2c-bb87973658e0 has no valid date (undefined), falling back to asset.fileCreatedAt ```

deekerman commented

Author

Owner

@pypeaday commented on GitHub (Sep 19, 2024):

can the xmp files or whatever be written somewhere else?

@pypeaday commented on GitHub (Sep 19, 2024): can the xmp files or whatever be written somewhere else?

deekerman commented

Author

Owner

@basicfu commented on GitHub (Sep 27, 2024):

+1 can the xmp files or whatever be written somewhere else?
I need to back up my gallery, but I don't want to see the xmp file when previewing through NAS

@basicfu commented on GitHub (Sep 27, 2024): +1 can the xmp files or whatever be written somewhere else? I need to back up my gallery, but I don't want to see the xmp file when previewing through NAS

deekerman commented

Author

Owner

@bo0tzz commented on GitHub (Sep 28, 2024):

They're sidecar files, storing them somewhere disconnected from the media doesn't really make sense.

@bo0tzz commented on GitHub (Sep 28, 2024): They're _sidecar_ files, storing them somewhere disconnected from the media doesn't really make sense.

deekerman commented

Author

Owner

@acbarrentine commented on GitHub (Oct 5, 2024):

When the tag feature first went in, I was able to tag external (read-only) pictures. In subsequent updates, I have lost that functionality. Tags appear to apply, then disappear a few seconds later. In the log, I see the same kind of warnings reported above.

I could make the file system writeable, but I'd prefer not to. Something knows tags for my external files in that read-only file system made during the span where it worked. I suspect something went wrong with a revision and it is now trying to send that data to the .xmp files incorrectly

@acbarrentine commented on GitHub (Oct 5, 2024): When the tag feature first went in, I was able to tag external (read-only) pictures. In subsequent updates, I have lost that functionality. Tags appear to apply, then disappear a few seconds later. In the log, I see the same kind of warnings [reported above](https://github.com/immich-app/immich/issues/10538#issuecomment-2361328995). I could make the file system writeable, but I'd prefer not to. _Something_ knows tags for my external files in that read-only file system made during the span where it worked. I suspect something went wrong with a revision and it is now trying to send that data to the .xmp files incorrectly

deekerman commented

Author

Owner

@Stronginthearm42 commented on GitHub (Oct 9, 2024):

When the tag feature first went in, I was able to tag external (read-only) pictures. In subsequent updates, I have lost that functionality. Tags appear to apply, then disappear a few seconds later. In the log, I see the same kind of warnings reported above.

I could make the file system writeable, but I'd prefer not to. Something knows tags for my external files in that read-only file system made during the span where it worked. I suspect something went wrong with a revision and it is now trying to send that data to the .xmp files incorrectly

+1
It seems to me that the "correct" option would be to store changed metadata for ro assets in the db.

@Stronginthearm42 commented on GitHub (Oct 9, 2024): > When the tag feature first went in, I was able to tag external (read-only) pictures. In subsequent updates, I have lost that functionality. Tags appear to apply, then disappear a few seconds later. In the log, I see the same kind of warnings [reported above](https://github.com/immich-app/immich/issues/10538#issuecomment-2361328995). > > I could make the file system writeable, but I'd prefer not to. _Something_ knows tags for my external files in that read-only file system made during the span where it worked. I suspect something went wrong with a revision and it is now trying to send that data to the .xmp files incorrectly +1 It seems to me that the "correct" option would be to store changed metadata for ro assets in the db.

deekerman commented

Author

Owner

@pyorot commented on GitHub (Oct 9, 2024):

+1 It seems to me that the "correct" option would be to store changed metadata for ro assets in the db.

Ah but then the user would need to be told that that new metadata would only be reflected in immich rather than portably

@pyorot commented on GitHub (Oct 9, 2024): > +1 It seems to me that the "correct" option would be to store changed metadata for ro assets in the db. Ah but then the user would need to be told that that new metadata would only be reflected in immich rather than portably

deekerman commented

Author

Owner

@Stronginthearm42 commented on GitHub (Oct 10, 2024):

Yes, but since the library is read-only, this shouldn't be surprising.

@Stronginthearm42 commented on GitHub (Oct 10, 2024): Yes, but since the library is read-only, this shouldn't be surprising.

deekerman commented

Author

Owner

@anthonynogales commented on GitHub (Oct 27, 2024):

I think that all libraries, not just read-only ones, should have metadata like descriptions, tags, and ratings stored in the database by default. This approach would provide a central, reliable repository for metadata management, and users could then elect to export this metadata to individual XMP sidecar files for all images or an individual image if desired. This flexibility would allow for more efficient organization and metadata handling across all library types without altering the original files unless chosen explicitly by the user.

@anthonynogales commented on GitHub (Oct 27, 2024): I think that all libraries, not just read-only ones, should have metadata like descriptions, tags, and ratings stored in the database by default. This approach would provide a central, reliable repository for metadata management, and users could then elect to export this metadata to individual XMP sidecar files for all images or an individual image if desired. This flexibility would allow for more efficient organization and metadata handling across all library types without altering the original files unless chosen explicitly by the user.

deekerman commented

Author

Owner

@jrasm91 commented on GitHub (Dec 28, 2024):

If you re-run metadata extraction on the file again it would silently overwrite the value in the database if it never was persisted to an xmp file

@jrasm91 commented on GitHub (Dec 28, 2024): If you re-run metadata extraction on the file again it would silently overwrite the value in the database if it never was persisted to an xmp file

deekerman commented

Author

Owner

@patmagauran commented on GitHub (Feb 6, 2025):

As a potential workaround to this, I setup an overlay mount for immich to access my external library from. That way the underlying files aren't modified; however sidecar data is still maintained by immich. I haven't been running it for long enough to notice any side-effects, but it should work well for a layer of separation.

For docker compose:

volumes:
  media_data_overlay:
    driver: local
    driver_opts:
      type: overlay
      o: lowerdir=/YOUR_EXTERNAL_DATA,upperdir=$PWD/immich_media,workdir=$PWD/immich_work
      device: overlay
services:
  immich-server:
....
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - media_data_overlay:/media_data   # This line mounts it to the container

The overlayFS works by taking the contents of the 'lower' directory and creating a virtual file system on top which is accessible from the container. The 'upperdir' houses all the changed files and the 'workdir' houses files that are in the process of being changed (kind of like file level COW).

Since Immich doesn't write metadata to the original file, the overlay should take up very little space as it would only store xmps (As well as any changes due to deletion or editing).

Credit for how to use the overlayfs as a volume mount: https://stackoverflow.com/questions/29550736/can-i-mount-docker-host-directory-as-copy-on-write-overlay

@patmagauran commented on GitHub (Feb 6, 2025): As a potential workaround to this, I setup an overlay mount for immich to access my external library from. That way the underlying files aren't modified; however sidecar data is still maintained by immich. I haven't been running it for long enough to notice any side-effects, but it should work well for a layer of separation. For docker compose: ``` volumes: media_data_overlay: driver: local driver_opts: type: overlay o: lowerdir=/YOUR_EXTERNAL_DATA,upperdir=$PWD/immich_media,workdir=$PWD/immich_work device: overlay services: immich-server: .... volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - media_data_overlay:/media_data # This line mounts it to the container ``` The overlayFS works by taking the contents of the 'lower' directory and creating a virtual file system on top which is accessible from the container. The 'upperdir' houses all the changed files and the 'workdir' houses files that are in the process of being changed (kind of like file level COW). Since Immich doesn't write metadata to the original file, the overlay should take up very little space as it would only store xmps (As well as any changes due to deletion or editing). Credit for how to use the overlayfs as a volume mount: https://stackoverflow.com/questions/29550736/can-i-mount-docker-host-directory-as-copy-on-write-overlay

deekerman commented

Author

Owner

@pyorot commented on GitHub (Feb 7, 2025):

i like the idea of implementing this as a persistent overlay, as long as there’s a system by which the data could optionally be merged into the originals (same as if the mount were rw rather than ro). bo0tzz suggested to me just now to use Exiftool, but the question is how much this process can be automated. for example, whether this overlay xmp set can be exported into a set of files that retain adequate links with their assets to then be mergeable in one step rather than #number-of-assets steps

@pyorot commented on GitHub (Feb 7, 2025): i like the idea of implementing this as a persistent overlay, as long as there’s a system by which the data could optionally be merged into the originals (same as if the mount were rw rather than ro). bo0tzz suggested to me just now to use Exiftool, but the question is how much this process can be automated. for example, whether this overlay xmp set can be exported into a set of files that retain adequate links with their assets to then be mergeable in one step rather than #number-of-assets steps

deekerman commented

Author

Owner

@Nightmaren commented on GitHub (Feb 22, 2025):

I have been using the overlayFS method heavily over the past week. I have been adding files to my external library's directory, scan them via Immich, then proceed to go through each one and manually tag them. Everything is working as expected, with the sidecars being created in the upper directory.

However, one side-effect I have found pertains to the way the external library is scanned. I typically have Immich rescanning it daily on a cron job, but even if I ask it to scan manually via the admin panel, it will not notice the presence/absence of additional photos. Upon restarting, and then rescanning, photos are updated as expected.

I believe this has to do with the way docker manages the lower directory's files when using the overlayFS.

@Nightmaren commented on GitHub (Feb 22, 2025): I have been using the overlayFS method heavily over the past week. I have been adding files to my external library's directory, scan them via Immich, then proceed to go through each one and manually tag them. Everything is working as expected, with the sidecars being created in the upper directory. However, one side-effect I have found pertains to the way the external library is scanned. I typically have Immich rescanning it daily on a cron job, but even if I ask it to scan manually via the admin panel, it will not notice the presence/absence of additional photos. Upon restarting, and *then* rescanning, photos are updated as expected. I believe this has to do with the way docker manages the lower directory's files when using the overlayFS.

deekerman commented

Author

Owner

@alextran1502 commented on GitHub (Nov 9, 2025):

~~This is no longer relevant with the new implementation, please open a new issue if it is still persisted~~

@alextran1502 commented on GitHub (Nov 9, 2025): ~~This is no longer relevant with the new implementation, please open a new issue if it is still persisted~~

deekerman commented

Author

Owner

@mikehardy commented on GitHub (Jan 6, 2026):

Very interesting! Just ran into this myself. I understand that philosophically the sidecar file should live next to the file. Adults can disagree but a firm opinion is clarifying, so - fine as far as it goes.

Seems a popup would be great as a warning, warning the non-documentation-reading regular users of our systems to be aware of the limitation.

Or, an implementation that went ahead and updated the database, followed by a popup on "refresh metadata" saying "this asset is from an external library that is read-only but has local modifications, refreshing metadata will overwrite the local modifications". In fact, it could even detect the changes between what the metadata refresh would do and what is local and show people a little diff panel per-changed-element similar to Calibre's metadata updater as you review downloaded metadata for changes (example image of same here in form of reasonably safe image search result link - was best I could find: https://duckduckgo.com/?q=calibre+review+downloaded+metadata+download+accept&iar=images&t=brave&iai=https%3A%2F%2Fwww.pctips.com%2Fwp-content%2Fuploads%2F2024%2F05%2Fcalibre-downloaded-metadata.jpg )

But it is easy for users to dream up features but PRs as well as dev time for feature maintenance are always scarce of course.

So, for the workaround - @Nightmaren - seems like you are trying the docker-side overlay FS style. You mention you have a need to restart the immich stack prior to rescans. Have you tried the similar-but-reversed host-side overlay FS style? It was mentioned in the stackoverflow (https://stackoverflow.com/a/50917037/9910298) as a separate answer and may give us the best of all worlds - read-only base image repository, writeable XMPs, and no need to restart the immich stack prior to external rescans.

If no one else has tried it I might. My repo is on a synology NAS which is linux I've got root to, so I should be able to make it work prior to the FS sharing out on the network to immich.

@mikehardy commented on GitHub (Jan 6, 2026): Very interesting! Just ran into this myself. I understand that philosophically the *side*car file should live next to the file. Adults can disagree but a firm opinion is clarifying, so - fine as far as it goes. Seems a popup would be great as a warning, warning the non-documentation-reading regular users of our systems to be aware of the limitation. Or, an implementation that went ahead and updated the database, followed by a popup on "refresh metadata" saying "this asset is from an external library that is read-only but has local modifications, refreshing metadata will overwrite the local modifications". In fact, it could even detect the changes between what the metadata refresh *would* do and what is local and show people a little diff panel per-changed-element similar to Calibre's metadata updater as you review downloaded metadata for changes (example image of same here in form of reasonably safe image search result link - was best I could find: https://duckduckgo.com/?q=calibre+review+downloaded+metadata+download+accept&iar=images&t=brave&iai=https%3A%2F%2Fwww.pctips.com%2Fwp-content%2Fuploads%2F2024%2F05%2Fcalibre-downloaded-metadata.jpg ) But it is easy for users to dream up features but PRs as well as dev time for feature maintenance are always scarce of course. So, for the workaround - @Nightmaren - seems like you are trying the docker-side overlay FS style. You mention you have a need to restart the immich stack prior to rescans. Have you tried the similar-but-reversed host-side overlay FS style? It was mentioned in the stackoverflow (https://stackoverflow.com/a/50917037/9910298) as a separate answer and may give us the best of all worlds - read-only base image repository, writeable XMPs, and no need to restart the immich stack prior to external rescans. If no one else has tried it I might. My repo is on a synology NAS which is linux I've got root to, so I should be able to make it work prior to the FS sharing out on the network to immich.

deekerman commented

Author

Owner

@P3TE04 commented on GitHub (Jan 26, 2026):

@mikehardy did that work for you? I think this is the next solution I try

@P3TE04 commented on GitHub (Jan 26, 2026): @mikehardy did that work for you? I think this is the next solution I try

deekerman commented

Author

Owner

@mikehardy commented on GitHub (Jan 26, 2026):

Hey @P3TE04 👋

If you mean this:

Have you tried the similar-but-reversed host-side overlay FS style? It was mentioned in the stackoverflow (https://stackoverflow.com/a/50917037/9910298) as a separate answer and may give us the best of all worlds - read-only base image repository, writeable XMPs, and no need to restart the immich stack prior to external rescans.

If no one else has tried it I might. My repo is on a synology NAS which is linux I've got root to, so I should be able to make it work prior to the FS sharing out on the network to immich.

I have not yet - my family is currently operating under the knowledge no metadata edits will be saved, and that's bought me time to do billable work instead.

I'm still very interested in trying it, just no time yet. If you do try it and have results to share I'd love to hear it personally

Cheers

@mikehardy commented on GitHub (Jan 26, 2026): Hey @P3TE04 👋 If you mean this: > Have you tried the similar-but-reversed host-side overlay FS style? It was mentioned in the stackoverflow (https://stackoverflow.com/a/50917037/9910298) as a separate answer and may give us the best of all worlds - read-only base image repository, writeable XMPs, and no need to restart the immich stack prior to external rescans. > > If no one else has tried it I might. My repo is on a synology NAS which is linux I've got root to, so I should be able to make it work prior to the FS sharing out on the network to immich. I have not yet - my family is currently operating under the knowledge no metadata edits will be saved, and that's bought me time to do billable work instead. I'm still very interested in trying it, just no time yet. If you do try it and have results to share I'd love to hear it personally Cheers

deekerman commented

Author

Owner

@P3TE04 commented on GitHub (Jan 30, 2026):

@mikehardy it looks like they may have solved our entire problem with the newest update! unless I'm doing something wrong. It does however seem that my files are still read only in immich, and yet I can add tags juuust fine :)

@P3TE04 commented on GitHub (Jan 30, 2026): @mikehardy it looks like they may have solved our entire problem with the newest update! unless I'm doing something wrong. It does however seem that my files are still read only in immich, and yet I can add tags juuust fine :)

deekerman commented

Author

Owner

@mikehardy commented on GitHub (Jan 30, 2026):

Tags were always in the db its if you try to change things that were in exif like location or time I thought?

@mikehardy commented on GitHub (Jan 30, 2026): Tags were always in the db its if you try to change things that were in exif like location or time I thought?

deekerman commented