External library scanning (Scan New Library Files) aborts upon Access Denied #3872

Open
opened 2026-02-20 02:17:55 -05:00 by deekerman · 13 comments
Owner

Originally created by @psla on GitHub (Sep 15, 2024).

Originally assigned to: @etnoy on GitHub.

The bug

  1. Restrict access to some folders in the external library
  2. Go to "external libraries"
  3. Call "Scan New Library Files"
  4. Observe error
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Object:
{
  "id": "27af95c1-57fc-4885-9b55-f78a9bc36276",
  "refreshModifiedFiles": false,
  "refreshAllFiles": false
}

Expected behavior: scanning of the library should continue

I understand that generally you probably want to fail upon errors, but this seems a valid use case (to me). Some photographs/content may be more sensitive, and may not be shared with others (including immich). It would be nice if Immich gracesfully fell back access denied errors.

The OS that Immich Server is running on

Debian Linux #redacted# 6.1.0-23-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux

Version of Immich Server

1.115.0

Version of Immich Mobile App

n/a

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.3"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /mnt/nas/public/wspolne/Fotografie:/mnt/media/Fotografie:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always
  backup:
    container_name: immich_db_dumper
    image: prodrigestivill/postgres-backup-local
    env_file:
      - .env
    environment:
      POSTGRES_HOST: database
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      SCHEDULE: "@daily"
      BACKUP_NUM_KEEP: 2
      BACKUP_DIR: /db_dumps
    volumes:
      - /mnt/newhome/unprivileged-docker/immich/db_dumps:/db_dumps
    depends_on:
      - database

volumes:
  pgdata:
  model-cache:

Your .env content

DB_HOSTNAME=#redacted#
DB_USERNAME=#redacted#
DB_PASSWORD=#redacted#
DB_DATABASE_NAME=immich



REDIS_HOSTNAME=immich_redis






UPLOAD_LOCATION=/mnt/nas/srv/immich/pictures


TYPESENSE_API_KEY=#




PUBLIC_LOGIN_PAGE_MESSAGE=


IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003




IMMICH_API_URL_EXTERNAL=#redacted#

Reproduction steps

  1. Restrict access to some folders in the external library
  2. Go to "external libraries"
  3. Call "Scan New Library Files"
  4. Observe error

Relevant log output

[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM'
[Nest] 7  - 09/15/2024, 10:27:38 PM   ERROR [Microservices:JobService] Object:
{
  "id": "27af95c1-57fc-4885-9b55-f78a9bc36276",
  "refreshModifiedFiles": false,
  "refreshAllFiles": false
}


### Additional information

_No response_
Originally created by @psla on GitHub (Sep 15, 2024). Originally assigned to: @etnoy on GitHub. ### The bug 0) Restrict access to some folders in the external library 1) Go to "external libraries" 2) Call "Scan New Library Files" 3) Observe error ``` [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Object: { "id": "27af95c1-57fc-4885-9b55-f78a9bc36276", "refreshModifiedFiles": false, "refreshAllFiles": false } ``` Expected behavior: scanning of the library should continue I understand that generally you probably want to fail upon errors, but this seems a valid use case (to me). Some photographs/content may be more sensitive, and may not be shared with others (including immich). It would be nice if Immich gracesfully fell back access denied errors. ### The OS that Immich Server is running on Debian Linux #redacted# 6.1.0-23-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.99-1 (2024-07-15) x86_64 GNU/Linux ### Version of Immich Server 1.115.0 ### Version of Immich Mobile App n/a ### Platform with the issue - [ ] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.3" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /mnt/nas/public/wspolne/Fotografie:/mnt/media/Fotografie:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3 restart: always database: container_name: immich_postgres image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - pgdata:/var/lib/postgresql/data restart: always backup: container_name: immich_db_dumper image: prodrigestivill/postgres-backup-local env_file: - .env environment: POSTGRES_HOST: database POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_USER: ${DB_USERNAME} POSTGRES_PASSWORD: ${DB_PASSWORD} SCHEDULE: "@daily" BACKUP_NUM_KEEP: 2 BACKUP_DIR: /db_dumps volumes: - /mnt/newhome/unprivileged-docker/immich/db_dumps:/db_dumps depends_on: - database volumes: pgdata: model-cache: ``` ### Your .env content ```Shell DB_HOSTNAME=#redacted# DB_USERNAME=#redacted# DB_PASSWORD=#redacted# DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis UPLOAD_LOCATION=/mnt/nas/srv/immich/pictures TYPESENSE_API_KEY=# PUBLIC_LOGIN_PAGE_MESSAGE= IMMICH_WEB_URL=http://immich-web:3000 IMMICH_SERVER_URL=http://immich-server:3001 IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003 IMMICH_API_URL_EXTERNAL=#redacted# ``` ### Reproduction steps 0) Restrict access to some folders in the external library 1) Go to "external libraries" 2) Call "Scan New Library Files" 3) Observe error ### Relevant log output ```shell [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Unable to run job handler (library/library-refresh): Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/mnt/media/Fotografie/102SSCAM' [Nest] 7 - 09/15/2024, 10:27:38 PM ERROR [Microservices:JobService] Object: { "id": "27af95c1-57fc-4885-9b55-f78a9bc36276", "refreshModifiedFiles": false, "refreshAllFiles": false } ``` ``` ### Additional information _No response_
Author
Owner

@alexantao commented on GitHub (Dec 16, 2024):

I don't know if I have the same issue.
I've just installed immich (v1.122.3) on a small box I have and mounted a NAS to a local directory as RO, since I want to make shure nothing will me modified on this disk.

Configured my external library, but no photos are found.
Saw on the logs the same access denied errors. But it's odd, because it does has acess, just that it's RO.

[Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Refreshing library ed72481e-f76d-437e-acb2-bb535f199c72 for new assets [Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Scanning library ed72481e-f76d-437e-acb2-bb535f199c72 for removed assets [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Unable to run job handler (library/library-queue-sync-files): Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Object:
And I tested entering bash from docker, accessed the library can also cat a JPG file inside it. Access was OK.
Can it be the same issue ?

@alexantao commented on GitHub (Dec 16, 2024): I don't know if I have the same issue. I've just installed immich (v1.122.3) on a small box I have and mounted a NAS to a local directory as RO, since I want to make shure nothing will me modified on this disk. Configured my external library, but no photos are found. Saw on the logs the same access denied errors. But it's odd, because it does has acess, just that it's RO. `[Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Refreshing library ed72481e-f76d-437e-acb2-bb535f199c72 for new assets [Nest] 7 - 12/16/2024, 11:39:10 AM LOG [Microservices:LibraryService] Scanning library ed72481e-f76d-437e-acb2-bb535f199c72 for removed assets [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Unable to run job handler (library/library-queue-sync-files): Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Error: EACCES: permission denied, scandir '/FOTOS/2004/01 - Jan' [Nest] 7 - 12/16/2024, 11:39:10 AM ERROR [Microservices:JobService] Object: ` And I tested entering bash from docker, accessed the library can also cat a JPG file inside it. Access was OK. Can it be the same issue ?
Author
Owner

@bo0tzz commented on GitHub (Feb 2, 2025):

Per this comment from @zackpollard, do we want to close this issue?

@bo0tzz commented on GitHub (Feb 2, 2025): Per [this comment](https://github.com/immich-app/immich/pull/13306#issuecomment-2473087839) from @zackpollard, do we want to close this issue?
Author
Owner

@etnoy commented on GitHub (Feb 2, 2025):

I'd say keep this open until we can get fast-glob to fix this one

@etnoy commented on GitHub (Feb 2, 2025): I'd say keep this open until we can get fast-glob to fix this one
Author
Owner

@biappi commented on GitHub (Sep 21, 2025):

After reading this comment i'm not sure if the following falls in the same umbrella as this issue, sorry if otherwise. But after working around EACCESS, i had the same issue with ENOTDIR as follows:

[Nest] 7  - 09/21/2025, 1:20:07 PM   ERROR [Microservices:{"id":"83d57784-995d-4206-bb67-3ea95cb5e5e3"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: ENOTDIR: not a directory, scandir '/ext-lib/Backups/Thala - 0/.wine/dosdevices/z:/dev/fd/54'
@biappi commented on GitHub (Sep 21, 2025): After reading [this comment](https://github.com/mrmlnc/fast-glob/pull/485#issuecomment-2639918116) i'm not sure if the following falls in the same umbrella as this issue, sorry if otherwise. But after working around `EACCESS`, i had the same issue with `ENOTDIR` as follows: [Nest] 7 - 09/21/2025, 1:20:07 PM ERROR [Microservices:{"id":"83d57784-995d-4206-bb67-3ea95cb5e5e3"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: ENOTDIR: not a directory, scandir '/ext-lib/Backups/Thala - 0/.wine/dosdevices/z:/dev/fd/54'
Author
Owner

@conchyliculture commented on GitHub (Sep 30, 2025):

I would also like the scanning to skip over directory/files with access errors and go to the next instead of stopping at the first error

immich_server            | [Nest] 7  - 09/30/2025, 7:22:22 PM   ERROR [Microservices:{"id":"633694ff-43de-4ab8-811b-1a8a21bf4a95"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old'
immich_server            | Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old'
@conchyliculture commented on GitHub (Sep 30, 2025): I would also like the scanning to skip over directory/files with access errors and go to the next instead of stopping at the first error ``` immich_server | [Nest] 7 - 09/30/2025, 7:22:22 PM ERROR [Microservices:{"id":"633694ff-43de-4ab8-811b-1a8a21bf4a95"}] Unable to run job handler (LibrarySyncFilesQueueAll): Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old' immich_server | Error: EACCES: permission denied, scandir '/mnt/photos/20020815_old' ```
Author
Owner

@etnoy commented on GitHub (Sep 30, 2025):

Unfortunately, this is a bug with fast-glob that we rely on to do file scans. We have an open PR with the upstream code to allow custom error handlers, but so far nothing.

https://github.com/mrmlnc/fast-glob/pull/485

Until that is fixed, any error that occurs during scan will cause the scan to fail.

@etnoy commented on GitHub (Sep 30, 2025): Unfortunately, this is a bug with fast-glob that we rely on to do file scans. We have an open PR with the upstream code to allow custom error handlers, but so far nothing. https://github.com/mrmlnc/fast-glob/pull/485 Until that is fixed, any error that occurs during scan will cause the scan to fail.
Author
Owner

@benmccann commented on GitHub (Nov 6, 2025):

One option would be to use fdir since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors

fdir supports early termination, but not streaming yet. There's an open PR for streaming support. @etnoy is streaming support a blocker in terms of performance? Maybe it'd be better to switch to fdir for stability and then upgrade it for streaming performance once that's merged and released?

@benmccann commented on GitHub (Nov 6, 2025): One option would be to use `fdir` since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors `fdir` supports early termination, but not streaming yet. There's an open PR for streaming support. @etnoy is streaming support a blocker in terms of performance? Maybe it'd be better to switch to `fdir` for stability and then upgrade it for streaming performance once that's merged and released?
Author
Owner

@etnoy commented on GitHub (Nov 6, 2025):

One option would be to use fdir since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors

fdir supports early termination, but not streaming yet. There's an open PR for streaming support. @etnoy is streaming support a blocker in terms of performance? Maybe it'd be better to switch to fdir for stability and then upgrade it for streaming performance once that's merged and released?

I think the easier route here is to fork fast-glob and add the three-ish lines to it to give it a better error handler.

Yes, streaming is critical. Without it, there's no way to scan really large libraries (over 1M assets), and performance will be terrible. I've been able to optimize performance by streaming in large chunks. See https://github.com/immich-app/immich/pull/14456

@etnoy commented on GitHub (Nov 6, 2025): > One option would be to use `fdir` since it supports ignoring of errors: https://github.com/thecodrr/fdir/blob/master/documentation.md#witherrors > > `fdir` supports early termination, but not streaming yet. There's an open PR for streaming support. [@etnoy](https://github.com/etnoy) is streaming support a blocker in terms of performance? Maybe it'd be better to switch to `fdir` for stability and then upgrade it for streaming performance once that's merged and released? I think the easier route here is to fork fast-glob and add the three-ish lines to it to give it a better error handler. Yes, streaming is critical. Without it, there's no way to scan really large libraries (over 1M assets), and performance will be terrible. I've been able to optimize performance by streaming in large chunks. See https://github.com/immich-app/immich/pull/14456
Author
Owner

@benmccann commented on GitHub (Nov 6, 2025):

I'll keep pushing for streaming in tinyglobby then. Most of the JS ecosystem has moved off of fast-glob now with its downloads coming mainly from legacy projects. It's a big supply chain risk because it has so many dependencies and moving to tinyglobby would really help with that and it's also faster as well

@benmccann commented on GitHub (Nov 6, 2025): I'll keep pushing for streaming in tinyglobby then. Most of the JS ecosystem has moved off of fast-glob now with its downloads coming mainly from legacy projects. It's a big supply chain risk because it has so many dependencies and moving to tinyglobby would really help with that and it's also faster as well
Author
Owner

@valentino-saitz-optris commented on GitHub (Jan 16, 2026):

Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained.

@valentino-saitz-optris commented on GitHub (Jan 16, 2026): Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained.
Author
Owner

@etnoy commented on GitHub (Jan 16, 2026):

Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained.

Agreed, but we are relying on features in fast-glob that I haven't found anywhere else. Last I checked, tinyglobby can't be used since it lacks features.

@etnoy commented on GitHub (Jan 16, 2026): > Fast-glob hasn't had any activity for a whole year at this point. Relying on a library with a sole maintainer that won't react to pull requests and hasn't had nearly any activity on Github this past year seems like a bad idea. Meanwhile tinyglobby is actively maintained. Agreed, but we are relying on features in fast-glob that I haven't found anywhere else. Last I checked, tinyglobby can't be used since it lacks features.
Author
Owner

@Thornsnake commented on GitHub (Feb 10, 2026):

We just ran into this issue setting up the external library scans for our company's image network storage. Not every department is supposed to have access to another department's images, so access denials are a natural part of the scan.

Since this issue is ongoing since 2024, is there any hope for a timely fix or alternative?

EDIT

There seems to be an open pull request already handling this issue here: https://github.com/mrmlnc/fast-glob/pull/485
Any idea when it might be merged?

@Thornsnake commented on GitHub (Feb 10, 2026): We just ran into this issue setting up the external library scans for our company's image network storage. Not every department is supposed to have access to another department's images, so access denials are a natural part of the scan. Since this issue is ongoing since 2024, is there any hope for a timely fix or alternative? **EDIT** There seems to be an open pull request already handling this issue here: https://github.com/mrmlnc/fast-glob/pull/485 Any idea when it might be merged?
Author
Owner

@etnoy commented on GitHub (Feb 10, 2026):

We just ran into this issue setting up the external library scans for our company's image network storage. Not every department is supposed to have access to another department's images, so access denials are a natural part of the scan.

Since this issue is ongoing since 2024, is there any hope for a timely fix or alternative?

EDIT

There seems to be an open pull request already handling this issue here: mrmlnc/fast-glob#485 Any idea when it might be merged?

Hello! I have lost hope of getting the pr merged, the project seems abandoned at this point. It's unfortunate because it has a few features we rely on that no other glob package provides.

Ideally we'd get another, maintained, glob package to add the async scan feature. Then we could switch to that.

That would be the ideal way forward and it would be a great way to help immich out.

@etnoy commented on GitHub (Feb 10, 2026): > We just ran into this issue setting up the external library scans for our company's image network storage. Not every department is supposed to have access to another department's images, so access denials are a natural part of the scan. > > Since this issue is ongoing since 2024, is there any hope for a timely fix or alternative? > > **EDIT** > > There seems to be an open pull request already handling this issue here: [mrmlnc/fast-glob#485](https://github.com/mrmlnc/fast-glob/pull/485) Any idea when it might be merged? Hello! I have lost hope of getting the pr merged, the project seems abandoned at this point. It's unfortunate because it has a few features we rely on that no other glob package provides. Ideally we'd get another, maintained, glob package to add the async scan feature. Then we could switch to that. That would be the ideal way forward and it would be a great way to help immich out.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/immich#3872
No description provided.