starred/photoprism

Fork 0

mirror of https://github.com/photoprism/photoprism.git synced 2026-03-02 22:57:18 -05:00

task "convert" takes a very long time to sort through the already encoded files #2039

New issue

Open

opened 2026-02-20 01:05:00 -05:00 by deekerman · 2 comments

deekerman commented

2026-02-20 01:05:00 -05:00

Owner

Originally created by @EuPhobos on GitHub (Mar 2, 2024).

For example in start i have no problem, but now i have lots of video files, and even if only one video has been added to the library. the "convert" command lasted forever, for some reason, photoprism goes through the already converted videos, up to 10 seconds pass from video to video, after which it writes in the log "does not require transcoding" and moves on to the next one, throughout the already transcoded library, several tens of hours pass until it finds a new single added file.

1. What is not working as documented?

All work, but very slow.

2. How can we reproduce it?

Steps to reproduce the behavior:

Get big library of video files.
Add new one.
Run "photoprism convert" command

3. What behavior do you expect?

Indexing of images takes place instantly, photoprism already knows which ones have been added and which ones are new. The same behavior would be ideal for videos.

4. What could be the cause of your problem?

I think there is no record of which video has already been transcoded and which is new.

5. Can you provide us with example files for testing, error logs, or screenshots?

6. Which software versions do you use?

Build [231128-f48ff16ef] AMD64
11.2.2-MariaDB
Linux Debian 12

7. On what kind of device is PhotoPrism installed?

Intel(R) Pentium(R) Silver N6005 @ 2.00GHz
16Gb RAM
Photoprism and MariaDB: SSD M.2-NVMe
Media library: HDD 7200 rpm
All in 1 machine

8. Do you use a Reverse Proxy, Firewall, VPN, or CDN?

Yes nginx as proxy, but he does not participate in transcoding problem.

Originally created by @EuPhobos on GitHub (Mar 2, 2024). For example in start i have no problem, but now i have lots of video files, and even if only one video has been added to the library. the "convert" command lasted forever, for some reason, photoprism goes through the already converted videos, up to 10 seconds pass from video to video, after which it writes in the log "does not require transcoding" and moves on to the next one, throughout the already transcoded library, several tens of hours pass until it finds a new single added file. --------------------------------------------------------------------------- #### 1. What is not working as documented? All work, but very slow. #### 2. How can we reproduce it? Steps to reproduce the behavior: 1. Get big library of video files. 2. Add new one. 3. Run "photoprism convert" command #### 3. What behavior do you expect? Indexing of images takes place instantly, photoprism already knows which ones have been added and which ones are new. The same behavior would be ideal for videos. #### 4. What could be the cause of your problem? I think there is no record of which video has already been transcoded and which is new. #### 5. Can you provide us with example files for testing, error logs, or screenshots? ![Снимок экрана_2024-03-02_23-06-38](https://github.com/photoprism/photoprism/assets/3438730/9848f603-dc11-46b3-bcc9-dcd261eb855f) #### 6. Which software versions do you use? Build [231128-f48ff16ef] AMD64 11.2.2-MariaDB Linux Debian 12 #### 7. On what kind of device is PhotoPrism installed? Intel(R) Pentium(R) Silver N6005 @ 2.00GHz 16Gb RAM Photoprism and MariaDB: SSD M.2-NVMe Media library: HDD 7200 rpm All in 1 machine #### 8. Do you use a Reverse Proxy, Firewall, VPN, or CDN? Yes nginx as proxy, but he does not participate in transcoding problem.

deekerman added the

enhancement

help wanted

needs-analysis

labels

2026-02-20 01:05:00 -05:00

deekerman commented

2026-02-20 01:05:01 -05:00

Author

Owner

@andrewlow commented on GitHub (Mar 22, 2024):

I was hitting exactly this problem myself just today - and searched for a matching issue :)

You can get help from the convert command

$ docker exec photoprism photoprism convert --help
NAME:
   photoprism convert - Converts files in other formats to JPEG and AVC as needed

USAGE:
   photoprism convert [command options] [subfolder]

OPTIONS:
   --ext value, -e value  only process files with the specified extensions, e.g. mp4
   --force, -f            replace existing JPEG files in the sidecar folder

It would be wonderful to add an additional filter command. I would suggest --since

Ideally the since flag would take input in easy to input human terms

--since 7d would calculate todays date, minus 7 days, and then exclude any movie file in the library which was older than that.

It would still require that the library be scanned entirely, but would likely speed up the checking if there is an existing encoding or not. (I'm guessing.. )

For very large libraries (100,000 files) - you still need to scan each of those, and determine if they are a movie file - then do a date check.

However, right now - for each of those movie files - the system needs to probe to see if there is an optimized version of that file.

It should be much faster to scan every file - ignoring any older than the date.
Then check to see if it is a movie file
Then if there is an optimized version.

@andrewlow commented on GitHub (Mar 22, 2024): I was hitting exactly this problem myself just today - and searched for a matching issue :) You can get help from the convert command ``` $ docker exec photoprism photoprism convert --help NAME: photoprism convert - Converts files in other formats to JPEG and AVC as needed USAGE: photoprism convert [command options] [subfolder] OPTIONS: --ext value, -e value only process files with the specified extensions, e.g. mp4 --force, -f replace existing JPEG files in the sidecar folder ``` It would be wonderful to add an additional filter command. I would suggest `--since` Ideally the `since` flag would take input in easy to input human terms `--since 7d` would calculate todays date, minus 7 days, and then exclude any movie file in the library which was older than that. It would still require that the library be scanned entirely, but would likely speed up the checking if there is an existing encoding or not. (I'm guessing.. ) For very large libraries (100,000 files) - you still need to scan each of those, and determine if they are a movie file - then do a date check. However, right now - for each of those movie files - the system needs to probe to see if there is an optimized version of that file. It should be much faster to scan every file - ignoring any older than the date. Then check to see if it is a movie file Then if there is an optimized version.

deekerman commented

2026-02-20 01:05:01 -05:00

Author

Owner

@EuPhobos commented on GitHub (May 5, 2024):

Maybe I don't understand something. But at the moment I have 30k files from the bottom 4000 videos. And at the moment, it is no longer possible to use the photo album.
For example, when adding another 5 new video files, it should take a day before I can share new videos, because convert starts going through all 4000 files in search of new videos to convert.
And the strangest thing is, why does it take about 10 seconds from file to file to understand that the video does not need to be converted? What exactly does photoprism do at the same time, where does it take so much time?
Why not just run convert on new video files when indexing? After all, indexing works correctly.

@EuPhobos commented on GitHub (May 5, 2024): Maybe I don't understand something. But at the moment I have 30k files from the bottom 4000 videos. And at the moment, it is no longer possible to use the photo album. For example, when adding another 5 new video files, it should take a day before I can share new videos, because convert starts going through all 4000 files in search of new videos to convert. And the strangest thing is, why does it take about 10 seconds from file to file to understand that the video does not need to be converted? What exactly does photoprism do at the same time, where does it take so much time? Why not just run convert on new video files when indexing? After all, indexing works correctly.