mirror of
https://github.com/photoprism/photoprism.git
synced 2026-03-02 22:57:18 -05:00
Vips: Error buffer fills up when images have no interop-index #2456
Labels
No labels
ai
android
api
auth
awesome
bug
bug
ci
cli
config
database
declined
deprecated
docker
docs 📚
documents
duplicate
easy
enhancement
enhancement
enhancement
epic
faces
feedback wanted
frontend
hacktoberfest
help wanted
idea
in-progress
incomplete
index
invalid
ios
labels
live
live
low-priority
macos
member-feature
metadata
mobile
nas
needs-analysis
no-coding-required
no-coding-required
observability
performance
places
please-test
plus-feature
priority
pro-feature
question
raspberry-pi
raw
released
released
released
research
resolved
security
sharing
tested
tests
third-party-issue
thumbnails
upgrade
upstream-issue
ux
vector
video
waiting
won't fix
won't fix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/photoprism#2456
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @ptr727 on GitHub (Dec 19, 2025).
Originally assigned to: @ptr727 on GitHub.
Before You Continue
What Is Not Working as Documented?
I started with a discussion but now I believe this is a bug.
I have been scanning external media for more than day, and noticed that the log has not made any progress.
Looking at
htopI can see 13darktable_cliprocessing that have been running for longer than a day.Those 13 processes for those 13 files are consuming CPU but never complete.
How Can We Reproduce It?
What Behavior Do You Expect?
Scan completes successfully and starts recognizing faces.
What Could Be the Cause?
darktable-cliis hanging.Logs, Sample Files, or Screenshots
docker logs photoprism_server > photoprism.log 2>&1:photoprism.zip
Which Software Versions Do You Use?
On What Device Is PhotoPrism Installed?
Do You Use a Reverse Proxy, Firewall, VPN, or CDN?
@lastzero commented on GitHub (Dec 19, 2025):
Other than setting a timeout, there is probably not much we can do. It would be best to report this issue directly to the Darktable maintainers.
@lastzero commented on GitHub (Dec 19, 2025):
Other than setting a timeout, there is probably not much we can do. It would be best to report this issue directly to the Darktable maintainers.
@ptr727 commented on GitHub (Dec 19, 2025):
Maybe some kind of watchdog timer on calling 3rd party processes would be good, at least periodically log when still waiting for the process to return?
Do you build darktable from source or are you using a released version, if released, what version, if from source, what tag?
@ptr727 commented on GitHub (Dec 19, 2025):
I see that you are using a very old darktable 5.0.1, while the current version is 5.2.1?
@lastzero commented on GitHub (Dec 19, 2025):
If you run PhotoPrism using our official Docker image, Darktable comes preinstalled as a system package (Ubuntu 25.10 right now, until the next version is available and so on).
@ptr727 commented on GitHub (Dec 19, 2025):
Got it.
@ptr727 commented on GitHub (Dec 19, 2025):
Killing those hanging processes did not resume processing, assume the logic does not monitor for the process being killed, restarted container, waiting to see if it is going to hang again at those same files.
@gi-man commented on GitHub (Dec 20, 2025):
Since we can process the dng using the current dt, I think the correct action is to upgrade the container to a more recent version.
@ptr727 commented on GitHub (Dec 21, 2025):
I discovered thousands of "hidden" DNG files that appear to fail rendering JPEGs. So I wonder if the darktable CLI hang is just a symptom of a different issue.
I already attached full logs here, when I go to the errors tab, I see a few interesting errors:
The DB deadlock is the most interesting one.
I am on vacation so recalling from memory, will add screenshots in a few days.
@ptr727 commented on GitHub (Dec 25, 2025):
Errors:
This is not in the log I attached, I do not know where this is stored?
This, and other errors, are in the log I attached:
@lastzero commented on GitHub (Dec 25, 2025):
Could not create preview image is what is causing your issues. The database lock error may be a side effect of marking all the failed pictures as hidden/broken. As explained in our troubleshooting guide, more detailed information is available through the Docker service logs (enable trace log mode). For security reasons, you won't be able to see them through the Web UI since they might expose server internals.
@gi-man commented on GitHub (Dec 25, 2025):
I have a few questions around the intent of using darktablecli in photoprism. From the thread above, it seems like it is just using dt to generate a generic jpg from the dng (or likely any raw image). It is using the default dt processing modules, which I think it might not be ideal. I'm assuming the intent is to show a preview of the image from that raw. The colors might be muted using this approach.
I think it would be best for photoprism to just extract the embedded jpg from the raw using exiftool or exiv2 or ImageMagick. This should 1) give a better looking image (the jpg generated by the camera) and 2) likely have less cpu usage in the photoprism server.
If there is still a desire to use darktablecli, since the raw file could have an issue (eg. dng), photoprism should kill the processing if an image is not generated within X amount of time.
@lastzero commented on GitHub (Dec 25, 2025):
In some cases, extracting a thumbnail from RAW files would alternatively be possible. Do you have detailed information on which cameras and formats support this, and if all RAW will always include a JPEG with the same resolution?
@gi-man commented on GitHub (Dec 25, 2025):
I believe all raws have this and some have multiple sizes. I think you can request exittool to give a specific size.
@lastzero commented on GitHub (Dec 25, 2025):
I don't believe so, and we can't release this as a stable & tested version if we are not sure.
@ptr727 commented on GitHub (Dec 25, 2025):
I ran re-created the reported failed JPEGs using darktable while attached to the container.
This is one of the files that was "hanging":
This is one of the hidden/error images:
pictures_elmer/Sondela April 2017/IMG_1812.dngThis is one of the error HEIC images:
In all cases darktable was able to successfully create the JPEG image. (I did
"the path names with spaces).So, my guess is the problem lies elsewhere?
@ptr727 commented on GitHub (Dec 25, 2025):
I rsync'd all the DNG and troublesome HEIC files to a separate path preserving directory structure, reset photoprism with debug logging, external
:romounting only the test directory.So far DNG's are ok, will report more errors as processing continues?
The
IMG_2582.HEICfile is again reporting an error, I can open and view it fine, and I can get exif info, nothing obvious in log why a preview can't be created?The image is of an underage child so not going to share.
@ptr727 commented on GitHub (Dec 26, 2025):
After some time, all DNG's imported and created previews, so my suspicion is there is some kind of synchronization or deadlock problem that resulted in the initial issue.
Still issue with the same JPEG and HEIC files:
P1101490.JPG:
IMG_2582-3348.HEIC:
Reported error creating preview files open and render find in Windows Photos app.
@lastzero commented on GitHub (Dec 29, 2025):
@ptr727 Although I didn't find a clear problem in the Vips code, it seemed that the error buffer could fill up when an interop index was missing. The above comments therefore improve handling to avoid this issue.
You can test it using the preview builds we provide:
Should you still encounter problems converting or indexing HEIC files, try to manually run the
/usr/local/bin/heif-deccommand inside the container:heif-decdoes not work, then it is an upstream issue withlibheif.@ptr727 commented on GitHub (Dec 29, 2025):
I tested one of the failing HEIC images with the release version container:
So it seems the issue is not with
heif-dec.If you provide a private upload I will share the image with you to test?
P.s. I still get the error adding the image with the
&in the path, this is a JPEG, any troubleshooting I can do with this one?@lastzero commented on GitHub (Dec 29, 2025):
So, it seems that the "issue" is that the image is actually an image sequence. I've never encountered such a file before, so it's not surprising that it's not working as expected. Can you provide us with a test sample?
@ptr727 commented on GitHub (Dec 29, 2025):
@lastzero commented on GitHub (Dec 29, 2025):
Yes, you just need to contact us under the linked email address (or look it up under Contact on our homepage).
@ptr727 commented on GitHub (Dec 29, 2025):
Done