mirror of
https://github.com/photoprism/photoprism.git
synced 2026-03-02 22:57:18 -05:00
Videos: Improve FFmpeg parameters for Intel QSV hardware transcoding #1386
Labels
No labels
ai
android
api
auth
awesome
bug
bug
ci
cli
config
database
declined
deprecated
docker
docs 📚
documents
duplicate
easy
enhancement
enhancement
enhancement
epic
faces
feedback wanted
frontend
hacktoberfest
help wanted
idea
in-progress
incomplete
index
invalid
ios
labels
live
live
low-priority
macos
member-feature
metadata
mobile
nas
needs-analysis
no-coding-required
no-coding-required
observability
performance
places
please-test
plus-feature
priority
pro-feature
question
raspberry-pi
raw
released
released
released
research
resolved
security
sharing
tested
tests
third-party-issue
thumbnails
upgrade
upstream-issue
ux
vector
video
waiting
won't fix
won't fix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/photoprism#1386
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @sunbeam60 on GitHub (Apr 5, 2022).
Originally assigned to: @lastzero on GitHub.
The current parameters cause ffmpeg to fail on an Synology 718+ w/Intel J3455 - I will open a bug for this.
I think what is happening is that the current invocation of ffmpeg attempts to create a hardware device called qsv (-init_hw_device) which it attempts to set to just "hw" ... but as far as I understand -init_hw_device, the short-hand device name you are creating has to be set to an actual device (quoting from ffmpeg.org's site):
So I think the reason ffmpeg spits out
Failed to set value 'qsv=hw' for option 'init_hw_device': Generic error in an external libraryis that hw is not an actual device.-filter_hw_device is then trying to pass a device to all filters in ffmpeg's filter-chain ... I presume this was meant to be the 'qsv' device (which is incorrectly set), not the generic 'hw' device which doesn't actually exist. From ffmpeg.org:
So I simply cannot understand how anyone has been able to get HW decoding/encoding to work using these parameters, whatever their configuration. As far as I can decode ffmpeg's docs, this won't work on any system.
If I remove the init_hw_device and -filter_hw_device gubbins and simply invoke...
... which is PhotoPrism's invocation but without a named hardware device being passed to all filters (or at least without the attempt to create a named hardware device and pass it to all filters, ffmpeg works.
If I additionally remove the -qsv_device parameter, ffmpeg continues happily and at same speed as with the -qsv_device parameter. It's only when I switch to
-c:v h264that the speed dramatically drops due to libx264 now being used instead of the QSV hardware (3 fps instead of 20 fps).Originally posted by @sunbeam60 in https://github.com/photoprism/photoprism/issues/1337#issuecomment-1088648066
@sunbeam60 commented on GitHub (Apr 5, 2022):
https://user-images.githubusercontent.com/17337714/161757718-a5d07b8a-d266-4680-a3d8-77e21fe931db.mp4
@sunbeam60 commented on GitHub (Apr 5, 2022):
I think all that is required is to remove the following parameters
-init_hw_device qsv=hw -filter_hw_device hwwhich are erroneous.@lastzero commented on GitHub (Apr 5, 2022):
Removed
filter_hw_deviceand started a new preview build for testing... works this way on my Core i7-10700T.Thanks for your advice and becoming a sponsor! 👍
@lastzero commented on GitHub (Apr 5, 2022):
Note that this change also removes the
PHOTOPRISM_FFMPEG_BUFFERSconfig option. It turned out to be specific to thev4l2encoder commonly used on the Raspberry Pi 4. Now has a fixed value as for custom parameters of the other encoders.@sunbeam60 commented on GitHub (Apr 5, 2022):
Ran a test now and it's not good news :(
... and, indeed, when I try to play the file in the UI, the sidecar file is now obviously present and plays.
So, I was extremely cocksure that I'd found the issue, but it seems I've been proven wrong, or at least come up against another issue. Both are run inside the docker container, but something is preventing the VAAPI device being created when invoked from PhotoPrism but not when invoked from bash, inside the container.
I'll need to think this through and family calls so might return to it tonight and redo the steps above just to make sure I've not done something silly. As it stands, I've teetered over the limit of my docker + linux knowledge. Perhaps something is brought in when bashing into the container that doesn't exist outside this...
@lastzero commented on GitHub (Apr 5, 2022):
Permission issues? Running the command as root in the terminal while PhotoPrism is running as non-root user?
@sunbeam60 commented on GitHub (Apr 5, 2022):
🤦♂️
Yes, it could be. I’m certainly root when logging into the container. Not sure what that implies for PhotoPrism if so.
@sunbeam60 commented on GitHub (Apr 6, 2022):
Trying to re-run the container with:
.. and unfortunately get the same result.
But my plex container runs with a restricted user, non-root/admin, and employes ffmpeg transcoding just fine.
Any ffmpeg experts around?
@sunbeam60 commented on GitHub (Apr 6, 2022):
Couple of issues/PRs in Jellyfin that could relate to this:
https://github.com/linuxserver/docker-jellyfin/issues/111
https://github.com/linuxserver/docker-jellyfin/pull/112
... it could be a Synology specific issue, although why it works when I bash into the shell and run the same command line is still beyond me.
Will try to set permissions on the render devices (by bashing into the container, running chmod before making a video serve request on the UI), but it will have to be after work - lunch break is over :)
@sunbeam60 commented on GitHub (Apr 6, 2022):
Ok, it's probably a Synology issue and probably related to the jellyfin issues I posted in the comment above.
I had forgotten that root and admin is not the same on a Synology. When I change the PhotoPrism container to run as UID=0 (root), the ffmpeg invocation using QSV works fine and indeed the video transcodes MUCH quicker.
This is why the call succeeds when bashing into the container - one arrives as root when running
docker exec -it photoprism bash.So the answer is something along the lines of what jellyfin is doing in https://github.com/linuxserver/docker-jellyfin/pull/112/files. If you look at the comments in https://github.com/linuxserver/docker-jellyfin/issues/26:
It's exactly the same issue as I've described.
So: iterate over the drivers and ensure the current user has access to them. This is a solution specific to Synology - or at least all devices where the renderD128 doesn't have RW permissions by default.
@lastzero up to you whether you want to add a specific fix for Synology like what Jellyfin is doing.
Until then, Synology users who wish to transcode with QSV need to run as root.
@lastzero commented on GitHub (Apr 6, 2022):
We can't make any more changes for the next release. However, I took the opportunity to refactor video transcoding along the way when implementing this today:
There used to be an automatic fallback to the software encoder, which should now work again if hardware transcoding failed. Logging has been improved too. Happy indexing/testing! :)
@sunbeam60 commented on GitHub (Apr 6, 2022):
Understood - I'll open a specific improvement issue for the Synology fix so you can prioritise this one separately.
Automatic fallback is awesome 👍
@sunbeam60 commented on GitHub (Apr 6, 2022):
Synology improvement proposal here: https://github.com/photoprism/photoprism/issues/2228
@Dulanic commented on GitHub (May 28, 2022):
I am seeing issues /w QSV, it is complaining about incorrect parameters. This is an example, but it happened a bunch of times on my server.
PHOTOPRISM_INIT="gpu tensorflow"@lastzero commented on GitHub (May 28, 2022):
Would be great if you can share more details, such as your Linux kernel version / operating system, CPU type and model, and anything else that could help to reproduce this. It may also help to change the user PhotoPrism is running as in case this is a permissions issue.
@Dulanic commented on GitHub (May 28, 2022):
CPU: Intel(R) Core(TM) i5-9400
Kernel: 5.16.14-051614-generic
I can run ffmpeg as software and it runs no problem, i use my main user 1000. It's not a permission issue. Well at least a file permission problem....
Plex transcodes fines, but I know they use a customized ffmpeg. I will try to identify what the issue is, but no luck so far.
@lastzero commented on GitHub (May 28, 2022):
You could try one of our other Docker images based on a different Debian version as it could depend on that as well.
@Dulanic commented on GitHub (May 28, 2022):
No change /w bullseye or jammy.
@lastzero commented on GitHub (May 28, 2022):
Just to be sure: you installed the drivers and shred ALL required devices as shown in our example docker-compose.yml?
@Dulanic commented on GitHub (May 28, 2022):
This is my full .env file for photoprism:
My photoprism in my docker-compose.
@lastzero commented on GitHub (May 29, 2022):
What logs do you see when starting the container for the first time? You should see information about the drivers getting installed. Note init only runs once on the first start the variable was defined. Otherwise use docker-compose up --force-recreate
Also note that while I've added magic to make the installation work as UID 1000, you should generally use the PHOTOPRISM_UID env variable for switching to the UID after init as installing drivers requires the container to start as root. Starting as UID 1000 and expecting distribution packages and device access to work 100% is experimental to say the least.
@lastzero commented on GitHub (May 29, 2022):
I recommend trying as root, just to exclude the possibility of a permission or privilege issue after all, since these are the single most common cause for problems. Especially for transcoding and device access.
@rezzalex commented on GitHub (May 29, 2022):
Hello everyone,
I can't succeed to make hardware acceleration works on my NAS Asustor 5202T, the CPU is an Intel® Celeron™ CPU @ 2.00GHz
and this NAS is supposed to have hardware transcoding functions.
my log when finishing a transcode is
time="2022-05-29T07:56:10Z" level=info msg="libx264: created VID_20220526_165402.mp4.avc [13m20.938821874s]"my docker compose file contains this :
PHOTOPRISM_FFMPEG_ENCODER: "intel" PHOTOPRISM_INIT: "gpu tensorflow"I tried also previously with
PHOTOPRISM_INIT: "intel-graphics tensorflow-amd64-avx2"withtout any successhere are my logs when first starting the container :
started 220527-bookworm as root (amd64-prod) u make: Entering directory '/scripts' /scripts/install-gpu.sh Get:1 http://deb.debian.org/debian bookworm InRelease [130 kB] Get:2 http://security.debian.org/debian-security bookworm-security InRelease [35.6 kB] Get:3 http://deb.debian.org/debian bookworm-updates InRelease [36.8 kB] Get:4 http://deb.debian.org/debian bookworm-backports InRelease [37.0 kB] Get:5 http://deb.debian.org/debian bookworm/non-free amd64 Packages [99.9 kB] Get:6 http://deb.debian.org/debian bookworm/contrib amd64 Packages [55.3 kB] Get:7 http://deb.debian.org/debian bookworm/main amd64 Packages [8465 kB] Fetched 8860 kB in 2s (4292 kB/s) Reading package lists... debconf: delaying package configuration, since apt-utils is not installed (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 23816 files and directories currently installed.) Preparing to unpack .../archives/dpkg_1.21.8_amd64.deb ... Unpacking dpkg (1.21.8) over (1.21.7) ... Setting up dpkg (1.21.8) ... (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 23816 files and directories currently installed.) Preparing to unpack .../archives/lsb-base_11.2_all.deb ... Unpacking lsb-base (11.2) over (11.1.0) ... Setting up lsb-base (11.2) ... (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 23816 files and directories currently installed.) Preparing to unpack .../libcups2_2.4.2-1_amd64.deb ... Unpacking libcups2:amd64 (2.4.2-1) over (2.4.1op1-2) ... Preparing to unpack .../libldap-2.5-0_2.5.12+dfsg-2_amd64.deb ... Unpacking libldap-2.5-0:amd64 (2.5.12+dfsg-2) over (2.5.11+dfsg-1) ... Setting up libldap-2.5-0:amd64 (2.5.12+dfsg-2) ... Setting up libcups2:amd64 (2.4.2-1) ... Processing triggers for libc-bin (2.33-7) ... GPU detected: i915 Installing Intel Drivers... debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package i965-va-driver-shaders:amd64. (Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 23816 files and directories currently installed.) Preparing to unpack .../00-i965-va-driver-shaders_2.4.1-1_amd64.deb ... Unpacking i965-va-driver-shaders:amd64 (2.4.1-1) ... Selecting previously unselected package libigdgmm12:amd64. Preparing to unpack .../01-libigdgmm12_22.1.2+ds1-1_amd64.deb ... Unpacking libigdgmm12:amd64 (22.1.2+ds1-1) ... Selecting previously unselected package intel-media-va-driver-non-free:amd64. Preparing to unpack .../02-intel-media-va-driver-non-free_22.4.0+ds1-1_amd64.deb ... Unpacking intel-media-va-driver-non-free:amd64 (22.4.0+ds1-1) ... Selecting previously unselected package libllvm11:amd64. Preparing to unpack .../03-libllvm11_1%3a11.1.0-6+b2_amd64.deb ... Unpacking libllvm11:amd64 (1:11.1.0-6+b2) ... Selecting previously unselected package libigc1:amd64. Preparing to unpack .../04-libigc1_1.0.8744-4_amd64.deb ... Unpacking libigc1:amd64 (1.0.8744-4) ... Selecting previously unselected package libclang-cpp11. Preparing to unpack .../05-libclang-cpp11_1%3a11.1.0-6+b2_amd64.deb ... Unpacking libclang-cpp11 (1:11.1.0-6+b2) ... Selecting previously unselected package libllvmspirvlib11:amd64. Preparing to unpack .../06-libllvmspirvlib11_11.0.0-2+b1_amd64.deb ... Unpacking libllvmspirvlib11:amd64 (11.0.0-2+b1) ... Selecting previously unselected package libopencl-clang11:amd64. Preparing to unpack .../07-libopencl-clang11_11.0.0-2+b1_amd64.deb ... Unpacking libopencl-clang11:amd64 (11.0.0-2+b1) ... Selecting previously unselected package libigdfcl1:amd64. Preparing to unpack .../08-libigdfcl1_1.0.8744-4_amd64.deb ... Unpacking libigdfcl1:amd64 (1.0.8744-4) ... Selecting previously unselected package intel-opencl-icd. Preparing to unpack .../09-intel-opencl-icd_21.32.20609-3_amd64.deb ... Unpacking intel-opencl-icd (21.32.20609-3) ... Selecting previously unselected package libva-wayland2:amd64. Preparing to unpack .../10-libva-wayland2_2.14.0-1_amd64.deb ... Unpacking libva-wayland2:amd64 (2.14.0-1) ... Selecting previously unselected package mesa-va-drivers:amd64. Preparing to unpack .../11-mesa-va-drivers_21.3.8-1_amd64.deb ... Unpacking mesa-va-drivers:amd64 (21.3.8-1) ... Selecting previously unselected package vainfo. Preparing to unpack .../12-vainfo_2.12.0+ds1-1_amd64.deb ... Unpacking vainfo (2.12.0+ds1-1) ... Setting up i965-va-driver-shaders:amd64 (2.4.1-1) ... Setting up libigdgmm12:amd64 (22.1.2+ds1-1) ... Setting up libllvm11:amd64 (1:11.1.0-6+b2) ... Setting up libva-wayland2:amd64 (2.14.0-1) ... Setting up mesa-va-drivers:amd64 (21.3.8-1) ... Setting up libigc1:amd64 (1.0.8744-4) ... Setting up intel-media-va-driver-non-free:amd64 (22.4.0+ds1-1) ... Setting up libllvmspirvlib11:amd64 (11.0.0-2+b1) ... Setting up libclang-cpp11 (1:11.1.0-6+b2) ... Setting up vainfo (2.12.0+ds1-1) ... Setting up libopencl-clang11:amd64 (11.0.0-2+b1) ... Setting up libigdfcl1:amd64 (1.0.8744-4) ... Setting up intel-opencl-icd (21.32.20609-3) ... Processing triggers for libc-bin (2.33-7) ... Done. make: Leaving directory '/scripts' init: tensorflow make: Entering directory '/scripts' /scripts/install-tensorflow.sh auto Detecting driver... Installing TensorFlow 1.15.2 for AMD64 in "/usr"... Downloading amd64 libs from "https://dl.photoprism.app/tensorflow/amd64/libtensorflow-amd64-1.15.2.tar.gz". Please wait. Extracting "/tmp/amd64/libtensorflow-amd64-1.15.2.tar.gz" to "/usr". Running "ldconfig". Done. make: Leaving directory '/scripts' Problems? Our Troubleshooting Checklists help you quickly diagnose and solve them: https://docs.photoprism.app/getting-started/troubleshooting/ file umask....: "0002" (u=rwx,g=rwx,o=rx) home directory: /photoprism assets path...: /opt/photoprism/assets storage path..: /photoprism/storage config path...: default cache path....: default backup path...: /photoprism/storage/backups import path...: /photoprism/import originals path: /photoprism/originals switching to uid 999:999 /opt/photoprism/bin/photoprism start@lastzero commented on GitHub (May 29, 2022):
Hardware transcoding is one of the special features that causes a very high support and maintenance overhead for our team. To account for that, it's currently only available to sponsors.
@Dulanic commented on GitHub (May 29, 2022):
Changed to root...
runnng convert to get more low level details now...
@Dulanic commented on GitHub (May 29, 2022):
This took longer than expected.... I think running it as root worked... so I assume don't use user in docker-compose lol.
@Dulanic commented on GitHub (May 29, 2022):
One last follow up, using PHOTOPRISM_UID causes it to break again so Ill just stay root.
@blue-kaleidoscope commented on GitHub (Jun 16, 2022):
Hi, I'm also having issues with using QSV transcoding on my QNAP-TS451+ which has an Intel Celeron J1900. According to Intel it supports QSV.
Using QNAP's (catastrophic) QPhoto I can transcode iPhone videos quite fast. So I assume QSV is working there. The only thing to watch out: You have to use QPhoto as a root user.
I believe Photoprism is by far a better solution but I could use your help with hardware transcoding.
When executing ffmpeg as root user inside the container as follows I get an error message:
An excerpt of my
docker-compose.ymllooks as follows:When powering up photoprism I see the following messages:
Am I doing everything correctly? It looks like I am running it as root. Looking forward to your support.