Corrupt transcoded video with rkmpp decoder. #4154

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Oct 31, 2024):

Hi, Are you sure this is fixed?
You mentioned in #13785 that HW decoding is untested.
The problem I reported only affects HW decoding.

@kaaku3 commented on GitHub (Oct 31, 2024): Hi, Are you sure this is fixed? You mentioned in #13785 that HW decoding is untested. The problem I reported only affects HW decoding.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Oct 31, 2024):

I tested this separately and I think it's caused by this issue. You're most likely seeing tone-mapping artifacts that will be fixed by setting the peak luminance explicitly. Since the linked PR does just this, it should fix the issue.

@mertalev commented on GitHub (Oct 31, 2024): I tested this separately and I think it's caused by [this](https://github.com/jellyfin/jellyfin-ffmpeg/issues/482) issue. You're most likely seeing tone-mapping artifacts that will be fixed by setting the peak luminance explicitly. Since the linked PR does just this, it should fix the issue.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Oct 31, 2024):

Thankyou very much for getting back to me, I appreciate all the hard work.

@kaaku3 commented on GitHub (Oct 31, 2024): Thankyou very much for getting back to me, I appreciate all the hard work.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 7, 2024):

HI, Unfortunately I do still have the same problem on hardware decoded transcoded videos in v1.120.
It has significantly improved but is still happening in low light videos

@kaaku3 commented on GitHub (Nov 7, 2024): HI, Unfortunately I do still have the same problem on hardware decoded transcoded videos in v1.120. It has significantly improved but is still happening in low light videos

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 7, 2024):

Hmm, I guess peak=100 (1000 nits) is off for certain videos. Can you share the output of mediainfo <path> or ffprobe -show_streams <path> for an affected video? Maybe we can use a different number based on certain metadata.

@mertalev commented on GitHub (Nov 7, 2024): Hmm, I guess peak=100 (1000 nits) is off for certain videos. Can you share the output of `mediainfo <path>` or `ffprobe -show_streams <path>` for an affected video? Maybe we can use a different number based on certain metadata.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 7, 2024):

Also, are the videos experiencing issues in 1.120.0 the same videos that looked corrupt in earlier versions? Or do different videos look corrupt now?

@mertalev commented on GitHub (Nov 7, 2024): Also, are the videos experiencing issues in 1.120.0 the same videos that looked corrupt in earlier versions? Or do different videos look corrupt now?

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 8, 2024):

Media Info

General
Complete name : PXL_20241105_062315977.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 54.8 MiB
Duration : 7 s 250 ms
Overall bit rate : 63.4 Mb/s
Frame rate : 59.859 FPS
Encoded date : 2024-11-05 06:23:25 UTC
Tagged date : 2024-11-05 06:23:25 UTC
xyz : +35.6247+139.4298/
com.android.manufacturer : Google
com.android.model : Pixel 6a

Video
ID : 3
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L5.1@High
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 7 s 250 ms
Bit rate : 63.1 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Rotation : 90°
Frame rate mode : Variable
Frame rate : 59.859 FPS
Minimum frame rate : 59.367 FPS
Maximum frame rate : 60.362 FPS
Real frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.127
Stream size : 54.5 MiB (100%)
Title : VideoHandle
Language : English
Encoded date : 2024-11-05 06:23:25 UTC
Tagged date : 2024-11-05 06:23:25 UTC
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 7 s 243 ms
Bit rate mode : Constant
Bit rate : 192 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 170 KiB (0%)
Title : SoundHandle
Language : English
Encoded date : 2024-11-05 06:23:25 UTC
Tagged date : 2024-11-05 06:23:25 UTC

Other
Type : meta
Duration : 7 s 250 ms
Bit rate mode : Variable

@kaaku3 commented on GitHub (Nov 8, 2024): Media Info General Complete name : PXL_20241105_062315977.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/mp41) File size : 54.8 MiB Duration : 7 s 250 ms Overall bit rate : 63.4 Mb/s Frame rate : 59.859 FPS Encoded date : 2024-11-05 06:23:25 UTC Tagged date : 2024-11-05 06:23:25 UTC xyz : +35.6247+139.4298/ com.android.manufacturer : Google com.android.model : Pixel 6a Video ID : 3 Format : HEVC Format/Info : High Efficiency Video Coding Format profile : Main@L5.1@High Codec ID : hvc1 Codec ID/Info : High Efficiency Video Coding Duration : 7 s 250 ms Bit rate : 63.1 Mb/s Width : 3 840 pixels Height : 2 160 pixels Display aspect ratio : 16:9 Rotation : 90° Frame rate mode : Variable Frame rate : 59.859 FPS Minimum frame rate : 59.367 FPS Maximum frame rate : 60.362 FPS Real frame rate : 60.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Bits/(Pixel*Frame) : 0.127 Stream size : 54.5 MiB (100%) Title : VideoHandle Language : English Encoded date : 2024-11-05 06:23:25 UTC Tagged date : 2024-11-05 06:23:25 UTC Color range : Full Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709 Codec configuration box : hvcC Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Codec ID : mp4a-40-2 Duration : 7 s 243 ms Bit rate mode : Constant Bit rate : 192 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 170 KiB (0%) Title : SoundHandle Language : English Encoded date : 2024-11-05 06:23:25 UTC Tagged date : 2024-11-05 06:23:25 UTC Other Type : meta Duration : 7 s 250 ms Bit rate mode : Variable

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 8, 2024):

Hmm, it doesn't seem like this is an HDR video. Could you follow these steps:

Change immich's log level to debug in the admin settings under Log Settings
Run transcoding for this asset, making sure you have hardware decoding enabled
In the terminal, run docker logs immich_server
Find the latest log that shows an ffmpeg command (has /usr/bin/ffmpeg in it)

I'm curious to see what the specific command is to see where this is going wrong.

@mertalev commented on GitHub (Nov 8, 2024): Hmm, it doesn't seem like this is an HDR video. Could you follow these steps: 1. Change immich's log level to debug in the admin settings under Log Settings 2. Run transcoding for this asset, making sure you have hardware decoding enabled 3. In the terminal, run `docker logs immich_server` 4. Find the latest log that shows an `ffmpeg` command (has `/usr/bin/ffmpeg` in it) I'm curious to see what the specific command is to see where this is going wrong.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

I set log level to debug, but unfortunately i do not see ffmpeg anything.
I tried verbose and the only thing I see that is remotely related in the log is:
[Nest] 17 - 11/09/2024, 1:18:31 PM VERBOSE [Api:LoggingInterceptor~p0aolibi] {"assetIds":[],"name":"transcode-video"}

@kaaku3 commented on GitHub (Nov 9, 2024): I set log level to debug, but unfortunately i do not see ffmpeg anything. I tried verbose and the only thing I see that is remotely related in the log is: [Nest] 17 - 11/09/2024, 1:18:31 PM VERBOSE [Api:LoggingInterceptor~p0aolibi] {"assetIds":[],"name":"transcode-video"}

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

Disregard that, the problem was the option "refresh encoded videos" is not re-encoding the video... maybe i misunderstood the meaning of that feature.

I made a new video in low light and uploaded it with hardware decoding on, the result was corruption towards the end of the file.

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4

I then renamed the file and reuploaded it with hardware decoding off and the resulting file was fine.

ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4

Media info for file

General
Complete name : Kirk/2024/2024-11-09/reupload.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 78.3 MiB
Duration : 10 s 515 ms
Overall bit rate : 62.5 Mb/s
Frame rate : 59.536 FPS
Movie name : Re-upload
Encoded date : 2024-11-09 13:35:28 UTC
Tagged date : 2024-11-09 13:35:28 UTC
xyz : +35.6879+139.4576/
com.android.model : Pixel 7a
com.android.manufacturer : Google

Video
ID : 3
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 10 s 515 ms
Bit rate : 62.2 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Rotation : 90°
Frame rate mode : Variable
Frame rate : 59.536 FPS
Minimum frame rate : 42.959 FPS
Maximum frame rate : 96.774 FPS
Real frame rate : 60.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.126
Stream size : 78.0 MiB (100%)
Language : English
Encoded date : 2024-11-09 13:35:28 UTC
Tagged date : 2024-11-09 13:35:28 UTC
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 10 s 505 ms
Bit rate mode : Constant
Bit rate : 192 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 246 KiB (0%)
Language : English
Encoded date : 2024-11-09 13:35:28 UTC
Tagged date : 2024-11-09 13:35:28 UTC

Other
Type : meta
Duration : 10 s 515 ms
Bit rate mode : Variable

I think you are right they are not HDR, but with the latest update corruption only seems to happen in low light videos, where as before the update it seemed more random.

edit. Just ran a few tests and the artifacts and problems seem to happen when in low light but then a strong light source appears in the video, causing looped video, artifacts and strange colours.

@kaaku3 commented on GitHub (Nov 9, 2024): Disregard that, the problem was the option "refresh encoded videos" is not re-encoding the video... maybe i misunderstood the meaning of that feature. I made a new video in low light and uploaded it with hardware decoding on, the result was corruption towards the end of the file. ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4 I then renamed the file and reuploaded it with hardware decoding off and the resulting file was fine. ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4 Media info for file General Complete name : Kirk/2024/2024-11-09/reupload.mp4 Format : MPEG-4 Format profile : Base Media Codec ID : isom (isom/iso2/mp41) File size : 78.3 MiB Duration : 10 s 515 ms Overall bit rate : 62.5 Mb/s Frame rate : 59.536 FPS Movie name : Re-upload Encoded date : 2024-11-09 13:35:28 UTC Tagged date : 2024-11-09 13:35:28 UTC xyz : +35.6879+139.4576/ com.android.model : Pixel 7a com.android.manufacturer : Google Video ID : 3 Format : HEVC Format/Info : High Efficiency Video Coding Format profile : Main@L6@Main Codec ID : hvc1 Codec ID/Info : High Efficiency Video Coding Duration : 10 s 515 ms Bit rate : 62.2 Mb/s Width : 3 840 pixels Height : 2 160 pixels Display aspect ratio : 16:9 Rotation : 90° Frame rate mode : Variable Frame rate : 59.536 FPS Minimum frame rate : 42.959 FPS Maximum frame rate : 96.774 FPS Real frame rate : 60.000 FPS Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Bits/(Pixel*Frame) : 0.126 Stream size : 78.0 MiB (100%) Language : English Encoded date : 2024-11-09 13:35:28 UTC Tagged date : 2024-11-09 13:35:28 UTC Color range : Full Color primaries : BT.709 Transfer characteristics : BT.709 Matrix coefficients : BT.709 Codec configuration box : hvcC Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Codec ID : mp4a-40-2 Duration : 10 s 505 ms Bit rate mode : Constant Bit rate : 192 kb/s Channel(s) : 2 channels Channel layout : L R Sampling rate : 48.0 kHz Frame rate : 46.875 FPS (1024 SPF) Compression mode : Lossy Stream size : 246 KiB (0%) Language : English Encoded date : 2024-11-09 13:35:28 UTC Tagged date : 2024-11-09 13:35:28 UTC Other Type : meta Duration : 10 s 515 ms Bit rate mode : Variable I think you are right they are not HDR, but with the latest update corruption only seems to happen in low light videos, where as before the update it seemed more random. edit. Just ran a few tests and the artifacts and problems seem to happen when in low light but then a strong light source appears in the video, causing looped video, artifacts and strange colours.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 9, 2024):

In the transcoding settings under Advanced, could you set the keyframe interval to 60? I wonder if that has an effect here.

@mertalev commented on GitHub (Nov 9, 2024): In the transcoding settings under Advanced, could you set the keyframe interval to 60? I wonder if that has an effect here.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

Wow yes, its seems fine with "Maximum keyframe interval" set to 60, thank you so much.
Is it bad to keep it at 60 as opposed to auto?

@kaaku3 commented on GitHub (Nov 9, 2024): Wow yes, its seems fine with "Maximum keyframe interval" set to 60, thank you so much. Is it bad to keep it at 60 as opposed to auto?

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 9, 2024):

It will make the videos very slightly bigger, but otherwise no downside as far as I know.

@mertalev commented on GitHub (Nov 9, 2024): It will make the videos very slightly bigger, but otherwise no downside as far as I know.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

Ok, thankyou.
keeping the file size down was the reason i'm transcoding at all, as my mobile data plan is slow sometimes.

Not sure if it's related but i just noticed a very very thin line at the top and bottom of the video that is not there when I use software decoding.

@kaaku3 commented on GitHub (Nov 9, 2024): Ok, thankyou. keeping the file size down was the reason i'm transcoding at all, as my mobile data plan is slow sometimes. Not sure if it's related but i just noticed a very very thin line at the top and bottom of the video that is not there when I use software decoding.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 9, 2024):

The default in your case is 256, for reference. Can you try setting it to, say, 128? I'm thinking of changing the default for RKMPP to remedy this, but also trying to minimize the impact to file size.

@mertalev commented on GitHub (Nov 9, 2024): The default in your case is 256, for reference. Can you try setting it to, say, 128? I'm thinking of changing the default for RKMPP to remedy this, but also trying to minimize the impact to file size.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 9, 2024):

cc @nyanmisaka if you have any insight into what's causing this behavior

@mertalev commented on GitHub (Nov 9, 2024): cc @nyanmisaka if you have any insight into what's causing this behavior

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

After deleting and re-uploading the same to files one a dark video and one bright.
with 60 still sometimes is causing an issue on both bright and dark video.
128 was very bad.
Haven't seen a problem with it set to 32 after a few uploads.
The line I mentioned only affects hardware decoded videos and only visible on the android mobile app, not on web.

@kaaku3 commented on GitHub (Nov 9, 2024): After deleting and re-uploading the same to files one a dark video and one bright. with 60 still sometimes is causing an issue on both bright and dark video. 128 was very bad. Haven't seen a problem with it set to 32 after a few uploads. The line I mentioned only affects hardware decoded videos and only visible on the android mobile app, not on web.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

Since software decode is working fine with rkmpp hardware encode on its defaults, is there not any settings to play with on the hardware decode side?
Just after some googling it seems not everyone is getting perfect results with ffmpeg and drm_prime, could that be the problem?
The hardware decoder or ffmpegs compatibility with drm_prime? I don't really understand any of this but is there an alternative to drm_prime with rkmpp?

@kaaku3 commented on GitHub (Nov 9, 2024): Since software decode is working fine with rkmpp hardware encode on its defaults, is there not any settings to play with on the hardware decode side? Just after some googling it seems not everyone is getting perfect results with ffmpeg and drm_prime, could that be the problem? The hardware decoder or ffmpegs compatibility with drm_prime? I don't really understand any of this but is there an alternative to drm_prime with rkmpp?

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 9, 2024):

A max of 32 for a 60 FPS video is a keyframe roughly every 0.5s. This is okay, but I think it's the lowest that's reasonable to use.

What's particularly weird is that this is an encoding setting - it shouldn't be fixed by using software decoding. It's a combination of the frames being outputted by hardware decoding and the keyframe interval that's causing this issue.

I think drm_prime is needed for good performance so filters can directly access the frames in the GPU without copying them.

@mertalev commented on GitHub (Nov 9, 2024): A max of 32 for a 60 FPS video is a keyframe roughly every 0.5s. This is okay, but I think it's the lowest that's reasonable to use. What's particularly weird is that this is an encoding setting - it shouldn't be fixed by using software decoding. It's a combination of the frames being outputted by hardware decoding and the keyframe interval that's causing this issue. I think drm_prime is needed for good performance so filters can directly access the frames in the GPU without copying them.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

I'd like to help in anyway I can but I'm not very knowledgeable but i can follow instructions, do any of the devs have access to an rkmpp device?

@kaaku3 commented on GitHub (Nov 9, 2024): I'd like to help in anyway I can but I'm not very knowledgeable but i can follow instructions, do any of the devs have access to an rkmpp device?

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

I hope it doesn't seem like i'm complaining about the issue too much, I understand you must be busy on other things.
I love immich and i'm just grateful that rkmpp encode works at all, otherwise i'd have to buy a more power hungry device.

@kaaku3 commented on GitHub (Nov 9, 2024): I hope it doesn't seem like i'm complaining about the issue too much, I understand you must be busy on other things. I love immich and i'm just grateful that rkmpp encode works at all, otherwise i'd have to buy a more power hungry device.

deekerman commented

Author

Owner

@nyanmisaka commented on GitHub (Nov 9, 2024):

What’s the SoC model and rockchip linux kernel version? (uname -a)

Can u share a sample clip and the full ffmpeg command line?

I may have time to take a look tomorrow.

@nyanmisaka commented on GitHub (Nov 9, 2024): What’s the SoC model and rockchip linux kernel version? (uname -a) Can u share a sample clip and the full ffmpeg command line? I may have time to take a look tomorrow.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 9, 2024):

No worries! It's great that you're reporting on this so we know about it and can fix it.

I think the best option would be to open an issue in jellyfin-ffmpeg as I believe most/all of the RKMPP decoders, filters and encoders for FFmpeg are implemented by them. A zipped sample video would be particularly helpful.

@mertalev commented on GitHub (Nov 9, 2024): No worries! It's great that you're reporting on this so we know about it and can fix it. I think the best option would be to open an issue in [jellyfin-ffmpeg](https://github.com/jellyfin/jellyfin-ffmpeg) as I believe most/all of the RKMPP decoders, filters and encoders for FFmpeg are implemented by them. A zipped sample video would be particularly helpful.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 9, 2024):

According to OP, this command is problematic:

ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4

This command works fine:

ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4

And changing the first command to use -g 32 instead of -g 256 also works fine.

Also, it seems there's a line at the top and bottom of the video when hardware decoding that doesn't happen with software decoding. I think this happens with -g 32 as well, but I'm not sure.

@mertalev commented on GitHub (Nov 9, 2024): According to OP, this command is problematic: ``` ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/library/Kirk/2024/2024-11-09/PXL_20241109_133516985.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/f2/b0/f2b017d0-38f9-4331-8f71-d7c71e48b0c2.mp4 ``` This command works fine: ``` ffmpeg -i upload/library/Kirk/2024/2024-11-09/reupload.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale=720:-2 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/c160480f-610d-42d9-8cea-829489f5a58e/8d/4e/8d4e5afc-9692-4169-b813-67ba68b96fdd.mp4 ``` And changing the first command to use `-g 32` instead of `-g 256` also works fine. Also, it seems there's a line at the top and bottom of the video when hardware decoding that doesn't happen with software decoding. I think this happens with `-g 32` as well, but I'm not sure.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 9, 2024):

uname -a
Linux immich-server 5.10.160-legacy-rk35xx #1 SMP Wed May 15 03:04:45 UTC 2024 aarch64 GNU/Linux
Its actually a rk3588s.
This is my sample video of a messy (excuse the mess) dark room that constantly causes issues.
2 files one with -g256 and one with -g 32.
In this instance -g 32 is also corrupt (up to this point -g32 was actually fine)
It seems hit or miss.

Thankyou for the command line noted above and the description is perfect, the line thing happens no matter the -g setting and is only visible in mobile app (android) maybe web viewer clips it off?

@kaaku3 commented on GitHub (Nov 9, 2024): uname -a Linux immich-server 5.10.160-legacy-rk35xx #1 SMP Wed May 15 03:04:45 UTC 2024 aarch64 GNU/Linux Its actually a rk3588s. This is my sample video of a messy (excuse the mess) dark room that constantly causes issues. 2 files one with -g256 and one with -g 32. In this instance -g 32 is also corrupt (up to this point -g32 was actually fine) It seems hit or miss. Thankyou for the command line noted above and the description is perfect, the line thing happens no matter the -g setting and is only visible in mobile app (android) maybe web viewer clips it off?

deekerman commented

Author

Owner

@w00tlarr commented on GitHub (Nov 10, 2024):

Hiya - running into this issue too but with Intel iGPU hardware acceleration. No issues encoding without iGPU.

@w00tlarr commented on GitHub (Nov 10, 2024): Hiya - running into this issue too but with Intel iGPU hardware acceleration. No issues encoding without iGPU.

deekerman commented

https://github.com/user-attachments/assets/2c0df285-4d35-440c-94cf-ba0dae5469d1

Author

Owner

@w00tlarr commented on GitHub (Nov 10, 2024):

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.

@w00tlarr commented on GitHub (Nov 10, 2024): Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync. https://github.com/user-attachments/assets/2c0df285-4d35-440c-94cf-ba0dae5469d1

deekerman commented

2026-02-20 03:01:09 -05:00

Author

Owner

@mertalev commented on GitHub (Nov 10, 2024):

Thanks for the report, but I don't think this is the same issue. The issue in question should be specific to RKMPP and (I think) fixed by this PR.

Please make a separate issue for this. You can set the log level to debug under Log Settings, then select Refresh encoded videos for an affected asset. Find the FFmpeg command used for this video in the server logs with docker logs immich_server. In the issue, provide a sample video that causes this output and the accompanying FFmpeg command. Also include information about the OS, kernel version and specific processor.

@mertalev commented on GitHub (Nov 10, 2024): Thanks for the report, but I don't think this is the same issue. The issue in question should be specific to RKMPP and (I think) fixed by [this](https://github.com/jellyfin/jellyfin-ffmpeg/pull/498) PR. Please make a separate issue for this. You can set the log level to debug under Log Settings, then select `Refresh encoded videos` for an affected asset. Find the FFmpeg command used for this video in the server logs with `docker logs immich_server`. In the issue, provide a sample video that causes this output and the accompanying FFmpeg command. Also include information about the OS, kernel version and specific processor.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 10, 2024):

Great to hear a fix is on the way, thankyou so much.
(I was beginning to wonder if my chip was faulty)

@kaaku3 commented on GitHub (Nov 10, 2024): Great to hear a fix is on the way, thankyou so much. (I was beginning to wonder if my chip was faulty)

deekerman commented

2026-02-20 03:01:09 -05:00

Author

Owner

@kaaku3 commented on GitHub (Nov 10, 2024):

So the reason it didn't happen with software decode was because it was slower?

@kaaku3 commented on GitHub (Nov 10, 2024): So the reason it didn't happen with software decode was because it was slower?

deekerman commented

https://github.com/intel/media-driver/issues/1665

Author

Owner

@nyanmisaka commented on GitHub (Nov 11, 2024):

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.

playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg.

@nyanmisaka commented on GitHub (Nov 11, 2024): > Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync. > > playback.mp4 This is a completely different issue. Due to the driver/kernel rather than FFmpeg. https://github.com/intel/media-driver/issues/1665

deekerman commented

Author

Owner

@nyanmisaka commented on GitHub (Nov 11, 2024):

Great to hear a fix is on the way, thankyou so much. (I was beginning to wonder if my chip was faulty)

The version in the link above should fix this artifact. You can try to install it manually and try again.

As for the green line I couldn't reproduce it in Chrome, PotPlayer, MPC-BE and even FFplay. So it is a problem specific to the built-in video player of your client platform.

720x406 hevc main

@nyanmisaka commented on GitHub (Nov 11, 2024): > Great to hear a fix is on the way, thankyou so much. (I was beginning to wonder if my chip was faulty) The version in the link above should fix this artifact. You can try to install it manually and try again. As for the green line I couldn't reproduce it in Chrome, PotPlayer, MPC-BE and even FFplay. So it is a problem specific to the built-in video player of your client platform. 720x406 hevc main ![720x406](https://github.com/user-attachments/assets/f5ffe03c-5e12-4bb9-a5d4-2e7cdb987908)

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 11, 2024):

Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update.
The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded.

You can see it here top left, and bottom middle in this screenshot

@kaaku3 commented on GitHub (Nov 11, 2024): Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update. The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded. You can see it here top left, and bottom middle in this screenshot ![Screenshot_20241111-175451](https://github.com/user-attachments/assets/36069bd0-197b-4300-b43e-03124d416c6f)

deekerman commented

Author

Owner

@nyanmisaka commented on GitHub (Nov 11, 2024):

Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update. The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded.

You can see it here top left, and bottom middle in this screenshot

It's best to switch to a different client and try again.

@nyanmisaka commented on GitHub (Nov 11, 2024): > Thankyou, I'm not nearly versed enough with docker to update ffmpeg from source,I will wait for this to reach immich in an update. The line, I was only about to reproduce in the immich app and only with files hardware decided and encoded. > > You can see it here top left, and bottom middle in this screenshot It's best to switch to a different client and try again.

deekerman commented

Author

Owner

@nyanmisaka commented on GitHub (Nov 11, 2024):

For example, I cannot reproduce it on iPhone/iOS 17.

@nyanmisaka commented on GitHub (Nov 11, 2024): For example, I cannot reproduce it on iPhone/iOS 17. ![IMG_8534](https://github.com/user-attachments/assets/e6230fc3-ecd3-4e1c-ad03-0ff264442c19)

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 11, 2024):

Ok, I will try it with the updated immich client, once the fix reaches it maybe it's not an issue anymore.
If it's still an issue I will open up an issue with the immich mobile app for android.

@kaaku3 commented on GitHub (Nov 11, 2024): Ok, I will try it with the updated immich client, once the fix reaches it maybe it's not an issue anymore. If it's still an issue I will open up an issue with the immich mobile app for android.

deekerman commented

Author

Owner

@w00tlarr commented on GitHub (Nov 11, 2024):

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.
playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg.

intel/media-driver#1665

Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR.

Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration.

Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed.

@w00tlarr commented on GitHub (Nov 11, 2024): > > Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync. > > playback.mp4 > > This is a completely different issue. Due to the driver/kernel rather than FFmpeg. > > [intel/media-driver#1665](https://github.com/intel/media-driver/issues/1665) Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR. Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration. Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed.

deekerman commented

Author

Owner

@nyanmisaka commented on GitHub (Nov 12, 2024):

Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync.
playback.mp4

This is a completely different issue. Due to the driver/kernel rather than FFmpeg.
intel/media-driver#1665

Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR.

Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration.

Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed.

I don't have an Intel GPU that supports SRIOV at the moment (Intel ARC A380), you could try looking in the Intel repo to provide some context. But anyway this issue is not related to the downstream software, as the same FFmpeg commands run perfectly on the host.

@nyanmisaka commented on GitHub (Nov 12, 2024): > > > Sample video after Intel iGPU hardware transcode using VAAPI or QuickSync. > > > playback.mp4 > > > > > > This is a completely different issue. Due to the driver/kernel rather than FFmpeg. > > [intel/media-driver#1665](https://github.com/intel/media-driver/issues/1665) > > Thanks. Yeah it's that bug with the Intel Media Driver link you provided. Not sure if I should open a separate bug now as it seems to be a very very old issue with Proxmox + SRIOV + my new iPhone 16 with HDR videos. I just noticed this issue because of my new iPhone is recording HDR videos vs. my old iPhone XR. > > Disabling the Tone-Mapping in the Immich Video Transcoding setting now works properly with hardware acceleration. > > Anyways, sorry for hijacking this thread as it read like the same issue with me. I can open a new bug thread for tips to other folks, if needed. I don't have an Intel GPU that supports SRIOV at the moment (Intel ARC A380), you could try looking in the Intel repo to provide some context. But anyway this issue is not related to the downstream software, as the same FFmpeg commands run perfectly on the host.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 12, 2024):

@kaaku3 The bug fix for the RKMPP issue should be in 1.120.2. Can you try with that?

@mertalev commented on GitHub (Nov 12, 2024): @kaaku3 The bug fix for the RKMPP issue should be in 1.120.2. Can you try with that?

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 12, 2024):

@mertalev that's great news, unfortunately my armbian server that immich was running went down last night.
I didn't touch anything, I don't know why.
You think publicly putting out a link to a video on immich was a bad idea?
I'm sure people have better things to do then mess with my server probably over thinking it.
Anyway I think I will need to put the immich backup restore feature to use today.
I will let you know if it's fixed as soon as i can.
Thankyou fire your help.

@kaaku3 commented on GitHub (Nov 12, 2024): @mertalev that's great news, unfortunately my armbian server that immich was running went down last night. I didn't touch anything, I don't know why. You think publicly putting out a link to a video on immich was a bad idea? I'm sure people have better things to do then mess with my server probably over thinking it. Anyway I think I will need to put the immich backup restore feature to use today. I will let you know if it's fixed as soon as i can. Thankyou fire your help.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 12, 2024):

Sorry to hear that! There are lots of reasons it could fail, like power loss. But in general, make sure the server is hardened with a reverse proxy, fail2ban, etc. if you're exposing it to the internet. Definitely don't expose Immich (or any other service) directly.

@mertalev commented on GitHub (Nov 12, 2024): Sorry to hear that! There are lots of reasons it could fail, like power loss. But in general, make sure the server is hardened with a reverse proxy, fail2ban, etc. if you're exposing it to the internet. Definitely don't expose Immich (or any other service) directly.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 12, 2024):

I was using nginx, I will look into fail2ban too thankyou.

@kaaku3 commented on GitHub (Nov 12, 2024): I was using nginx, I will look into fail2ban too thankyou.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 13, 2024):

Hi, I started fresh. I had a lot of problems seems the newer kernel don't fully support hardware acceleration... i'm not quite sure.
I went with Joshua Rieks ubuntu with legacy kernel, that apparent everything should work out of the box, but i had problems with opencl. It seems to be working now, but I'm not 100%. Is there anyway from immich log to tell if its falling back to software encode. It doesn't seem to be going as fast as it was before. Also immich restore didn't work for me so i'm uploading everything.

this is the ffmpeg from immich log

ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/a7/f0/a7f09b0f-653d-4920-8a95-ec929b890cc8.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/f9/57/f9578312-e623-4a61-9667-607baed5a02e.mp4

anyway to tell from that that it isn't falling back to software?
I'm currently waiting on the problem file to be transcoded.

edit.

I there are errors

[Nest] 7 - 11/14/2024, 1:30:47 AM ERROR [Microservices:MediaRepository] handler_name : SoundHandle
vendor_id : [0][0][0][0]
Stream #0:20x3: Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), yuvj420p(pc, bt709, left), 3840x2160, 62337 kb/s, SAR 1:1 DAR 16:9, 59.86 fps, 59.94 tbr, 90k tbn (default)
Metadata:
creation_time : 2024-09-27T11:15:49.000000Z
handler_name : VideoHandle
vendor_id : [0][0][0][0]
[out#0/mp4 @ 0x3b53a170fc0] Adding streams from explicit maps...
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Created video stream from input stream 0:2
[hevc_rkmpp @ 0x3b53a2d1d80] Found SoC name from device-tree: 'rockchip,rk3588s-orangepi-5 rockchip,rk3588'
[hevc_rkmpp @ 0x3b53a2d1d80] Picked up an existing RKMPP hardware device
[aost#0:1/aac @ 0x3b53a1f0780] Created audio stream from input stream 0:1
Stream mapping:
Stream #0:2 -> #0:0 (hevc (hevc_rkmpp) -> hevc (hevc_rkmpp))
Stream #0:1 -> #0:1 (aac (native) -> aac (native))
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Starting thread...
[aost#0:1/aac @ 0x3b53a1f0780] Starting thread...
[vf#0:0 @ 0x3b53a127d40] Starting thread...
[af#0:1 @ 0x3b53a127e80] Starting thread...
[vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Starting thread...
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Starting thread...
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Starting thread...
Press [q] to stop, [?] for help
[hevc_rkmpp @ 0x3b53a2d1d80] Noticed an info change
[hevc_rkmpp @ 0x3b53a2d1d80] Configured with size: 3840x2160 | pix_fmt: drm_prime | sw_pix_fmt: nv12
[graph 0 input from stream 0:2 @ 0x3b542090300] w:3840 h:2160 pixfmt:drm_prime tb:1/90000 fr:60000/1001 sar:1/1 csp:bt709 range:pc
[Parsed_scale_rkrga_0 @ 0x3b542090240] w:3840 h:2160 fmt:nv12 -> w:1280 h:720 fmt:nv12
[graph 0 input from stream 0:2 @ 0x3b542090300] video frame properties congruent with link at pts_time: 0
[hevc_rkmpp @ 0x3b53a2d0500] Using input frames context (format drm_prime) with hevc_rkmpp encoder.
[hevc_rkmpp @ 0x3b53a2d0500] Rate Control mode is set to CQP
[hevc_rkmpp @ 0x3b53a2d0500] QP Init/Max/Min/Max_I/Min_I is set to 28/28/28/28/28
[hevc_rkmpp @ 0x3b53a2d0500] Tier is set to 1
[hevc_rkmpp @ 0x3b53a2d0500] Profile is set to MAIN
[hevc_rkmpp @ 0x3b53a2d0500] Level is set to 51
[hevc_rkmpp @ 0x3b53a2d0500] Configured with size: 1280x720 | pix_fmt: drm_prime | sw_pix_fmt: nv12
[graph_1_in_0:1 @ 0x3b540090300] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:stereo
Output #0, mp4, to 'upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4':
Metadata:
major_brand : isom
minor_version : 131072
compatible_brands: isomiso2mp41
com.android.capture.fps: 60.000000
location : +35.6813+139.5357/
location-eng : +35.6813+139.5357/
com.android.manufacturer: Google
com.android.model: Pixel 6a
encoder : Lavf61.1.100
Stream #0:0(eng): Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), drm_prime(pc, bt709, progressive, left), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 59.94 fps, 60k tbn (default)
Metadata:
creation_time : 2024-09-27T11:15:49.000000Z
handler_name : VideoHandle
vendor_id : [0][0][0][0]
encoder : Lavc61.3.100 hevc_rkmpp
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default)
Metadata:
creation_time : 2024-09-27T11:15:49.000000Z
handler_name : SoundHandle
vendor_id : [0][0][0][0]
encoder : Lavc61.3.100 aac
[out#0/mp4 @ 0x3b53a170fc0] Starting thread...
frame= 138 fps=0.0 q=-0.0 size= 1024KiB time=00:00:02.02 bitrate=4133.1kbits/s speed=4.06x
[hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error submitting video frame to the encoder
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error encoding a frame: Generic error in an external library
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Task finished with error code: -542398533 (Generic error in an external library)
[vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Terminating thread with return code -542398533 (Generic error in an external library)
[vf#0:0 @ 0x3b53a127d40] All consumers returned EOF
[vf#0:0 @ 0x3b53a127d40] Terminating thread with return code 0 (success)
[vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Decoder returned EOF, finishing
[vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Terminating thread with return code 0 (success)
[vist#0:2/hevc @ 0x3b53a050d80] All consumers of this stream are done
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Terminating thread with return code 0 (success)
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder thread received EOF packet
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder returned EOF, finishing
[aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Terminating thread with return code 0 (success)
[af#0:1 @ 0x3b53a127e80] Filtergraph returned EOF, finishing
[af#0:1 @ 0x3b53a127e80] All consumers returned EOF
[aost#0:1/aac @ 0x3b53a1f0780] Encoder thread received EOF
[aost#0:1/aac @ 0x3b53a1f0780] Terminating thread with return code 0 (success)
[out#0/mp4 @ 0x3b53a170fc0] All streams finished
[out#0/mp4 @ 0x3b53a170fc0] Terminating thread with return code 0 (success)
[af#0:1 @ 0x3b53a127e80] Terminating thread with return code 0 (success)
[mp4 @ 0x3b53a190e00] Starting second pass: moving the moov atom to the beginning of the file
[AVIOContext @ 0x3b53a237fc0] Statistics: 2787923 bytes read, 0 seeks
[AVIOContext @ 0x3b53a237c00] Statistics: 5582216 bytes written, 4 seeks, 25 writeouts
[out#0/mp4 @ 0x3b53a170fc0] Output file #0 (upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4):
[out#0/mp4 @ 0x3b53a170fc0] Output stream #0:0 (video): 283 frames encoded; 278 packets muxed (2720898 bytes);
[out#0/mp4 @ 0x3b53a170fc0] Output stream #0:1 (audio): 192 frames encoded (196608 samples); 193 packets muxed (66981 bytes);
[out#0/mp4 @ 0x3b53a170fc0] Total: 471 packets (2787879 bytes) muxed
[out#0/mp4 @ 0x3b53a170fc0] video:2657KiB audio:65KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.230928%
frame= 278 fps=272 q=-0.0 Lsize= 2729KiB time=00:00:04.12 bitrate=5425.4kbits/s speed=4.04x
[aac @ 0x3b53a2d2480] Qavg: 832.837
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input file #0 (upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/40/f9/40f9b5f3-8512-4058-9ffe-b71c12358bac.mp4):
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:1 (audio): 193 packets read (99038 bytes); 192 frames decoded; 0 decode errors (196608 samples);
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:2 (video): 303 packets read (39917376 bytes); 289 frames decoded; 0 decode errors;
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Total: 496 packets (40016414 bytes) demuxed
[AVIOContext @ 0x3b53a230180] Statistics: 40437714 bytes read, 4 seeks
Conversion failed!

I don't think i can test this right now

@kaaku3 commented on GitHub (Nov 13, 2024): Hi, I started fresh. I had a lot of problems seems the newer kernel don't fully support hardware acceleration... i'm not quite sure. I went with Joshua Rieks ubuntu with legacy kernel, that apparent everything should work out of the box, but i had problems with opencl. It seems to be working now, but I'm not 100%. Is there anyway from immich log to tell if its falling back to software encode. It doesn't seem to be going as fast as it was before. Also immich restore didn't work for me so i'm uploading everything. this is the ffmpeg from immich log ffmpeg -n 10 /usr/bin/ffmpeg -hwaccel rkmpp -hwaccel_output_format drm_prime -afbc rga -noautorotate -i upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/a7/f0/a7f09b0f-653d-4920-8a95-ec929b890cc8.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:2 -map 0:1 -g 256 -tag:v hvc1 -v verbose -vf scale_rkrga=720:-2:format=nv12:afbc=1 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/f9/57/f9578312-e623-4a61-9667-607baed5a02e.mp4 anyway to tell from that that it isn't falling back to software? I'm currently waiting on the problem file to be transcoded. edit. I there are errors [Nest] 7 - 11/14/2024, 1:30:47 AM ERROR [Microservices:MediaRepository] handler_name : SoundHandle vendor_id : [0][0][0][0] Stream #0:2[0x3](eng): Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), yuvj420p(pc, bt709, left), 3840x2160, 62337 kb/s, SAR 1:1 DAR 16:9, 59.86 fps, 59.94 tbr, 90k tbn (default) Metadata: creation_time : 2024-09-27T11:15:49.000000Z handler_name : VideoHandle vendor_id : [0][0][0][0] [out#0/mp4 @ 0x3b53a170fc0] Adding streams from explicit maps... [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Created video stream from input stream 0:2 [hevc_rkmpp @ 0x3b53a2d1d80] Found SoC name from device-tree: 'rockchip,rk3588s-orangepi-5 rockchip,rk3588' [hevc_rkmpp @ 0x3b53a2d1d80] Picked up an existing RKMPP hardware device [aost#0:1/aac @ 0x3b53a1f0780] Created audio stream from input stream 0:1 Stream mapping: Stream #0:2 -> #0:0 (hevc (hevc_rkmpp) -> hevc (hevc_rkmpp)) Stream #0:1 -> #0:1 (aac (native) -> aac (native)) [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Starting thread... [aost#0:1/aac @ 0x3b53a1f0780] Starting thread... [vf#0:0 @ 0x3b53a127d40] Starting thread... [af#0:1 @ 0x3b53a127e80] Starting thread... [vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Starting thread... [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Starting thread... [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Starting thread... Press [q] to stop, [?] for help [hevc_rkmpp @ 0x3b53a2d1d80] Noticed an info change [hevc_rkmpp @ 0x3b53a2d1d80] Configured with size: 3840x2160 | pix_fmt: drm_prime | sw_pix_fmt: nv12 [graph 0 input from stream 0:2 @ 0x3b542090300] w:3840 h:2160 pixfmt:drm_prime tb:1/90000 fr:60000/1001 sar:1/1 csp:bt709 range:pc [Parsed_scale_rkrga_0 @ 0x3b542090240] w:3840 h:2160 fmt:nv12 -> w:1280 h:720 fmt:nv12 [graph 0 input from stream 0:2 @ 0x3b542090300] video frame properties congruent with link at pts_time: 0 [hevc_rkmpp @ 0x3b53a2d0500] Using input frames context (format drm_prime) with hevc_rkmpp encoder. [hevc_rkmpp @ 0x3b53a2d0500] Rate Control mode is set to CQP [hevc_rkmpp @ 0x3b53a2d0500] QP Init/Max/Min/Max_I/Min_I is set to 28/28/28/28/28 [hevc_rkmpp @ 0x3b53a2d0500] Tier is set to 1 [hevc_rkmpp @ 0x3b53a2d0500] Profile is set to MAIN [hevc_rkmpp @ 0x3b53a2d0500] Level is set to 51 [hevc_rkmpp @ 0x3b53a2d0500] Configured with size: 1280x720 | pix_fmt: drm_prime | sw_pix_fmt: nv12 [graph_1_in_0:1 @ 0x3b540090300] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:stereo Output #0, mp4, to 'upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4': Metadata: major_brand : isom minor_version : 131072 compatible_brands: isomiso2mp41 com.android.capture.fps: 60.000000 location : +35.6813+139.5357/ location-eng : +35.6813+139.5357/ com.android.manufacturer: Google com.android.model: Pixel 6a encoder : Lavf61.1.100 Stream #0:0(eng): Video: hevc (Main), 1 reference frame (hvc1 / 0x31637668), drm_prime(pc, bt709, progressive, left), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 2000 kb/s, 59.94 fps, 60k tbn (default) Metadata: creation_time : 2024-09-27T11:15:49.000000Z handler_name : VideoHandle vendor_id : [0][0][0][0] encoder : Lavc61.3.100 hevc_rkmpp Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, delay 1024, 128 kb/s (default) Metadata: creation_time : 2024-09-27T11:15:49.000000Z handler_name : SoundHandle vendor_id : [0][0][0][0] encoder : Lavc61.3.100 aac [out#0/mp4 @ 0x3b53a170fc0] Starting thread... frame= 138 fps=0.0 q=-0.0 size= 1024KiB time=00:00:02.02 bitrate=4133.1kbits/s speed=4.06x [hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1 [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error submitting video frame to the encoder [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Error encoding a frame: Generic error in an external library [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Task finished with error code: -542398533 (Generic error in an external library) [vost#0:0/hevc_rkmpp @ 0x3b53a1f2880] Terminating thread with return code -542398533 (Generic error in an external library) [vf#0:0 @ 0x3b53a127d40] All consumers returned EOF [vf#0:0 @ 0x3b53a127d40] Terminating thread with return code 0 (success) [vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Decoder returned EOF, finishing [vist#0:2/hevc @ 0x3b53a050d80] [dec:hevc_rkmpp @ 0x3b53a2357c0] Terminating thread with return code 0 (success) [vist#0:2/hevc @ 0x3b53a050d80] All consumers of this stream are done [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Terminating thread with return code 0 (success) [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder thread received EOF packet [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Decoder returned EOF, finishing [aist#0:1/aac @ 0x3b53a050c00] [dec:aac @ 0x3b53a237200] Terminating thread with return code 0 (success) [af#0:1 @ 0x3b53a127e80] Filtergraph returned EOF, finishing [af#0:1 @ 0x3b53a127e80] All consumers returned EOF [aost#0:1/aac @ 0x3b53a1f0780] Encoder thread received EOF [aost#0:1/aac @ 0x3b53a1f0780] Terminating thread with return code 0 (success) [out#0/mp4 @ 0x3b53a170fc0] All streams finished [out#0/mp4 @ 0x3b53a170fc0] Terminating thread with return code 0 (success) [af#0:1 @ 0x3b53a127e80] Terminating thread with return code 0 (success) [mp4 @ 0x3b53a190e00] Starting second pass: moving the moov atom to the beginning of the file [AVIOContext @ 0x3b53a237fc0] Statistics: 2787923 bytes read, 0 seeks [AVIOContext @ 0x3b53a237c00] Statistics: 5582216 bytes written, 4 seeks, 25 writeouts [out#0/mp4 @ 0x3b53a170fc0] Output file #0 (upload/encoded-video/0175ee76-7502-46c4-9685-08b06e26c9dc/d1/6d/d16d540c-1124-4d90-847d-53fb08e97a41.mp4): [out#0/mp4 @ 0x3b53a170fc0] Output stream #0:0 (video): 283 frames encoded; 278 packets muxed (2720898 bytes); [out#0/mp4 @ 0x3b53a170fc0] Output stream #0:1 (audio): 192 frames encoded (196608 samples); 193 packets muxed (66981 bytes); [out#0/mp4 @ 0x3b53a170fc0] Total: 471 packets (2787879 bytes) muxed [out#0/mp4 @ 0x3b53a170fc0] video:2657KiB audio:65KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.230928% frame= 278 fps=272 q=-0.0 Lsize= 2729KiB time=00:00:04.12 bitrate=5425.4kbits/s speed=4.04x [aac @ 0x3b53a2d2480] Qavg: 832.837 [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input file #0 (upload/upload/0175ee76-7502-46c4-9685-08b06e26c9dc/40/f9/40f9b5f3-8512-4058-9ffe-b71c12358bac.mp4): [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:1 (audio): 193 packets read (99038 bytes); 192 frames decoded; 0 decode errors (196608 samples); [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Input stream #0:2 (video): 303 packets read (39917376 bytes); 289 frames decoded; 0 decode errors; [in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x3b53a080380] Total: 496 packets (40016414 bytes) demuxed [AVIOContext @ 0x3b53a230180] Statistics: 40437714 bytes read, 4 seeks Conversion failed! I don't think i can test this right now

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 13, 2024):

It seems like this is the relevant line:

[hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1

Does it fail for all videos, or only certain ones?

@mertalev commented on GitHub (Nov 13, 2024): It seems like this is the relevant line: ``` [hevc_rkmpp @ 0x3b53a2d0500] Failed to get key input frame from packet meta: -1 ``` Does it fail for all videos, or only certain ones?

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 13, 2024):

Thankyou for pointing that out,it seems to be only one file.
I was starting to think something was up with the drivers.
Looks as tho it was uploaded via an immich client on my wife's phone, is it possible it was something up with the upload...
It fell back to software transcode and passed that ok

@kaaku3 commented on GitHub (Nov 13, 2024): Thankyou for pointing that out,it seems to be only one file. I was starting to think something was up with the drivers. Looks as tho it was uploaded via an immich client on my wife's phone, is it possible it was something up with the upload... It fell back to software transcode and passed that ok

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 13, 2024):

Thanks for clarifying. I'm assuming the corruption issue is resolved now?

I'm not sure what the issue is for that particular video, but if it worked with software decoding then it's most likely a bug.

@mertalev commented on GitHub (Nov 13, 2024): Thanks for clarifying. I'm assuming the corruption issue is resolved now? I'm not sure what the issue is for that particular video, but if it worked with software decoding then it's most likely a bug.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 13, 2024):

Yes, I just transcoded that file and it's fixed, thankyou so much.
There is the line still but maybe that's a different issue.
I got a few more errors like the last
I will start again with my system make sure there is nothing up with the GPU drivers.
You wouldn't have any recommendations on a base image for orange pi 5 would you?

@kaaku3 commented on GitHub (Nov 13, 2024): Yes, I just transcoded that file and it's fixed, thankyou so much. There is the line still but maybe that's a different issue. I got a few more errors like the last I will start again with my system make sure there is nothing up with the GPU drivers. You wouldn't have any recommendations on a base image for orange pi 5 would you?

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 13, 2024):

ubuntu-rockchip tends to be recommended as the best for hardware acceleration support.

@mertalev commented on GitHub (Nov 13, 2024): [ubuntu-rockchip](https://github.com/Joshua-Riek/ubuntu-rockchip) tends to be recommended as the best for hardware acceleration support.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 14, 2024):

Ok, thankyou

@kaaku3 commented on GitHub (Nov 14, 2024): Ok, thankyou

deekerman commented

2026-02-20 03:01:13 -05:00

Author

Owner

@mertalev commented on GitHub (Nov 14, 2024):

You can also try downgrading back to 1.120.1 to see if you still get the same error. That would make it clear whether the issue is due to the different server environment or the FFmpeg change.

@mertalev commented on GitHub (Nov 14, 2024): You can also try downgrading back to 1.120.1 to see if you still get the same error. That would make it clear whether the issue is due to the different server environment or the FFmpeg change.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 14, 2024):

I just wiped it ;), I'll keep that in mind if it persists

@kaaku3 commented on GitHub (Nov 14, 2024): I just wiped it ;), I'll keep that in mind if it persists

deekerman commented

2026-02-20 03:01:13 -05:00

Author

Owner

@kaaku3 commented on GitHub (Nov 15, 2024):

Sorry to keeps bothering you here... I think the base system is fine now, but in immich when i run transcoding despite having hardware decoding enable... I don't think it is.
here is the ffmepg output
ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/admin/2020/2020-10-24/PXL_20201024_021840973.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:1 -map 0:0 -g 256 -tag:v hvc1 -v verbose -vf scale=-2:720 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/a32baa7e-7326-459b-a8ab-f08034213bb7/14/de/14de9676-c1c3-4e73-80e8-e3b614773b6d.mp4

edit...

I just noticed this

[Nest] 7 - 11/15/2024, 11:39:53 AM DEBUG [Microservices:MediaService] OpenCL not available for transcoding, so RKMPP acceleration will use CPU decoding

So i think its just me and i need to get opencl working on my system

@kaaku3 commented on GitHub (Nov 15, 2024): Sorry to keeps bothering you here... I think the base system is fine now, but in immich when i run transcoding despite having hardware decoding enable... I don't think it is. here is the ffmepg output ffmpeg -n 10 /usr/bin/ffmpeg -i upload/library/admin/2020/2020-10-24/PXL_20201024_021840973.mp4 -y -c:v hevc_rkmpp -c:a aac -movflags faststart -fps_mode passthrough -map 0:1 -map 0:0 -g 256 -tag:v hvc1 -v verbose -vf scale=-2:720 -level 153 -rc_mode CQP -qp_init 28 upload/encoded-video/a32baa7e-7326-459b-a8ab-f08034213bb7/14/de/14de9676-c1c3-4e73-80e8-e3b614773b6d.mp4 edit... I just noticed this [Nest] 7 - 11/15/2024, 11:39:53 AM DEBUG [Microservices:MediaService] OpenCL not available for transcoding, so RKMPP acceleration will use CPU decoding So i think its just me and i need to get opencl working on my system

deekerman commented

2026-02-20 03:01:13 -05:00

Author

Owner

@mertalev commented on GitHub (Nov 17, 2024):

Yes, the issue is the lack of OpenCL. There's a PR to loosen this restriction so it tries to still use hardware decoding and only do tone-mapping on CPU (if it's an HDR video).

The Failed to get key input frame from packet meta: -1 error you mentioned should be fixed in the next release.

@mertalev commented on GitHub (Nov 17, 2024): Yes, the issue is the lack of OpenCL. There's a PR to loosen this restriction so it tries to still use hardware decoding and only do tone-mapping on CPU (if it's an HDR video). The `Failed to get key input frame from packet meta: -1` error you mentioned should be fixed in the next release.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 17, 2024):

Thankyou, yeah I was keeping an eye on jellyfin ffmpeg and I hoped that's what that commit meant.

edit...

I don't know how or why, but i after reinstalling everything opencl related, it is working now and hardware decoding is working too.

Thanks for all your help.

@kaaku3 commented on GitHub (Nov 17, 2024): Thankyou, yeah I was keeping an eye on jellyfin ffmpeg and I hoped that's what that commit meant. edit... I don't know how or why, but i after reinstalling everything opencl related, it is working now and hardware decoding is working too. Thanks for all your help.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 20, 2024):

Hi I just wanted to add that I still have the same
Failed to get key input frame from packet meta: -1 in the latest update.

@kaaku3 commented on GitHub (Nov 20, 2024): Hi I just wanted to add that I still have the same Failed to get key input frame from packet meta: -1 in the latest update.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Nov 20, 2024):

Unfortunately the fixed version was merged two hours after the release :/

Maybe we can do a patch release.

@mertalev commented on GitHub (Nov 20, 2024): Unfortunately the fixed version was merged two hours after the release :/ Maybe we can do a patch release.

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Nov 20, 2024):

Good to hear the fix is still on the way. Thankyou

@kaaku3 commented on GitHub (Nov 20, 2024): Good to hear the fix is still on the way. Thankyou

deekerman commented

Author

Owner

@mertalev commented on GitHub (Dec 6, 2024):

Can you confirm if the issue is fully fixed now on 1.122.1?

@mertalev commented on GitHub (Dec 6, 2024): Can you confirm if the issue is fully fixed now on 1.122.1?

deekerman commented

Author

Owner

@kaaku3 commented on GitHub (Dec 7, 2024):

Hi, I'm currently away right now.
I clicked re-transcode all in web interface and kept an eye on it via SSH on my phone. For some reason my system suspended... Maybe I pocket typed something... I'm not sure.
I was able to check that it did about 4 thousand odd transcodes without error before it suspended though, so seems fixed.
Thankyou so much and thankyou for all the hard work.
I'm sorry I can't answer more definitely right now.

@kaaku3 commented on GitHub (Dec 7, 2024): Hi, I'm currently away right now. I clicked re-transcode all in web interface and kept an eye on it via SSH on my phone. For some reason my system suspended... Maybe I pocket typed something... I'm not sure. I was able to check that it did about 4 thousand odd transcodes without error before it suspended though, so seems fixed. Thankyou so much and thankyou for all the hard work. I'm sorry I can't answer more definitely right now.

deekerman commented

Author

Owner

@mertalev commented on GitHub (Dec 7, 2024):

Tentatively closing this issue since it sounds like it's fixed

@mertalev commented on GitHub (Dec 7, 2024): Tentatively closing this issue since it sounds like it's fixed

deekerman commented