mirror of
https://github.com/motioneye-project/motioneye.git
synced 2026-03-02 22:57:06 -05:00
Why is the recording size with passthrough 1/8 of that with overlay? #1681
Labels
No labels
Android app
Arch Linux
CI/CD
CSS
FreeBSD
HTML/HTTP
Home Assistant addon
JavaScript
Python
Raspberry Pi
Stale No Activity 60 Days
bug
code format
dependencies
dev branch
docker
documentation
duplicate
enhancement
feature
help wanted
i18n/l10n
invalid
legacy motionEye
meta
motion
motionEyeOS
notourproblem
python update
question
question
security
troubleshooting
wontfix
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/motioneye#1681
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @hspaay on GitHub (Jun 24, 2020).
Hi, I'm a bit stumped why with passthrough enabled the video size is so much smaller than with it disabled. I like the motion detection overlay so prefer to keep it disabled.
The camera is encoding with h264, 3 FPS and an image size of 1080 and with passthrough enabled the recording size is 250K per second. With passthrough disabled and using movie format h264 the same camera produces videos with about 2MB per second. I see no difference in video quality between the two.
It looks like encoding configuration by motion(eye) is not very efficient. Any suggestions how to improve this?
Thanks!
@zagrim commented on GitHub (Jun 25, 2020):
With video passthrough, the resulting file is encoded based on the parameters set by the camera (in your case with 3 FPS and an image size of 1080, and whatever quality setting for the codec). With overlay, Motion (well, ffmpeg that is run by Motion) is encoding the resulting file based on the parameters set by MotionEye (with whatever FPS, image size and encoding quality settings set there). You don't mention the FPS and image for the latter, but I'd assume they are matching with the ones of the camera, but if the quality is "high enough" even with the camera feed, having higher quality (encoding-wise) set for the video re-encoded by Motion would only result in a larger file. So try and see if lowering the quality ("Movies" -> "Movie Quality") evens out the size difference.
However, the overlays are adding information in the frames, so even theoretically the result should be a bit larger keeping the overall quality identical. Also, if the original camera feed is already highly packed (i.e. of low quality due to losing plenty of information), the decoded sequence of frames that Motion (ffmpeg) is re-encoding is not anymore nearly the same as the sequence of frames taken from the camera image sensor, and that also has some effect to both the resulting video file size and the visual quality of it. Basically, re-encoding always reduces perceived quality (unless the original encoding didn't lose nearly any information). But since you are saying that there's no noticeable difference in quality between the videos, the size difference should be just due to the one with overlays being "of too high quality".
You can also compare the encoding parameters used to produce the files by running e.g.
ffprobe -show_streams myvideofile.mp4(look at the details under "[STREAM]", "profile" is one that can play big part on the resulting file size).@hspaay commented on GitHub (Jun 25, 2020):
Hello @zagrim, thank you for the assist. This is quite helpful information. Running ffprobe especially was an eye opener.
I'm not sure where the movie output FPS settings is in motioneye. The 'Movies' section has the format set to H.264 and quality to 75%, but there is no FPS option here. The Video Streaming section is set to 5 FPS, but I think that is only for viewing a live stream. Both images are set to 1920x1080. The movie_codec option in the motion configuration file is set to 'mp4'.
ffprobe on the camera video shows a bit rate of 2106kb/s and 3fps which is what the camera is configured as, so makes sense.
ffprobe on the re-coded motion(eye) video shows a bit rate of 16655 kb/s and 2fps (!) wow, not sure where that comes from but it explains the file size difference. It is also odd seeing such a high bit rate with only 2 fps. Surely the overlay can't be adding that much, especially since there is little extra information. Any ideas?
Attached the ffprobe output for the camera and the motion video output. Note that these are two different videos but taken around the same time and of the same camera, so the video content is similar with little movement in the picture
ffprobe-camera-stream - 34 seconds.txt
ffprobe-motioneye stream - 13 sec.txt
The ffprobe header on the camera stream:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './camera-stream - 34 seconds.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.20.100
Duration: 00:00:34.23, start: 0.499000, bitrate: 2106 kb/s
Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 2105 kb/s, 3.01 fps, 3 tbr, 90k tbn, 6 tbc (default)
Metadata:
handler_name : VideoHandler
ffprobe header on motion stream:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './motioneye stream - 13 sec.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.20.100
Duration: 00:00:13.14, start: 0.000000, bitrate: 16655 kb/s
Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, 16655 kb/s, 2.05 fps, 2 tbr, 16384 tbn, 4 tbc (default)
Metadata:
handler_name : VideoHandler
@zagrim commented on GitHub (Jun 25, 2020):
There's Frame Rate option in "Video Device", I think that dictates the movie frame rate.
The difference in bitrate is somewhat due to the difference in "profile": the camera stream has "Main" while the one from Motioneye has "Constrained Baseline" (which is more "low-end" and thus likely to cause bigger files that can be played with older hardware, see here for more on h.264 profiles). The way Motion sets quality parameters for
ffmpegis a bit unclear to me (I don't know if all there is is what can be read from ffmpeg_set_quality here or if there is something more to it) and anyway the details of that depend on the exact build of Motion. I've seen people reporting even High 4:4:4 Predictive being produced when the quality slider is at 100%, but where that came, I don't know.What versions of MotionEye, Motion and ffmpeg you have? I'm wondering if you might have an old version of ffmpeg or something like that. I can't compare this to my setup since I'm using
movie_passthrough, but I'd expectffmpegto be able to produce way smaller files with that framerate and quality setting.@hspaay commented on GitHub (Jun 26, 2020):
Oh wow, the profiles is new to me and this is very interesting.
First to answer your question about versions, ubuntu 19.10; motioneye 0.42; motion 4.2.2-1build1; ffmpeg 7:4.1.4.-1build2. All these are the latest version available on ubuntu 19.10.
Back to profiles. I took the motion encoded video and simply converted it:
ffmpeg motioneye-stream.mp4 test.mp4The motion file was 27MB (13 seconds) and the ffmpeg output file only 3.9MB. The output file has profile High, level 4.0. This is a great result and begs for more investigation with different profiles vs cpu. The plot thickens!
So, why does motion(eye) choose Constrained Baseline?
Trying to read the code you highlighted (nice find), the decision to use FF_PROFILE_H264_HIGH is when the codec_id is MY_CODEC_ID_H264 and 'ffmpeg->preferred_codec' is H264OMX. It is unclear where either parameter is coming from. In motioneye there is a movie format parameters, set to H264. Changing it to H264/OMX did not reduce movie size so that wasn't it. The movie quality is set to 75% btw and the frame rate to 2fps.
There is also a comment saying: "other H264 encoders quality via CRF", which looks to be the calculated quality.
Okay, well that narrows the question down a bit to, "How does motion(eye) determine the H264 profile used". Or is that motion's domain? There doesn't seem to be a profile option in motion.conf
Thanks again for pointing into the right direction. It would be good to hear what profiles other people see in the video file. You'd think that everyone is interested in reducing the file size drastically. Maybe it is just my setup?
@zagrim commented on GitHub (Jun 26, 2020):
I don't think MotionEye has any control on the h.264 profile, it is likely set by
ffmpegbased on some options Motion passes to it. I think it should be possible to see what is the actual command line that is executed whenffmpegis being called to assemble to video on the detected motion, but couldn't get that to happen on my test setup. You might get better help if you asked this from the Motion project (see https://github.com/ccrisan/motioneye/wiki/Report-Motion-Issue for details).@zagrim commented on GitHub (Jun 26, 2020):
On my test setup (Debian 10, Motion 4.1.1, ffmpeg 4.1.4-1~deb10u1, MotionEye from the head of dev branch but it really shouldn't matter much) I get this kind of output:
Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x960, 8553 kb/s, 3.01 fps, 11.92 tbr, 10240 tbn, 20 tbc (default)The resolution isn't quite the same as yours (didn't want to break my "production" setup by altering camera configuration), but for 3 fps and Constrained Baseline profile I get way lower bitrate. Estimating very roughly, my resolution + framerate should produce something like 10% lower bitrate if everything else was equal, but in this case I got nearly 50% lower bitrate. Of course everything else isn't equal (starting from a different camera and a different scene) but I also used 75% quality setting in MotionEye.
@hspaay commented on GitHub (Jun 26, 2020):
Yes I think you're right. It is time to continue the question at the motion forum.
The bit rate you are seeing is about half of what I'm seeing and indeed probably due to the lower resolution. How did you get to expecting a 10% lower bitrate with 40% lower resolution? Update: Oh never mind, at 3fps vs 2fps. Got it.
What I'm seeing is around 1.5-2MByte per second of recorded video at 1080. Purely based on resolution I would expect that you would see around 800KB/sec with passthrough disabled. If that is true then you could also benefit from significant lower storage size if we can figure out how to enable the High profile. I'm speculating but this might be the case for many motion(eye) users. It might not matter to you though if you are using passthrough mode everywhere.
Just for fun, how many cameras do you have in your production setup and what cpu? My motioneye system uses 8 cameras all of them set to 1080 and 2 or 3 fps. Half of them have motion detection enabled. Any more it would not keep up and skip motion detection. The CPU is a Intel core i3-5010U, dual core/4 threads with 2GB of RAM, so nothing special. When not recording 'top' shows 75% idle and motion itself takes around 100% cpu (out of 400%). When motioneye runs, that drops to 50% idle and 175% cpu for the motion process. The meyectl process then takes 25% cpu. I'm pretty happy with it except for the disk space consumption.
@zagrim commented on GitHub (Jun 27, 2020):
My production setup is two cameras with 1280x960@10fps, doing motion detection on a 704x576@4fps substream. MotionEye is running on a RPi3 which barely keeps from boiling in the summertime thanks to passthrough video (system temperature between 60 and 70 degrees Celsius) even though CPU is always >75% idle. I'm using a 120 GB SSD attached via USB for storage. My only woe with disk space consumption is the high number of false positives due to dancing tree shadows in the scene, which I hope to eliminate some day using some sort of an AI solution.
@hspaay commented on GitHub (Jun 27, 2020):
I've got the same problem with dancing tree shadows. 100+ false positives on a sunny and windy day is no exception. The motion detection algorithm that motion uses is not really suitable for outdoor use. I usually block out most of the image except for a few key locations. It helps but is far from ideal. I've used zoneminder for a while and its 'blob' algorithm was much more resilient against false positives of shadows. Like yourself I'm hoping to use an AI solution. I bought a nVidia Jetson Nano ($100) which has 128 CUDA cores and hope to run Tensorflow or something like it. This object detection project looks promising.
Anyways, I'm getting off-topic. Back to the problem at hand :) Ciao