starred/rustdesk-rustdesk

Fork 0

mirror of https://github.com/rustdesk/rustdesk.git synced 2026-03-02 19:26:56 -05:00

Low-latency streaming #66

New issue

Closed

opened 2026-02-20 22:21:42 -05:00 by deekerman · 10 comments

deekerman commented

2026-02-20 22:21:42 -05:00

Owner

Originally created by @dantmnf on GitHub (Jun 17, 2021).

The current screen-to-screen latency can be up to 120ms on a wired LAN connection. After skimming through the code, I found some points that could be improved.

Screen capturing (on Windows)

The good old GDI approach suffers from performance issues since the introduction of desktop composition (Windows Vista). It cannot achieve 60 fps on a 3840x2160 desktop. I found some traces of DXGI desktop duplication in comments, looking forward to seeing the implementation.

Encoder tuning

There is not so much useful information about the input-to-output delay of libvpx available. I also see some attempts to use NVENC. Generally speaking, it is better to use platform hardware-accelerated encoding API instead. (e.g. Media Foundation, VideoToolbox, VA-API, v4l2m2m)

Low-latency rendering

This may require playing with platform-native windowing system APIs. (e.g. DXGI_SWAP_EFFECT_FLIP_DISCARD on Windows)

Some game streaming solutions are doing well on this, like Parsec.

Originally created by @dantmnf on GitHub (Jun 17, 2021). The current screen-to-screen latency can be up to 120ms on a wired LAN connection. After skimming through the code, I found some points that could be improved. * Screen capturing (on Windows) The good old GDI approach suffers from performance issues since the introduction of desktop composition (Windows Vista). It cannot achieve 60 fps on a 3840x2160 desktop. I found some traces of DXGI desktop duplication in comments, looking forward to seeing the implementation. * Encoder tuning There is not so much useful information about the input-to-output delay of libvpx available. I also see some attempts to use NVENC. Generally speaking, it is better to use platform hardware-accelerated encoding API instead. (e.g. Media Foundation, VideoToolbox, VA-API, v4l2m2m) * Low-latency rendering This may require playing with platform-native windowing system APIs. (e.g. DXGI_SWAP_EFFECT_FLIP_DISCARD on Windows) Some game streaming solutions are doing well on this, like Parsec.

deekerman

2026-02-20 22:21:42 -05:00

closed this issue
added the
enhancement
label

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@rustdesk commented on GitHub (Jun 17, 2021):

Screen capturing

I did using DXGI before, but there was memory leak, so switched to GDI as a workaround.

@rustdesk commented on GitHub (Jun 17, 2021): > * Screen capturing I did using DXGI before, but there was memory leak, so switched to GDI as a workaround.

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@rustdesk commented on GitHub (Jun 17, 2021):

I agree with you. Once I get fund support, I will make RustDesk much better.

@rustdesk commented on GitHub (Jun 17, 2021): I agree with you. Once I get fund support, I will make RustDesk much better.

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@rustdesk commented on GitHub (Jul 7, 2021):

https://github.com/H-M-H/Weylus#hardware-acceleration-2
https://github.com/H-M-H/Weylus/blob/master/lib/encode_video.c
https://github.com/moonlight-stream

@rustdesk commented on GitHub (Jul 7, 2021): https://github.com/H-M-H/Weylus#hardware-acceleration-2 https://github.com/H-M-H/Weylus/blob/master/lib/encode_video.c https://github.com/moonlight-stream

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@rustdesk commented on GitHub (Jul 22, 2021):

https://github.com/bryal/dxgcap-rs ref for memory leak
Display rotation: https://github.com/bryal/dxgcap-rs/blob/master/src/lib.rs#L409

@rustdesk commented on GitHub (Jul 22, 2021): https://github.com/bryal/dxgcap-rs ref for memory leak Display rotation: https://github.com/bryal/dxgcap-rs/blob/master/src/lib.rs#L409

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@rustdesk commented on GitHub (Jan 9, 2022):

Split socket read/write

@rustdesk commented on GitHub (Jan 9, 2022): Split socket read/write

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@andrewmd5 commented on GitHub (Jun 2, 2022):

You could also plug in Rainway: https://rainway.com/blog/2022/06/01/free-dev-tier/

@andrewmd5 commented on GitHub (Jun 2, 2022): You could also plug in Rainway: https://rainway.com/blog/2022/06/01/free-dev-tier/

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@rustdesk commented on GitHub (Jun 2, 2022):

Rainway

Too many such commercial SDK vendors in the market.

@rustdesk commented on GitHub (Jun 2, 2022): > Rainway Too many such commercial SDK vendors in the market.

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@andrewmd5 commented on GitHub (Jun 2, 2022):

Rainway

Too many such commercial SDK vendors in the market.

Well you mentioned Parsec as inspiration, and Rainway provides that core functionality in a pluggable way while also providing necessary infrastructure. Rainway itself spun out of my old FOSS TeamViewer alternative https://github.com/Ulterius

So given Rainway is free for FOSS projects and is built on Rust it seemed like a good pairing. IIRC there are no other people offering SDKs for this.

@andrewmd5 commented on GitHub (Jun 2, 2022): > > Rainway > > Too many such commercial SDK vendors in the market. Well you mentioned Parsec as inspiration, and Rainway provides that core functionality in a pluggable way while also providing necessary infrastructure. Rainway itself spun out of my old FOSS TeamViewer alternative https://github.com/Ulterius So given Rainway is free for FOSS projects and is built on Rust it seemed like a good pairing. IIRC there are no other people offering SDKs for this.

deekerman commented

2026-02-20 22:21:45 -05:00

Author

Owner

@rustdesk commented on GitHub (Jun 2, 2022):

Ulterius

Good job. How do you decode video frame on web? We are using vp9 wasm, but it is slow.

@rustdesk commented on GitHub (Jun 2, 2022): > Ulterius Good job. How do you decode video frame on web? We are using vp9 wasm, but it is slow.

deekerman commented

2026-02-20 22:21:46 -05:00

Author

Owner

@andrewmd5 commented on GitHub (Jun 2, 2022):

Ulterius

Good job. How do you decode video frame on web? We are using vp9 wasm, but it is slow.

VP9 is going to have a lot of overhead for the majority of deployments due the lack of hardware decoding and encoding capabilities on most vendors. If you setup a Rainway account we're happy to allocate time to how you might plug it into your existing solution to take advantage of our real-time media streaming capabilities.

@andrewmd5 commented on GitHub (Jun 2, 2022): > > Ulterius > > Good job. How do you decode video frame on web? We are using vp9 wasm, but it is slow. VP9 is going to have a lot of overhead for the majority of deployments due the lack of hardware decoding and encoding capabilities on most vendors. If you setup a Rainway account we're happy to allocate time to how you might plug it into your existing solution to take advantage of our real-time media streaming capabilities.

deekerman referenced this issue

2026-02-20 23:06:36 -05:00

App crash when connect AndroidTV 11 device (or the other non-phone devices) #619