starred/rustdesk-rustdesk
1
0
Fork 0
mirror of https://github.com/rustdesk/rustdesk.git synced 2026-03-02 19:26:56 -05:00
Code Issues 77 Projects Packages Wiki Activity

Unsupported display server type tty when running without display manager #72

New issue
Closed
opened 2026-02-20 22:21:52 -05:00 by deekerman · 24 comments
deekerman commented 2026-02-20 22:21:52 -05:00
Owner
Copy link

Originally created by @mtoohey31 on GitHub (Jun 18, 2021).

Hello! I'm starting my window manager with startx instead of with a display manager, and the display server type is incorrectly identified as tty instead of x11. I took a look at the method being used to detect the display server and the issue isn't in the implementation of the method you're using to check it, but with the result of the loginctl command. One alternative method might be checking the WAYLAND_DISPLAY and DISPLAY environment variables, as suggested here: https://unix.stackexchange.com/a/559950

I haven't tested it much at all, here's a mockup of what that might look like (in src/platform/linux.rs), I haven't been able to test it though cause it won't build on my system for other reasons even without these modifications.

pub fn get_display_server() -> String {
    match std::env::var("WAYLAND_DISPLAY") {
        Some(display) => "wayland",
        _ => match std::env::var("DISPLAY") {
            Some(display) => "x11",
            _ => "unknown",
        },
    }
}

Let me know if there's any other information I can provide, or testing I can do!

Originally created by @mtoohey31 on GitHub (Jun 18, 2021). Hello! I'm starting my window manager with `startx` instead of with a display manager, and the display server type is incorrectly identified as `tty` instead of `x11`. I took a look at the method being used to detect the display server and the issue isn't in the implementation of the method you're using to check it, but with the result of the `loginctl` command. One alternative method might be checking the `WAYLAND_DISPLAY` and `DISPLAY` environment variables, as suggested here: https://unix.stackexchange.com/a/559950 I haven't tested it much at all, here's a mockup of what that might look like (in `src/platform/linux.rs`), I haven't been able to test it though cause it won't build on my system for other reasons even without these modifications. ```rust pub fn get_display_server() -> String { match std::env::var("WAYLAND_DISPLAY") { Some(display) => "wayland", _ => match std::env::var("DISPLAY") { Some(display) => "x11", _ => "unknown", }, } } ``` Let me know if there's any other information I can provide, or testing I can do!
deekerman 2026-02-20 22:21:52 -05:00
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 18, 2021):

noted with thanks

@rustdesk commented on GitHub (Jun 18, 2021): noted with thanks
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 18, 2021):

Wondering if you can work on it. You do not need to work on this big project. You can write standalone functions get_display_server, get_env and get_active_userid for non-display-manager.

@rustdesk commented on GitHub (Jun 18, 2021): Wondering if you can work on it. You do not need to work on this big project. You can write standalone functions `get_display_server`, `get_env` and `get_active_userid` for non-display-manager.
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@mtoohey31 commented on GitHub (Jun 18, 2021):

Sure, I can try giving it a go tomorrow.

@mtoohey31 commented on GitHub (Jun 18, 2021): Sure, I can try giving it a go tomorrow.
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@mtoohey31 commented on GitHub (Jun 19, 2021):

Alright, here's a new version of get_display_server_of_session that works without a display-manager:

fn get_display_server_of_session(session: &str) -> String {
    if let Ok(output) = std::process::Command::new("loginctl")
        .args(vec!["show-session", "-p", "Type", session])
        .output()
    // Check session type of the session
    {
        let display_server = String::from_utf8_lossy(&output.stdout)
            .replace("Type=", "")
            .trim_end()
            .into();
        if display_server == "tty" {
            // If the type is tty...
            if let Ok(output) = std::process::Command::new("loginctl")
                .args(vec!["show-session", "-p", "TTY", session])
                .output()
            // Get the tty number
            {
                let tty: String = String::from_utf8_lossy(&output.stdout)
                    .replace("TTY=", "")
                    .trim_end()
                    .into();
                if let Ok(Some(xorg_results)) =
                    run_cmds(format!("ps -e | grep \"{}.\\\\+Xorg\"", tty))
                // And check if Xorg is running on that tty
                {
                    if xorg_results.trim_end().to_string() != "" {
                        // If it is, manually return "x11", otherwise return tty
                        "x11".to_owned()
                    } else {
                        display_server
                    }
                } else {
                    // If any of these commands fail just fall back to the display server
                    display_server
                }
            } else {
                display_server
            }
        } else {
            // If the session is not a tty, then just return the type as usual
            display_server
        }
    } else {
        "".to_owned()
    }
}

Basically it just adds a check for whether the session type is tty, and if it is, it checks if Xorg is running on that tty. It's a bit of a mess of nested if statements so feel free to refactor that if you have a neater way of doing things. I believe this fixes the issue, let me know if changes to those are functions are necessary though. I haven't gotten to testing it in the main application, but I've tested it on its own and it correctly identifies the display server now on my setup which is Arch and startx (x11), stock Manjaro KDE (x11), and stock Fedora 34 (wayland).

Lmk if that looks ok and if you'd like me to open a PR or if you have something else in mind. Also, random fun fact, it looks like Teamviewer has this same issue according to this comment: https://aur.archlinux.org/packages/teamviewer/?O=40&PP=10#comment-763211.

@mtoohey31 commented on GitHub (Jun 19, 2021): Alright, here's a new version of `get_display_server_of_session` that works without a display-manager: ```rust fn get_display_server_of_session(session: &str) -> String { if let Ok(output) = std::process::Command::new("loginctl") .args(vec!["show-session", "-p", "Type", session]) .output() // Check session type of the session { let display_server = String::from_utf8_lossy(&output.stdout) .replace("Type=", "") .trim_end() .into(); if display_server == "tty" { // If the type is tty... if let Ok(output) = std::process::Command::new("loginctl") .args(vec!["show-session", "-p", "TTY", session]) .output() // Get the tty number { let tty: String = String::from_utf8_lossy(&output.stdout) .replace("TTY=", "") .trim_end() .into(); if let Ok(Some(xorg_results)) = run_cmds(format!("ps -e | grep \"{}.\\\\+Xorg\"", tty)) // And check if Xorg is running on that tty { if xorg_results.trim_end().to_string() != "" { // If it is, manually return "x11", otherwise return tty "x11".to_owned() } else { display_server } } else { // If any of these commands fail just fall back to the display server display_server } } else { display_server } } else { // If the session is not a tty, then just return the type as usual display_server } } else { "".to_owned() } } ``` Basically it just adds a check for whether the session type is tty, and if it is, it checks if Xorg is running on that tty. It's a bit of a mess of nested if statements so feel free to refactor that if you have a neater way of doing things. I believe this fixes the issue, let me know if changes to those are functions are necessary though. I haven't gotten to testing it in the main application, but I've tested it on its own and it correctly identifies the display server now on my setup which is Arch and startx (x11), stock Manjaro KDE (x11), and stock Fedora 34 (wayland). Lmk if that looks ok and if you'd like me to open a PR or if you have something else in mind. Also, random fun fact, it looks like Teamviewer has this same issue according to this comment: https://aur.archlinux.org/packages/teamviewer/?O=40&PP=10#comment-763211.
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 19, 2021):

get_display_server_of_session

Good job, but you also need to modify get_active_userid, and get_env, RustDesk uses get_env to get $DISPLAY and $XAUTHORITY to make xdo and screen capture work. get_active_userid get the user id of the active X session.

@rustdesk commented on GitHub (Jun 19, 2021): > get_display_server_of_session Good job, but you also need to modify `get_active_userid`, and `get_env`, RustDesk uses `get_env` to get `$DISPLAY` and `$XAUTHORITY` to make xdo and screen capture work. `get_active_userid` get the user id of the active X session.
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 19, 2021):

Have you test if you can get correct $DISPLAY and $XAUTHORITY with old get_env and your new get_display_server_of_session?

@rustdesk commented on GitHub (Jun 19, 2021): Have you test if you can get correct `$DISPLAY` and `$XAUTHORITY` with old get_env and your new `get_display_server_of_session`?
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 19, 2021):

Seems get_active_userid can work, but I am not sure if get_env works with your new get_display_server_of_session.

@rustdesk commented on GitHub (Jun 19, 2021): Seems `get_active_userid` can work, but I am not sure if `get_env` works with your new `get_display_server_of_session`.
deekerman commented 2026-02-20 22:21:54 -05:00
Author
Owner
Copy link

@mtoohey31 commented on GitHub (Jun 19, 2021):

No I have not tried that approach cause I assumed it would encounter issues when the program is running as a systemd service. I'm currently building the full application so I can test it properly to see what works and what doesn't.

@mtoohey31 commented on GitHub (Jun 19, 2021): No I have not tried that approach cause I assumed it would encounter issues when the program is running as a systemd service. I'm currently building the full application so I can test it properly to see what works and what doesn't.
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 19, 2021):

building the full application

Thanks

@rustdesk commented on GitHub (Jun 19, 2021): > building the full application Thanks
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@mtoohey31 commented on GitHub (Jun 19, 2021):

K I've got it built and I did some testing and it works just fine without changing any other functions. I don't think the bug was anything in the actual functionality, it was just these checks here that were preventing the program from trying to connect, because get_display_server was returning tty instead of x11, which is now fixed:

src/ui.rs
450:            if dtype != "x11" {
451:                return format!("Unsupported display server type {}, x11 expected!", dtype);

src/server/connection.rs
402:            if dtype != "x11" {
404:                    "Unsupported display server type {}, x11 expected",

I've got the changes in this branch here. Let me know if there's anything else you'd like me to check before opening a PR.

@mtoohey31 commented on GitHub (Jun 19, 2021): K I've got it built and I did some testing and it works just fine without changing any other functions. I don't think the bug was anything in the actual functionality, it was just these checks here that were preventing the program from trying to connect, because `get_display_server` was returning `tty` instead of `x11`, which is now fixed: ``` src/ui.rs 450: if dtype != "x11" { 451: return format!("Unsupported display server type {}, x11 expected!", dtype); src/server/connection.rs 402: if dtype != "x11" { 404: "Unsupported display server type {}, x11 expected", ``` I've got the changes in this branch [here](https://github.com/mtoohey31/rustdesk/tree/display-server-fix). Let me know if there's anything else you'd like me to check before opening a PR.
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 19, 2021):

Awesome work Mattherw. pr please. If you can help, please also look into #56, much more challenge. Although now RustDesk is growng, we don't have enough cash but I can sponsor u anyway, Thanks for your time!

@rustdesk commented on GitHub (Jun 19, 2021): Awesome work Mattherw. pr please. If you can help, please also look into #56, much more challenge. Although now RustDesk is growng, we don't have enough cash but I can sponsor u anyway, Thanks for your time!
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@mtoohey31 commented on GitHub (Jun 19, 2021):

👍 PR created, I can maybe take a look tomorrow, I'd want to figure out how I'd go about it or whether it's even doable before agreeing to anything though. One other thing I was wondering is whether you'd be interested in including a Dockerfile for building the application, in case people want to use that method. I put a rough one together based on the build scripts in the README to test these changes and I figure it might be easier than trying to follow the scripts manually for a lot of people, it might also cut down on the number of build-related issues. Lmk if you're interested, if so I can clean it up and add documentation.

@mtoohey31 commented on GitHub (Jun 19, 2021): :+1: PR created, I can maybe take a look tomorrow, I'd want to figure out how I'd go about it or whether it's even doable before agreeing to anything though. One other thing I was wondering is whether you'd be interested in including a Dockerfile for building the application, in case people want to use that method. I put a rough one together based on the build scripts in the README to test these changes and I figure it might be easier than trying to follow the scripts manually for a lot of people, it might also cut down on the number of build-related issues. Lmk if you're interested, if so I can clean it up and add documentation.
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 20, 2021):

Dockerfile

clean it up and add documentation.

Please go ahead. Very interested.

@rustdesk commented on GitHub (Jun 20, 2021): > Dockerfile > clean it up and add documentation. Please go ahead. Very interested.
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 20, 2021):

I'd want to figure out how I'd go about it or whether it's even doable before agreeing to anything though.

Thanks :)

@rustdesk commented on GitHub (Jun 20, 2021): > I'd want to figure out how I'd go about it or whether it's even doable before agreeing to anything though. Thanks :)
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (Jun 20, 2021):

doable

There is Wayland support (screen capture and input control) in https://github.com/H-M-H/Weylus/ with rust, I believe it is doable.

@rustdesk commented on GitHub (Jun 20, 2021): > doable There is Wayland support (screen capture and input control) in https://github.com/H-M-H/Weylus/ with rust, I believe it is doable.
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@lstep commented on GitHub (May 21, 2022):

Hello,

I just downloaded version 1.1.9 (latest, as of today), on an Ubuntu Linux, starting from tty with startx, I thought the bug would be corrected on version 1.1.9, but I still get the error message Unsupported display server tty, x11 expected :-(.

$ ps -e |grep tty
   4000 tty1     00:00:00 login
   4089 tty1     00:00:00 bash
   4683 tty2     07:33:45 Xorg

I guess that's because I switched to tty2 before running startx, and your code only checks the tty that has the login process (as I see from the patch).

$ rustdesk --version
1.1.9
@lstep commented on GitHub (May 21, 2022): Hello, I just downloaded version 1.1.9 (latest, as of today), on an Ubuntu Linux, starting from tty with `startx`, I thought the bug would be corrected on version 1.1.9, but I still get the error message `Unsupported display server tty, x11 expected` :-(. ``` $ ps -e |grep tty 4000 tty1 00:00:00 login 4089 tty1 00:00:00 bash 4683 tty2 07:33:45 Xorg ``` I guess that's because I switched to `tty2` before running `startx`, and your code only checks the tty that has the login process (as I see from the patch). ``` $ rustdesk --version 1.1.9 ```
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (May 22, 2022):

Really? @mtoohey31 could you please take a look?

@rustdesk commented on GitHub (May 22, 2022): Really? @mtoohey31 could you please take a look?
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@mtoohey31 commented on GitHub (May 22, 2022):

Sorry, I've been using Wayland (and haven't been using rustdesk) for a while now, so I'm probably not the best person to test this as it would require a lot of setup. However, I could be wrong, but I don't think from looking at it that get_display_server_of_session is limited operating on any specific tty, I'm pretty sure it will operate on the session its called with, regardless of what tty it corresponds to. The function that calls it (get_display_server) will always call it with the session of seat0 though, so maybe that's related?

Looking back on the code for get_display_server_of_session this many months later it could definitely use a rewrite; there's got to be a way to accomplish this without running so many shell commands. But again, I don't have the setup to test it...

@mtoohey31 commented on GitHub (May 22, 2022): Sorry, I've been using Wayland (and haven't been using rustdesk) for a while now, so I'm probably not the best person to test this as it would require a lot of setup. However, I could be wrong, but I don't think from looking at it that `get_display_server_of_session` is limited operating on any specific tty, I'm pretty sure it will operate on the session its called with, regardless of what tty it corresponds to. The function that calls it (`get_display_server`) will always call it with the session of seat0 though, so maybe that's related? Looking back on the code for `get_display_server_of_session` this many months later it could definitely use a rewrite; there's got to be a way to accomplish this without running so many shell commands. But again, I don't have the setup to test it...
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@rustdesk commented on GitHub (May 22, 2022):

get_display_server_of_session

Thanks

@rustdesk commented on GitHub (May 22, 2022): > get_display_server_of_session Thanks
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@lstep commented on GitHub (May 23, 2022):

From what I see in the code (but I'm not a Rust user), it gets the tty where the login process is running, but from what I see from my list of processes, I only have a login process on tty1, but (I don't know why), none shown on tty2, although I am logged in on tty2 (did a ALT-F2 to change).

$ ps -e |grep tty
   4000 tty1     00:00:00 login
   4089 tty1     00:00:00 bash
   4683 tty2     07:33:45 Xorg
@lstep commented on GitHub (May 23, 2022): From what I see in the code (but I'm not a Rust user), it gets the tty where the login process is running, but from what I see from my list of processes, I only have a login process on tty1, but (I don't know why), none shown on tty2, although I *am* logged in on tty2 (did a ALT-F2 to change). ``` $ ps -e |grep tty 4000 tty1 00:00:00 login 4089 tty1 00:00:00 bash 4683 tty2 07:33:45 Xorg ```
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@schrmh commented on GitHub (Feb 16, 2025):

Looks like it is still an issue. (Also tried the flatpak, as recommended in the Artix issue.)
Also using startx,
then Arch Linux as a distro (so systemd)
and FVWM3 as a WM.

And it would be nice if there was any option to ignore that (setting, button, command line switch) since it blocks incoming connections....

@schrmh commented on GitHub (Feb 16, 2025): Looks like it is still an issue. (Also tried the flatpak, as recommended in the Artix issue.) Also using startx, then Arch Linux as a distro (so systemd) and FVWM3 as a WM. And it would be nice if there was any option to ignore that (setting, button, command line switch) since it blocks incoming connections....
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@drlat commented on GitHub (Jun 6, 2025):

I'm also using startx on archlinux. I use stumpwm. I'm surprised that this isn't fixed, since the code to fix has already been written. Is there any way around this problem.

@drlat commented on GitHub (Jun 6, 2025): I'm also using startx on archlinux. I use stumpwm. I'm surprised that this isn't fixed, since the code to fix has already been written. Is there any way around this problem.
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@GameKyuubi commented on GitHub (Jun 10, 2025):

Still busted. With XFCE via startxfce4, RustDesk is unusable. Not sure why this is such a problem; a lot of people who would use RDP also use TTY for stuff.

@GameKyuubi commented on GitHub (Jun 10, 2025): Still busted. With XFCE via startxfce4, RustDesk is unusable. Not sure why this is such a problem; a lot of people who would use RDP also use TTY for stuff.
deekerman commented 2026-02-20 22:21:55 -05:00
Author
Owner
Copy link

@JSouthGB commented on GitHub (Dec 4, 2025):

@schrmh @drlat @GameKyuubi

I also use startx. Setting XDG_SESSION_TYPE=x11 eliminated the error.

I found this comment posted by @DrewRidley

Try setting,

XDG_SESSION_TYPE=wayland

Or,

XDG_SESSION_TYPE=x11

when running rustdesk. It should solve this problem.

@JSouthGB commented on GitHub (Dec 4, 2025): @schrmh @drlat @GameKyuubi I also use startx. Setting `XDG_SESSION_TYPE=x11` eliminated the error. [I found this comment](https://github.com/rustdesk/rustdesk/issues/7531#issuecomment-2686806280) posted by @DrewRidley > Try setting, > > ``` > XDG_SESSION_TYPE=wayland > ``` > > Or, > > ``` > XDG_SESSION_TYPE=x11 > ``` > > when running rustdesk. It should solve this problem.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
avatar
copilot/fix-discussion-14282-issue
copilot/fix-h265-image-quality-issue
fix-mobile-mouse-hover-issue-8789
ios
sciterjs
nightly
1.4.5
1.4.4
1.4.3
1.4.2
1.4.1
1.4.0
1.3.9
1.3.8
1.3.7
1.3.6
1.3.5
1.3.4
1.3.3
1.3.2
1.3.1
1.3.0
1.2.7
1.2.6
1.2.5
1.2.4
fdroid-version
1.2.3-2
1.2.3-1
1.2.3
1.2.2
1.2.1
1.2.0
1.1.9
1.1.8
1.1.6
1.1.5
1.1.4
1.1.3
1.1.2
Labels
Clear labels   
bug

   
documentation

   
duplicate

   
enhancement

   
enhancement

   
enhancement

   
good first issue

   
help wanted

   
invalid

   
question

   
unreproducible

   
wontfix
No labels
bug
documentation
duplicate
enhancement
enhancement
enhancement
good first issue
help wanted
invalid
question
unreproducible
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/rustdesk-rustdesk#72
No description provided.