If there are more than 174 organizations per user, the synchronization no longer works. Limit 174 organizations per user? #1801

Closed
opened 2026-02-20 08:12:42 -05:00 by deekerman · 17 comments
Owner

Originally created by @mhmario on GitHub (Dec 11, 2023).

Subject of the issue

Currently I have the following problem: A user has 174 organizations. If this user has created another organization, synchronization will no longer work. All 175 organizations no longer appear in the web browser. When I try to synchronize in the Bitwarden APP, the error message appears: “Failed to synchronize”.

If I delete an organization using the admin token, everything works again without any problems.

Vaultwarden runs in a Docker container within ProxMox on the current version 2023.10.0 with the MySQL DB. In the OPNsense it is configured with Nginx.

Do you have a solution for the problem?

Deployment environment

2023.10.0

2023.10.0

  • vaultwarden version: 2023.10.0
  • Install method: Docker image

  • Clients used:

  • Google Chrome

  • Firefox

  • Windows-App

  • iOS APP

  • Reverse proxy and version:

  • Nginx on OPNsnese

  • MySQL/MariaDB or PostgreSQL version:

  • MySQL

  • Other relevant details:

Steps to reproduce

Everything is set to default. If you create the 175 organization for a user, the error occurs.

Expected behaviour

Actual behaviour

Troubleshooting data

Originally created by @mhmario on GitHub (Dec 11, 2023). <!-- # ### NOTE: Please update to the latest version of vaultwarden before reporting an issue! This saves you and us a lot of time and troubleshooting. See: * https://github.com/dani-garcia/vaultwarden/issues/1180 * https://github.com/dani-garcia/vaultwarden/wiki/Updating-the-vaultwarden-image # ### --> <!-- Please fill out the following template to make solving your problem easier and faster for us. This is only a guideline. If you think that parts are unnecessary for your issue, feel free to remove them. Remember to hide/redact personal or confidential information, such as passwords, IP addresses, and DNS names as appropriate. --> ### Subject of the issue Currently I have the following problem: A user has 174 organizations. If this user has created another organization, synchronization will no longer work. All 175 organizations no longer appear in the web browser. When I try to synchronize in the Bitwarden APP, the error message appears: “Failed to synchronize”. If I delete an organization using the admin token, everything works again without any problems. Vaultwarden runs in a Docker container within ProxMox on the current version 2023.10.0 with the MySQL DB. In the OPNsense it is configured with Nginx. Do you have a solution for the problem? ### Deployment environment <!-- ========================================================================================= Preferably, use the `Generate Support String` button on the admin page's Diagnostics tab. That will auto-generate most of the info requested in this section. ========================================================================================= --> <!-- The version number, obtained from the logs (at startup) or the admin diagnostics page --> 2023.10.0 <!-- This is NOT the version number shown on the web vault, which is versioned separately from vaultwarden --> 2023.10.0 <!-- Remember to check if your issue exists on the latest version first! --> * vaultwarden version: 2023.10.0 <!-- How the server was installed: Docker image, OS package, built from source, etc. --> * Install method: Docker image * Clients used: <!-- web vault, desktop, Android, iOS, etc. (if applicable) --> * Google Chrome * Firefox * Windows-App * iOS APP * Reverse proxy and version: <!-- if applicable --> * Nginx on OPNsnese * MySQL/MariaDB or PostgreSQL version: <!-- if applicable --> * MySQL * Other relevant details: ### Steps to reproduce <!-- Tell us how to reproduce this issue. What parameters did you set (differently from the defaults) and how did you start vaultwarden? --> Everything is set to default. If you create the 175 organization for a user, the error occurs. ### Expected behaviour <!-- Tell us what you expected to happen --> ### Actual behaviour <!-- Tell us what actually happened --> ### Troubleshooting data <!-- Share any log files, screenshots, or other relevant troubleshooting data -->
deekerman 2026-02-20 08:12:42 -05:00
Author
Owner

@BlackDex commented on GitHub (Dec 11, 2023):

What are the logs telling? And if possible also client issue?
Does this also happen at the browser/web-vault?
If so, what does the Developer Console say?

@BlackDex commented on GitHub (Dec 11, 2023): What are the logs telling? And if possible also client issue? Does this also happen at the browser/web-vault? If so, what does the Developer Console say?
Author
Owner

@mhmario commented on GitHub (Dec 11, 2023):

Hi BlackDex

Here are the logs

[2023-12-11 12:48:52.735][request][INFO] GET /api/config
[2023-12-11 12:48:52.735][response][INFO] (config) GET /api/config => 200 OK
[2023-12-11 12:48:54.492][request][INFO] GET /api/devices/knowndevice
[2023-12-11 12:48:54.492][response][INFO] (get_known_device) GET /api/devices/knowndevice => 200 OK
[2023-12-11 12:48:55.460][request][INFO] POST /api/organizations
[2023-12-11 12:48:55.462][response][INFO] (create_organization) POST /api/organizations => 200 OK
[2023-12-11 12:48:55.489][request][INFO] POST /identity/connect/token
[2023-12-11 12:48:55.490][response][INFO] (login) POST /identity/connect/token => 200 OK
[2023-12-11 12:48:55.650][rocket::data::data::][ERROR] Failed to read into peek buffer: Custom { kind: Other, error: hyper::Error(Body, Custom { kind: UnexpectedEof, error: IncompleteBody }) }.
[2023-12-11 12:48:55.650][request][INFO] POST /identity/connect/token
[2023-12-11 12:48:55.650][vaultwarden::api::identity::
][WARN] Data guard Form < ConnectData > failed: Errors([Error { name: Some("grant_type"), value: None, kind: Missing, entity: Field }]).
[2023-12-11 12:48:55.650][response][INFO] (login) POST /identity/connect/token => 422 Unprocessable Entity
[2023-12-11 12:49:01.248][request][INFO] POST /identity/accounts/prelogin
[2023-12-11 12:49:01.248][response][INFO] (prelogin) POST /identity/accounts/prelogin => 200 OK
[2023-12-11 12:49:01.502][request][INFO] POST /identity/connect/token
[2023-12-11 12:49:01.542][vaultwarden::api::identity][INFO] User xxx logged in successfully. IP: x.x.x.x
[2023-12-11 12:49:01.542][response][INFO] (login) POST /identity/connect/token => 200 OK
[2023-12-11 12:49:16.358][vaultwarden::api::notifications][INFO] Closing WS connection from
[2023-12-11 12:49:16.358][rocket::server][ERROR] Upgraded websocket I/O handler failed: WebSocket protocol error: Sending after closing is not allowed
[2023-12-11 12:50:59.771][request][INFO] GET /api/devices/knowndevice
[2023-12-11 12:50:59.771][response][INFO] (get_known_device) GET /api/devices/knowndevice => 200 OK
[2023-12-11 12:51:06.313][request][INFO] POST /identity/accounts/prelogin
[2023-12-11 12:51:06.314][response][INFO] (prelogin) POST /identity/accounts/prelogin => 200 OK
[2023-12-11 12:51:06.541][request][INFO] POST /identity/connect/token
[2023-12-11 12:51:06.581][vaultwarden::api::identity][INFO] User xxxx logged in successfully. IP: x.x.x.x
[2023-12-11 12:51:06.581][response][INFO] (login) POST /identity/connect/token => 200 OK
[2023-12-11 12:51:06.731][request][INFO] POST /identity/connect/token
[2023-12-11 12:51:06.732][response][INFO] (login) POST /identity/connect/token => 200 OK

Does this also happen at the browser/web-vault? -> Yes, that happens everywhere. Web browser, iOS APP, Windows APP

Developer Console -> Unfortunately I can't say that because I don't know how to use it

@mhmario commented on GitHub (Dec 11, 2023): Hi BlackDex Here are the logs [2023-12-11 12:48:52.735][request][INFO] GET /api/config [2023-12-11 12:48:52.735][response][INFO] (config) GET /api/config => 200 OK [2023-12-11 12:48:54.492][request][INFO] GET /api/devices/knowndevice [2023-12-11 12:48:54.492][response][INFO] (get_known_device) GET /api/devices/knowndevice => 200 OK [2023-12-11 12:48:55.460][request][INFO] POST /api/organizations [2023-12-11 12:48:55.462][response][INFO] (create_organization) POST /api/organizations => 200 OK [2023-12-11 12:48:55.489][request][INFO] POST /identity/connect/token [2023-12-11 12:48:55.490][response][INFO] (login) POST /identity/connect/token => 200 OK [2023-12-11 12:48:55.650][rocket::data::data::_][ERROR] Failed to read into peek buffer: Custom { kind: Other, error: hyper::Error(Body, Custom { kind: UnexpectedEof, error: IncompleteBody }) }. [2023-12-11 12:48:55.650][request][INFO] POST /identity/connect/token [2023-12-11 12:48:55.650][vaultwarden::api::identity::_][WARN] Data guard `Form < ConnectData >` failed: Errors([Error { name: Some("grant_type"), value: None, kind: Missing, entity: Field }]). [2023-12-11 12:48:55.650][response][INFO] (login) POST /identity/connect/token => 422 Unprocessable Entity [2023-12-11 12:49:01.248][request][INFO] POST /identity/accounts/prelogin [2023-12-11 12:49:01.248][response][INFO] (prelogin) POST /identity/accounts/prelogin => 200 OK [2023-12-11 12:49:01.502][request][INFO] POST /identity/connect/token [2023-12-11 12:49:01.542][vaultwarden::api::identity][INFO] User xxx logged in successfully. IP: x.x.x.x [2023-12-11 12:49:01.542][response][INFO] (login) POST /identity/connect/token => 200 OK [2023-12-11 12:49:16.358][vaultwarden::api::notifications][INFO] Closing WS connection from [2023-12-11 12:49:16.358][rocket::server][ERROR] Upgraded websocket I/O handler failed: WebSocket protocol error: Sending after closing is not allowed [2023-12-11 12:50:59.771][request][INFO] GET /api/devices/knowndevice [2023-12-11 12:50:59.771][response][INFO] (get_known_device) GET /api/devices/knowndevice => 200 OK [2023-12-11 12:51:06.313][request][INFO] POST /identity/accounts/prelogin [2023-12-11 12:51:06.314][response][INFO] (prelogin) POST /identity/accounts/prelogin => 200 OK [2023-12-11 12:51:06.541][request][INFO] POST /identity/connect/token [2023-12-11 12:51:06.581][vaultwarden::api::identity][INFO] User xxxx logged in successfully. IP: x.x.x.x [2023-12-11 12:51:06.581][response][INFO] (login) POST /identity/connect/token => 200 OK [2023-12-11 12:51:06.731][request][INFO] POST /identity/connect/token [2023-12-11 12:51:06.732][response][INFO] (login) POST /identity/connect/token => 200 OK Does this also happen at the browser/web-vault? -> Yes, that happens everywhere. Web browser, iOS APP, Windows APP Developer Console -> Unfortunately I can't say that because I don't know how to use it
Author
Owner

@BlackDex commented on GitHub (Dec 11, 2023):

I actually think the body size is too large and causes your reverse proxy to truncate or something the body part.
Both sending and receiving maybe.

Please check the config of your reverse proxy and the limits it has configured (could be defaults).

@BlackDex commented on GitHub (Dec 11, 2023): I actually think the body size is too large and causes your reverse proxy to truncate or something the body part. Both sending and receiving maybe. Please check the config of your reverse proxy and the limits it has configured (could be defaults).
Author
Owner

@mhmario commented on GitHub (Dec 12, 2023):

Hello, we checked this today. The default value for us is 200M. Even after changing to client_max_body_size 525M; didn't bring any change. We got the value of 525M from the proxy examples https://github.com/dani-garcia/vaultwarden/wiki/Proxy-examples

@mhmario commented on GitHub (Dec 12, 2023): Hello, we checked this today. The default value for us is 200M. Even after changing to client_max_body_size 525M; didn't bring any change. We got the value of 525M from the proxy examples https://github.com/dani-garcia/vaultwarden/wiki/Proxy-examples
Author
Owner

@dani-garcia commented on GitHub (Dec 12, 2023):

@BlackDex We're adding a list of all the organization IDs to the encoded JWT token (orgowner, orgadmin, orguser,orgmanager), could we have gone over some limit query parameter/header limit that is truncating the request in the clients?

I don't think we ever use those fields for any reason, they're just there to match what the official server is doing, and the clients don't use them either, so if they are the cause we could just remove them.

@dani-garcia commented on GitHub (Dec 12, 2023): @BlackDex We're adding a list of all the organization IDs to the encoded JWT token (orgowner, orgadmin, orguser,orgmanager), could we have gone over some limit query parameter/header limit that is truncating the request in the clients? I don't think we ever use those fields for any reason, they're just there to match what the official server is doing, and the clients don't use them either, so if they are the cause we could just remove them.
Author
Owner

@BlackDex commented on GitHub (Dec 12, 2023):

@dani-garcia, That could be indeed. Also using a different encoding for the JWT could help in the size maybe. But not too much I think.
If we do not use it in any way, then it's fine to remove, but i thought we use it it some auth functions though.

If still needed, we could see if compressing some parts or the whole token could be an option.

@BlackDex commented on GitHub (Dec 12, 2023): @dani-garcia, That could be indeed. Also using a different encoding for the JWT could help in the size maybe. But not too much I think. If we do not use it in any way, then it's fine to remove, but i thought we use it it some auth functions though. If still needed, we could see if compressing some parts or the whole token could be an option.
Author
Owner

@mhmario commented on GitHub (Dec 13, 2023):

Hello everyone,

first of all, thank you very much for the quick and uncomplicated support. Unfortunately, we cannot continue to use the system at the moment because we can no longer create organizations.

Can you estimate how long it will take until the problem is fixed? Or do you have a workaround that could help us in the short term? I would also be happy to quickly install a patch or something similar at short notice.

I would also like to honor your support with a donation. Can you issue a donation receipt for a German company?

@mhmario commented on GitHub (Dec 13, 2023): Hello everyone, first of all, thank you very much for the quick and uncomplicated support. Unfortunately, we cannot continue to use the system at the moment because we can no longer create organizations. Can you estimate how long it will take until the problem is fixed? Or do you have a workaround that could help us in the short term? I would also be happy to quickly install a patch or something similar at short notice. I would also like to honor your support with a donation. Can you issue a donation receipt for a German company?
Author
Owner

@BlackDex commented on GitHub (Dec 13, 2023):

Im not sure how many users are in all those organizations, but if not all users are in there, and only an admin, then a quick workaround would be creating a second owner which creates a new org and is not a member of all the others.

@BlackDex commented on GitHub (Dec 13, 2023): Im not sure how many users are in all those organizations, but if not all users are in there, and only an admin, then a quick workaround would be creating a second owner which creates a new org and is not a member of all the others.
Author
Owner

@mhmario commented on GitHub (Dec 13, 2023):

Thank you for the idea. But unfortunately that is not manageable. We manage a lot of organizations and would lose track. Every time you have to think about which user you have to login in with in order to get the organization's access data.

Can you estimate how much time you need for the bug fix?

@mhmario commented on GitHub (Dec 13, 2023): Thank you for the idea. But unfortunately that is not manageable. We manage a lot of organizations and would lose track. Every time you have to think about which user you have to login in with in order to get the organization's access data. Can you estimate how much time you need for the bug fix?
Author
Owner

@BlackDex commented on GitHub (Dec 13, 2023):

Not really from the top of my head. I need to check if it really isn't used anywhere or if there is a different solution.
As mentioned it would be a temp workaround for now.

I See compression is not an option in a default way at least, the create doesn't support that, so that is at least out of the question.

@BlackDex commented on GitHub (Dec 13, 2023): Not really from the top of my head. I need to check if it really isn't used anywhere or if there is a different solution. As mentioned it would be a temp workaround for now. I See compression is not an option in a default way at least, the create doesn't support that, so that is at least out of the question.
Author
Owner

@BlackDex commented on GitHub (Dec 13, 2023):

I quickly tested removing those items from the JWT struct, and it seems to be working just fine.
I also checked the client code, and can't find that it is used anywhere as @dani-garcia already mentioned.

Ill create a PR for this.

@BlackDex commented on GitHub (Dec 13, 2023): I quickly tested removing those items from the JWT struct, and it seems to be working just fine. I also checked the client code, and can't find that it is used anywhere as @dani-garcia already mentioned. Ill create a PR for this.
Author
Owner

@BlackDex commented on GitHub (Dec 13, 2023):

@mhmario When the new testing containers are done building, somewhere within the next hour, could you test that container please? And let us know if that solves your issue?

If not, please reopen this issue.

Update:
It's this job btw: https://github.com/dani-garcia/vaultwarden/actions/runs/7198422630

@BlackDex commented on GitHub (Dec 13, 2023): @mhmario When the new testing containers are done building, somewhere within the next hour, could you test that container please? And let us know if that solves your issue? If not, please reopen this issue. Update: It's this job btw: https://github.com/dani-garcia/vaultwarden/actions/runs/7198422630
Author
Owner

@mhmario commented on GitHub (Dec 13, 2023):

@BlackDex

naturally. Our system admin will test this first thing tomorrow and I'll let you know right away.

I would like to thank you very much in advance for this quick help

@mhmario commented on GitHub (Dec 13, 2023): @BlackDex naturally. Our system admin will test this first thing tomorrow and I'll let you know right away. I would like to thank you very much in advance for this quick help
Author
Owner

@mhmario commented on GitHub (Dec 14, 2023):

Perfect!!! It works perfectly 😊. Wow was a service. I would like to spend. Can you issue an invoice?

@mhmario commented on GitHub (Dec 14, 2023): Perfect!!! It works perfectly 😊. Wow was a service. I would like to spend. Can you issue an invoice?
Author
Owner

@BlackDex commented on GitHub (Dec 14, 2023):

@mhmario I'm glad it all works great.

We do not really have an invoice system since this is just an Open Source project and do not charge anyone for fixes or additions.
I do know that Liberapay provides an invoice for donations made.

Both me and dani have a Librepay page.

@BlackDex commented on GitHub (Dec 14, 2023): @mhmario I'm glad it all works great. We do not really have an invoice system since this is just an Open Source project and do not charge anyone for fixes or additions. I do know that Liberapay provides an invoice for donations made. Both me and dani have a Librepay page. - https://liberapay.com/BlackDex - https://liberapay.com/dani-garcia
Author
Owner

@mhmario commented on GitHub (Oct 4, 2024):

Hello everyone,

Unfortunately I have to report again now. We have now created almost 200 organizations. When we log in to Vaultwarden now, the loading process takes about 70 - 80 seconds before we see the passwords. Vaultwarden is probably loading all organizations in the background. Can you improve that? It would help if all organizations and collections weren't always fully loaded in the background

Vaultwarden

Best regards

Mario

@mhmario commented on GitHub (Oct 4, 2024): Hello everyone, Unfortunately I have to report again now. We have now created almost 200 organizations. When we log in to Vaultwarden now, the loading process takes about 70 - 80 seconds before we see the passwords. Vaultwarden is probably loading all organizations in the background. Can you improve that? It would help if all organizations and collections weren't always fully loaded in the background ![Vaultwarden](https://github.com/user-attachments/assets/c40944d9-e2cb-4e9f-8bf8-0d03d09aa05a) Best regards Mario
Author
Owner

@BlackDex commented on GitHub (Oct 4, 2024):

Vaultwarden does not load anything.
Vaultwarden returns all data which a user has access to during a sync request. The clients then need to handle this.

There is no lazy loading of vault items. And if anything like that need to be added, it needs to be done by the clients.
I'm afraid we can't do anything here. Maybe future web-vaults have some optimizations for this, but currently we are stuck at v2024.6.2 because we do not have the new roles/collections feature in Vaultwarden.

@BlackDex commented on GitHub (Oct 4, 2024): Vaultwarden does not load anything. Vaultwarden returns all data which a user has access to during a sync request. The clients then need to handle this. There is no lazy loading of vault items. And if anything like that need to be added, it needs to be done by the clients. I'm afraid we can't do anything here. Maybe future web-vaults have some optimizations for this, but currently we are stuck at v2024.6.2 because we do not have the new roles/collections feature in Vaultwarden.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/vaultwarden#1801
No description provided.