Remote MySQL Connection drops and never comes back

deekerman commented

2026-02-20 08:05:38 -05:00

Owner

Originally created by @misilot on GitHub (Oct 28, 2020).

Subject of the issue

Connection to remote mysql just drops for some reason and never comes back

Your environment

Bitwarden_rs version:

Install method: Docker Compose
Reverse proxy and version: Apache 2.4.29
Version of mysql/postgresql:
Other relevant information:

Steps to reproduce

My connection string in docker-compose.yml
DATABASE_URL: "mysql://bitwarden_rs:password@mysql.example.net:53306/bitwarden_rs"

Expected behaviour

Connect all happy as can be?

Actual behaviour

Eventually runs if I do docker compose down; docker compose up -d

Relevant logs

bitwardenrs | [2020-10-28 09:40:17.254][r2d2][ERROR] Unknown MySQL server host 'mysql.server.net' (-3)

After restarting I see this. I have to do a complete down and up to get bitwarden_rs working again.
bitwardenrs | [2020-10-28 12:55:10.450][bitwarden_rs::util][WARN] Can't connect to database, retrying: DieselConError.
bitwardenrs | [CAUSE] BadConnection(
bitwardenrs | "Unknown MySQL server host 'mysql.server.net' (-3)",
bitwardenrs | )

Originally created by @misilot on GitHub (Oct 28, 2020).   ### Subject of the issue Connection to remote mysql just drops for some reason and never comes back ### Your environment   * Bitwarden_rs version:  * Install method: Docker Compose * Reverse proxy and version: Apache 2.4.29 * Version of mysql/postgresql: * Other relevant information: ### Steps to reproduce  My connection string in docker-compose.yml DATABASE_URL: "mysql://bitwarden_rs:password@mysql.example.net:53306/bitwarden_rs" ### Expected behaviour  Connect all happy as can be? ### Actual behaviour  Eventually runs if I do docker compose down; docker compose up -d ### Relevant logs  bitwardenrs | [2020-10-28 09:40:17.254][r2d2][ERROR] Unknown MySQL server host 'mysql.server.net' (-3) After restarting I see this. I have to do a complete down and up to get bitwarden_rs working again. bitwardenrs | [2020-10-28 12:55:10.450][bitwarden_rs::util][WARN] Can't connect to database, retrying: DieselConError. bitwardenrs | [CAUSE] BadConnection( bitwardenrs | "Unknown MySQL server host \'mysql.server.net\' (-3)", bitwardenrs | )

deekerman

2026-02-20 08:05:38 -05:00

closed this issue
added the
question

troubleshooting
labels

deekerman commented

2026-02-20 08:05:39 -05:00

Author

Owner

@dani-garcia commented on GitHub (Nov 7, 2020):

So restarting bitwarden_rs doesn't fix it, you need to restart the whole docker compose group? Sounds some kind of networking issue in docker, can you share your compose file?

@dani-garcia commented on GitHub (Nov 7, 2020): So restarting bitwarden_rs doesn't fix it, you need to restart the whole docker compose group? Sounds some kind of networking issue in docker, can you share your compose file?

deekerman commented

2026-02-20 08:05:39 -05:00

Author

Owner

@misilot commented on GitHub (Nov 7, 2020):

#---
#Docker-compose file for Bitwarden_rs
#---
version: "3"
services:
  bitwardenrs:
    restart: always
    # Dani Garcia image https://github.com/dani-garcia/bitwarden_rs
    image: "bitwardenrs/server:latest"
    container_name: bitwardenrs
    environment:
      # Timezone settings, important for Fail2ban to work
      TZ: 'America/Chicago'
      # Logging connection attemps
      LOG_FILE: '/data/bitwarden.log'
      EXTENDED_LOGGING: 'true'
      LOG_LEVEL: 'warn'
      # Beef up a bit
      ROCKET_WORKERS: 20
      WEBSOCKET_ENABLED: 'true'
      # Hardening a bit
      SIGNUPS_ALLOWED: 'false'
      ADMIN_TOKEN: '<removed>'
      DOMAIN: 'https://key.domain'
      SHOW_PASSWORD_HINT: 'false'
      DISABLE_ICON_DOWNLOAD: 'true'
      SMTP_HOST: 'apprelay.smtp.domain'
      SMTP_PORT: 25
      SMTP_SSL: 'false'
      SMTP_FROM: 'adm@domain'
      SMTP_FROM_NAME: 'IT'
      DUO_IKEY: 'dafsdf'
      DUO_SKEY: 'fasfsad'
      DUO_HOST: 'api-.duosecurity.com'
      INVITATION_ORG_NAME: 'Password Management'
      ORG_CREATION_USERS: 'libadm@domain'
    networks:
      - bitwarden_net
    ports:
      - '9080:80'
      - '9082:3012'
    volumes:
      - "/klib/data/bw-data:/data"

volumes:
  Certfiles:

networks:
  bitwarden_net:

DATABASE_URL: "mysql://bitwarden_rs:password@mysql.example.net:53306/bitwarden_rs" was in there as well, but I had to switch to sqlite backend for now.

I did end up removing the networks: bitwarden_net configuration currently as well trying to troubleshoot #1203

My docker-compose file is based on https://github.com/sosandroid/docker-bitwarden_rs-caddy-synology/blob/master/docker-compose_bitwarden-caddy.yml

@misilot commented on GitHub (Nov 7, 2020): ```yaml #--- #Docker-compose file for Bitwarden_rs #--- version: "3" services: bitwardenrs: restart: always # Dani Garcia image https://github.com/dani-garcia/bitwarden_rs image: "bitwardenrs/server:latest" container_name: bitwardenrs environment: # Timezone settings, important for Fail2ban to work TZ: 'America/Chicago' # Logging connection attemps LOG_FILE: '/data/bitwarden.log' EXTENDED_LOGGING: 'true' LOG_LEVEL: 'warn' # Beef up a bit ROCKET_WORKERS: 20 WEBSOCKET_ENABLED: 'true' # Hardening a bit SIGNUPS_ALLOWED: 'false' ADMIN_TOKEN: '<removed>' DOMAIN: 'https://key.domain' SHOW_PASSWORD_HINT: 'false' DISABLE_ICON_DOWNLOAD: 'true' SMTP_HOST: 'apprelay.smtp.domain' SMTP_PORT: 25 SMTP_SSL: 'false' SMTP_FROM: 'adm@domain' SMTP_FROM_NAME: 'IT' DUO_IKEY: 'dafsdf' DUO_SKEY: 'fasfsad' DUO_HOST: 'api-.duosecurity.com' INVITATION_ORG_NAME: 'Password Management' ORG_CREATION_USERS: 'libadm@domain' networks: - bitwarden_net ports: - '9080:80' - '9082:3012' volumes: - "/klib/data/bw-data:/data" volumes: Certfiles: networks: bitwarden_net: ``` DATABASE_URL: "mysql://bitwarden_rs:password@mysql.example.net:53306/bitwarden_rs" was in there as well, but I had to switch to sqlite backend for now. I did end up removing the networks: bitwarden_net configuration currently as well trying to troubleshoot #1203 My docker-compose file is based on https://github.com/sosandroid/docker-bitwarden_rs-caddy-synology/blob/master/docker-compose_bitwarden-caddy.yml

deekerman commented

2026-02-20 08:05:40 -05:00

Author

Owner

@jean-io commented on GitHub (Nov 8, 2020):

Strange, I often get theses errors in my logs:

[2020-11-08 12:02:19.733][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.733][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.734][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.734][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away
[2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away

But bitwarden_rs has no trouble re-opening connections to my remote MySQL server and there is no bug visible to end users.

Have you tried to add the hostname of your mysql server to container's /etc/hosts file?

My docker compose file has a volume for /etc/hosts, it helps on managing remote IPs (warning: it may not be best practices):

volumes:
      - '/etc/hosts:/etc/hosts'

@jean-io commented on GitHub (Nov 8, 2020): Strange, I often get theses errors in my logs: ``` [2020-11-08 12:02:19.733][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.733][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.734][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.734][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away [2020-11-08 12:02:19.735][r2d2][ERROR] MySQL server has gone away ``` But bitwarden_rs has no trouble re-opening connections to my remote MySQL server and there is no bug visible to end users. Have you tried to add the hostname of your mysql server to container's `/etc/hosts` file? My docker compose file has a volume for `/etc/hosts`, it helps on managing remote IPs **(warning: it may not be best practices)**: ``` volumes: - '/etc/hosts:/etc/hosts' ```

deekerman commented

2026-02-20 08:05:40 -05:00

Author

Owner

@sgissi commented on GitHub (Nov 11, 2020):

Sounds like a DNS issue to resolve the MySQL host name indeed. Is the DNS entry somehow dynamic (e.g. provided by Docker, Kubernetes, Cloud provider, etc)? When you have the issue, can you attach to the container and try to resolve the MySQL's FQDN? How often the problem appears?

@sgissi commented on GitHub (Nov 11, 2020): Sounds like a DNS issue to resolve the MySQL host name indeed. Is the DNS entry somehow dynamic (e.g. provided by Docker, Kubernetes, Cloud provider, etc)? When you have the issue, can you attach to the container and try to resolve the MySQL's FQDN? How often the problem appears?

deekerman commented

2026-02-20 08:05:40 -05:00

Author

Owner

@misilot commented on GitHub (Nov 11, 2020):

@sgissi hostname for me the hostname was static and not provided by docker. The server is running as a standalone server. The machine running docker has other applications (not via docker) as well as other systems that talk to and use mysql constantly, and those never seem to skip a beat.

The problem was occurring every few hours. I wasn't using the system all the time but when it failed the connection, I wasn't able to login to the vault using my credentials until I stopped and started the container again.

I thought I tried doing a nslookup but the command didn't work (it has been a little while since I tried). Should that work, or is there another way to test and resolve the FQDN when it fails. I can look at creating a new container that uses the mysql backend to test again. We switched to sqlite as we wanted to make the switch sooner rather than later.

@misilot commented on GitHub (Nov 11, 2020): @sgissi hostname for me the hostname was static and not provided by docker. The server is running as a standalone server. The machine running docker has other applications (not via docker) as well as other systems that talk to and use mysql constantly, and those never seem to skip a beat. The problem was occurring every few hours. I wasn't using the system all the time but when it failed the connection, I wasn't able to login to the vault using my credentials until I stopped and started the container again. I thought I tried doing a nslookup but the command didn't work (it has been a little while since I tried). Should that work, or is there another way to test and resolve the FQDN when it fails. I can look at creating a new container that uses the mysql backend to test again. We switched to sqlite as we wanted to make the switch sooner rather than later.

deekerman commented

2026-02-20 08:05:40 -05:00

Author

Owner

@BlackDex commented on GitHub (Nov 18, 2020):

@misilot to do a DNS lookup from within the container you could run the following command on the docker host.

# IPv4 only
docker exec -it bitwarden getent ahosts mysql.example.net
# IPv6 when available
docker exec -it bitwarden getent hosts mysql.example.net

Now i actually do not know if the mysql library uses IPv6 to connect, but just to be sure you can check if it resolves to something.

There also is a DNS check in the admin interface under the diagnostics page which does a DNS check for github.com.

Other then that i have no clue on what it could be. Maybe enabling debug log_level for a while could result in some more info.

@BlackDex commented on GitHub (Nov 18, 2020): @misilot to do a DNS lookup from within the container you could run the following command on the docker host. ```bash # IPv4 only docker exec -it bitwarden getent ahosts mysql.example.net # IPv6 when available docker exec -it bitwarden getent hosts mysql.example.net ``` Now i actually do not know if the mysql library uses IPv6 to connect, but just to be sure you can check if it resolves to something. There also is a DNS check in the admin interface under the diagnostics page which does a DNS check for github.com. Other then that i have no clue on what it could be. Maybe enabling debug log_level for a while could result in some more info.

deekerman commented

2026-02-20 08:05:40 -05:00

Author

Owner

@BlackDex commented on GitHub (Dec 12, 2020):

@misilot, have you solved this issue? Is this still relevant using the latest or testing version?

@BlackDex commented on GitHub (Dec 12, 2020): @misilot, have you solved this issue? Is this still relevant using the `latest` or `testing` version?

deekerman commented

2026-02-20 08:05:40 -05:00

Author

Owner

@misilot commented on GitHub (Dec 16, 2020):

No I haven't been able to, we switched to sqllite and that appears to be
working for fine for us. If you want you can close this, and I can try and
circle back to it in the near future.

On Sat, Dec 12, 2020 at 10:50 AM Mathijs van Veluw notifications@github.com
wrote:

@misilot https://github.com/misilot, have you solved this issue? Is
this still relevant using the latest or testing version?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dani-garcia/bitwarden_rs/issues/1204#issuecomment-743782807,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AALBHSDV3YJLNX6EOUA7OODSUONLVANCNFSM4TCWMTEQ
.

@misilot commented on GitHub (Dec 16, 2020): No I haven't been able to, we switched to sqllite and that appears to be working for fine for us. If you want you can close this, and I can try and circle back to it in the near future. On Sat, Dec 12, 2020 at 10:50 AM Mathijs van Veluw <notifications@github.com> wrote: > @misilot <https://github.com/misilot>, have you solved this issue? Is > this still relevant using the latest or testing version? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/dani-garcia/bitwarden_rs/issues/1204#issuecomment-743782807>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AALBHSDV3YJLNX6EOUA7OODSUONLVANCNFSM4TCWMTEQ> > . >

deekerman commented

2026-02-20 08:05:40 -05:00

Author

Owner

@BlackDex commented on GitHub (Jan 31, 2021):

Closing this issue as requested.

@BlackDex commented on GitHub (Jan 31, 2021): Closing this issue as requested.

deekerman referenced this issue

2026-02-20 08:08:12 -05:00

Env-Variable - Renaming SMTP_EXPLICIT_TLS #1194