[WARN] tls handshake with 127.0.0.1:XXXXX failed: received corrupt message of type InvalidContentType #1786

Closed
opened 2026-02-20 08:12:36 -05:00 by deekerman · 8 comments
Owner

Originally created by @rwjack on GitHub (Nov 28, 2023).

Subject of the issue

Container shows as unhealthy on portainer, even though everything is working. Unsure why I'm getting these WARN logs, the curl healthcheck works within the container.

[2023-11-28 13:04:14.301][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:39154 failed: received corrupt message of type InvalidContentType
[2023-11-28 13:05:14.348][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:51004 failed: received corrupt message of type InvalidContentType
[2023-11-28 13:05:30.237][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:34490 failed: cannot decrypt peer's message
[2023-11-28 13:06:14.389][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:43980 failed: received corrupt message of type InvalidContentType
[2023-11-28 13:07:14.432][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:51626 failed: received corrupt message of type InvalidContentType

Deployment environment

  • vaultwarden version:
    1.30.1
  • Install method:
    Docker

  • Clients used:
    Irellevant

  • Reverse proxy and version:
    Irellevant

  • Other relevant details:

version: "3"

volumes:
  data:

services:
  bitwarden:
    image: vaultwarden/server:latest
    restart: unless-stopped

    container_name: bitwarden
    hostname: bitwarden

    environment:
      - TZ=[redacted]

    ports:
      - "[redacted]:80"
      #- "[redacted]:3012" WSS disabled due to bw extension data leakage via GET request

    volumes:
      - data:/data/
      - ./certs/:/etc/ssl/custom/
      - ./.env:/.env:ro

Relevant .env changes:

DOMAIN=https://[redacted]

## Rocket specific settings
## See https://rocket.rs/v0.4/guide/configuration/ for more details.
# ROCKET_ADDRESS=0.0.0.0
# ROCKET_PORT=80  # Defaults to 80 in the Docker images, or 8000 otherwise.
# ROCKET_WORKERS=10
ROCKET_TLS={certs="/etc/ssl/custom/[redacted].pem",key="/etc/ssl/custom/[redacted]-key.pem"}

Steps to reproduce

Expected behaviour

Container to show as healthy

Actual behaviour

Container shows as unhealthy, even though everything is working.

Troubleshooting data

root@bitwarden:/# curl --insecure --fail --silent --show-error https://localhost:80/alive || exit 1
"2023-11-28T12:06:47.277011Z"root@bitwarden:/# echo $?
0
Originally created by @rwjack on GitHub (Nov 28, 2023). <!-- # ### NOTE: Please update to the latest version of vaultwarden before reporting an issue! This saves you and us a lot of time and troubleshooting. See: * https://github.com/dani-garcia/vaultwarden/issues/1180 * https://github.com/dani-garcia/vaultwarden/wiki/Updating-the-vaultwarden-image # ### --> <!-- Please fill out the following template to make solving your problem easier and faster for us. This is only a guideline. If you think that parts are unnecessary for your issue, feel free to remove them. Remember to hide/redact personal or confidential information, such as passwords, IP addresses, and DNS names as appropriate. --> ### Subject of the issue <!-- Describe your issue here. --> Container shows as unhealthy on portainer, even though everything is working. Unsure why I'm getting these WARN logs, the curl healthcheck works within the container. ``` [2023-11-28 13:04:14.301][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:39154 failed: received corrupt message of type InvalidContentType [2023-11-28 13:05:14.348][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:51004 failed: received corrupt message of type InvalidContentType [2023-11-28 13:05:30.237][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:34490 failed: cannot decrypt peer's message [2023-11-28 13:06:14.389][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:43980 failed: received corrupt message of type InvalidContentType [2023-11-28 13:07:14.432][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:51626 failed: received corrupt message of type InvalidContentType ``` ### Deployment environment <!-- ========================================================================================= Preferably, use the `Generate Support String` button on the admin page's Diagnostics tab. That will auto-generate most of the info requested in this section. ========================================================================================= --> <!-- The version number, obtained from the logs (at startup) or the admin diagnostics page --> <!-- This is NOT the version number shown on the web vault, which is versioned separately from vaultwarden --> <!-- Remember to check if your issue exists on the latest version first! --> * vaultwarden version: 1.30.1 <!-- How the server was installed: Docker image, OS package, built from source, etc. --> * Install method: Docker * Clients used: <!-- web vault, desktop, Android, iOS, etc. (if applicable) --> Irellevant * Reverse proxy and version: <!-- if applicable --> Irellevant * Other relevant details: ``` version: "3" volumes: data: services: bitwarden: image: vaultwarden/server:latest restart: unless-stopped container_name: bitwarden hostname: bitwarden environment: - TZ=[redacted] ports: - "[redacted]:80" #- "[redacted]:3012" WSS disabled due to bw extension data leakage via GET request volumes: - data:/data/ - ./certs/:/etc/ssl/custom/ - ./.env:/.env:ro ``` Relevant .env changes: ``` DOMAIN=https://[redacted] ## Rocket specific settings ## See https://rocket.rs/v0.4/guide/configuration/ for more details. # ROCKET_ADDRESS=0.0.0.0 # ROCKET_PORT=80 # Defaults to 80 in the Docker images, or 8000 otherwise. # ROCKET_WORKERS=10 ROCKET_TLS={certs="/etc/ssl/custom/[redacted].pem",key="/etc/ssl/custom/[redacted]-key.pem"} ``` ### Steps to reproduce <!-- Tell us how to reproduce this issue. What parameters did you set (differently from the defaults) and how did you start vaultwarden? --> ### Expected behaviour Container to show as healthy ### Actual behaviour <!-- Tell us what actually happened --> Container shows as unhealthy, even though everything is working. ### Troubleshooting data <!-- Share any log files, screenshots, or other relevant troubleshooting data --> ``` root@bitwarden:/# curl --insecure --fail --silent --show-error https://localhost:80/alive || exit 1 "2023-11-28T12:06:47.277011Z"root@bitwarden:/# echo $? 0 ```
Author
Owner

@BlackDex commented on GitHub (Dec 1, 2023):

It seems to work fine for me.
Try setting the LOG_LEVEL to debug and see if you get some more information.
I tested it with curl versions, 7.68.0, 7.81.0, 7.88.1 and 8.4.0, they all worked fine.

Tested both the FQDN and localhost nothing breaks.

It could be your host platform which causes some (Open)SSL client settings to be different host-wide, and causes this to happen.
The certs might be unsupported in some way, or not have a full chain available?

Also, i see you use port 80, while it looks like you do a port-proxy via Docker. Not sure if you use a reverse proxy in-front of Vaultwarden, if so, it could also be your reverse proxy which is causes an issue.

I'm not able to reproduce this on my side.

@BlackDex commented on GitHub (Dec 1, 2023): It seems to work fine for me. Try setting the `LOG_LEVEL` to debug and see if you get some more information. I tested it with curl versions, `7.68.0`, `7.81.0`, `7.88.1` and `8.4.0`, they all worked fine. Tested both the FQDN and `localhost` nothing breaks. It could be your host platform which causes some (Open)SSL client settings to be different host-wide, and causes this to happen. The certs might be unsupported in some way, or not have a full chain available? Also, i see you use port `80`, while it looks like you do a port-proxy via Docker. Not sure if you use a reverse proxy in-front of Vaultwarden, if so, it could also be your reverse proxy which is causes an issue. I'm not able to reproduce this on my side.
Author
Owner

@rwjack commented on GitHub (Dec 1, 2023):

Try setting the LOG_LEVEL to debug and see if you get some more information.

Let me give it a shot.


It could be your host platform which causes some (Open)SSL client settings to be different host-wide, and causes this to happen.

Debian 11 with the latest version of Docker, I doubt that's the issue.


The certs might be unsupported in some way, or not have a full chain available?

I don't think they do have a full chain, it's just a regular certificate generated by mkcert


Also, i see you use port 80, while it looks like you do a port-proxy via Docker. Not sure if you use a reverse proxy in-front of Vaultwarden, if so, it could also be your reverse proxy which is causes an issue.

Port 80 is the default for the container. I expose another port on the Debian host, which is open only to the reverse proxy, but I don't see how that's related, as the healthcheck should curl "localhost", since ROCKET_ADDRESS (by default I hope) is configured to 0.0.0.0.

addr="${ROCKET_ADDRESS}"
if [ -z "${addr}" ] || [ "${addr}" = '0.0.0.0' ] || [ "${addr}" = '::' ]; then
    addr='localhost'
fi
@rwjack commented on GitHub (Dec 1, 2023): > Try setting the LOG_LEVEL to debug and see if you get some more information. Let me give it a shot. --- > It could be your host platform which causes some (Open)SSL client settings to be different host-wide, and causes this to happen. Debian 11 with the latest version of Docker, I doubt that's the issue. --- > The certs might be unsupported in some way, or not have a full chain available? I don't think they do have a full chain, it's just a regular certificate generated by [mkcert](https://github.com/FiloSottile/mkcert/) --- > Also, i see you use port 80, while it looks like you do a port-proxy via Docker. Not sure if you use a reverse proxy in-front of Vaultwarden, if so, it could also be your reverse proxy which is causes an issue. Port 80 is the default for the container. I expose another port on the Debian host, which is open only to the reverse proxy, but I don't see how that's related, as the healthcheck should curl "localhost", since `ROCKET_ADDRESS` (by default I hope) is configured to `0.0.0.0`. ``` addr="${ROCKET_ADDRESS}" if [ -z "${addr}" ] || [ "${addr}" = '0.0.0.0' ] || [ "${addr}" = '::' ]; then addr='localhost' fi ```
Author
Owner

@rwjack commented on GitHub (Dec 1, 2023):

@BlackDex Though I didn't just try to run the healthcheck manually.

This is the error:

root@bitwarden:/# ./healthcheck.sh
curl: (1) Received HTTP/0.9 when not allowed

And the timestamp matches a corresponding error in the container logs:

[2023-12-01 12:41:08.865][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:34488 failed: received corrupt message of type InvalidContentType

Relevant issue: https://github.com/curl/curl/issues/12183

@rwjack commented on GitHub (Dec 1, 2023): @BlackDex Though I didn't just try to run the healthcheck manually. This is the error: ``` root@bitwarden:/# ./healthcheck.sh curl: (1) Received HTTP/0.9 when not allowed ``` And the timestamp matches a corresponding error in the container logs: ``` [2023-12-01 12:41:08.865][rocket_http::tls::listener][WARN] tls handshake with 127.0.0.1:34488 failed: received corrupt message of type InvalidContentType ``` Relevant issue: https://github.com/curl/curl/issues/12183
Author
Owner

@BlackDex commented on GitHub (Dec 1, 2023):

Ah! Looks like ROCKET_TLS is not seen as a env variable to the healthcheck.sh script.
Probably because you are using a .env file there which isn't seen by the script.

That scenario isn't covered it looks like.

@BlackDex commented on GitHub (Dec 1, 2023): Ah! Looks like `ROCKET_TLS` is not seen as a env variable to the healthcheck.sh script. Probably because you are using a `.env` file there which isn't seen by the script. That scenario isn't covered it looks like.
Author
Owner

@rwjack commented on GitHub (Dec 1, 2023):

Okay, definitely a minor bug, I fixed it by adding this in my docker-compose:

    env_file:
      - .env

Not sure why I didn't set it like that in the first place, that's how I do it everywhere. Anyways, all good now.

This is probably the reason:

## By default, Vaultwarden expects for this file to be named ".env" and located
## in the current working directory.
@rwjack commented on GitHub (Dec 1, 2023): Okay, definitely a minor bug, I fixed it by adding this in my docker-compose: ``` env_file: - .env ``` Not sure why I didn't set it like that in the first place, that's how I do it everywhere. Anyways, all good now. This is probably the reason: ``` ## By default, Vaultwarden expects for this file to be named ".env" and located ## in the current working directory. ```
Author
Owner

@BlackDex commented on GitHub (Dec 1, 2023):

That is if not running via Docker. But that could be cleared up. Since there also is a ENV variable which can relocate the .env file location ;). Which doesn't make it more clear haha.

@BlackDex commented on GitHub (Dec 1, 2023): That is if not running via Docker. But that could be cleared up. Since there also is a ENV variable which can relocate the `.env` file location ;). Which doesn't make it more clear haha.
Author
Owner

@rwjack commented on GitHub (Dec 1, 2023):

Gotcha, thanks for the help!

Even though my issue is resolved, do you still want to keep this open as a reminder?

@rwjack commented on GitHub (Dec 1, 2023): Gotcha, thanks for the help! Even though my issue is resolved, do you still want to keep this open as a reminder?
Author
Owner

@BlackDex commented on GitHub (Dec 1, 2023):

Yes please, this is something we need to fix i think.

@BlackDex commented on GitHub (Dec 1, 2023): Yes please, this is something we need to fix i think.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/vaultwarden#1786
No description provided.