Allow to send notifications when UK database is down #4534

Open
opened 2026-02-28 04:06:22 -05:00 by deekerman · 5 comments
Owner

Originally created by @jcjveraa on GitHub (Dec 17, 2025).

Somewhat related as a topic to #6046

🛡️ Security Policy

📝 Description

Tested on external Mariadb, perhaps the same issue on others.
When the uptime kuma database (it's own database) is not available, UK stops functioning completely. It does not send a notification to inform UK itself is having issues.

👟 Reproduction steps

  1. Run UK with an external MariaDB database
  2. Take down the database.
  3. Verify UK became unresponsive, and no notifications are triggered.

👀 Expected behavior

UK will send a notification to all notification channels that UK itself is having issues/is down.

😓 Actual Behavior

No notifications were sent

🐻 Uptime-Kuma Version

2.0.2

💻 Operating System and Arch

Debian Trixie (13.2), x86_64
Kernel Debian 6.12.57-1 (2025-11-05) x86_64

🌐 Browser

not relevant

🖥️ Deployment Environment

  • Runtime Environment:
    • Docker: Version 26.1.5+dfsg1, build a72d7cd (default Debian Trixie package)
    • Docker Compose: Version 2.26.1-4 (default Debian Trixie package)
    • Portainer (BE/CE): N/A
    • MariaDB: Version 11.8.3-MariaDB (default Debian Trixie package)
    • Node.js: Version 20.19.5 (as provided by 'your' docker image)
    • Kubernetes (K3S/K8S): N/A
  • Database:
    • MariaDB: External
  • Database Storage:
    • Filesystem:
      • Linux: ext4
    • Storage Medium: NVMe
  • Uptime Kuma Setup:
    • Number of monitors: 4
    • Nextcloud Talk notifications

📝 Relevant log output

redacted my server IP to 192.168.x.x

2025-12-15T19:24:19+01:00 [MONITOR] ERROR: Please report to https://github.com/louislam/uptime-kuma/issues
2025-12-15T19:24:19+01:00 [MONITOR] INFO: Try to restart the monitor
Trace: Error: connect EHOSTUNREACH 192.168.x.x:3306
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) {
  errno: -113,
  code: 'EHOSTUNREACH',
  syscall: 'connect',
  address: '192.168.x.x',
  port: 3306,
  fatal: true
}
    at Timeout.safeBeat [as _onTimeout] (/app/server/model/monitor.js:1021:25)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2025-12-15T19:25:24+01:00 [MONITOR] ERROR: Please report to https://github.com/louislam/uptime-kuma/issues
2025-12-15T19:25:24+01:00 [MONITOR] INFO: Try to restart the monitor
Trace: Error: connect EHOSTUNREACH 192.168.x.x:3306
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) {
  errno: -113,
  code: 'EHOSTUNREACH',
  syscall: 'connect',
  address: '192.168.x.x',
  port: 3306,
  fatal: true
}
    at process.unexpectedErrorHandler (/app/server/server.js:1893:13)
    at process.emit (node:events:524:28)
    at emitUnhandledRejection (node:internal/process/promises:250:13)
    at throwUnhandledRejectionsMode (node:internal/process/promises:385:19)
    at processPromiseRejections (node:internal/process/promises:470:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)
If you keep encountering errors, please report to https://github.com/louislam/uptime-kuma/issues
Trace: Error: connect EHOSTUNREACH 192.168.x.x:3306
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) {
  errno: -113,
  code: 'EHOSTUNREACH',
  syscall: 'connect',
  address: '192.168.x.x',
  port: 3306,
  fatal: true
}
    at Timeout.safeBeat [as _onTimeout] (/app/server/model/monitor.js:1021:25)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Originally created by @jcjveraa on GitHub (Dec 17, 2025). ### 📑 I have found these related issues/pull requests Somewhat related as a topic to #6046 ### 🛡️ Security Policy - [x] I have read and agree to Uptime Kuma's [Security Policy](https://github.com/louislam/uptime-kuma/security/policy). ### 📝 Description Tested on external Mariadb, perhaps the same issue on others. When the uptime kuma database (it's own database) is not available, UK stops functioning completely. It does not send a notification to inform UK itself is having issues. ### 👟 Reproduction steps 1. Run UK with an external MariaDB database 2. Take down the database. 3. Verify UK became unresponsive, and no notifications are triggered. ### 👀 Expected behavior UK will send a notification to all notification channels that UK itself is having issues/is down. ### 😓 Actual Behavior No notifications were sent ### 🐻 Uptime-Kuma Version 2.0.2 ### 💻 Operating System and Arch Debian Trixie (13.2), x86_64 Kernel Debian 6.12.57-1 (2025-11-05) x86_64 ### 🌐 Browser not relevant ### 🖥️ Deployment Environment - **Runtime Environment**: - Docker: Version `26.1.5+dfsg1`, build `a72d7cd` (default Debian Trixie package) - Docker Compose: Version `2.26.1-4` (default Debian Trixie package) - Portainer (BE/CE): N/A - MariaDB: Version ` 11.8.3-MariaDB` (default Debian Trixie package) - Node.js: Version 20.19.5 (as provided by 'your' docker image) - Kubernetes (K3S/K8S): N/A - **Database**: - MariaDB: External - **Database Storage**: - **Filesystem**: - Linux: ext4 - **Storage Medium**: NVMe - **Uptime Kuma Setup**: - Number of monitors: `4` - Nextcloud Talk notifications ### 📝 Relevant log output redacted my server IP to 192.168.x.x ``` 2025-12-15T19:24:19+01:00 [MONITOR] ERROR: Please report to https://github.com/louislam/uptime-kuma/issues 2025-12-15T19:24:19+01:00 [MONITOR] INFO: Try to restart the monitor Trace: Error: connect EHOSTUNREACH 192.168.x.x:3306 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) { errno: -113, code: 'EHOSTUNREACH', syscall: 'connect', address: '192.168.x.x', port: 3306, fatal: true } at Timeout.safeBeat [as _onTimeout] (/app/server/model/monitor.js:1021:25) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2025-12-15T19:25:24+01:00 [MONITOR] ERROR: Please report to https://github.com/louislam/uptime-kuma/issues 2025-12-15T19:25:24+01:00 [MONITOR] INFO: Try to restart the monitor Trace: Error: connect EHOSTUNREACH 192.168.x.x:3306 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) { errno: -113, code: 'EHOSTUNREACH', syscall: 'connect', address: '192.168.x.x', port: 3306, fatal: true } at process.unexpectedErrorHandler (/app/server/server.js:1893:13) at process.emit (node:events:524:28) at emitUnhandledRejection (node:internal/process/promises:250:13) at throwUnhandledRejectionsMode (node:internal/process/promises:385:19) at processPromiseRejections (node:internal/process/promises:470:17) at process.processTicksAndRejections (node:internal/process/task_queues:96:32) If you keep encountering errors, please report to https://github.com/louislam/uptime-kuma/issues Trace: Error: connect EHOSTUNREACH 192.168.x.x:3306 at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1611:16) { errno: -113, code: 'EHOSTUNREACH', syscall: 'connect', address: '192.168.x.x', port: 3306, fatal: true } at Timeout.safeBeat [as _onTimeout] (/app/server/model/monitor.js:1021:25) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) ```
Author
Owner

@jcjveraa commented on GitHub (Dec 17, 2025):

Meant to have this here as a Draft Issue, but that's not a n option - will complete with full environment details within the hour. done

@jcjveraa commented on GitHub (Dec 17, 2025): ~Meant to have this here as a Draft Issue, but that's not a n option - will complete with full environment details within the hour.~ done
Author
Owner

@CommanderStorm commented on GitHub (Dec 17, 2025):

That would make sense indeed.

@CommanderStorm commented on GitHub (Dec 17, 2025): That would make sense indeed.
Author
Owner

@jcjveraa commented on GitHub (Dec 17, 2025):

The solution is simple but perhaps implementation wise tricky, I haven’t looked at the source code yet, but if the auth tokens/urls/ etc for sending the notification (I’ll just call them “notification config”) are in the db this might pose a challenge. A solution could then be to cache all notification config in memory - it should not be a massive amount of data, a few kB per channel at most.

Otherwise it might make sense to add this as an (invisible?) regular “monitor”, with the special property that anytime a notification channel is added, that channel gets added to this monitor as well. The logic being that any channel that wants to get notified of some monitored app being down, probably also wants a notification if the monitor itself is down.

@jcjveraa commented on GitHub (Dec 17, 2025): The solution is simple but perhaps implementation wise tricky, I haven’t looked at the source code yet, but if the auth tokens/urls/ etc for sending the notification (I’ll just call them “notification config”) are in the db this might pose a challenge. A solution could then be to cache all notification config in memory - it should not be a massive amount of data, a few kB per channel at most. Otherwise it might make sense to add this as an (invisible?) regular “monitor”, with the special property that anytime a notification channel is added, that channel gets added to this monitor as well. The logic being that any channel that wants to get notified of some monitored app being down, probably also wants a notification if the monitor itself is down.
Author
Owner

@CommanderStorm commented on GitHub (Dec 17, 2025):

You are right, we likely should just crash loop. The db being down when we try to load which notifications exist is not recoverable.
I would expect to currently either crash loop or at least not have a positive health check.

@CommanderStorm commented on GitHub (Dec 17, 2025): You are right, we likely should just crash loop. The db being down when we try to load which notifications exist is not recoverable. I would expect to currently either crash loop or at least not have a positive health check.
Author
Owner

@CommanderStorm commented on GitHub (Jan 18, 2026):

looking at https://github.com/louislam/uptime-kuma/pull/6596 it seems that it can be hacked in, but this does not seem to be cleanly possible.
We need a bit of architectural rethinking before this is possible.

I am going to leave this issue open if someone wants to look into what our options are to solve this and which way we need to refactor.
Be warned: this likely requires a bit of a larger refactoring.

@CommanderStorm commented on GitHub (Jan 18, 2026): looking at https://github.com/louislam/uptime-kuma/pull/6596 it seems that it can be hacked in, but this does not seem to be cleanly possible. We need a bit of architectural rethinking before this is possible. I am going to leave this issue open if someone wants to look into what our options are to solve this and which way we need to refactor. Be warned: this likely requires a bit of a larger refactoring.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/uptime-kuma#4534
No description provided.