improving performance of the 24h dashboard chart for extreme monitor counts #4204

Open
opened 2026-02-28 03:54:36 -05:00 by deekerman · 15 comments
Owner

Originally created by @592774447 on GitHub (Jul 15, 2025).

Image

🏷️ Feature Request Type

Dashboard

🔖 Feature description

Historical data indicates whether "recent" can be set as the default value to reduce data queries and increase access speed

✔️ Solution

Historical data indicates whether "recent" can be set as the default value to reduce data queries and increase access speed

Alternatives

No response

📝 Additional Context

This only applies to extreme monitor counts

Originally created by @592774447 on GitHub (Jul 15, 2025). ### 📑 I have found these related issues/pull requests <img width="2473" height="468" alt="Image" src="https://github.com/user-attachments/assets/5b445517-bfff-4639-850d-9f790d26b308" /> ### 🏷️ Feature Request Type Dashboard ### 🔖 Feature description Historical data indicates whether "recent" can be set as the default value to reduce data queries and increase access speed ### ✔️ Solution Historical data indicates whether "recent" can be set as the default value to reduce data queries and increase access speed ### ❓ Alternatives _No response_ ### 📝 Additional Context This only applies to extreme monitor counts
Author
Owner

@CommanderStorm commented on GitHub (Jul 15, 2025):

I have no clue what you want to convey. Could you go over the template and fill out all the fields (what you would like changed how, alternatives, duplicates)?

@CommanderStorm commented on GitHub (Jul 15, 2025): I have no clue what you want to convey. Could you go over the template and fill out all the fields (what you would like changed how, alternatives, duplicates)?
Author
Owner

@592774447 commented on GitHub (Jul 15, 2025):

Can the '24h' in the image be set to 'Recent' as the default value

@592774447 commented on GitHub (Jul 15, 2025): Can the '24h' in the image be set to 'Recent' as the default value
Author
Owner

@CommanderStorm commented on GitHub (Jul 16, 2025):

Which version are you using? The performance should be quite close in v2.0

@CommanderStorm commented on GitHub (Jul 16, 2025): Which version are you using? The performance should be quite close in v2.0
Author
Owner

@592774447 commented on GitHub (Jul 16, 2025):

v2.0beta.3

@592774447 commented on GitHub (Jul 16, 2025): v2.0beta.3
Author
Owner

@CommanderStorm commented on GitHub (Jul 16, 2025):

I'd still like the default to be 24h.
That is imo the better experience.

If you or someone else wants to look into where this is noticably slower (it should not), that would be appreciated.
I am currently preparing for my exams again, so I can't do this.

@CommanderStorm commented on GitHub (Jul 16, 2025): I'd still like the default to be 24h. That is imo the better experience. If you or someone else wants to look into where this is noticably slower (it should not), that would be appreciated. I am currently preparing for my exams again, so I can't do this.
Author
Owner

@592774447 commented on GitHub (Jul 16, 2025):

I just want to ask if there is a way to customize and modify this default value, rather than changing it in the official version. Because I have over a thousand monitoring items, if I query a large amount of database chart data when logging into the front end, it will be slower. The front-end page takes 5-10 seconds or even longer to display the data

Image

@592774447 commented on GitHub (Jul 16, 2025): I just want to ask if there is a way to customize and modify this default value, rather than changing it in the official version. Because I have over a thousand monitoring items, if I query a large amount of database chart data when logging into the front end, it will be slower. The front-end page takes 5-10 seconds or even longer to display the data ![Image](https://github.com/user-attachments/assets/58dc62f7-9fed-4c03-a6d8-a4a1752c18b6)
Author
Owner

@CommanderStorm commented on GitHub (Jul 17, 2025):

You are just reaching the limit of what we were designed to support. I am not sure if changing the default for large monitor counts would improve performance.

We just have not tested against such a large amount of monitors.

@CommanderStorm commented on GitHub (Jul 17, 2025): You are just reaching the limit of what we were designed to support. I am not sure if changing the default for large monitor counts would improve performance. We just have not tested against such a large amount of monitors.
Author
Owner

@592774447 commented on GitHub (Jul 17, 2025):

Okay, I understand. My idea is to set the default query for charts to 24 hours of data. If we change it to, for example, a default query for 30 minutes of data, the database will reduce a significant amount of data queries, which should improve the speed of data retrieval. Because compared to 24 hours, 30 minutes will definitely query a lot less data

I found the following code in the src/components/PingChart.vue file

created() {
// Load chart period from storage if saved
let period = this.$root.storage()[chart-period-${this.monitorId}];
if (period != null) {
// Has this ever been not a string?
if (typeof period !== "string") {
period = period.toString();
}
this.chartPeriodHrs = period;
} else {
this.chartPeriodHrs = "0";
}
},

I tried to change this.chartPeriodHrs = "24h"; to this.chartPeriodHrs = "0";
Then, run "npm install && npm run build" to rebuild the frontend files in the dist directory. The default value has been successfully changed to "recent". After testing, it has improved the access speed slightly. However, if the number of monitoring items increases further, the speed may still decrease. It is recommended that the official team consider the scenario when there are too many monitoring items

@592774447 commented on GitHub (Jul 17, 2025): Okay, I understand. My idea is to set the default query for charts to 24 hours of data. If we change it to, for example, a default query for 30 minutes of data, the database will reduce a significant amount of data queries, which should improve the speed of data retrieval. Because compared to 24 hours, 30 minutes will definitely query a lot less data I found the following code in the src/components/PingChart.vue file created() { // Load chart period from storage if saved let period = this.$root.storage()[`chart-period-${this.monitorId}`]; if (period != null) { // Has this ever been not a string? if (typeof period !== "string") { period = period.toString(); } this.chartPeriodHrs = period; } else { this.chartPeriodHrs = "0"; } }, I tried to change this.chartPeriodHrs = "24h"; to this.chartPeriodHrs = "0"; Then, run "npm install && npm run build" to rebuild the frontend files in the dist directory. The default value has been successfully changed to "recent". After testing, it has improved the access speed slightly. However, if the number of monitoring items increases further, the speed may still decrease. It is recommended that the official team consider the scenario when there are too many monitoring items
Author
Owner

@592774447 commented on GitHub (Jul 17, 2025):

Just out of curiosity, I was wondering if adding a cache, such as Redis, to the program's middle tier would improve its speed. I'm not familiar with programming, so this is purely a casual question

@592774447 commented on GitHub (Jul 17, 2025): Just out of curiosity, I was wondering if adding a cache, such as Redis, to the program's middle tier would improve its speed. I'm not familiar with programming, so this is purely a casual question
Author
Owner

@592774447 commented on GitHub (Jul 17, 2025):

Because I feel that every time the frontend is accessed, the program queries the database completely, so when the data volume is large, it becomes very slow. If caching is added in the middle, or the frontend prioritizes displaying data, monitoring items that haven't finished querying can be displayed as loading or something similar. It shouldn't be that all data needs to be fully queried before the frontend page is displayed

@592774447 commented on GitHub (Jul 17, 2025): Because I feel that every time the frontend is accessed, the program queries the database completely, so when the data volume is large, it becomes very slow. If caching is added in the middle, or the frontend prioritizes displaying data, monitoring items that haven't finished querying can be displayed as loading or something similar. It shouldn't be that all data needs to be fully queried before the frontend page is displayed
Author
Owner

@CommanderStorm commented on GitHub (Jul 17, 2025):

Caching like you are proposing is not pheasable under the current architecture.
Feel free to look into where performance can be improved though.

I can review your PR, but won't invest engineering time myself into this.

I won't accept changing the default on this for a insignificant performance gain - it would have to be considerable and come with alternates that were considered.
There should not be much impact to using the 24h model.
I still think 24h should be the default.

@CommanderStorm commented on GitHub (Jul 17, 2025): Caching like you are proposing is not pheasable under the current architecture. Feel free to look into where performance can be improved though. I can review your PR, but won't invest engineering time myself into this. I won't accept changing the default on this for a insignificant performance gain - it would have to be considerable and come with alternates that were considered. There should not be much impact to using the 24h model. I still think 24h should be the default.
Author
Owner

@592774447 commented on GitHub (Jul 17, 2025):

Okay, I understand. By the way, regarding the function of retaining historical data, if I set it to 7, will expired data be automatically cleared? Does the system perform scheduled cleanup or does it retain data for each monitoring item for 7 days in real-time?

@592774447 commented on GitHub (Jul 17, 2025): Okay, I understand. By the way, regarding the function of retaining historical data, if I set it to 7, will expired data be automatically cleared? Does the system perform scheduled cleanup or does it retain data for each monitoring item for 7 days in real-time?
Author
Owner

@592774447 commented on GitHub (Jul 17, 2025):

Because every time I manually click to clear all statistical data, I still find that the 24-hour statistical data can be seen on the chart.

@592774447 commented on GitHub (Jul 17, 2025): Because every time I manually click to clear all statistical data, I still find that the 24-hour statistical data can be seen on the chart.
Author
Owner

@CommanderStorm commented on GitHub (Jul 18, 2025):

it is a new system, there might be bugs

@CommanderStorm commented on GitHub (Jul 18, 2025): it is a new system, there might be bugs
Author
Owner

@592774447 commented on GitHub (Jul 27, 2025):

Image

Recently, there have been frequent instances of extensive timeouts in monitoring during the early morning hours, with the error message: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?

It seems like the database connection pool is exhausted, which always occurs after the scheduled cleanup of historical data. I tried increasing the configuration value of mariadbPoolConfig in database.js, but it seems to have little effect

@592774447 commented on GitHub (Jul 27, 2025): <img width="2541" height="1625" alt="Image" src="https://github.com/user-attachments/assets/b523490a-5d3d-4b6b-891d-814119848c51" /> Recently, there have been frequent instances of extensive timeouts in monitoring during the early morning hours, with the error message: Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call? It seems like the database connection pool is exhausted, which always occurs after the scheduled cleanup of historical data. I tried increasing the configuration value of mariadbPoolConfig in database.js, but it seems to have little effect
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/uptime-kuma#4204
No description provided.