u/joeshiett

Almost going insane using Zabbix with timescaleDB

Hello,

I’m having trouble with my self-hosted timescaleDB (96GB RAM 12vCPU and 1.5TB) keeping up with data from my Zabbix server. We have close to 20k hosts on Zabbix, all being polled. With compression set after every 2 days, timescale can’t compress enough, my storage gets filled quickly. I have done as much optimizations as I possibly can, I just can’t stop the DB from being filled so quickly (less than a week). I have 1.5TB worth of storage. Also my query performance is poor as well, despite having direct DB connections enabled on Grafana. I’m considering moving to using victoriaMetrics or clickhouse not too totally sure which to go for.

What would you recommend? Thank you.

reddit.com
u/joeshiett — 3 days ago
▲ 10 r/zabbix

Zabbix Grafana Dashboards are really slow for me

Helll everyone,

I have 10s of thousands of hosts on Zabbix, I have some Grafana dashboards created to visualize some metrics concerning my hosts. My dashboards take a really long time to display information, considering it has to go through 100s of thousands of items via the Zabbix API (connected as a data source in Grafana) to get the info I need.

What are some ways have you employed to get data to be quickly visualized on your Grafana dashboard, considering the amount of hosts, and items per host that I have?

I was thinking of adding some cache layer between the Zabbix API, and Grafana, but I’m not sure that may be a viable solution as well.

Your responses would be much appreciated!

reddit.com
u/joeshiett — 13 days ago
▲ 12 r/zabbix

How are you handling Zabbix at scale?

Hello, I’m quite new to Zabbix, I’m working on a production setup so far I’ve run into issues such as slow queries on Grafana or timeout errors. I have over 5,000 hosts connected. I’m trying to build dashboards on Grafana to visualize the data I need, but I keep running into timeout issues or 500 error issues in Grafana.

I’m running timescaleDB on CloudnativePG, I’m not hitting any resource limits on the database or server side. I’m just not sure how to set things up to run with the scale we have. Any tips will be highly appreciated!

reddit.com
u/joeshiett — 26 days ago