Tencent Cloud CKafka Monitoring Integration

Utilize Site24x7 Tencent Cloud CKafka Monitoring Integration to:

Track produce and consume message rates and throughput to monitor data pipeline health.
Monitor message heap (backlog) and max consume latency to detect consumer lag.
Observe disk usage and partition counts for storage and scaling management.
Correlate produce and consume request rates to balance producer and consumer workloads.

This help document contains instructions to set up Tencent Cloud CKafka Monitoring Integration and related information.

Setup and configuration

During your Tencent Cloud monitor setup, if you have selected CKafka in the Services/Resources drop-down, all your CKafka resources will be auto-detected and monitored without the need to manually create a monitor for each resource.

If you haven't selected CKafka during Tencent Cloud monitor creation, follow these instructions:

Log in to your Site24x7 account.
Go to Cloud > Tencent Cloud, select your Tencent Cloud monitor, then go to any of the dashboards on the left pane.
Click the hamburger icon and select Edit, which brings you to the Edit Tencent Cloud Monitor page.
On the Edit Tencent Cloud Monitor page, select CKafka from the Services/Resources drop-down and click Save.
After successful configuration, go to Cloud > Tencent Cloud > CKafka. Now you can view the discovered resources.

Data collection frequency

Performance metrics of your Tencent Cloud CKafka resources will be collected every five minutes and updated in the Site24x7 portal every five minutes by default; metrics are also collected based on the selected poll interval.

Supported metrics

Metric name	Description	Unit
Consumed Messages	Total number of messages consumed by all consumer groups in the collection interval	Count
Consume Request Rate	Number of consume (fetch) requests made per second	Requests/second
Consume Traffic	Volume of data consumed by consumer groups per second	MB/second
Disk Usage	Percentage of total disk capacity currently used by CKafka data	Percent
Max Consume Latency	Maximum lag in milliseconds between message production and consumption	Milliseconds
Message Heap	Number of unconsumed messages currently accumulated in the topic	Count
Partition Count	Total number of partitions across all topics in the CKafka instance	Count
Produced Messages	Total number of messages produced by all producers in the collection interval	Count
Produce Request Rate	Number of produce requests made per second	Requests/second
Produce Traffic	Volume of data produced and written to CKafka per second	MB/second

Threshold configuration

Global configuration

Go to the Admin section in the left navigation pane.
Select Configuration Profiles from the left pane and choose the Threshold and Availability (+) tab from the drop-down menu. Click Add Threshold Profile from the top-right corner.
Set the monitor type as Tencent Cloud CKafka. Now you can set the threshold values for all the metrics mentioned above.

Monitor-level configuration

Go to Cloud > Tencent Cloud > CKafka.
Choose a resource for which you would like to set a threshold, then click the hamburger icon . Select Edit, which directs you to the Edit Tencent Cloud CKafka Monitor page.
You can set the threshold values for the metrics by selecting Threshold and Availability. You can also configure IT Automation at the attribute level.

IT Automation

Site24x7's IT Automation tools help with automatically resolving performance degradation issues. When a breach occurs, the alarm engine continuously examines the system events for which thresholds have been defined and performs the mapped automation.

How to configure IT Automation for a monitor

Configuration Rules

With Site24x7's Configuration Rules, you can set parameters like Threshold Profile, Notification Profile, Tags, and Monitor Group for multiple monitors and automate the configuration settings of your monitoring resources.

How to add a Configuration Rule