Tencent Cloud EMR Monitoring Integration
Utilize Site24x7 Tencent Cloud EMR Monitoring Integration to:
- Monitor HDFS capacity utilization to prevent storage exhaustion in big data clusters.
- Track DataNode count and block distribution for cluster health and data integrity.
- Observe total file counts to understand workload density across the HDFS namespace.
- Alert on capacity thresholds before EMR jobs fail due to storage constraints.
This help document contains instructions to set up Tencent Cloud EMR Monitoring Integration and related information.
Setup and configuration
During your Tencent Cloud monitor setup, if you have selected EMR in the Services/Resources drop-down, all your EMR resources will be auto-detected and monitored without the need to manually create a monitor for each resource.
If you haven't selected EMR during Tencent Cloud monitor creation, follow these instructions:
- Log in to your Site24x7 account.
- Go to Cloud > Tencent Cloud, select your Tencent Cloud monitor, then go to any of the dashboards on the left pane.
- Click the hamburger icon
and select Edit, which brings you to the Edit Tencent Cloud Monitor page. - On the Edit Tencent Cloud Monitor page, select EMR from the Services/Resources drop-down and click Save.
- After successful configuration, go to Cloud > Tencent Cloud > EMR. Now you can view the discovered resources.
Data collection frequency
Performance metrics of your Tencent Cloud EMR resources will be collected every five minutes and updated in the Site24x7 portal every five minutes by default; metrics are also collected based on the selected poll interval.
Supported metrics
| Metric name | Description | Unit |
|---|---|---|
| HDFS Capacity Total | Total storage capacity available in the HDFS distributed file system | GB |
| HDFS Capacity Used | Amount of HDFS storage currently in use by data and replicas | GB |
| DataNode Count | Number of DataNode daemons currently active and registered with the NameNode | Count |
| Total Blocks | Total number of data blocks stored across all DataNodes in HDFS | Count |
| Total Files | Total number of files and directories in the HDFS namespace | Count |
Threshold configuration
Global configuration
- Go to the Admin section in the left navigation pane.
- Select Configuration Profiles from the left pane and choose the Threshold and Availability (+) tab from the drop-down menu. Click Add Threshold Profile from the top-right corner.
- Set the monitor type as Tencent Cloud EMR. Now you can set the threshold values for all the metrics mentioned above.
Monitor-level configuration
- Go to Cloud > Tencent Cloud > EMR.
- Choose a resource for which you would like to set a threshold, then click the hamburger icon . Select Edit, which directs you to the Edit Tencent Cloud EMR Monitor page.
- You can set the threshold values for the metrics by selecting Threshold and Availability. You can also configure IT Automation at the attribute level.
IT Automation
Site24x7's IT Automation tools help with automatically resolving performance degradation issues. When a breach occurs, the alarm engine continuously examines the system events for which thresholds have been defined and performs the mapped automation.
How to configure IT Automation for a monitor
Configuration Rules
With Site24x7's Configuration Rules, you can set parameters like Threshold Profile, Notification Profile, Tags, and Monitor Group for multiple monitors and automate the configuration settings of your monitoring resources.
How to add a Configuration Rule
Related links:
How to add a Tencent Cloud monitor.
List of Tencent Cloud service types monitored.
How to configure IT Automations for a monitor.
How to add a Configuration Rule.