Go to All Forums

Introducing enhanced EC2 monitoring - Combine your CloudWatch and agent driven metrics now! (Updated with alert control for spot and reserved instances)


 
Hello folks!
We are happy to announce that our all new (EC2 CloudWatch + Server Agent) integration will be made generally available to all Site24x7 users. For those of you, who are using AWS infrastructure monitoring to monitor EC2 instances, it might still look same to you on the surface, but a whole lot of work has gone underneath the hood to efficiently map your EC2 instance monitor, with the agent driven metrics of the server monitor. From tackling thresholds, notification settings, to AutoScaling instances our development team has worked really hard to make this enhancement possible. With this integration we will be giving you, a new set capabilities to monitor Amazon EC2 instances in a more simple and efficient manner. 
 
Couple of notable improvements for existing users who are monitoring EC2 instances with our agent based server monitoring.
 
By combining the EC2 metadata with the system agent, you will be able to solve a couple of niggling issues, which generally crop up when you try to monitor dynamic AWS environments with standalone server agents.
 
  • No more licensing headaches - Your terminated AutoScaling instances won't be eating up valuable licensing space anymore, instead they will get suspended in the Site24x7 console
  • Site24x7 will be able to distinguish an instance termination from an instance crash
  • Performance charts for all your EC2 related metrics will now be available in a single unified view - no need to toggle between tabs 
  • Automatic clean up - Your terminated EC2 instances will be automatically removed from the Site24x7 console.
  • Reduced alert fatigue no more redundant alerts when AutoScaling instance gets terminated or when agent communication failure occurs. 
  • Thresholds, notifications, maintenance - now all can be set up and configured on a single monitor.
 
 
For AWS EC2 monitoring users, you can now enrich your standard CloudWatch metrics with additional system and application metrics from the agent. You can get the following capabilities.
 
  • Process and service level granularity: Identify the top process by CPU/Memory over a period of time, monitor CPU, memory, object handles and thread count for each running process and service 
  • System load:  Analyze the load average on your system
  • Analyze filesystem usage: Monitor free space usage and individual disk utilization for your attached EBS volumes
  • Memory utilization: view memory breakup,  assess instance memory usage, get alerted in the event of memory swap or page outs
  • Check network traffic: monitor outgoing and incoming network packets for each network interface. Get alerted when error packets gets generated.
  • Plugins: Open ecosystem to monitor various application services, use our out-of-the plugins or write your own.
  • Root cause analysis report reduce mean time to repair (MTTR)
  • Automatically collect metrics if you're running IIS and SQL server
  • Heartbeat checks every 1 minuteCheck with server team if 1 minute heartbeat is correct?in other words the availability of your instance is monitored every 1 minute.
  • Extensive alerting for a host of new system level parameters, improved performance reports and a whole lot more.

How to enable this enhancement?
If you're already monitoring EC2 instances with the server agent, and want to try our integration, don't fret, we have got you covered. Our development team has already put in all the necessary provisions in place to seamlessly sync your existing server monitoring data with the AWS EC2 monitor. All we need is your go ahead. We believe in the popular adage of starting fresh, so when syncing is done, your outage history from the server monitor won't be ported to the newly integrated monitor. If you want the old outages to be carried over, you can get in touch with us.
 
To know more about our one click integrate, read our product documentation.
 
To our existing EC2 monitoring users, if you want to enable this integration,  all you have to do is install the agent on your EC2 instance, once done you'll start to see all your EC2 related data in  a matter of minutes. You can also use configuration management tools like Chef, Puppet, Ansible or Saltstack, to automatically deploy the monitoring agent on your instance when it gets provisioned.
 

(Important!) Automatic integration

Going forward, this integration will be handled automatically in the client. For example imagine a scenario, where you are monitoring a group of 10 EC2 instances standalone with our server agents. Then you decide to try our CloudWatch integration, so you connect your AWS account with Site24x7, auto discover all your EC2 instances and add them as EC2 CloudWatch monitors. Once this is done, all the agent data corresponding to the various instances will be automatically ported from the server monitor to the integrated EC2 CloudWatch monitor and your server monitor will cease to exist.

This automatic integration will also be applicable, when you are using our CloudWatch integration and decide to deploy agents on the said instances. 

  • All charting and graphs pertaining to CPU usage, memory, disk utilization, system load, idle/busy time, etc will be ported to the EC2 summary page present in the EC2 CloudWatch monitor
  • In addition to EC2 summary, volumes and EC2 inventory tabs, the integrated EC2 CloudWatch monitor will contain additional tabs like CPU, Memory, Disk, Processes, Plugins, Checks, Network Sys logs and Process viewer
  • Configuration and threshold profiles will also be ported.

Pricing and licensing details

The AWS EC2 monitor and the server agent installed on it, will be considered as two basic monitors. The agent also comes along with 2 plugins, these plugins can either be our prebuilt plugins or custom plugins written by yourself.  Once you have installed the agent, you can execute our out of the box plugins, to collect custom metrics for various application services running on your AWS stack.
 
CloudWatch based AWS EC2 Monitoring (1 Basic Monitor) + Server Agent based monitoring (1 Basic Monitor) + 2 Plugins  = 2 basic monitors
 
Essentially you will getting standard EC2 CloudWatch metrics, system level performance counters from the agent and application performance metrics. Combine that with our comprehensive dashboards, tailor made alerts, performance reports and data retention, you will be getting one powerful unified EC2 monitoring tool.
 
Compatibility requirements
The integration is only available for Linux and Windows based EC2 instances.
To integrate the server monitor with the EC2 CloudWatch monitor, you need to install the updated Linux and Windows agent
Linux - version 16.1.0 and above
Windows - version 18.1.5  and above 
 
Resources 
Read our product documentation to know more about the additional monitoring capabilities and how you can enable them.
 
Go ahead take our new enhancement for a spin, install the agent or integrate your existing server monitor, and post your feedback and comments on our forums. If you have any queries or questions about installation or licensing please get in touch with our support.
Reply 0
Replies (7)

Re: Introducing enhanced EC2 monitoring – Combine your CloudWatch and agent driven metrics now!​

This is a perfect feature! Thanks! Helps a great deal for reporting as well.

One minor request. In the monitor overview, it would be great if there was a visual difference between just an EC2 Instance monitor, and an integrated monitor. That way we can easily tell what is what, and also spot whether we have forgotten to deploy any agents to servers.
Reply 0

Re: Re: Introducing enhanced EC2 monitoring – Combine your CloudWatch and agent driven metrics now!​

Hi Tom,

Thanks a lot for taking your time in checking out our new feature. Do stay updated with us on this , as we are building new stuffs on top of this integration.

With respect to your query on classification between the EC2 instances that are integrated and the ones that are not, we have provided the agent hosted machine name along with the agent version in the overview page at the top left corner(I have attached a sample to provide some clarity). 

But we do understand that we can do something better about this. We will see on how to project this better!!

Regards,

Ananthkumar K S


Attachments
EC_Server_Integrated.png
Size: 149.01 KB
Reply 0

Re: Introducing enhanced EC2 monitoring – Combine your CloudWatch and agent driven metrics now!​

Hi Ananthkumar,

That is indeed correct, and that is the only place I know where to verify this. It would be very handy if we could see it from the Monitor Status page where all monitors are lised. Can be as simple as giving the "EC2 Instance" text in that list another color.
Reply 0

Re: Introducing enhanced EC2 monitoring – Combine your CloudWatch and agent driven metrics now!​

Hi Tom ,

We understand your concern with respect to monitor status page.

We will look into this right away.

Regards,

Ananthkumar K S
Reply 0

Re: Re: Introducing enhanced EC2 monitoring ? Combine your CloudWatch and agent driven metrics now!?

Hi Tom,

 

We have tried to classify the integrated instance by giving you some classification in listing page. 

Hope it helps. 

 

ServerAgentClassifier

 

Regards,

 

Ananthkumar K S

 

Reply 0


Configuration support to mute alerts for Spot and Reserved instances

In response to feedback received from a customer, regarding constant alerting for spot instance interruptions we're adding more alert options and control to improve our signal-to-noise ratio. After today's release, you can navigate to the Edit section of a monitored EC2 instance, access the attached threshold profile, and configure whether you want to be alerted for spot instance interruptions and scheduled reserved instance termination. By opting out you neither won't be bombarded with unnecessary alerts nor would you deplete your alert credits, when instance termination occurs.

By default, the toggle will be set to "Yes" for an already discovered and monitored EC2 instance. If you don't care about these attributes, then you can switch the toggle to "No" and save the profile. However, for a newly connected AWS account, the toggle for both the attributes - "Notify for Spot Instance Termination" and "Notify for Scheduled Instance Termination" will be set to "No" in the threshold profile.

EC2 instance threshold profile

Help resource

EC2 instance threshold configuration

If you have any feedback or suggestions on how best to take spot or reserved instance monitoring forward, please get in touch with our support or reply in this forum thread.

Reply 3