Since there is an alert generated for disk. The ideal way is to Mark it as maintenance and look into the issue. Monitoring will still continue though the server (in this can EC2 instance) is in maintenance state. However, you will not get alerted if there is a CPU threshold breach. We'll check on our end if it can be supported.
Meanwhile, may I ask what would you generally do if you get a disk alert? Possibly a disk capacity alert?
Product Manager, Site24x7