Google Cloud Vertex AI GenAI Model monitoring integration

Monitor your Google Cloud Vertex AI generative AI model deployments to track invocation volume, latency, token and character throughput, and provisioned capacity utilization. Google Cloud Vertex AI GenAI Model monitor lets you ensure your generative AI workloads stay within capacity and perform reliably.

Benefits

Monitoring Vertex AI GenAI Model ensures that your service remains reliable, secure, and high-performing. Key benefits include:

• Identify capacity errors or latency spikes before they impact end users.
• Track token and character throughput against provisioned limits to fine-tune capacity.
• Automated alerts and IT automation help maintain SLA compliance and reduce downtime.
• Consolidate GenAI Model metrics with other GCP services for a single pane of glass view.
• Bulk configuration and agent upgrades ensure smooth monitoring across multiple model deployments.

Use cases

Here are some use cases where Google Cloud Vertex AI GenAI Model monitoring will be helpful:

• Monitor invocation rate and model latency to detect performance degradation in real time.
• Capture capacity and system error counts to quickly identify throughput bottlenecks.
• Visualize token and character throughput utilization to plan capacity and optimize scaling.
• Set thresholds for first token latency to ensure contractual obligations are met.
• Track user error counts to strengthen request validation and client-side reliability.
• Integrate GenAI Model monitoring with other GCP services (Vertex AI Vector Search, BigQuery, Cloud Run) for holistic visibility.

Discovery

Google Cloud has no inventory of the generative models a project uses. Site24x7 adds a model when it has been invoked in the last 7 days. Models you have never called, or haven’t called recently, don’t appear. Each monitor is named <model ID> (<region>), for example gemini-2.5-flash (us-central1). The same model used in two regions appears as two monitors.

Setup and configuration

• Adding Google Cloud Vertex AI GenAI Model while configuring a new Google Cloud monitor
  

  If you have not configured a Google Cloud monitor yet, add one by following the steps below:

  1. Go to Cloud > GCP > Add GCP Monitor or Admin > Cloud Monitoring > Google Cloud Platform (GCP).
  2. Provide a unique display name for identification purposes.
  3. Upload the JSON file that contains the private key of the service account to authenticate and perform resource discovery.
  4. Select Vertex AI GenAI Model from the Select the Resources for Monitoring list.
  5. Select existing Notification Profiles, User Alerts Groups, Tags, and IT Automation Templates or add new ones. You can also integrate alarms with your preferred third-party service.
  6. Click Start GCP Monitoring.
• Adding Google Cloud Vertex AI GenAI Model to an existing Google Cloud monitor
  

  If you already have a Google Cloud monitor configured for the service account, you can add Google Cloud Vertex AI GenAI Model by following the steps below:

  1. Go to Cloud > GCP and select your GCP monitor.
  2. Click the hamburger icon next to Service View and select Edit, which brings you to the Edit GCP Monitor page.
  3. On the Edit GCP Monitor page, select Vertex AI GenAI Model from the Select the Resources for Monitoring list and click Save.
  4. After successful configuration, go to Cloud > GCP > Vertex AI GenAI Model. Now you can view the discovered Google Cloud Vertex AI GenAI Model resources.

Licensing

Each Google Cloud Vertex AI GenAI Model monitor consumes one basic monitor license.

Polling frequency

Google Cloud Vertex AI GenAI Model monitor collects minute-wise metric data with about a 3-minute delay, and the statuses of your Google Vertex AI GenAI Model resources are reported every five minutes.

Supported metrics

Configuration details

Threshold configuration

• Global configuration
  1. In the web client, go to the Admin section on the left navigation pane.
  2. Select Configuration Profiles from the left pane and select Threshold and Availability from the drop-down menu.
  3. Click Add Threshold Profile in the top-right corner.
  4. For Monitor Type, select Vertex AI GenAI Model.
  5. Specify the threshold values for the required metrics, then click Save.
• Monitor-level configuration
  1. In the web client, go to Cloud > GCP > Vertex AI GenAI Model.
  2. Select a resource you would like to set a threshold for, then click the hamburger icon.
  3. Select Edit, which directs you to the Edit Vertex AI GenAI Model Monitor page.
  4. Specify the threshold values for the required metrics, then click Save.

In the Edit Monitor page, you can set alerts for idle resources.

• Idle Days Before Retirement: the number of days without any invocation after which the monitor is marked Resource Removed. It is then deleted or suspended, according to the Remove terminated resources setting of the GCP monitor. If it was deleted, it is added again automatically the next time the model is invoked; a suspended monitor stays suspended. Allowed range: 30–42 days (default 30), because Google Cloud keeps Vertex AI usage metrics for only 6 weeks. The check runs once a day.

The monitor stays Up while it’s monitored, and it becomes Resource Removed only through idle retirement. Set thresholds on metrics such as capacity, system and user errors.

IT Automation

IT Automation tools automatically resolve performance degradation issues. These tools react to events proactively rather than waiting for manual intervention. The IT Automation tools automate repetitive tasks and automatically remediate threshold breaches. The alarm engine continually evaluates system events for which thresholds are set and executes the mapped automation when there is a breach.

Configuration Rules

Editing multiple monitors to associate different monitor groups or add a different tag can be a tedious process. With Configuration Rules, you can automate the configuration settings of your monitoring resources. Create custom rules to track configuration changes continuously and achieve the ideal configuration settings.

Summary

The Summary tab will give you the performance data organized by time for the metrics listed above. To view the summary:

1. Go to Cloud > GCP > Vertex AI GenAI Model.
2. Select a resource.
3. Click the Summary tab.

Configuration Details

The Configuration Details tab provides details on the configurations of application instances. To get the configuration details:

1. Go to Cloud > GCP > Vertex AI GenAI Model.
2. Select a resource.
3. Click the Configuration Details tab.

Reports

Gain in-depth data about the various parameters of your monitored resources and accentuate your service performance using our insightful reports.

To view reports for a Google Vertex AI GenAI Model resource:

1. Go to the Reports section on the left navigation pane.
2. Select Vertex AI GenAI Model from the menu on the left.
3. You can find the Availability Summary Report, Performance Report, and Inventory Report for one selected monitor. Or you can get the Summary Report, Availability Summary Report, Health Trend Report, and Performance Report for all the Google Vertex AI GenAI Model monitors.

You can also get reports from the Summary tab of the Google Vertex AI GenAI Model monitor:

1. Click the Summary tab.
2. Get the Availability Summary Report of the monitor by clicking Availability.
3. You can also find the Performance Report of the monitor by clicking any chart title.

Related content

• Monitor Google Cloud Platform
• List of Google Cloud resources monitored
• Google Cloud service quota monitoring