Lightlines
By google

gke-ai-troubleshooting-tpu-metrics-monitoring

google2.7k installs

Diagnose GKE TPU slowdowns and interruptions.

What it does

Diagnoses infrastructure causes of GKE TPU workload interruptions and performance issues using system metrics.

When to use it

Use for metric-based monitoring of TPU utilization, memory, node health, node-pool availability, interruptions, and recovery; not for non-TPU workloads or non-metric debugging.

How to use it

Give it a GKE TPU monitoring or troubleshooting request; it gathers cluster context and diagnoses infrastructure issues using metrics.

Uses

Latest review

What you need · 1

Requires a TPU cluster with GKE System Metrics enabled and container port 8431 exposed. Runtime metrics require GKE 1.27.4-gke.900 or later; node-status queries require 1.32.1-gke.1357001 or later.


Access · 2

This skill

GKE

Read

Inspects GKE cluster configuration

Cloud Monitoring

Read

Reads TPU health metrics