Wavefront Alert Queries for Kubernetes Monitoring

Recently, we have setup monitoring of Kubernetes using wavefront. Here are some of the useful alerts and their queries:

POD Memory Utilization:
ts(“kubernetes.pod.memory.working_set”, namespace_name=”xyz”)/ts(“kubernetes.pod.memory.limit”, namespace_name=”xyz”) * 100

Kong POD CPU Utilization:
ts(“kubernetes.pod.cpu.usage_rate”, namespace_name=”xyz”)/ts(“kubernetes.pod.cpu.limit”, namespace_name=”xyz”) * 100

Kong Replica Count Mismatch:
ts(“kubernetes.deployment.desired_replicas”, namespace_name=”xyz”) – ts(“kubernetes.deployment.available_replicas” and namespace_name=”xyz”)

New Pod Created/Pod Deleted:
highpass(0, ts(“kubernetes.pod.uptime”, namespace_name=”xyz”) < 630000)

Container Restart:
mdiff(10m, ts(“kubernetes.pod.restart_count”, namespace_name=”xyz”))

Related Post