Kubernetes monitoring from cluster to code

What is Kubernetes monitoring?

Kubernetes monitoring is the continuous collection and analysis of telemetry from Kubernetes clusters: the control plane, nodes, pods, containers and the applications running inside them. Its goal is to answer three questions at any moment: is the cluster healthy, are workloads getting the resources they need, and are the services users depend on working as expected?

Kubernetes makes this harder than classic host monitoring. Pods are short-lived, IP addresses change, workloads scale up and down automatically, and a single request can cross dozens of containers on different nodes. Static dashboards built around host names break down quickly. Effective Kubernetes monitoring therefore relies on labels and metadata, automatic discovery and correlation across metrics, logs, events and traces.

Why Kubernetes monitoring matters

Kubernetes is designed to self-heal, which can hide problems. A container that is OOMKilled and restarted every few minutes may never trigger an outage alert, yet it causes failed requests and growing latency. A misconfigured resource limit can throttle CPU on a critical service while the node looks idle. Without visibility, teams discover these issues only through customer complaints.

Good Kubernetes monitoring helps platform teams keep clusters stable and efficient, helps developers understand how their code behaves in production, and gives engineering leaders the data to plan capacity and control spend.

Key Kubernetes metrics and signals

A useful way to structure Kubernetes monitoring is by layer, applying the USE method (utilization, saturation, errors) to resources and the RED method (rate, errors, duration) to services.

• Control plane: API server request latency and error rate, etcd leader changes and disk sync duration, scheduler pending pods.
• Nodes: CPU and memory utilization, disk and inode pressure, network errors, node conditions such as NotReady or MemoryPressure.
• Pods and containers: restarts, OOM kills, CPU throttling, memory working set versus limits, readiness and liveness probe failures.
• Workloads: desired versus available replicas, rollout progress, failed Jobs, HPA scaling activity.
• Applications: request rate, error rate and latency per service, ideally from distributed traces.
• Events and logs: scheduling failures, image pull errors, evictions and application log errors with pod and namespace metadata.

How Obseria handles Kubernetes monitoring

Obseria installs with a single Helm chart. The Obseria Agent runs as a DaemonSet and collects node and container metrics, container logs, Kubernetes events, profiles and traces. If you already run the OpenTelemetry Collector or OpenTelemetry SDKs, Obseria ingests that data natively over OTLP. Most teams see their first traces within about 15 minutes.

Obseria TopoGraph builds a live map of clusters, namespaces, workloads and service dependencies, so every signal carries its context. When a pod restarts, you see the deployment it belongs to, the recent rollout, the logs before the crash and the downstream services affected. Obseria Intelligence uses this topology for causal analysis and runs agentic investigations that end in a chain of evidence instead of a list of correlated charts.

All data lands in Obseria Lake, an index-free lakehouse, so you can query it ad hoc with OQL, PromQL or SQL:

fetch logs
| filter k8s.namespace == "checkout" and level == "error"
| summarize count(), by:{k8s.pod.name}

Obseria Pipeline lets you drop noisy debug logs, mask PII and route data at ingest, and filtered data costs only ingestion.

Kubernetes monitoring best practices

1. Label consistently. Use standard labels for team, service, version and environment so data can be grouped and owned.
2. Alert on symptoms, not every event. Page on SLO burn rates and user-facing errors; send node and pod warnings to tickets or dashboards.
3. Watch requests and limits. Compare actual usage with requests to reduce waste and avoid throttling and OOM kills.
4. Correlate deployments. Annotate rollouts so regressions are visible immediately after a release.
5. Trace across services. Metrics tell you that something is slow; traces tell you where.
6. Control data volume. Filter and sample at ingest so monitoring cost scales with value, not with cluster size.