What Is OpenTelemetry? The Complete Guide to Observability

OpenTelemetry has quickly become one of the most important infrastructure projects in the cloud-native ecosystem. If you work in software engineering today — whether you are an SRE, a backend developer, a platform engineer, or a DevOps practitioner — understanding OpenTelemetry is no longer optional. It is the new baseline for how modern teams instrument, collect, and export observability data.

This guide explains what OpenTelemetry is, how it works, and how to put it into production. We cover the full architecture from API to wire protocol, walk through concrete code examples in Python, Node.js, and Java, and provide reference tables for semantic conventions and SDK stability across languages. By the end, you will have everything you need to evaluate OpenTelemetry for your own stack and make an informed decision about how to adopt it.

1. What Is OpenTelemetry?

OpenTelemetry (often shortened to OTel) is an open-source observability framework — a vendor-neutral collection of APIs, SDKs, tools, and a wire protocol — for generating, collecting, and exporting telemetry data from software systems. It supports three categories of telemetry signal: distributed traces, metrics, and logs.

It is a graduated project of the CNCF (Cloud Native Computing Foundation) and is, by contributor count and release velocity, the second-most active CNCF project in existence — behind only Kubernetes itself. Over 1,000 individual contributors from hundreds of organisations — including Google, Microsoft, AWS, Datadog, Splunk, Elastic, and Lightstep — maintain and advance the project.

The core value proposition can be stated in a single sentence: instrument your application once, and send the resulting telemetry to any backend that understands OTLP. In practical terms, this means you write instrumentation code once against the stable OTel API, and you can route the resulting traces, metrics, and logs to obseria.io today, switch to a different backend next year, and never touch your application code again.

This portability is what makes OpenTelemetry categorically different from every proprietary agent that came before it. With Datadog, New Relic, or Dynatrace agents, your instrumentation code is tightly coupled to the vendor. Switching means months of re-instrumentation work across every service. With OpenTelemetry, the investment in instrumentation is yours permanently.

2. A Brief History: From OpenCensus and OpenTracing to OpenTelemetry

To understand why OpenTelemetry is designed the way it is, it helps to know how it came to exist. The story begins with two separate open-source initiatives launched in 2016 that, despite solving overlapping problems, were incompatible with each other.

Google open-sourced OpenCensus — a set of libraries for collecting distributed traces and metrics — drawing on internal experience from Google's Dapper tracing infrastructure. Around the same time, Ben Sigelman and colleagues at Lightstep released OpenTracing, a CNCF-hosted API specification focused exclusively on distributed tracing with an API-first philosophy.

Both projects attracted significant adoption, but the duplication of effort created real problems for the ecosystem. Library authors had to choose one standard or maintain two separate instrumentation paths. Vendor backends had to support both formats. Practitioners had to evaluate which project their organisation should standardise on — a decision that could easily be made wrong.

The merger announcement in May 2019 was a turning point for the entire observability industry. Instead of two half-complete, incompatible standards, the industry now had a single, well-governed framework backed by essentially every major vendor and cloud provider simultaneously. The competitive dynamics that used to produce incompatible telemetry formats were replaced by a collaborative model where even competing vendors contribute to the same standard — because they all benefit from customers being able to instrument their applications once and evaluate multiple backends fairly.

3. OpenTelemetry Architecture: Four Layers That Work Together

OpenTelemetry is not a single library or binary. It is a specification that defines four distinct layers, each with a specific responsibility. Understanding which layer does what prevents the most common integration mistakes.

A typical production flow looks like this: your application code calls the OTel API, the SDK processes and batches the resulting telemetry, sends it over OTLP to a local Collector agent, the Collector applies tail-based sampling and fan-out routing, and the processed telemetry is exported to your observability backend. For development and simpler workloads, the Collector is optional — you can export directly from the SDK to any OTLP-compatible backend.

4. The Three Observability Signals: Traces, Metrics, and Logs

OpenTelemetry covers all three of the signals that modern observability practitioners consider essential. Each signal answers a different class of question about your system, and they are most powerful when used together and correlated across the same request context.

Distributed Traces

A trace records the complete end-to-end journey of a single request through your system — crossing service boundaries, database calls, message queue interactions, and external API calls. Each discrete unit of work within a trace is called a span. Spans are arranged in a parent-child tree structure that represents causality: if span B was triggered by span A, B is a child of A.

Every span carries a timing record (start time and duration), a status (OK, Error, or Unset), a human-readable name, and an arbitrary set of key-value attributes. The combination of the span tree and its attributes is what allows you to answer questions like "which downstream service caused this request to take 3.2 seconds?" or "which database query is responsible for the latency spike on our checkout endpoint?"

Context propagation is what makes distributed tracing work across service boundaries. When service A calls service B over HTTP or gRPC, it injects trace context (a trace ID and parent span ID) into the request headers using the W3C TraceContext standard. Service B extracts that context and uses it to create child spans that belong to the same trace. This propagation happens automatically when you use OTel auto-instrumentation.

from opentelemetry import trace

tracer = trace.get_tracer("checkout-service", "2.1.0")

with tracer.start_as_current_span("process-payment") as span:
    span.set_attribute("payment.method", "stripe")
    span.set_attribute("payment.amount_cents", 4999)
    span.set_attribute("user.tier", "premium")
    span.set_attribute("order.id", order_id)

    result = stripe_client.charge(amount=4999, currency="eur")

    span.set_attribute("payment.status", result.status)
    span.set_attribute("payment.transaction_id", result.id)

    if result.status != "succeeded":
        span.set_status(trace.StatusCode.ERROR, result.error_message)
        span.record_exception(PaymentException(result.error_message))

Metrics

Metrics capture aggregated numeric measurements over time. Unlike traces — which record individual requests — metrics summarise behaviour across many requests into statistical aggregates: request rates, error percentages, latency percentiles, queue depths, and resource utilisation.

OpenTelemetry supports four metric instrument types: Counters (monotonically increasing values, like requests processed), Gauges (point-in-time measurements, like current memory usage), Histograms (distributions with configurable bucket boundaries, like request latency), and UpDown Counters (values that can increase or decrease, like active connections). All four instruments share the same API across all supported languages.

Metrics are essential for alerting and capacity planning because they are cheap to store and query at scale. A trace for every request at 100,000 RPS is expensive; a histogram of request latencies updated 100,000 times per second is trivial. The right observability strategy uses metrics for alerting and dashboards, and traces for deep-dive investigation once an alert fires.

from opentelemetry import metrics

meter = metrics.get_meter("order-service", "1.0.0")

# Counter: total orders since service started
orders_counter = meter.create_counter(
    name="orders.processed",
    unit="1",
    description="Total number of orders successfully processed",
)

# Histogram: request latency distribution
latency_histogram = meter.create_histogram(
    name="order.processing.duration",
    unit="ms",
    description="Time taken to fully process an order",
)

# Usage: record with attributes for dimensional breakdowns
orders_counter.add(1, {"region": "eu-west-1", "tier": "premium", "payment_method": "stripe"})
latency_histogram.record(142, {"region": "eu-west-1", "tier": "premium"})

Logs

Logs are the oldest form of telemetry and the one most engineers are already producing. OpenTelemetry's approach to logs is deliberately pragmatic: rather than asking you to replace your existing logging library with an OTel-native logger, it provides a Logs Bridge API that wraps your existing framework (Python's logging module, SLF4J in Java, Winston or Pino in Node.js, Serilog in .NET) and adds OTel context — specifically the active trace_id and span_id — to every log record automatically.

The result is that every log line your application emits is automatically correlated with the request that produced it. In your observability backend, you can click on a trace span and instantly see all the logs generated during that span's execution — without adding a single line of custom instrumentation to your logging calls.

5. OpenTelemetry vs. Proprietary Vendor Agents

Every major APM vendor — Datadog, New Relic, Dynatrace, AppDynamics — ships a proprietary agent. These agents can be quick to get started with and often provide auto-instrumentation that works well out of the box. But there are fundamental trade-offs that become more significant as your organisation grows and your observability requirements mature.

The most important trade-off is portability. A proprietary agent embeds vendor-specific instrumentation logic into your application. When you change backends — whether because of pricing, features, or organisational requirements — you typically need to re-instrument every service from scratch. For a company with dozens of services across multiple languages, that is a multi-month engineering project.

The clearest way to think about this trade-off: proprietary agents offer a slightly lower initial setup cost in exchange for a permanent and growing future cost in the form of vendor dependency. OpenTelemetry inverts this — the initial setup may take slightly longer, but the long-term cost is dramatically lower because you own your instrumentation permanently.

For organisations at scale — with many services, multiple teams, and significant observability budgets — OpenTelemetry is almost always the right choice. The portability alone protects you from vendor pricing changes, and the standardised attribute schema means your data is consistent and queryable regardless of which backend you use.

6. Language and SDK Support

OpenTelemetry provides official SDKs for all major programming languages. Stability is tracked per signal (traces, metrics, logs) and varies by language. The table below reflects the state as of August 2026.

Status as of August 2026. Always check opentelemetry.io/docs/languages for the latest stability declarations before making production decisions.

For most production workloads, the languages most teams care about — Java, Python, Node.js, Go, and .NET — all have fully stable GA implementations across all three signals. The remaining languages are progressing quickly; the OTel maintainers follow a strict stability policy, which means Beta signals may have API changes but are generally safe for internal use.

Auto-instrumentation deserves special mention. For languages that support it (primarily Java, Python, Node.js, and .NET), you can get production-quality distributed traces, metrics, and logs from your application with zero changes to your application code. The Java Javaagent, for example, instruments over 150 frameworks automatically — covering everything from Spring Boot REST controllers to JDBC database calls to Kafka producers and consumers.

7. Semantic Conventions: Why Consistent Attribute Names Matter

Semantic conventions are one of the most underrated parts of OpenTelemetry — and the most common source of problems for teams that skip them. They define a standardised vocabulary of attribute names that all OTel-instrumented systems should use for common concepts: HTTP methods, database queries, messaging topics, cloud provider details, and more.

The problem semantic conventions solve is fragmentation. Without a standard, one team might record a database query as sql.query, another as db.query_text, and a third as query. Your observability backend cannot automatically recognise these as the same concept, so cross-service queries fail, automatic dashboards break, and AI-powered analysis produces incorrect results.

When every team follows semantic conventions, your backend can automatically correlate data across services, generate meaningful service maps, identify slow database queries across all services, and power AI-driven root cause analysis — without any manual configuration.

The service.name resource attribute is the single most important attribute in your entire telemetry setup. It is how your observability backend knows which service produced which spans, metrics, and logs. Every service must have a unique, stable service name set — either via the OTEL_SERVICE_NAME environment variable or programmatically in your SDK initialisation.

8. The OpenTelemetry Collector

The OpenTelemetry Collector is a standalone binary that acts as a vendor-agnostic telemetry pipeline. It receives data from your application SDKs or other sources, applies configurable processing, and exports the processed data to one or more observability backends. You can deploy it as a DaemonSet agent on every Kubernetes node, as a centralised deployment for tail-based sampling, or both simultaneously.

The Collector is built around three component types that chain together into named pipelines:

One of the most important use cases for the Collector is tail-based sampling. Head-based sampling — where you decide to keep or drop a trace at the moment the first span starts — means you make sampling decisions before you know whether the trace is interesting. A 10% head-based sample will drop 90% of your slow requests, failed transactions, and unusual traces. Tail-based sampling waits until the full trace is assembled, then makes the keep/drop decision based on the complete picture. You can always keep all traces with errors, all traces over a latency threshold, and a representative sample of normal traffic.

# Production Collector config: SDK → Collector → obseria.io
receivers:
  otlp:
    protocols:
      grpc:  { endpoint: "0.0.0.0:4317" }
      http:  { endpoint: "0.0.0.0:4318" }

processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 1500          # hard cap; refuse data above this
    spike_limit_mib: 400     # headroom above the soft limit

  batch:
    timeout: 5s              # flush at least every 5 seconds
    send_batch_size: 8000    # target batch size in spans
    send_batch_max_size: 10000

  # Remove high-cardinality attributes that inflate metric costs
  transform/sanitise:
    trace_statements:
      - context: span
        statements:
          - delete_key(attributes, "user.id")
          - delete_key(attributes, "http.request.header.authorization")

exporters:
  otlp/obseria:
    endpoint: ingest.obseria.io:4317
    compression: gzip
    headers:
      Authorization: "Bearer ${env:OBSERIA_API_KEY}"
    sending_queue:
      enabled: true
      queue_size: 5000
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_elapsed_time: 300s

service:
  pipelines:
    traces:
      receivers:  [otlp]
      processors: [memory_limiter, transform/sanitise, batch]
      exporters:  [otlp/obseria]
    metrics:
      receivers:  [otlp]
      processors: [memory_limiter, batch]
      exporters:  [otlp/obseria]
    logs:
      receivers:  [otlp]
      processors: [memory_limiter, batch]
      exporters:  [otlp/obseria]

9. Getting Started: Instrument Your First Service in 5 Minutes

The fastest path to production traces is auto-instrumentation. You do not need to write a single span manually. For Python, Node.js, and Java, the following commands get you from zero to sending live traces in under five minutes.

All three examples use the OTEL_* environment variables to configure the SDK. This means zero configuration code in your application — just set the variables in your container spec, systemd unit, or CI/CD pipeline.

# ── Python ──────────────────────────────────────────────────────
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap --action=install   # installs framework integrations

OTEL_SERVICE_NAME=my-python-service \
OTEL_SERVICE_VERSION=1.0.0 \
OTEL_DEPLOYMENT_ENVIRONMENT=production \
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.obseria.io \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <api-key>" \
opentelemetry-instrument python app.py

# Instruments: Django, Flask, FastAPI, SQLAlchemy, psycopg2, redis,
#              requests, httpx, celery, boto3 — automatically.

# ── Node.js ─────────────────────────────────────────────────────
npm install @opentelemetry/auto-instrumentations-node

OTEL_SERVICE_NAME=my-node-service \
OTEL_SERVICE_VERSION=1.0.0 \
OTEL_DEPLOYMENT_ENVIRONMENT=production \
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.obseria.io \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <api-key>" \
node --require @opentelemetry/auto-instrumentations-node/register app.js

# Instruments: Express, Fastify, Koa, http/https, pg, mysql2,
#              ioredis, mongodb, grpc, aws-sdk — automatically.

# ── Java ────────────────────────────────────────────────────────
# Download the javaagent once; reuse across all services
curl -Lo opentelemetry-javaagent.jar \
  https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

java -javaagent:opentelemetry-javaagent.jar \
  -DOTEL_SERVICE_NAME=my-java-service \
  -DOTEL_SERVICE_VERSION=1.0.0 \
  -DOTEL_DEPLOYMENT_ENVIRONMENT=production \
  -DOTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.obseria.io \
  -DOTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer <api-key>" \
  -jar my-app.jar

# Instruments: Spring Boot, Quarkus, Micronaut, Vert.x, Jetty,
#              JDBC, Hibernate, Kafka, RabbitMQ, gRPC — automatically.

Once your service is running with any of the above configurations, traces will appear in your obseria.io account within seconds. The auto-instrumentation captures incoming HTTP requests, outgoing HTTP calls, database queries, and cache operations — giving you a complete picture of request flow without a single line of custom instrumentation code.

10. Best Practices and Common Mistakes

After working with hundreds of teams instrumenting their systems with OpenTelemetry, we have seen the same mistakes appear repeatedly. The table below documents the most frequent ones and the recommended fix.

11. Frequently Asked Questions