OpenTelemetry Collector

What is the OpenTelemetry Collector?

The OpenTelemetry Collector is the data plane of OpenTelemetry. Applications instrumented with OpenTelemetry SDKs can send data straight to a backend, but in production most teams put a Collector in between. It decouples applications from backends, offloads batching and retries from the application process, and gives platform teams one place to filter, enrich, sample and route telemetry.

The Collector is a single Go binary configured with YAML. The project ships several distributions: otelcol (core components), otelcol-contrib (a large set of community components), and a Kubernetes-focused distribution. With the OpenTelemetry Collector Builder (ocb) you can also compile a custom distribution that contains only the components you need.

Components: receivers, processors, exporters

• Receivers get data into the Collector. They either listen (for example the otlp receiver on gRPC port 4317 and HTTP port 4318) or pull (for example prometheus scraping endpoints, hostmetrics, filelog for log files, kubeletstats).
• Processors transform data in flight: memory_limiter protects the process from running out of memory, batch groups data for efficient export, k8sattributes and resourcedetection add metadata, filter and transform drop or rewrite data, tail_sampling keeps traces based on their outcome.
• Exporters send data out, most commonly with otlp or otlphttp, but also to Prometheus remote write, Kafka, files or vendor-specific endpoints.
• Connectors join two pipelines, acting as exporter of one and receiver of another. The spanmetrics connector, for example, turns spans into request, error and duration metrics.
• Extensions add capabilities outside the data flow, such as health_check, pprof or authentication.

How pipelines work

Components only become active when they are referenced in the service.pipelines section. Each pipeline handles one signal type (traces, metrics or logs) and lists its receivers, an ordered list of processors and its exporters. Processors run in the order given, which matters: memory_limiter should come first so the Collector can refuse data under memory pressure before doing any work. Batching usually comes late in the chain; recent Collector versions can also batch and queue inside exporters.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 80
    spike_limit_percentage: 20
  k8sattributes: {}
  batch: {}
exporters:
  otlphttp:
    endpoint: https://otlp.example.com
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [memory_limiter, k8sattributes, batch]
      exporters: [otlphttp]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, k8sattributes, batch]
      exporters: [otlphttp]

The endpoint above is illustrative. The same receiver can feed several pipelines, and one pipeline can fan out to several exporters, which is how teams dual-ship data during a migration.

Agent vs. gateway deployment

Pattern	Where it runs	Typical tasks
Agent	On every host, as a Kubernetes DaemonSet or as a sidecar	Receive local OTLP, scrape host and kubelet metrics, tail log files, add host and pod metadata
Gateway	A central, horizontally scaled deployment per cluster or region	Tail-based sampling, PII redaction, routing to several backends, central credentials

Most production setups combine both: agents close to the workload, gateways in front of the backend. Tail-based sampling has one extra requirement: all spans of a trace must reach the same gateway instance, which is why agents typically use the loadbalancing exporter to route by trace ID.

Scaling and sizing

Agents scale automatically with the number of nodes. Gateways are usually stateless and can scale horizontally behind a load balancer, with one important exception: components that keep state, such as tail sampling or cumulative-to-delta conversion, need consistent routing so that related data always lands on the same instance. Size gateways by measuring throughput and memory per replica under realistic load rather than by guessing, and leave headroom for traffic spikes during incidents, when telemetry volume often rises sharply.

Best practices

1. Always configure memory_limiter and set container memory limits, so overload leads to back pressure instead of crashes.
2. Monitor the Collector itself. Its internal telemetry exposes accepted, refused and failed items per component; alert on rising export failures and queue size.
3. Pin versions and build lean distributions. Fewer components mean a smaller attack surface and fewer surprises on upgrade.
4. Use persistent queues for exporters when a short backend outage must not lose data.
5. Keep configuration in Git and roll it out like code, for example with the OpenTelemetry Operator on Kubernetes.

For a deeper checklist, see OpenTelemetry Collector production best practices.

Common mistakes

• Defining components but forgetting to add them to a pipeline, so they silently do nothing.
• Running tail sampling on several gateway replicas without trace-ID-aware load balancing.
• Exposing receivers on 0.0.0.0 without network policies or authentication.

How Obseria helps

Obseria accepts OTLP natively, so any Collector can export to it with a standard otlp or otlphttp exporter. Teams with existing OpenTelemetry SDKs typically see a first trace within 15 minutes. Obseria Pipeline can take over filtering, enrichment and PII masking at ingest, and because data stays in OpenTelemetry formats you can export it via OTLP at any time. See OpenTelemetry in Obseria.