Structured logging is the practice of emitting log events as machine-readable records with named fields, typically JSON, rather than free-form text. Each event carries a timestamp, severity, message and context such as service name, trace ID and request attributes, so logs can be filtered, aggregated and correlated with traces without fragile parsing.
What is structured logging?
Traditional logs are lines of text written for humans: 2026-10-15 09:12:03 ERROR payment failed for order 8812 after 3 retries. To answer a question such as "how many payment failures per provider in the last hour", someone has to write and maintain a regular expression for that exact sentence. Structured logging emits the same event as a record with explicit fields:
{"timestamp":"2026-10-15T09:12:03.512Z","severity":"ERROR",
"message":"payment failed","service.name":"checkout",
"order.id":"8812","payment.provider":"acme","retry.count":3,
"trace_id":"4bf92f3577b34da6a3ce929d0e0e4736",
"span_id":"00f067aa0ba902b7"}Now every field is directly queryable, the message stays stable when someone rewords it, and the trace ID links the log line to the request that produced it. JSON is the most common format; logfmt (key=value pairs) is a lighter alternative that stays readable in a terminal.
Why structured logging matters
- Reliable querying: filtering by
payment.providerworks regardless of wording or log library. - Aggregation: counts, rates and distributions can be computed from log fields, which turns logs into an analytical data source.
- Correlation: with
trace_idandspan_idin every record, you can jump from a slow span to its logs and back. - Cheaper processing: pipelines can route, sample or mask fields precisely instead of guessing with patterns.
- Privacy controls: sensitive fields can be identified and redacted by name.
The OpenTelemetry log data model
OpenTelemetry defines a common data model for logs so that records from any language or source can be handled uniformly. Its main fields are:
| Field | Purpose |
|---|---|
| Timestamp / ObservedTimestamp | When the event occurred and when it was collected |
| SeverityText / SeverityNumber | Original level name and a normalized number from 1 to 24 (TRACE 1–4, DEBUG 5–8, INFO 9–12, WARN 13–16, ERROR 17–20, FATAL 21–24) |
| Body | The message itself, a string or structured value |
| Attributes | Event-specific key-value pairs |
| TraceId / SpanId / TraceFlags | Link to the active trace context |
| Resource / InstrumentationScope | Who produced the log, e.g. service.name, k8s.pod.name |
OpenTelemetry log bridges connect existing libraries such as Log4j, Logback, Python logging or Winston to the SDK, so trace context is attached automatically and logs are exported via OTLP alongside traces and metrics.
Designing good log fields
- Use consistent names across services, ideally aligned with OpenTelemetry semantic conventions (
http.route,user_agent.original,exception.type). - Keep the message constant and put variable data into fields. "payment failed" with
order.idas a field beats a message that embeds the ID. - Keep types stable. If
retry.countis a number in one service and a string in another, aggregations break. - Log events, not narratives. One well-structured record at the end of an operation is often more useful than ten lines tracing each step.
- Choose levels deliberately. ERROR should mean someone may need to act; routine retries are WARN or INFO.
Moving an existing system to structured logs
Few teams can rewrite every log statement at once, and they do not need to. A pragmatic migration has three steps. First, switch each service's log output to a JSON encoder or formatter; most libraries support this through configuration, and the existing messages simply become the message field. Second, add shared context automatically: service name, environment, version and the active trace context through an OpenTelemetry log bridge or a logging middleware. Third, refactor the log statements that matter most, typically errors and key business events, so their variable parts become fields. For legacy components you cannot change, parse their text output once in the Collector or pipeline and emit structured records from there.
Common mistakes
- Embedding JSON inside a text message, which produces escaped strings that nobody can query.
- Logging personal data, secrets or full request bodies; mask them at the source or in the pipeline.
- Turning every log field into an indexed label or metric dimension, which explodes cardinality and cost.
- Different timestamp formats and time zones across services; use ISO 8601 in UTC or epoch nanoseconds.
- Leaving DEBUG logging on in production at full volume, so that a few noisy services dominate ingest.
How Obseria helps
Obseria Log Analytics stores logs in Obseria Lake with schema-on-read, so JSON fields are queryable without defining indexes upfront, and unstructured lines can be parsed at query time when needed. Obseria Pipeline masks PII, drops noise and routes logs at ingest, and logs carrying trace IDs are linked to their traces automatically. A query such as fetch logs | filter service == "checkout" | summarize count(), by:{payment.provider} runs directly on the structured fields. To see how logs fit with the other signals, read what is observability.
