Black-box monitoring checks a system from the outside, the way a user or client sees it, without any knowledge of its internals. Probes send real requests, such as HTTP calls, DNS lookups, TCP connects or scripted user journeys, and verify availability, correctness and response time. It detects symptoms that are visible to users right now.
What is black-box monitoring?
Black-box monitoring observes a system purely through its externally visible behaviour. A probe sends a request to an endpoint, waits for the answer and checks whether it arrived, whether it was correct and how long it took. The probe does not know which services, databases or queues sit behind the endpoint; it treats the system as a closed box.
The opposite is white-box monitoring, which relies on telemetry emitted from inside the system: metrics, logs, traces and profiles. Google's SRE book describes the two as complementary. Black-box monitoring is symptom-oriented and answers "is it broken for users right now?". White-box monitoring answers "why is it broken?" and can also reveal problems that have not reached users yet. The distinction is part of the broader observability vs. monitoring discussion.
How black-box monitoring works
A black-box check usually consists of four parts:
- A target, such as a URL, hostname, IP and port, DNS record or TLS endpoint.
- A probe that runs on a schedule from one or more locations, for example every minute from several regions.
- Assertions that define success: an HTTP status code, a string in the response body, a valid certificate, a response time below a threshold.
- An alerting rule that fires when checks fail, typically only if several consecutive checks or several locations fail, to avoid paging on a single network glitch.
Simple probes test a single request, like a health check endpoint or a TCP connect. Scripted browser checks go further and replay a user journey, such as logging in, searching and adding an item to a cart. This is usually called synthetic monitoring.
Black-box vs. white-box monitoring
| Aspect | Black-box | White-box |
|---|---|---|
| Perspective | External, as a user or client | Internal, from the running code and infrastructure |
| Data | Probe results: success, status, response time | Metrics, logs, traces, profiles |
| Detects | Outages, DNS, TLS, CDN and routing problems | Causes, degradations, problems before they hit users |
| Blind spots | Why something fails; paths the probe does not test | Failures outside instrumented code, e.g. DNS or a load balancer |
A key strength of black-box monitoring is that it works even when your internal telemetry does not. If a cluster goes down completely, internal metrics may simply stop arriving, while an external probe reports the outage clearly.
Black-box checks are also the most direct way to measure what customers and contracts care about. An availability figure in an SLA is usually defined in terms of successful requests from the outside, and probe results provide an independent record of it. At the same time, probes only see the paths they test. A check against the home page says nothing about a broken payment provider, which is why critical journeys need their own checks and why probe coverage should be reviewed whenever new features ship.
Example: Prometheus blackbox exporter
The Prometheus blackbox exporter is a widely used open-source prober. An illustrative module that requires a 2xx response over valid TLS looks like this:
modules:
http_2xx_tls:
prober: http
timeout: 5s
http:
valid_status_codes: [] # defaults to 2xx
fail_if_not_ssl: true
fail_if_body_not_matches_regexp:
- "status.*ok"The exporter exposes probe_success and probe_duration_seconds, so an alert can fire when probe_success is 0 for several minutes. The same probe data can feed uptime reports and certificate expiry warnings.
Best practices
- Probe from outside your own network and from several locations, otherwise you test your internal network, not the user path.
- Check correctness, not just reachability. A 200 response with an error page is still an outage.
- Require multiple failures before paging to keep false alarms and alert fatigue low.
- Cover critical user journeys such as login and checkout, not only the home page.
- Pair every black-box alert with white-box data so the on-call engineer can move from symptom to cause quickly.
Common mistakes
- Relying only on black-box checks, which tell you that something is broken but not why.
- Probing a shallow endpoint that returns OK even when the database is unreachable.
- Running probes from the same data centre as the application.
- Generating test traffic that pollutes business analytics, instead of tagging it as synthetic.
How Obseria helps
Obseria runs HTTP, API and scripted browser checks as part of its synthetic monitoring and stores the results in Obseria Lake next to internal metrics, logs and traces. Failed checks can open incidents, trigger runbooks via Automation and update status pages, and the corresponding backend traces help explain what went wrong.
