Fighting Availability Bias in Observability¶

CPU usage is usually available before we know what success means for the service. Memory graphs appear before anyone has defined the user journey. Infrastructure dashboards are ready on the first day, while service-level measurements remain a backlog item.

This order is understandable. Platforms expose resource telemetry cheaply and consistently. User outcomes are specific to each service and require deliberate design.

The danger is that availability becomes importance. We pay more attention to the measurements that are easy to obtain, then build our understanding of the system around them.

That is availability bias in observability.

Reliability requires detecting meaningful change¶

A reliable system adjusts to changing conditions. In control-theory terms, it must detect a change in behavior and apply an action that moves the system back toward its desired outcome.

Measurement is therefore central to SRE. But the loop only works when the measurement represents something the controller needs to know.

Suppose a service has low CPU usage and plenty of free memory. Those facts may be accurate while:

Every request fails authorization.
A downstream dependency exceeds the caller's deadline.
Responses are fast but contain incorrect data.
Work is accepted into a queue and never completed.
One customer network cannot reach the service.
A browser workflow breaks after an interface change.

Resource telemetry describes conditions inside part of the system. It does not directly describe whether the service is producing its intended outcome.

The problem is not that CPU and memory are useless. The problem is treating them as primary evidence about behavior they do not measure.

The first dashboard becomes the mental model¶

Measurements shape how teams talk about systems. The signals placed on a default dashboard become the signals people inspect during an incident. The alerts enabled by a platform become the conditions people learn to call “unhealthy.”

This creates a reinforcing loop:

Easy signal -> prominent dashboard -> repeated attention
     ^                                  |
     +--------- assumed importance <----+

Over time, the team becomes skilled at explaining CPU spikes and memory pressure even when neither is the reason users are failing. Meanwhile, a critical behavior may remain unmeasured because no standard exporter could name it.

Availability bias is not only an individual cognitive error. It can be built into platform defaults, dashboard templates, alert catalogues, and review processes.

Start with the controlled outcome¶

The most effective countermeasure is to reverse the usual instrumentation sequence.

Do not begin with the metrics a machine or framework exposes. Begin with the outcome the service is expected to control.

For example:

A request must complete correctly within its useful deadline.
A scheduled job must produce an artifact before another process needs it.
A user must be able to authenticate and complete a critical workflow.
A message must be processed at least once without creating an unacceptable duplicate effect.

Once the outcome is explicit, ask which evidence reveals whether it is being produced. That evidence becomes the top of the measurement hierarchy.

Supporting signals then explain the outcome:

Outcome signals — success, correctness, latency, freshness, completion.
Path signals — dependency calls, queues, hand-offs, retries, fallbacks.
Component signals — local errors, saturation, internal state.
Resource signals — CPU, memory, disk, network, file descriptors.

The lower levels are not less technical or less valuable. They are lower because they explain the outcome rather than define it.

Measure from more than one place¶

Distributed systems do not have one universal vantage point. A service can be healthy from inside its cluster and unreachable from a customer network. A public endpoint can respond correctly while an internal dependency path is broken. One region can succeed while another times out.

Availability bias also appears as vantage-point bias: we trust the measurement taken from the place where measurement was easiest to deploy.

Synthetic monitoring and continuous verification help by placing known tests where relevant interactions occur. A worker in a private network can verify behavior that a public uptime service cannot see. A browser test can verify a journey that an endpoint check cannot represent.

The goal is not to probe from everywhere. It is to choose vantage points that correspond to meaningful service boundaries and failure domains.

Separate detection from diagnosis¶

One reason infrastructure signals become prominent is that teams ask one dashboard to do two jobs:

Detect that the service outcome is at risk.
Diagnose which internal condition caused the change.

These are different tasks.

Detection should begin as close as possible to the desired outcome. Diagnosis usually moves inward through paths, components, and resources. If the detection signal begins at the resource level, the team is forced to infer user impact before it even knows whether an incident exists.

A high CPU alert can be useful when high CPU reliably predicts imminent loss of a service objective and there is a safe action to take. Without that link, it is a diagnostic clue promoted into an interruption.

For every alert, ask:

Which desired outcome is threatened?
How strong is the relationship between this signal and that outcome?
What action should follow?
How will we verify that the action protected the outcome?

An alert without a plausible decision path is attention telemetry, not a control signal.

Defaults should expose their assumptions¶

Platform teams can reduce availability bias by treating default dashboards and alerts as opinionated products.

A default should say what it can and cannot establish. A host dashboard can state that it describes resource conditions, not service health. A Kubernetes dashboard can distinguish scheduling and runtime state from user-visible behavior. An alert template can require the adopting team to name the service objective and response action.

Useful platform defaults include:

A place for service-level outcome measurements alongside infrastructure signals.
Clear separation between symptom alerts and diagnostic dashboards.
Templates that begin with user journeys and critical operations.
Synthetic-check placement across relevant network and regional boundaries.
Review of unused dashboards and alerts that no longer support a decision.

The point is not to remove standard telemetry. It is to stop presenting standard telemetry as a complete model of every service.

More telemetry is not the same as better observation¶

When a system remains difficult to operate, adding telemetry feels like a safe response. Sometimes it is correct. Sometimes the real gap is a weak system model.

If a team cannot say which service boundary matters, no amount of tracing will choose it for them. If an alert has no safe response, a higher-resolution metric will not create one. If a retry policy destabilizes the service, collecting more retry counters is less valuable than understanding the feedback loop and changing the policy.

Better observation combines three things:

A model of the distributed system and its interactions.
Measurements selected for a specific outcome and decision.
Enough internal evidence to explain unexpected behavior.

Telemetry volume is not one of the goals.

A practical review for every signal¶

Teams can audit an observability surface with a short set of questions:

What does this signal measure directly?
Which service outcome does it help protect?
Is it used for detection, diagnosis, or verification?
From which vantage point is it measured?
Which failure modes can it not see?
What decision or action follows from a meaningful change?
Who owns that action?
How do we know the action worked?

If the answers are weak, the signal may still be useful for exploration. It should not automatically consume permanent dashboard space or generate an alert.

Make importance determine availability¶

We will always begin with some signals because they are cheap and standard. That is not a failure. The failure is allowing those signals to define the system we think we are operating.

Start with the behavior the service must preserve. Make those outcomes measurable, even when doing so requires application-specific work. Add path, component, and resource evidence to explain changes. Place active checks where important users and dependencies experience the system.

The objective is not to ignore readily available telemetry. It is to make the important measurements available too.