Sampling Is a Bet You Place Before You Know the Odds

Teams sample because keeping everything has been too expensive

Teams sample for financial reasons, because keeping every signal has simply cost too much for most observability budgets to absorb. Traditional observability stacks index heavily at ingest and store data in vendor-managed tiers, so cost grows with volume, and the rational response is to send less.

Seen in that light, sampling is a budget control presented as a configuration setting. Short retention windows and aggressive filtering serve the same financial purpose. If the cost structure changes, the trade-off that justified those controls changes with it, and the case for discarding data becomes much weaker.

DataObserve Sense makes full resolution the normal case

DataObserve Sense is designed so that keeping everything is the standard, affordable setup. Telemetry is stored in compressed, columnar Parquet, and a Rust query engine reads only the columns each query needs. The combination means less storage and less compute per gigabyte, which is where the economics of full fidelity begin.

Logs are searchable with familiar SQL, without a heavy index that has to be built and paid for up front. Telemetry can also sit in the customer's own cloud object storage bucket, billed at the provider's object-storage rates, including lower-cost lifecycle tiers. Logs, metrics, traces and AI telemetry share the same engine, so an organisation never pays three separate systems to store three views of one incident. Taken together, these choices allow Sense to avoid sampling entirely and keep every signal at full resolution.

Full fidelity makes correlation and AI investigation reliable

Correlation is the first capability to benefit. Sense auto-correlates signals across every layer of a request, from the browser click through the gateway and services to the database insert. Correlation helps only when every piece of the chain exists, and sampling breaks that chain at random points that nobody chooses in advance.

The same reasoning applies to automated investigation. The Sense AI SRE works on full-fidelity telemetry, and an agent investigating an incident is only as good as the evidence it can query. A sampled dataset caps the agent's accuracy before the investigation even starts.

Complete data also preserves the questions nobody planned for. The most valuable incident queries are often the ones nobody wrote a dashboard for, and full data keeps them answerable when they finally arise. The benefit extends well beyond the incident itself. When a security issue is discovered months late, or an auditor asks about last year, the relevant telemetry is still stored and still queryable.

Sense keeps telemetry in open formats that the customer owns

Full fidelity is half of the argument, and ownership is the other half. Sense is OpenTelemetry-native, so telemetry arrives over OTLP from any OpenTelemetry SDK or collector without a proprietary agent. Prometheus remote write, Fluent Bit, Vector and the major cloud logging services plug in directly. Data lands as open Parquet files in a bucket the customer owns, under the customer's own encryption keys. Because the data is already in open formats, it can be moved whenever the organisation decides to move it.

Sense deploys on Kubernetes with a single Helm command

Adoption is designed to be quick. On Kubernetes, the collector deploys with a single Helm command, and it begins collecting container logs, cluster events, metrics and zero-code traces for workloads that have not yet been instrumented. Dashboards and alert rules ship pre-built, and existing Grafana boards can be imported. Full fidelity therefore starts on day one, which spares the team a quarter of re-instrumentation work.

Recent incidents show what sampling has cost

A useful exercise is to review the last three significant incidents and ask, for each one, whether any data the team needed had been thrown away or had expired before anyone looked. If the answer is yes even once, sampling has only deferred its cost to the worst possible moment, when an outage was already under way and the evidence was gone.