Observability retention windows usually run from thirty days down to seven on the cheaper plans, and teams have learned to treat those windows as a law of nature. Logs and traces age out on schedule, and getting last quarter's data back means rehydrating an archive and then paying again once the wait is over. It is worth stating the cause plainly: retention limits exist because vendors pay for your storage. When the vendor holds the data, every extra day is a cost passed on to the customer, so windows stay short and long-term history becomes a premium feature. Once the organisation takes the storage back, the reason for those limits disappears.
Retention limits exist because vendors pay for your storage.
Short retention costs far more than the bill shows
The subscription bill is the visible cost of short retention, and the hidden costs are usually worse. Security incidents are often discovered late, and by the time investigators reach for the logs, the window has already closed. Regulations and audits can require multi-year retention, which leads many teams to run a second system in the form of a cold archive. That archive technically satisfies the requirement, yet it still cannot be queried when someone needs an answer.
Longer-term engineering work suffers in a similar way. Capacity planning and the tracking of performance regressions both depend on data that spans months or years, and seasonal patterns follow the same timescale. A thirty-day window cannot show last year's peak. The same constraint applies to AI-assisted operations, because an AI SRE that remembers past incidents is useful only if the telemetry from those incidents still exists.
Security
Breaches are often found late, after the window has closed
Compliance
Multi-year rules lead to a cold archive nobody can query
Engineering
Capacity, regressions and seasonality need months or years
AI operations
An AI SRE can only recall incidents whose telemetry still exists
Sense writes telemetry to a bucket the customer owns
DataObserve Sense allows an organisation to connect its own cloud bucket on Amazon S3 or Google Cloud Storage, and Azure Blob is supported as well. Sense writes telemetry to that bucket in compressed, columnar Parquet. From that point onward, the organisation's own policy decides how long data is kept, whether for years or indefinitely, with no forced expiry. Storage is billed at the cloud provider's object-storage rates, including lifecycle tiers such as S3 Glacier.
Old data also stays queryable, so telemetry from years ago runs through the same dashboards and alerts as data ingested yesterday. The storage format is what makes this practical. Columnar storage means a query reads only the columns it needs, and a Rust query engine keeps storage and compute per gigabyte low, so keeping more history does not have to mean paying more to search it. Setup involves creating a bucket in the organisation's own cloud account and granting Sense access to it, after which new telemetry lands in that bucket.
Step 1
Create a bucket in your own cloud account
Step 2
Grant Sense access to it
Step 3
New telemetry lands in that bucket
Owning the storage puts the security controls in the customer's hands
Moving storage into the customer's account changes who controls the data, and that matters as much as the cost. The organisation holds the encryption keys and sets access, which it can revoke at any time. Data residency becomes the organisation's decision as well, and telemetry is governed by its own access and retention policies. For regulated industries, that level of control often decides whether a tool is approved or blocked.
The same principle runs through the rest of Sense. Data arrives over standard OpenTelemetry without a proprietary agent, and it is stored as open Parquet files. If an organisation later changes its mind about any part of the stack, its history moves with it.
Owning the bucket changes several common use cases
Several everyday requirements become simpler once the bucket belongs to the organisation. Compliance retention works without a separate archive, because logs can be kept for as long as regulations or audits require. Security teams can search historical telemetry without rehydration delays when a breach is found late. Performance and capacity can be compared across years, since long-term trends remain visible far beyond a window of a few weeks. Throughout, the telemetry stays in an account the organisation controls and follows its own policies for as long as it chooses to keep the data.
Enterprises should own the operational memory their AI depends on
At DataObserve, we believe enterprises should own the intelligence they depend on, from the data it learns from to the infrastructure it runs on. Observability data is part of that intelligence. It is the operational memory of an organisation's systems, and increasingly it is the raw material that AI agents reason over. Renting that memory by the day, with a deletion date attached, never made much sense, and it makes even less sense as AI agents take on more operational work.
The contract and the incident history will settle the question
A simple review can show whether retention limits are costing an organisation more than it realises. The first step is to pull up the current observability contract and note the retention window along with the cost of extending it. The second is to find the date of the oldest incident the team still talks about. If that incident is older than the retention window, the answer to the question is already clear.
From the contract
- Retention window
- _____ days
- Cost to extend
- $_____
From the team
- Oldest incident still discussed
- _____ days ago
- Older than the window?
- Then the answer is already clear
Every signal can be kept in your own bucket for as long as you need, as described at dataobserve.io/products/sense.
We would be interested to hear how long your team actually needs to keep telemetry, since the answers tend to vary widely.