Shared agentsLogs
All indexed, no sampling.
- Observability & Monitoring
//Pipelines
Observability pipelines.Shape data before it lands.
Transform, enrich, redact, reduce, and parse your observability data with real-time and scheduled pipelines.
Privacy
Redact PII before it's stored
Sensitive fields are stripped or masked at ingestion, before data ever hits disk.
Real time
Parse and enrich in real time
VRL functions transform data the moment it arrives, no batch delay.
Formats
One pipeline, any format
JSON, syslog, CEF, and custom formats without a separate parser per source.
//Overview
What is an observability pipeline?
An observability pipeline processes telemetry between the source and storage: it parses, enriches, transforms, filters and redacts data to support your storage and data-handling policies.
DataObserve Sense runs real-time pipelines as data arrives and scheduled pipelines on stored data, using VRL functions, enrichment tables, dynamic lookups and stream operations to redact PII and shape data before it lands.
//Shape data before it costs you
Pipelines, transformations, and enrichment
Real-time pipelines
Parse, filter, and enrich streams the instant they arrive.
Scheduled pipelines
Aggregate large datasets and turn logs into metrics on your timeline.
VRL functions
Custom transformations for complex logic.
Enrichment tables
Location, user, and metadata from simple CSV lookups.
Dynamic lookups
External context pulled in via API at ingestion.
Stream operations
Filter, clone, and route streams to multiple destinations.
//Explore more withDataObserve
More from Sense · Observability & Monitoring
Metrics
PromQL and SQL, full resolution, retained.
- Observability & Monitoring
Traces
OpenTelemetry-native, no sampling.
- Observability & Monitoring
Frontend Monitoring
Real user monitoring, session to stack trace.
- Observability & Monitoring
//FAQs
Pipelines: questions teams ask
Which pipeline types are there?
Real-time pipelines process data as it arrives; scheduled pipelines run batch transformations.
What are enrichment tables?
CSV lookup tables that add context, queried from functions.
Can pipelines route conditionally?
Yes. VRL conditions route and process data selectively.
What is VRL?
VRL (Vector Remap Language) is a language for transforming observability data. DataObserve Sense pipelines use VRL functions to parse, reshape and route events.
Can pipelines remove personal data?
Yes. Pipelines can redact or mask PII before it is stored, which helps with privacy regulations such as GDPR.
