When I work with time series data, I start with the question window: last five minutes, last day, or last year. I recommend separating recent fast reads from long term rollups before cost gets out of control.
The answer changes over time
Time series data is about shape over time, not a single value in isolation.
How time turns readings into decisions
Time series data is about shape over time, not a single value in isolation.
Typical systems of record
When I talk about "Typical systems of record", I am checking whether Time Series Data Architecture can be traced back, trusted, and used by someone making a decision.
Timestream, InfluxDB, TimescaleDB, Prometheus with Thanos or Cortex, Azure Data Explorer, Bigtable, BigQuery, ClickHouse, Druid.
Provider options in practice
| Provider | Typical services |
|---|---|
| AWS | Kinesis, Timestream, CloudWatch, Managed Prometheus, Redshift. |
| Azure | Event Hubs, Azure Data Explorer, Azure Monitor, Synapse. |
| Google Cloud | Pub/Sub, Dataflow, Cloud Monitoring, Bigtable, BigQuery. |
Real company pattern
My recommendation in this time series pattern is to keep raw windows and rollups connected, because people will ask both what happened now and what changed over months.
Uber real time infrastructure is a public example of time sensitive data at scale: trip, driver, marketplace, and operational signals need fast processing because the business changes minute by minute.
Real world operating example
I use "Real world operating example" to keep Time Series Data Architecture grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.
A payments API records latency, error count, queue depth, and authorization rate every minute. Recent data powers alerts; hourly rollups support capacity planning; long term data supports reliability reviews.
Small implementation example
I use "Small implementation example" to keep Time Series Data Architecture grounded in a real system, because abstract patterns are too easy to agree with and too hard to operate.
with windows as (
select bin(time, 5m) as window_start,
service,
percentile(latency_ms, 95) as p95_latency
from api_metrics
where time > ago(30m)
group by bin(time, 5m), service
)
select service, count(*) as breach_windows
from windows
where p95_latency > 400
group by service
having count(*) >= 3;The query looks for repeated p95 latency breaches across five-minute windows. One spike is noise; three bad windows is an operational signal.
Operational traps
- Validate event time and arrival time separately.
- Define retention before the bill defines it for you.
- Roll up old data but preserve enough detail for incidents.