How we cut answer time from days to seconds
A deep dive into the query planner behind Orbit's natural-language engine, and the three architectural bets that made sub-5-second answers possible.
When we started Orbit, we set an aggressive internal bar: any question asked in plain English should return an answer in under five seconds, on datasets with billions of rows.
That constraint shaped everything about our architecture. Here are the three bets that got us there.
Bet 1: Pre-aggregate everything you can
The fastest query is the one you never run. Orbit maintains a tiered rollup system — hourly, daily and weekly aggregates — and the planner rewrites questions to hit the coarsest rollup that can answer them accurately.
Bet 2: Cache the plan, not just the result
Most systems cache query results. We cache query plans. When someone asks “why did signups drop?”, the decomposition into filters, joins and windows is often reusable even when the underlying data changes minute to minute.
Bet 3: Kill the warehouse round trip
For hot data (the last 30 days), we serve from an in-memory columnar store. Cold data spills to the warehouse, but the planner knows the boundary and only crosses it when a question genuinely needs history.
What’s next
We’re publishing a full technical write-up soon. If you’re building something similar, we’d love to compare notes — reach out any time.