I was a little disappointed to see that this was just a single-machine hierarchical time index on top of a pre-sorted append log.
It's not a bad architecture, and the lockless queuing system and shared memory is cute, but:
- It only supports time indexing. Querying by other fields (e.g. if you wanted to build a time histogram of when you saw a particular event) requires reading the entire dataset.
- It doesn't address replication/distributed indexing, which seems like a must.
- The use of calendar time over micro-timestamps also needlessly complicates things, though presumably it makes their queries more efficient (assuming people query by discreet time buckets like "yesterday").
It's not a bad architecture, and the lockless queuing system and shared memory is cute, but:
- It only supports time indexing. Querying by other fields (e.g. if you wanted to build a time histogram of when you saw a particular event) requires reading the entire dataset.
- It doesn't address replication/distributed indexing, which seems like a must.
- The use of calendar time over micro-timestamps also needlessly complicates things, though presumably it makes their queries more efficient (assuming people query by discreet time buckets like "yesterday").