NoSQL for Time Series Data Benchmark
May 26, 2010
Speed of processing is an important aspect in choosing a data storage for time series data. The fast the better.
So how fast it is when NoSQL meet time series data? The following was what I found.
Test env
Hardware/OS
- Ubuntu 2.6.32-19-generic SMP 64bit
- Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz
- 2G memory
- 5400RPM hard disk
we have compared:
- Toyko Cabinet 1.4.44, B+ tree database
- Toyko Cabinet 1.4.44, table databse
- mongoDB snapshot-v20100520, multi-collections
- mongoDB snapshot-v20100520, one-collection with index
- redis 1.3.8
ruby client:
- tokyocabinet-ruby-1.30
- mongo 1.0.1 with BSON_ext
- redis 1.0.4
benchmark code
I pushed it to github in case you want to roll you own results.
Results
write 1M records
build index
read last 30 days ohlc by symbol
read all ohlc by symbol
Storage size
Results clearly shows Toyko Cabinet BDB was the winner.
Notes
This wasn't a full benchmark for NoSQL database, but more specific for time series data.
