Beware of your Hype Value Stores
-
Upload
jeremie-bordier -
Category
Technology
-
view
5.423 -
download
0
description
Transcript of Beware of your Hype Value Stores
Beware of your Hype – Value store !
Ignite Velocity 2009
Jérémie BORDIER -
@ahfeel / [email protected]
GettheHype!
• Rela&onalDBMSareso1990…• Sosimple,sofast,lookspowerful!
2 Tokyo Tyrant
Redis
Scalaris
Project Voldemort Hypertable Cassandra
MemcacheDB
MongoDB
Persevere BerkeleyDB
CouchDB
Dynomite SimpleDB Dynamo
10000+writes/sec!
• RDBMSonlyperforms±300w/sec…wC?
3
Let’s do some simple math !
Stateoftheartmathema8cs
• AverageSATAIIdiskseek&me:±6ms• VerygoodSCSIdisk:±3ms
4
1 second / 3ms = ±300 REAL writes / sec
EventualPersistency
• Doesn’tsyncthefilesystem• Keepthatinmind!
YourserversWILLcrash,youWILLlosedata
5
Don’tcareaboutwrites…
• Ifcando30000+lookup/sec!
6
Not for long !
Eventualmemory
• ReliesonB+Trees• Mapallthedatastructuresinmemory
7
What if your data goes too big ?
Lookup…
• get(“elevator”)– Visit8nodes– Readthedatafromdisk
8
E L
A S
V E A
S
O T
O
M
R Data: 42
Whyelevator?
• Americanelevatorsarewayyyytoofast!• FeelslikeaNASAtraining
Makesmewanttothrowup:(
9
Hardwarelimit!
• Onverygoodhardware!
10
Up to 9 RANDOM I/Os per lookup !
1 second / (9 * 3ms) = 37 get / sec
Wedidthattoo:)
• Wehadsimilaralgorithms• Encounteredhorribleperfdecreases…
11
REDUCE I/O !
Ensure only 1 I/O per lookup.
Don’ttrustbenchmarks
• Afewmillionentriesisn’tenough• You’rebenchmarkingyourDisk/OScache:)
12
(Common, DON’T BENCHMARK IN RUBY…)
Comparewhat’scomparable
• Distributedcolumnstores– BigTablelikesystems
• Keyvaluestores– Tokyo,Dynamolikesystems
13
Distributedcolumnstores
14
Hypertable
Cassandra
SimpleDB
HBase Distributed
Replicated
Persistent
Eventually Persistent
Mature
Google Megastore
Key–valuestores
15
Mutable
Embedded Distributed
Replicated Persistent
Eventually Persistent
Immutable
Tokyo Cabinet
Redis
Scalaris
Voldemort MemcacheDB
MongoDB
Persevere BerkeleyDB
CouchDB
Dynomite
Mature
Howtochoose?
• Maturityispriceless• Mostsuitablestores:
– Persistent:BerkeleyDB,MySQL
– Ev.Persistent:TokyoCabinet– Ev.Persistent+Distributed+…:Voldemort– Distributedcolumnstore:Cassandra
16
YouarenotGoogle
• Buildsomething• Makeitwork
• Thinkaboutscaling• Thinkaboutbeinghyped
17
(Well, not all of you )
Why ?
Keyvaluepain…
• Youwillendupcrossingdata.• DoingjoinswithKVstores?
18
Only for ninjas !
Query8mejoins
• CodingwhatRDBMSaremadefor?…• Slow!
19
(you shouldn’t do this…)
Build8meschemaflaTening
• Notflexible!• NeedsMapReduce(Hadoop…)toscale
20
Think before doing this :)
Andfinally..
• HRDharddrive:160000randomIO/sec!!!• YahooannouncedopensourcingSherpa!• Resources
– hip://developer.yahoo.net/blog/archives/2009/06/nosql_meetup.html
– hip://metabrew.com/ar&cle/an&‐rdbms‐a‐list‐of‐distributed‐key‐value‐stores/
– hip://www.ryanpark.org/2008/04/top‐10‐avoid‐the‐simpledb‐hype.html
– hip://project‐voldemort.com/blog/2009/06/building‐a‐1‐tb‐data‐cycle‐at‐linkedin‐with‐hadoop‐and‐project‐voldemort/
21
Thanks ! Contact: @ahfeel :)