Redis ndc2013
-
Upload
dae-myung-kang -
Category
Technology
-
view
4.357 -
download
4
description
Transcript of Redis ndc2013
Redis Tips
• OpenSource Contributor • REDIS • TWEMPROXY
Before Presentation
Agenda • Dangerous Commands •Memory Policies •Replication •RDB
Dangerous Commands •Keys *(pattern)
•FlushAll
Dangerous Commands •Why? –Single Thread. –Each command should spend small time.
Keys * • Don’t use keys command in production. –“Warning: consider KEYS as a command that should only be used in production environments with extreme care” is written in redis.io keys commands manual.
Keys * di = dictGetSafeIterator(c->db->dict); allkeys = (pattern[0] == '*' && pattern[1] == '\0'); while((de = dictNext(di)) != NULL) { …… stringmatchlen(pattern,plen,key,sdslen(key),0) }
FlushAll Cache Item Count Time
Memcache 1,000,000 1~2ms
Redis 1,000,000 1000ms(1 second)
FlushAll • Redis’s FlushAll is slow and paused.
• But memcache’s flush is really fast.
–memcache doesn’t depend on Item count.
–Redis depends on Item count.
FlushAll • What is different from memcache’s flush.
• Memcache just sets time to flush.
• Redis deletes all items.
–It spends 1 second in 1M or 2M items
FlushAll-Redis long long emptyDb() { for (int j = 0; j < server.dbnum; j++) { dictEmpty(server.db[j].dict); dictEmpty(server.db[j].expires); } return removed; }
FlushAll-Redis
FlushAll-Memcache static void process_command(conn *c, char *command) { if (exptime > 0) settings.oldest_live = realtime(exptime) - 1; else /* exptime == 0 */ settings.oldest_live = current_time - 1; }
FlushAll-Memcache if (settings.oldest_live != 0 && settings.oldest_live <= current_time && it->time <= settings.oldest_live) { do_item_unlink(it, hv); do_item_remove(it); it = NULL; }
Redis Memory Policy • Volatile-lru • Allkeys-lru • Volatile-random • Allkeys-random • Volatile-ttl • noeviction
Redis Memory Policy • In 64bit, Memory Policy doesn’t work,
if you don’t set maxmemory parameter.
• In 32bit, maxmemory is set 4GB as default
Default Setting in 64 • There is no memory limitation.
• So If you insert bigger data than memory. It will be slow.
• But But But. Make sure Vm.overcommit_memory property.
Memory Policy • If you use Memory-Policy and
replication.
– It can cause some consistency problem.
Master Slave Delete propagation
Delete items because of its own memory policy
Replication •Support Chained Replication
•Common mistake
Replication •Support Chained Replication
Master
Slave Slave
Slave Slave
Replication • Common mistake. –Don’t forget “slaveof no one”
• Slave’s replicationCron keeps track of master’s health.
Replication
Master Slave replicationCron
Health check
Replication
Master Slave replicationCron
Health check
Replication
Master Slave replicationCron
When master reruns, Resync with Master
Replication
Master Slave replicationCron
Slave will has no data after resyncing
If master has no data.
Initial Replication Step
RDB • Good and Bad.
• It causes all of problems.
RDB - GOOD • Can store data into persistent layer.
RDB - BAD • Bad Performance in Large memory.
• Twice memory problem.
• Denying write problem when storing RDB fails.
• If you just want Cache. Turn off RDB
RDB – Large Memory • Performance is relevant to Memory
Size.
17.1 GB 34.2 GB 68.4 GB
117 GB
RDB - BAD • Using bgsave in slave.
• Turning off RDB.
• Using save in master or slave.
RDB – Twice Memory • fork() and COW(copy on write) Issue
–In Write Heavy System:
RDB – Twice Memory
RDB – Twice Memory
RDB – Twice Memory
Real Case Study • Background
–Can’t write to Redis Server
–Sentinel doesn’t find Server’s failure.
Real Case Study • Reason
– If redis fails to save RDB, Redis basically denies write operations from client.
– “MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.”
Real Case Study if (server.stop_writes_on_bgsave_err && server.saveparamslen > 0 && server.lastbgsave_status == REDIS_ERR && c->cmd->flags & REDIS_CMD_WRITE) { flagTransaction(c); addReply(c, shared.bgsaveerr); return REDIS_OK; }
Real Case Study • Solution #1
• Solution #2
config set stop-writes-on-bgsave-error no
Turn off RDB Setting
2.6.12 부터 conf 에서 stop-writes-on-bgsave-error 설정이 가능해짐.
SELECT Command • Don’t use “SELECT” Command.
–You will not be able to use “SELECT” in Redis Cluster.
• Redis Proxy can make some troubles.
Response Time • Why memcache’s response time is
uniformed? –Difference of Memory Allocator
Memcache Redis - Slab Allocator - Just malloc()
Redis VS Memcache • The popular question
–Redis and Memcache: What is better?
Memcache Redis - Uniformed Response
Time - More stable
- Data Structure - Persistent
Sentinel
Sentinel • Sentinel is Failover Solution for Redis.
Master Slave
Sentinel Sentinel periodically checks Redis Master
Sentinel
Master Slave
Sentinel Send “slaveof no one” to slave node
Sentinel
Master Slave
Sentinel Notify to client about changing master
Client
Sentinel redis 127.0.0.1:2003> psubscribe * Reading messages... (press Ctrl-C to quit) 1) "pmessage" 2) "*" 3) "+switch-master" 4) "resque 127.0.0.1 1999 127.0.0.1 2002"
Sharding or Clustering
Two Approches.
Client Library VS
Server Proxy
Client Library
NHN Case Study: http://helloworld.naver.com/helloworld/294797
Craiglist Case Study: http://blog.zawodny.com/2011/02/26/redis-sharding-at-craigslist/
Client Library ZooKeeper
or health Checker
Redis Library
Craigslist
NHN Application Servers
ZooKeeper
Redis Cluster Manager
Redis Shard-1
Redis Shard-2
Redis Shard-3
NHN
Client Library • Benefits –No hob(faster than server proxy)
• Liabilities
–Consistency Problem(Timing Issue)
Server Proxy
Redis Cloud Services
Twemproxy https://github.com/twitter/twemproxy
Twemproxy • Simple and fast memcache/redis proxy
Client Client Client Client
Twemproxy
Redis Redis Redis Redis
Twemproxy If you use redis as a store, Twemproxy might not be good for your purpose
Server Proxy • Benefits –Client doesn’t need any other libraries
• Liabilities
–Hob.(less speed and proxy is also SPOF)
Q & A
Thank you!