How to use redis

2012.05.31 www.uajjang.com 미디어 검색팀

정경석(kris.j)

What is

When is the right time to use

Advantages and disadvantages

Install

Run/Test

Redis Feature

Performance test

Cluster Recommendation

Summary

REmote DIctionary Server(REDIS) One of open source NoSQL(key–value store) In memory database Supports persistence Redis is written in ANSI C(Portabel) Sponsored by VMware Using github, blizzard, digg, flicker, bump… Last stable version 2.4.14(2012.05.31) Supports a rich set of data types ◦ List, Set, Sorted Set, Hash

Need a fast response

Need a Atomic operations

Need a transaction

Need a Publish/Subscribe

Support for persistence

Use case ◦ Job Queue

◦ Session store

◦ Real time ranking(for all users)

◦ Local Address book(Multi client windows, )

Advantages ◦ Speed(80000 TPS depends on System and Net)

◦ Command level Atomic Operation(tx operation)

◦ Lots of client Lib(Objective-C, C#, Java, node.js, Erlang, Ruby…)

◦ Supports a serverside locking

Disadvantages ◦ Snapshot IO overhead/Memory overhead

◦ Source compile install

◦ Whole dataset be loaded into main memory

Unix/Linux

Windows 64/32 ◦ Unofficial support

◦ Binary http://github.com/dmajkic/redis/downloads

◦ Source export git from https://github.com/dmajkic/redis.git and build

$ wget http://redis.googlecode.com/files/redis-2.4.14.tar.gz $ tar xzf redis-2.4.14.tar.gz $ cd redis-2.4.14 $ make

※ http://try.redis-db.com(ver 2.0.2) test site

// run redis server $ cd $REDIS_HOME/src $ ./redis-server Redis log… // run redis client $ $REDIS_HOME/src $ ./redis-cli redis 127.0.0.1:6379> set firstkey sample OK redis 127.0.0.1:6379> get firstkey "sample" redis 127.0.0.1:6379>

Supports a Expire

redis> set a 1 ok redis> expire a 5 ok redis> get a "1" … (5 second later) redis> (nil) redis>

redis> set a 1 OK redis> set b 12 OK redis> set c 56 OK redis> mget a b c 1) "1" 2) "12" 3) "56"

Supports a Multi get

※ Supports more then 100 commands.

Supports a LIST

redis>del nickname redis> lpush nickname kris (integer) 1 redis> lpush nickname peter (integer) 2 redis> lpush nickname tom (integer) 3 redis> lpush nickname jane (integer) 4 redis> lrange nickname 0 1 1) "jane" 2) "tom"

redis>del nickname redis> sadd nickname kris (integer) 1 redis> sadd nickname peter (integer) 1 redis> sadd nickname tom (integer) 1 redis> smembers nickname 1) "tom" 2) "peter" 3) "kris" redis> sismember nickname tom (integer) 1

Supports a SET

※ Supports Range search ※ Non-redundant data processing

Supports a HASH

redis>del nickname redis> hset user1 name kris (integer) 1 redis> hset user1 lastname jey (integer) 1 redis> hset user2 name tom (integer) 1 redis> hget user1 lastname "jey" redis> hgetall user1 1) "name" 2) "kris" 3) "lastname" 4) "jey"

redis>del nickname redis> zadd nickname 10 kris redis> zadd nickname 10 peter redis> zadd nickname 10 tom redis> zadd nickname 10 jane redis> zrevrange nickname 0 2 1) "tom" 2) "peter" 3) "kris" redis> zincrby nickname 1 kris "11" redis> zrevrange nickname 0 2 1) "kris" 2) "tom" 3) "peter"

Supports a Sorted SET

※ Real time ranking

Supports a WildCard KeySet

redis> mset node Node.js node06 Node.js ruby18 Ruby1.8 OK redis> mset ruby19 Ruby1.9 java Java6 mode9 temp OK redis> keys node* 1) "node06" 2) "node" redis> keys ?ode* 1) "mode9" 2) "node06" 3) "node"

※ Most command's time complexity is O(1), except ‘keys’ : O(N)

※ On desktop environment : 1million keys in 40ms

Supports a Blocking Queue

redis> lpush job-list batch01 redis> lpush job-list batch02

redis> brpop job-list 10 1) "job-list" 2) "batch01" redis> brpop job-list 10 0 (nil) (10.28s) redis> brpop job-list 0 1) "job-list" 2) "batch02" (5.82s)

※ Client 1 ※ Client 2

Supports a Publish/Subscribe(PSubscribe)

redis> publish public-notice 'test message' (integer) 1 redis> publish public-alert 'server will shutdon‘ (integer) 1

redis> psubscribe public* Reading messages... (press Ctrl-C to quit) 1) "psubscribe" 2) "public*" 3) (integer) 1 1) "pmessage" 2) "public*" 3) "public-notice" 4) "test message" 1) "pmessage" 2) "public*" 3) "public-alert" 4) "server will shutdon"

※ Client 1 ※ Client 2

Test ENV ◦ Server : 10.x.x.xx/10.x.x.xx(dual node v2.4.14)

◦ Client : Local PC(Thread 100/100M Lan)

◦ Command : get/set

Jedis Client

Redis의 TPS는 대부분 Network/CPU 성능에 좌우되며 설정에 따라서 더 빨라질 수 있음. 공식 사이트의 측정치는 80000 TPS.

Redis Server(Master)

Redis Server(Slave)

Real time replication

set

get

Master/Slave Snapshot on

16,000~ 20,000 TPS

Slave Snapshot on

Slave Snapshot on

Master Snapshot off

Read Read

Write

모든 데이터가 Memory에 적재되므로 적절한 용량산정 필요.(천만 Key일때 약 500MB메모리 소요)

Redis서버의 구현이 Single Thread로 구성되어 있으므로 물리 Core개수 만큼의 Instance 설치.

현재 release version 2.4.14에서는 Serverside Cluster(Sharding)가 구현되어 있지 않음.(Sharding을 사용하기 위해서는 별도의 구현 필요)

Persistence지원을 위하여 Snapshot 또는 AOP설정을 사용하여야 함.(Disk IO overhead 불가피)

◦ 실제 Field에서는 Master(Snapshot off) slave(on), slave(on)의 구성을 사용하고 있으며 Write는 Master, Read는 Slave에서 처리(성능 향상을 위함.)

Redis Server 재 시작시 최종 Snapshot/AOF의 데이터가 로드됨.

A common convention is to use “obj-type:id:field”

Amazon EC2 Server에서 서비스중인 사례. ◦ 40000~100000 TPS처리중임.

◦ 20GB의 데이터 셋

◦ Amazon EC2 Server의 Virtualize 특성 때문에 Snapshot 사용 문제

원인 fork() 시간이 너무 길게 걸림.)

◦ 위의 문제로 인하여 Snapshot을 off하고 AOF(Append Only File)기능을 사용함.

◦ 위와 같은 설정에서 서비스 이상없이 운영중임.

사용사례 Posting(1억2천만 사용자 대상 실시간 로그인 통계) ◦ http://blog.getspool.com/2011/11/29/fast-easy-realtime-

metrics-using-redis-bitmaps/

◦ Unique방문자수 집계

일간 : 50ms, 주간 : 392ms, 월간 : 1624ms