Hedis - GET HBase via Redis

Kewang

2

Live DEMOhttps://www.youtube.com/watch?v=aAgJaqvgqLU

Who I am

● 王慕羣● Java / Node.js / AngularJS● SQL-like / HBase

GitHub: kewang

Facebook: kewangtw

Linkedin: kewangtw

Slideshare: kewang

Mail: cpckewang@gmail.com

What Mitake is

三竹資訊

What Mitake is

三竹資訊● 企業簡訊平台

What Mitake is

三竹資訊● 企業簡訊平台● 市佔 100%的行動下單

What Mitake is

三竹資訊● 企業簡訊平台● 市佔 100%的行動下單● 市佔 70%的行動銀行

What Mitake is

三竹資訊● 企業簡訊平台● 市佔 100%的行動下單● 市佔 70%的行動銀行● 企業內部應用、產壽險、金融相關政府機關

What Mitake is

三竹資訊● 企業簡訊平台● 市佔 100%的行動下單● 市佔 70%的行動銀行● 企業內部應用、產壽險、金融相關政府機關● 雲端：Qmi、 TechMe

What Mitake is

三竹資訊● 企業簡訊平台● 市佔 100%的行動下單● 市佔 70%的行動銀行● 企業內部應用、產壽險、金融相關政府機關● 雲端：Qmi、TechMe● 其他：兌彩券、台灣匯率、三竹小股王、行動股市 ...等

11

Agenda

● Redis brief● Scenario● Hedis (aka Hyper Redis)● Benchmark● TODO list

12

Redis brief

13

Redis brief

14

Redis brief

● Key-value pair

15

Redis brief

● Key-value pair● Value data type: String, List, Sets, Ordered sets,

Hashes

16

Redis brief

● Key-value pair● Value data type: String, List, Sets, Ordered sets,

Hashes● Binary-safe

17

Redis brief

● Key-value pair● Value data type: String, List, Sets, Ordered sets,

Hashes● Binary-safe● Single thread

18

Redis brief

● Key-value pair● Value data type: String, List, Sets, Ordered sets,

Hashes● Binary-safe● Single thread● In-memory database

19

ScenarioGet hot data, etc.

20

Redis

21

Redis

Application server

Redis Database

Client

22

Redis - Send request to AP

Application server

Redis Database

Client (1)

23

Redis - AP gets data from Redis

Application server

Redis Database

(2)

Client (1)

24

Redis - Redis has no data

Application server

Redis Database

(2)

Client (1)(3

)

25

Redis - Back to get data from DB

Application server

Redis Database

(4)(2)

Client (1)(3

)

26

Redis - AP got data

Application server

Redis Database

(4)(5)(2)

Client (1)(3

)

27

Redis - Store data to Redis

Application server

Redis Database

(6) (4)(5)(2)

Client (1)(3

)

28

Redis - Return data to client

Application server

Redis Database

(6) (4)(5)(2)

Client (7)(1)

(3)

29

Hedis

30

Hedis

Application server

Hedis Database

Client

31

Hedis - Send request to AP

Application server

Hedis Database

Client(1)

32

Hedis - AP gets data from Hedis

Application server

Hedis Database

Client

(2)

(1)

33

Hedis - Turns to get from DB

Application server

Hedis Database

Client

(2)

(3)

(1)

34

Hedis - Store data to Hedis

Application server

Hedis Database

Client

(2)

(3)

(4)

(1)

35

Hedis - Return data to AP

Application server

Hedis Database

Client

(2)

(3)

(4)

(5)

(1)

36

Hedis - Return data to client

Application server

Hedis Database

Client

(2)

(3)

(4)

(5)

(1)

(6)

37

HedisHyper Redis

38

Architecture

39

Architecture

Hedis

40

Architecture

Redis

Hedis

41

Architecture

Redis Connector

Hedis

42

Architecture

Redis Connector

MySQLHedis

43

Architecture

Redis Connector

MySQL

HBase

Hedis

44

Architecture

Redis Connector

MySQL

HBase

others

Hedis

45

Architecture

Redis Connector

MySQL

HBase

others

Hedis

46

Architecture

Redis Connector

MySQL

HBase

others

Hedis

47

Architecture

Redis Connector

MySQL

HBase

others

Hedis

48

Architecture

Redis Connector

MySQL

HBase

others

Hedis

49

Architecture

● Core● Connector● Database

50

Architecture

● Core - Redis● Connector● Database

51

Architecture

● Core - Redis● Connector - connect Redis and database together● Database

52

Architecture

● Core - Redis● Connector - connect Redis and database together● Database - any database (includes RDBMS &

NoSQL)

53

Configuration

54

Configuration

55

How to start Hedis up ?

56

How to start Hedis up ?

57

Commands - only GET currently

58

Commands - only GET currently

● GET "cdh1://user@kewang"

59

Commands - only GET currently

● GET "cdh1://user@kewang"– No exists: turns to get "kewang" rowkey at "user" table

on "cdh1" connector

60

Commands - only GET currently

● GET "cdh1://user@kewang"– No exists: turns to get "kewang" rowkey at "user" table

on "cdh1" connector– Exists: gets directly

61

Commands - another GET

62

Commands - another GET

● GET "cdh1://!user@kewang"

63

Commands - another GET

● GET "cdh1://!user@kewang"

64

Commands - another GET

● GET "cdh1://!user@kewang"– Always gets "kewang" rowkey at "user" table on "cdh1"

connector directly

65

Commands - another GET

● GET "cdh1://!user@kewang"– Always gets "kewang" rowkey at "user" table on "cdh1"

connector directly– Use ruby's bang sign ( ! ) concept, it will modify original

value

66

Benchmark

67

Latency - Lower is better

68

Latency - Lower is better

69

Request - Higher is better

70

Request - Higher is better

71

TODO list

72

TODO list

73

TODO list

● Fix memory leak

74

TODO list

● Fix memory leak● Support multiple Redis version

75

TODO list

● Fix memory leak● Support multiple Redis version● Support more commands

76

TODO list

● Fix memory leak● Support multiple Redis version● Support more commands● Support more databases

77

TODO list

● Fix memory leak● Support multiple Redis version● Support more commands● Support more databases● Support git

78

TODO list

● Fix memory leak● Support multiple Redis version● Support more commands● Support more databases● Support git● Add dockerfile

79

TODO list

● Fix memory leak● Support multiple Redis version● Support more commands● Support more databases● Support git● Add dockerfile● Redesign official website

80

Live DEMO

81

References

● http://hedis.io● http://redis.io

82

WE'RE HIRING !!!Backend, System, Web, ... etc.

83