Geode Meetup Apachecon

1© Copyright 2013 Pivotal. All rights reserved. 1© Copyright 2013 Pivotal. All rights reserved.

Open Source Core GemFire

Introducing Project Geode

2© Copyright 2013 Pivotal. All rights reserved.

Agenda Intro

– History, Use Cases, Customers, 2015 Roadmap– Architecture Overview

Why OSS, Why Apache

Southwest experience

Code walk thru/deep dive– Build/source code– PDX - Serialization

– Transactions– Persistence & GII

Demo


Geode Team Members in the roomName Title Years with Technology

Catherine Johnson Product Manager 16 years GemFire, Coherence

Anthony Baker Software Engineer 3 years GemFire

Roman Shaposhnik Director of OS Pivotal 3 years in memory grids

Greg Chase Director of Community 20 years Poet, SAP, GemFire

Dan Smith Software Engineer 7 years GemFire

Jens Deppe Software Engineer 4 years GemFire

Swapnil Bawaskar Software Engineer 7 years GemFire

William Markito Enterprise Architect 6 years GemFire, Coherence


2004 2008 2014

• Massive increase in data volumes

• Falling margins per transaction

• Increasing cost of IT maintenance

• Need for elasticity in systems

• Financial Services Providers (Every major wall steet bank)

• Department of Defense

• Real Time response needs• Time to market constraints • Need for flexible data

models across enterprise• Distributed development• Persistence + In-memory

• Global data visibility needs• Fast Ingest needs for data• Need to allow devices to

hook into enterprise data• Always on

• Largest travel Portal• Airlines• Trade clearing• Online gambling

• Largest Telcos• Large mfrers• Largest Payroll processor• Auto insurance giants• Largest rail systems on

earth

Hybrid Transactional/Analytics grids

Our GemFire Journey Over The Years


Big Data Apps at Scale Have Unique Needs

Project Geode is the distributed, NoSQL, in-memory database for big data apps that need:

1. Scale-out performance

2. Consistent database operations across nodes

3. High availability, resilience, and elasticity

4. Powerful developer features

5. Easy administration of distributed nodes


1. Scale-Out PerformanceChina Railway Corporation

“The system is operating with solid performance and uptime. Now, we have a reliable, economically sound production system that supports record volumes and has room to grow”

Dr. Jiansheng Zhu, Vice Director of China Academy of Railway Sciences

• 4.5 million ticket purchases & 20 million users per day.

• Spikes of 15,000 tickets sold per minute, 40,000 visits per second.

In-Memory Storage

Optimized datadistribution

Elastic, linear scalability

Nodes

Ops

/ S

ec


2. Consistent Database Operations Across Globally Distributed Nodes

Indexing, triggers, event notification

Performance-optimizedpersistence

Configurableconsistency Partitioned Replicated Disabled

Distributed queries& regional functions

“Our global deployment of Geode’s distributed cache gives me a single version of the trade – resolving hard-to-test-for synchronization issues that exist within any globally distributed business application architecture”

Michael Benillouche, Global Head of Data ManaGEOent


3. High Availability and Resilence

“We can track and collect money at our 4,000+ kiosks and branches – even without a reliable Internet connection. Geode provides the core data grid and a significant amount of related functionality to help us handle this unreliable network problem”

Gustavo Valdez, Chief of Architecture and Development

• 19 million payment transactions per month

• 4000+ points of sale with intermittent Internet connectivity

Cluster resilience& failover


4. Powerful Developer Features

Data Structures:– User-defined objects– Complex object graphs– Documents (JSON)

Schema versioning– Multiple application versions can run

simultaneously against same data nodes

API’s– Java: Hashmap– Spring Data GemFire– Serialization API’s

Minimal to no code changes:– Web app session state caching– L2 Hibernate– Memchaced

Powerful application functions: – Data-aware functions– Scatter-gather functions– Object Query Language (OQL)– Publish & subscribe & continuous query

event framework– Reliable asynchronous event queues


5. Easy Administration of Distributed Data Grids

Auto tuning of distributed computing resources to optimize performance

Cluster monitoring dashboard– Cluster and node status & performance

Offline performance statistics analysis tool– View historical logs and events to diagnose performance and resource bottlenecks

Command-line tools for easy automation and scripting of administrative tasks


Deployment Flexibility for In-Memory Apps

Embedded Embedded, Clustered Tiered, Clustered

WEB SERVER

WEB SERVER

WEB SERVER

WEB SERVER

GEOCLIENT

WEB SERVER

GEOCLIENT

WEB SERVER

GEOCLIENT

GEOSERVER

GEOSERVER

GEOSERVER

Flexibility Flexibility Scale

Flexibility Scale Performance

Flexibility Scale Performance Availability Localization

WEB SERVER

WEB SERVER

WEB SERVER

WEB SERVER

WEB SERVER

WEB SERVER

GEOPEER

GEOPEER

GEOPEER

WEB SERVER

WEB SERVER

GEOCACHE


Difference between Geode and GemFire

Native Clients beyond Java– C++– C#

WAN connectivity between clusters

Continuous Queries from clients


Geode High Level Architecture


• Scaled from 256 clients and 2 servers to 1280 clients and 10 servers

• Partitioned region with redundancy and 1K data size

Horizontal Scaling for Geode Reads with Consistent Latency and CPU


Basic Design patterns


“low touch” Usage Patterns

Simple template for TCServer, TC, App servers

Shared nothing persistence, Global session stateHTTP Session manaGEOent

Set Cache in hibernate.cfg.xml

Support for query and entity cachingHibernate L2 Cache plugin

Servers understand the memcached wire protocol

Use any memcached clientMemcached protocol

<bean id="cacheManager" class="org.springframework.data.Geode.support.GeodeCacheManager"Spring Cache Abstraction


As embedded, clustered Java database

• Just deploy a JAR or WAR into clustered App nodes

• Just like H2 or Derby except data can be sync’d with DB is partitioned or replicated across the cluster

• Low cost and easy to manage


As a scalable OLTP data store

• Shared nothing persistence to disk• Backup and recovery• No Database to configure and be throttled by


To process app behavior in parallel

Map-reduce but based on simpler RPC


“Write thru” Distributed caching

• Pre-load using DDLUtils• for queries

• Lazily load using “RowLoader” for PK queries

• Configure LRU eviction or expiry for large data

• “Write thru” – participate in container transaction


Distributed caching with Async writes to DB

• Buffer high write rate from DB• Writes can be enqueued in

memory redundantly on multiple nodes

• Or, also be persisted to disk on each node

• Batches can be conflated and written to DB

• Pattern for “high ingest” into Data Warehouse


Real-time AnalyticsData stored within Geode in a “sliding window”Geode map-reduce style in-memory analytics can be performed with data locality

Ex: Violation of known trading patterns

Benefit: Early-warning indicators can be identified faster than waiting for analysis on just Pivotal HDBenefit: Real-time analytics can better influence what kind of big data analytics need to be performed Pivotal HD

Geode

Micro-batches

Analysis Tools

SlidingWindow

Real time analytics

Alerts

influence


What’s Next


Geode Roadmap for 2015

HDFS Integration

Off Heap Storage

Spark Integration

Lucene Indexing

Distributed Transactions


Why OSS, Why Apache?


Why OSS? Why Now? Why Apache?

Open Source Software is fundamentally changing buying patterns– Developers have to endorse product selection (No longer CIO handshake)– Community endorsement is key to product visibility– Open source credentials attract the best developers– Vendor credibility directly tied to street credibility of product

Align with the tides of history– Customers increasingly asking to participate in product development– Resume driven development forces customers to consider OSS products– Allow product development to happen with full transparency

Apache is where you go to build Open Source street cred– Transparent, meritocracy which puts developers in charge– Roman keeps shouting “Apache!” every few hours


Geode Will Be A Significant Apache Project

Over a 1000 person years invested into cutting edge R&D

Thousands of production customers in very demanding verticals

Cutting edge use cases that have shaped product thinking

Tens of thousands of distributed , scaled up tests that can randomize every aspect of the product

A core technology team that has stayed together since founding

Performance differentiators that are baked into every aspect of the product

28Pivotal Confidential–Internal Use Only 28Pivotal Confidential–Internal Use Only

Transactions

Swapnil Bawaskar

29Pivotal Confidential–Internal Use Only

Geode Transactions Across multiple Entries and Regions

Full ACID

Isolation level: Repeatable Read

JTA– Last Resource– Provider

Optimistic, conflict detection rather than locks

Faster than doing individual operations

Ability to suspend and resume

Work on Colocated data


Usage

TransactionManager provides methods to begin, commit, rollback, suspend, resume.

E.g.– TransactionManager txMgr = cache.getTransactionManager();– txMgr.begin();– Region1.put(k1, v1)– Region2.get(k2)– Region2.put(k2, v2)– txMgr.commit();

Single entry operations supported via ConcurrentMap methods– putIfAbsent(K, V)– replace(K, V, V)– remove(K, V)


Implementation

Repeatable Read ThreadLocal

At commit()– Grab a d-lock on key set. (tx with different key set can still execute concurrently)– Conflict detection Reference checks– Send the commit set to all replicas – no ack– Send a commit message– Recipients apply the commit only on getting the second message and keep track of last few transactions

Failure Scenarios– Replica fails No problem, it will do a GII operation when it starts up again– Coordinator fails Replicas gossip to arrive at the outcome of the transaction– If no member has commit message, some members may be missing commit set, abort transaction– If at-least one member has commit message, all members have commit set, apply transaction

32Pivotal Confidential–Internal Use Only 32Pivotal Confidential–Internal Use Only

Thanks!


Geode Demo


Post RegionPartitioned

People RegionPartitioned

Social Network

Person

Name: StringDescription:String

Post

Id: PostId(name, date)Text: String


Partition put

Client

Server 1

Server 2

Server 3

Bucket 1

Bucket 1

Bucket 2

Bucket 2

#(LOL)=1Put LOL


Partition put

Client

Server 1

Server 2

Server 3

LOL

LOL

Bucket 2

Bucket 2

ReplicateTo Secondary


public interface PersonRepository extends CrudRepository<Person, String> {}

“User” Use Case – Save Objects

@AutowiredPersonRepository people;

public static void main(String[] args) { people.save(new Person(name)); posts.save(new Post(new PostId(name, date), text));}

Nested Objects,Compound Keys


public interface PersonRepository extends CrudRepository<Person, String> {}

“User” Use Case – Save Objects

@AutowiredPersonRepository people;

public static void main(String[] args) { people.save(new Person(name)); posts.save(new Post(new PostId(name, date), text));}

Automatically SerializedWith PDX


<bean id="pdxSerializer" class="com.gemstone.gemfire.pdx.ReflectionBasedAutoSerializer">

<constructor-arg value="io.pivotal.happysocial.model.*"/></bean>

<gfe:cache pdx-serializer-ref="pdxSerializer"/>

<gfe:partitioned-region id="people" copies="1"/>

Configuration


• Find all of the posts for a user• Analyze their content

Data Analyst – Determine Sentiment


public interface PostRepository extends GemfireRepository<Post, PostId> { @Query("select * from /posts where id.person=$1") public Collection<Post> findPosts(String personName);}

First try – Just use a Query

Collection<Post> posts = postRepository.findPosts(personName);String sentiment = sentimentAnalyzer.analyze(posts);


public interface PostRepository extends GemfireRepository<Post, PostId> { @Query("select * from /posts where id.person=$1") public Collection<Post> findPosts(String personName);}

First try – Just use a Query

Collection<Post> posts = postRepository.findPosts(personName);String sentiment = sentimentAnalyzer.analyze(posts);

Query Nested Objects


Use an Index<gfe:index id="postAuthor" expression="id.person" from="/posts"/>


Still could be more efficient

Client

Server 1

Server 2

Server 3

Joe: LOL!!

Joe: LOL!!

EJ: arrg

Maya: Hii

Jess: sup

Jess: ok

Hitting multipleNodes

Bringing too muchData to the client


Colocate the data

Client

Server 1

Server 2

Server 3

Joe: LOL!! Joe: LOL!!

EJ: arrgMaya: Hii

Jess: sup Jess: ok

<gfe:partitioned-region id="posts" copies="1" colocated-with="people”> <gfe:partition-resolver ref="partitionResolver"/></gfe:partitioned-region>


Send behavior to data

Client

Server 1

Server 2

Server 3

Joe: LOL!! Joe: LOL!!

EJ: arrgMaya: Hii

Jess: sup Jess: ok

Execution functiongetSentimentOn Joe, Jess

Execute on Joe

Execute on Jess


Sample Function – Client Side@Component@OnRegion(region = "posts")public interface FunctionClient { public List<SentimentResult> getSentiment(@Filter Set<String> people);}


Sample Function – Server Side

@GemfireFunction(HA=true) public SentimentResult getSentiment(Region<PostId, Post> localPosts, @Filter Set<String> personNames) throws Exception { String personName = personNames.iterator().next(); Collection<Post> posts = localPosts.query("id.person='" personName + "'"); String sentiment = sentimentAnalyzer.analyze(posts); return new SentimentResult(sentiment, personName);}


Demo


Highly Available Asynchronous Events

LOL!!

sup

LOL!! sup

put

LOL!! sup

Primary Queue

Secondary Queue

Enqueue


Colocated, Parallel Delivery

LOL!!

supLOL!! supput

LOL!!

supLOL!! sup

Primary Queue (Partition 1)

Secondary Queue(Partition 1)

Primary Queue (Partition 2)


Modify k1->v5

Create k6->v6

Create k1->v1

Create k2->v2

Modifyk1->v3

Create k4->v4

Modify k1->v5

Create k6->v6

Shared Nothing Persistence

Put k6->v6k6->v6 k6->v6

Operation Logswith compaction


GemFire (Geode) 3.5-4.5X Faster Than Cassandra for YCSB

Geode Meetup Apachecon

Technology

Transcript of Geode Meetup Apachecon