The Potential of Big Data in the Cloud - README | SK플래닛...

The Potential of Big Data in the Cloud Juan Madera

Technology Consultant

[email protected]

Copyright © 2012 Accenture All rights reserved. Copyright © 2012 Accenture All rights reserved.

• How to apply Big Data & Analytics

• What is it? Definitions, Technology and Data Science

• The Big Data Market inside and outside the cloud

• Some use cases

2

Agenda


Resistance is futile Competitive advantage No one size fits all It’s different

Top 4 things about Big Data and Analytics

3


Source: An IDC White Paper - sponsored by EMC. As the Economy Contracts, the Digital Universe Expands. May 2009.

.

Complex, Unstructured

Relational

New kinds of data

Structured data vs. Unstructured data growth

Our ability

to analyze

Analysis

gap

4


Big Data Technologies

New technologies, new approaches

Source: Wordle for Credit Suisse, Does Size Matter Only?, September 2011

5

An Illustrative Customer Experience: We Detect a Customer’s Promotion

6

Existing Customer with a Current Account, Bank Detects Financial Improvement, Suggests Options (Customer Retention Scenario)

Very simple low-pass filter on transaction record

Comparisons made between Jane’s historical spending vs saving behaviour and those

of other customers

Jane has recently been promoted. An alert is triggered that her direct deposit amounts have jumped this month.

Financial recommendation system settles on advice to propose to Jane based on successful peers experiencing a similar trend.

• Improved Awareness of Customer: • Behavioural data captured

and stored for future use • Enhance segmentation and

enabling targeted offerings

• Improved Ability to Correlate Customers: • Allow for better targeting • Develop more agile

response capability

Social activity trends logged, fed back into a validation and improvement loop

Communications logged, retained for analysis,

incremental improvements

• Sentiment analysis: • Identify customer

perception about brand • Improve segmentation • Help with personalised

and targeted offerings

Bank engages Jane via web, SMS, and/or phone call to present suggestions and guidance, e.g., upgrading to a premium account.

Cu

sto

me

r Jo

urn

ey

Dat

a In

sigh

t B

usi

ne

ss

Val

ue

Jane enjoys better control and more financial security, broadcasts this success explicitly and implicitly.

Opportunity Detection

• Increased Customer Engagement: • An opportunity to improve

the relationship between the bank and its customer

Correlation and Prediction Proposition Reduced Churn

Web site screen shot

An Illustrative Customer Experience: Location-based Mobile Shopping Recommendations

7

Existing Customer with the Bank’s Mobile App Installed on his Mobile Device (Mobile Recommendations Scenario)

App sends home location of customer

Further calculations possible to compare customers on the

basis of daily routines

John is moving through town on foot, on transit, or in his car.

• Improved Data Quality: • Behavioural data captured

and stored for future use • Can be further analysed and

used to develop further offerings

• Improved Customer Insight: • Fuller understanding of

customer behaviour

Further analysis possible to improve targeting and

engagement

Records kept of which notifications result in

behavior and under what circumstances

• Improved brand perception: • Positive customer

experience of bank in the mobile space

• Cutting-edge tools

Mobile app raises a notification to John, and John tries out a new shop.

Cu

sto

me

r Jo

urn

ey

Dat

a In

sigh

t B

usi

ne

ss

Val

ue

• Improved Customer Insight: • More detailed analysis of

what drives customers financially and socially

John comes within a physical threshold of a shop where similar customers tend to shop but he does not.

Location Observation Proposition Correlation Reduced Churn

John finds mobile app useful and as a result has increased engagement with other offerings of the bank.

Bank storefront

An Illustrative Customer Experience: Suggesting Mortgage and Savings Plans for Newly Engaged Customers

8

Existing Customer with a Current Account, Bank Infers Future Marriage, Suggests Options (Mortgage and Savings Plan Scenario)

Comparing user behavior against historical library of spending behaviors of

all users

Outlier spending detected quickly and rules of

engagement applied automatically

Jim has been dating Julie. His spending habits have trended away from his usual nights out with friends, toward more romantic, pricier restaurants.

User-sim system recognizes this trend, and when Jim makes an extraordinarily large purchase at a local jeweler an alert is raised.

• Improved Awareness of Customer: • Behavioural data captured

and stored for future use • Enhance segmentation and

enabling targeted offerings

• Improved Ability to Flag Outlier Behaviour: • Possible to react quickly to

changing conditions and target more effectively

Social activity trends logged, fed back into a validation and improvement loop

Analysis used to predict customer’s future needs and

target appropriate offers

• Increased Customer Loyalty: • Long-term customers

provide the bank with even more opportunity to make smart suggestions

Analysis suggests that users with similar behaviour to Jim are likely to buy a house within 6 months. Jim currently does not have enough savings for a deposit so the bank emails a savings plan offer tailored to Jim’s needs.

Cu

sto

me

r Jo

urn

ey

Dat

a In

sigh

t B

usi

ne

ss

Val

ue

Jim enjoys an increased feeling of security as a customer of the bank, given their inclination to suggest ways he can save for his future.

Opportunity Detection

• Increased Cross Sell and Up Sell: • An opportunity to increase

cross sell and up sell rates to existing customers based on detailed analysis

Correlation and Prediction Proposition Increased Loyalty

Bank web site

Opportunity Areas

9

Sell more to existing

customers

Sell more to new customers

Retain more customers

Reduce risk exposure

Reduce cost to sell

Reduce cost to serve

• Proactively contact customers based on behavioural triggers and key life stages

• Improve action prompts based on social insight

• Provide personalised pricing based on recent circumstances and predicted changes

• Convert more leads into sales by using social data indicators during interactions

• Improve measurement and monitoring of cancellation propensity

• Proactively target customers with high risk of churn with specific high value services • Send pre-delinquency customer

messages • Add an additional layer ( of predicted

circumstances) in approval process of financial aid requests

• Pre-assess customers reducing invitations to non-eligible or bad debt customers

• Improve Forecast and planning process based on insight

• Proactively inform customers about service issues and next steps

• Include and generate relevant service prompts

• Use innovative technologies to store/retrieve data

Big Data


Big Data Analytics is a shift in the mindset of how we think about analytics as an internal component to the organization

Focuses on letting data be productized in a way that drives meaningful insights in a rapid fashion and innovation to exploit missed opportunities in areas previously unlooked

Big Data Analytics

What is it?

10


Everything will be analyzed

The three Vs

Structured Unstructured

Batch

Real-time

Velocity

Variety Source: IDC

Distributed,

ETL

Relational,

ETL

In-memory,

NoSQL, Event processing,

EDW

Event

processing, Distributed+

NoSQL

Volume

11


Big Data Analytics vs. traditional analytics

Where do they differ?

Technology Skills Processes & Organization

Big

Data

An

aly

tics

Tra

dit

ion

al

An

aly

tics

Assumes condensed,

structured, and feature rich datasets that can be modeled: relational

databases, data warehouses, dashboards

Basic knowledge of

reporting and analysis tools, few specialized resources

“Siloed” data

organizations Only specific “views” of

data visible across the enterprise

A stack of tools that

enables an organization to build a framework that allows them to extract

useful features from a large dataset to further

understand how to model their data.

Advanced analytical,

mathematical and statistical knowledge required to develop new

models – the data scientist

Data is productized and

shared across the enterprise

Dedicated data organizations with well-

defined data management processes and ownership

12


MapReduce and Hadoop

MapReduce revolutionized how we handle large amounts of data, Hadoop made it simple and affordable

• Originally designed and first developed in

Google as part of their efforts to more

efficiently index the web

• MapReduce splits input data into smaller

chunk that can be processed in parallel • Scales linearly with number of nodes

• Yahoo’s implementation of MapReduce

• Open source, top-level project in the

Apache Foundation

• Designed to run on commodity software

(Linux) and hardware (consumer-grade computers with directly attached storage)

• Large ecosystem of additional

components (both open source and

commercial) 13

Copyright © 2012 Accenture All rights reserved.

Analytics-Focused Massively Parallel Processing

(MPP) Software Platforms

Distributed In-memory

Big Data and Analytics in the Enterprise

Many technology choices in a rapidly changing environment. Which one is right for you?

Cloud

Hardware Optimized MPP Data Warehouses

Distributed Non-Relational Storage and Processing

Big Data-Enabled Intelligence and Analysis

14


Technology

Augmenting existing analytics with Big Data technologies

Emerging Data

Technologies

Existing Analytics Tools and

Investments

Big Data Analytics

15


It’s not just Hadoop

What are traditional analytics vendors doing about it?

Distributed In-memory

16


The impact of Big Data Analytics on our landscapes

Hybrid landscapes, where old and new converge

ERP CRM Web Logs Time

Series Files Social

Relational DBs

Enterprise DW

Real-time analytics

HDFS

HBase

MapReduce

Hive

Data Services (REST, WS)

Pig

ETL

Internal apps,

customer-facing

apps, mobile

apps Analysis tools (SAS, SPSS, R,

Tableau)

17


Data science

“The sexy job in the next 10 years will be statisticians”

– Hal Varian, Chief Economist at Google

Data scientists are the next-generation analytics professional, responsible for turning the data into insight

Data Science and the skill gap

Closing the loop – it’s not just about technology skills

18


Some examples

“Cool” Cloud Vendors of Big Data Analytics

Cloud Analytics reference models for Asset Management,

Banking, HighTech, Insurance and Retail

their business analytics platform is used by leading corporations

in many industries, including automotive, commercial real

estate, restaurants and entertainment, fast moving

consumer goods, retail franchising,

and telecommunications.

They leverage Force.com platform as a service as well as

traditional big data toolset to develop Geographical Intelligence

for sales reps.

They develope software for BI SaaS potential service

providers, both private or public.

19


Business challenge

• Database growth at 2 TB per month

• Traffic and Data size double every 6

months

• Total storage required reach 2

Petabytes in 2015

• Poor Oracle performance, very costly to

scale

• Siloed database systems

• Proliferation of home-grown tools

• Decentralized business rules and reporting data

Solving real problems with Big Data Analytics

Case study 1: Large storage systems vendor

Technologies used

• Processing – Hadoop, Hive, Pig, HBase

• Log processing – Flume

• Monitoring – Ganglia

• Business Intelligence – Pentaho

Delivered Results

• Highly scalable data processing platform

• Centralized data storage

• Cluster utilized by all teams and groups

• Increased efficiency of data consumption

• Foundation for BDaaS offering

20


Technologies • Processing – Hadoop, Hive

• Log archiving – Flume • Data retrieval – CouchDb

Delivered Results • Highly scalable data platform

• Various data mining and machine

learning algorithms

• Centralized data storage

• Cluster utilized by all teams and groups • Increased efficiency of data

consumption

• Innovation across all teams

• Established Central Analytics team and

private cloud

Business challenge

• Enormous amount of Customer,

Transaction and Click-through data.

• Inability of existing Relational stores to

power the various batch queries and

computations.

• Data residing in different stores spread

across the company


Case 2: Global retailer

21


Business Challenge

• Lack of agility in data processing and

analysis

• Business and Data Analysts forced to

wait inordinate amount of time to

explore the data

• Difficulty in ingesting new sources of

data without exhaustive ETL

processes

• Inability to apply advanced analytic

and statistical functions to a large data

set


Case 3: Large insurance company

Technologies used

• Processing – Hadoop, Hive, Pig,

• Analytics – Greenplum, R, Madlib

• Visualization – Tableau, Karmasphere, Alpine

Miner

Delivered Results

• Agile BI platform

• Multiple options for data ingestion and

processing for different business scenarios

• Hadoop as an economical platform for data

processing and Greenplum to ease, expedite

and enhance the data processing

22


Wrapping up

Big Data is challenging current patterns of thought

Cost-effective

computing and

storage

Data

“explosion”

Everything can be

stored

Cheap large scale

computing power

readily available

Data everywhere:

structured,

unstructured,

other people’s

data, geolocation

data

Big Data and Analytics

Resistance is futile

Are the path to competitive advantage and create value

There are many ways to go about

it Compared to traditional analytics,

they’re different; adapt or become irrelevant

23

Copyright © 2012 Accenture All rights reserved. Copyright © 2012 Accenture All rights reserved. 24

Accenture Technology Vision

http://bit.ly/accenturetechnologyvision2012

Strong advice on data for 2012



The Potential of Big Data in the Cloud - README | SK플래닛...

Documents

Transcript of The Potential of Big Data in the Cloud - README | SK플래닛...