How can we take advantage of Cloud computing & big data?

HOW CAN WE TAKE ADVANTAGE OF CLOUD COMPUTING & BIG DATA?

Jordi TorresUPC-BSC, February - 2013

MOBILE BUSINESS & APPSDESIGN MASTER DEGREE

CONTENT

1. New Business models2. How do we get here?3. What is Cloud Computing?4. What is Big Data?5. An opportunity for you

YESTERDAY’S BUSINESSFo

to: D

esig

nedI

nBar

celo

na.c

om

TODAY’S BUSINESS

OR NOT?

IS MUSIC A BUSINESS?

BON JOVIEx

empl

ede

Jos

ep V

alor

(IE

SE)

, 20

11

The most downloadesThe best-selling

20101986

BON JOVIEx

empl

ede

Jos

ep V

alor

(IE

SE)

, 20

11

Sold: 28.000.000

BON JOVIEx

empl

ede

Jos

ep V

alor

(IE

SE)

, 20

11

Sold: 28.000.000 Sold: ?

BON JOVIEx

empl

ede

Jos

ep V

alor

(IE

SE)

, 20

11

Revenues: 30.000.000$Concerts tour 2010Revenues: 70.000.000$

BON JOVIEx

empl

ede

Jos

ep V

alor

(IE

SE)

, 20

11

Revenues: 30.000.000$Concerts tour 2010Revenues: 70.000.000$

The business model changed!

THE BUSINESS MODEL

from physical movement to the movement of information

Courtesy: D

ani Urgell –

CIS

CODesintermediation

music: Itunes, spotifybooks: Amazoneducation: e-learning UOC, MITtravels: eDreamsRetail: Zara. . .

The businessmodel changed!

WHERE DATA ARE STORED?

WHERE DATA ARE STORED?Sou

rce:

App

le.c

om.

201

1

INTO CLOUD COMPUTING!

CLOUD COMPUTING

It’s here to stay

CLOUD FORECAST

Source: http://softwarestrategiesblog.com/2012/01/17/roundup-of-cloud-computing-forecasts-and-market-estimates-2012/

Gartner is predicting that the IaaS market to grow by 47.8% through 2015.

IaaSAccording to IDC, by 2015, about 24% of all new business software purchases will be of service‐enabled software with SaaS delivery being 13.1% of worldwide software spending.

SaaS

CLOUD FORECAST

Workload processed in the Cloud

Source: CISCO

http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns1175/Cloud_Index_White_Paper.pdf

CONTENT


HOW DO WE GET HERE?

Pilot ACE, UK.1950

EVERY 15 YEARS …

1950 2012

That has redefined IT and industry in general

TIC

A major technological jump in IT

MAINFRAMES

1950 1980 201019951965 2025

Mainframes

PERSONAL COMPUTERS

1950 1980 201019951965 2025

Mainframes

Personal

Computers

INTERNET

1950 1980 201019951965 2025

Mainframes

Personal

Computers

Internet

INDUSTRIALIZATION OF IT

1950 1980 201019951965 2025

Mainframes

Personal

Computers

Internet

Huge Data Centers

HUGE DATA CENTERS

28.000 m2

Source: http://www.fa

cebo

ok.com

/med

ia/

set/?set=a.190

8426

2096

5185

.470

08.140

3752

8934

5252

40 Mw

HUGE DATA CENTERSFoto: G

oogle

HUGE DATA CENTERSFoto: G

oogle

> football pitch x 4

HUGE DATA CENTERSFo

to: c

orte

sía

BSC

Industrializaton of IT: has arrived!

Source: http://www.google.com/about/datacenters/gallery/images

CONTENT


IT AS A SERVICE

Cloud Computing represents a majorchange in IT servicearchitecture, deliveryand consumption

Foto

: J.T

.

IT AS A SERVICE

Amazon Data Center

Foto: Tutorial SC2011 - Robert Grossman

(*) IaaS

IT AS A SERVICE

More flexible IT, which can adapt to businesses and where costs can be predicted

On-demand self-service

Rapid elasticity

Ubiquitous access

. . . .

Pay per use

EXAMPLE: WORKLOAD

SYSTEM CAPACITY

Demand

Time

Res

ourc

es

Capacity

RESOURCES NOT USED

Demand

Time

Res

ourc

es

CapacityResourcesnot used

WASTE OF RESOURCES

Demand

Time

Res

ourc

es

Capacity€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€€ €€€€€€€€€€€€ €€€€€€€€€ €€€€€€€€€€ €€€€€€€ €€€€€€€€ €€€€€ €€€€€€ €€€€ €€€€ €€

€€€ €

CUSTOMER BEHAVIOR

Res

ourc

es

Demand

Capacity

Time (days)

1 2 3

CLOUD COMPUTING ELASTICITY

Demand

Capacity

Time

Res

ourc

es

AGILITY FOR SUPPLYINGvs in‐house data center

Foto

s: c

orte

sía

BSC

TYPES OF CLOUDS

Software as a Service (SaaS)

Plataforma as a Service (PaaS)

Infraestructura as a Service (IaaS)

TYPES OF CLOUDS

models of service delivery

Server

Apps

VM

Frameworks

PaaS SaaS

Server

Apps

VM

Frameworks

IaaS

Server

Apps

VM

Frameworks

Tradicional

Server

Apps

VM

Frameworks

INFRAESTRUCTURE AS A SERVICE

Idea : Tutorial SC2011 - Robert Grossman

1 computer in a rack for 120 hours 120 computers in three

racks for 1 hour

Ex IaaS: Amazon Web Services

To create an AWS account go to http://aws.amazon.comTo launch an instance go to

GET STARTED WITH EC2

Follow the Wizard

CONNECT TO INSTANCE

e.g. SSH

TERMINATE INSTANCE

As soon as your instance starts to boot, you're billed for each hour or partial hour that you keep the instance running (even if the instance is idle). When you've decided that you no longer need the instance, you can terminate it.

CLOUD COMPUTING INHIBITORS

Security Privacy Lack of Standards

Data Integrity Regulatory Data

Recovery

Control VendorMaturity ...

CLOUD COMPUTING TODAY

CLOUD

Enterprise resourcesData+ Programs + Hardware

Sep 2011User

electronic devices with identification/computing/storage/communication capabilities

CLOUD COMPUTING TODAY

PRIVATE CLOUDPUBLIC CLOUD

Enterprise resourcesData+ Programs + Hardware

User

electronic devices with identification/computing/storage/communication capabilities

HYBRID CLOUD

Workload

HYBRID CLOUD

capacity

Workload

HYBRID CLOUD

capacity

HYBRID CLOUD

External Provider

EVERYTHING AS A SERVICE

IaaSPaaSSaaS

XaaSDRaaS

SUaaSBDaaS…

CLOUD CAN DRIVE BUSINESS INNOVATIONFo

to: D

esig

nedI

nBar

celo

na.c

om

Taking cost out of company Improve business processes and operational effectiveness Using the cloud to enable new business models

SUMMARY

Sou

rce:

Sec

urity

Gui

danc

e fo

r Critic

al A

reas

of

Focu

s in

Clo

ud

Com

putin

g ,

Apr

il 20

09.

Clo

ud S

ecur

ity A

llian

ce.

Hundreds ofcompanieson stage

CONTENT


EVERYBODY CONNECTED

4th year ESO secondary school:100% of students on Facbook

http

://w

ww.

sear

chen

gine

jour

nal.c

om/th

e-gr

owth

-of-s

ocia

l-med

ia-a

n-in

fogr

aphi

c/32

788

A MOBILE FUTURE

http://money.cnn.com/2011/02/09/technology/smartphones_eclipse_pcs/index.htm

INTERNET OF THINGS

Not only will we have mobile devices, but also

http://www.urbiotica.com

INTERNET OF THINGShttp://dow

nload.intel.com/new

sroom/kits/em

bedded/pdfs/ECG

_WhitePaper.pdf

OPEN DATA

BIG DATA?Do you need a definition? is data that becomes large

enough that it cannot be processed using conventional methods.

enough for you? :-)

Deluge of data created dailysocial networks, mobile phones, sensors, science, …

Source:http://www.datacenterknowledge.com/archives/2011/06/28/digital-universe-to-add-1-8-zettabytes-in-2011/?utm-source=feedburner&utm-medium=feed&utm-campaign=Feed:+DataCenterKnowledge+%28Data

“CHULETA”

1 Gigabyte (GB) = 1.000.000.000 byte1 Terabyte (TB) = 1.000 Gigabyte (GB)1 Petabyte (PB) = 1.000.000 Gigabyte (GB)1 Exabyte (EB) = 1.000.000.000 Gigabyte (GB)1 Zettabyte (ZB) = 1.000.000.000.000 (GB)

DEFINING BIG DATA

A more complete definition of Big Data is one that combines three attributes of data:

Source: IBM Book –

Understanding

Big Data

DEFINING BIG DATA

Volume: Large Volumes of data Terabytes, Petabytes, … Data that cannot be stored in conventional RDBMS

Variety Source data is diverse – Web Logs, Application Logs,

Machine generated data, Social network data, etc. Doesn't fall into neat relational structures – Unstructured,

Semi-structured Velocity Streaming data, Complex Event Processing data Velocity of incoming data and Speed of responding to it

WHY IS BIG DATA IMPORTANT

60%Potential increase in retailers

operating margins possible with big data

(*) Source: Big Data: The next frontier for innovation, competition and productivity – Mckinsey Global Institute, July 2011

40% projected growth in global data generated per year vs

5% projected growth in global IT spending

WHY IS BIG DATA IMPORTANT

Data is more important than ever, but the exponential growth of data has overwhelmed most company's ability to manage (and monetize it).

Big Data Challenges Processing such large data sets Storing and Managing Analyzing …

assume 100MB/sec

Storage affordable

assume 100MB/sec

But scanning disks…

assume 100MB/sec more than 5 hours

But scanning disks…

assume 20.000 disks: scanning 2 TB takes 1 second

Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg

SOLUTION: MASSIVE PARALLELISM

assume 20.000 disks: scanning 2 TB takes 1 second

Source: http://www.google.com/about/datacenters/gallery/images/_2000/IDI_018.jpg

Rethinking data processing is required: MapReduce, …

DATA PROCESSING CHALLENGES

DATA STORAGE CHALLENGES

New Storage technologies are required

RAM vs HHD

Present solutions:

Research:

HHD 100 cheaper than RAMBut 1000 times slower

Solid- state drive (SSD)Not volatile

80

Storage Class Memory (SCM)

Source: gigaom.com/cloud/big‐data‐and‐nosql‐march‐to‐the‐enterprise/

Relational DB can’t support everything:

NoSQL, …

DATA MANAGEMENT CHALLENGES

Solution: the so called “NoSQL systems”

BASE property is sometimes attributed to them(Basically Available, Soft state, Eventual consistency)

Example eventual consistency: Facebook!

DATA MANAGEMENT

82

Data deluge, is it enough?

“Data = Information”?

prediction using data models

The information is non actionable

knowledge

OBTAINING VALUE FROM DATA

86

4

Knowledge

Information

Data

+Volum

e

-+

-

Valu

e

For this we have Data Miningtechniques

• Association• Classification• Clustering• Prediction• ...

The majority of algorithms functionwell in thousands of registers, however at the moment they are impractical for thousands of milions.

CONTENT


AN OPPORTUNITY

AN OPPORTUNITY

1950 1980 201019951965 2025

Ap

plica

tio

ns

Thousandsof applications

Tens of thousandsof applications

Hundredsof applications

Hundreds of thousands

of applications

IT applications and services over the years

AN OPPORTUNITY

1950 1980 201019951965 2025

Ap

plica

tio

ns

Thousandsof applications

Tens of thousandsof applications

Hundredsof applications

Hundreds of thousands

of applications

we'll see millions of new applications and services built on top of these disruptive technologies

Millions ofapplications

AN OPPORTUNITY

1950 1980 201019951965

Millionsof users

Hundreds of Millionsof users

?

Thousandsof users

2025

Use

rs IT users growth

AN OPPORTUNITY

1950 1980 201019951965

Millionsof users

Hundreds of Millionsof users

Billions of users

Trillions of “things”

Thousandsof users

2025

Use

rs IT users growth

thanks to these millions of applications we'll see billions of users and trillions of “things” connected

BIG DATA TALENT

CONCLUSION

We have toprepare our businesses for

the explosion of the cloudin a world of mobile technology

and social networks, dominated by the

boom of available information and all in a sustainable way.

www.smartcityexpo.com www.bsc.es/eBusiness

Autonomic Systems and e‐Business Platforms research line at BSC/UPC

Thank you for yourattention

Questions?

www.JordiTorres.eu

@JordiTorresBCNBarcelona13-15 November 2012

www.smartcityexpo.com

How can we take advantage of Cloud computing & big data?

Technology

Transcript of How can we take advantage of Cloud computing & big data?