2008 WWFO PERFORMANCE - chiboug.org · 1 TPC ASE performance record ... • High query performance...
Transcript of 2008 WWFO PERFORMANCE - chiboug.org · 1 TPC ASE performance record ... • High query performance...
1 – Company Confidential – September 21, 2011
Smarter Analytics for Marketing Services
Providers
Introduction to Sybase IQ
Jason Lawrence –
Territory Account Manager
Doug Hoover – Principle Systems Consultant
Marketing Optimized:Intelligence to Action
Unleash Business Analytics Across the Enterprise with Sybase®
IQ
2 – Company Confidential – September 21, 2011
WHO IS SYBASE?Sybase delivers enterprise software to manage, analyze and mobilize information.
Sybase delivers mission‐critical
enterprise software to manage,
analyze and mobilize information.
Sybase is recognized globally as a
performance leader, proven in the
most data‐intensive industries and
across all systems, networks and
devices.
Over 27 years providing enterprise software ‐ 4000+ employees in 60 countries ‐ July 2010 – Became part of SAP
Excellent financial stability ‐
2009 Best year in company history ($1.132B)‐15% of Revenues are invested back into R&D
Market Leadership ‐
9 Gartner Magic Quadrants ‐
3 Forrester Waves‐ #1 IDC Mobile device mgmt‐
#1 LinuxWorld best database‐
#1 Best Wall St data solution‐
1 TPC ASE performance record‐
1 TPC SQL Anywhere price record‐
1
Guinness World Record
3 – Company Confidential – September 21, 20113
ONE SIZE DOES NOT FIT ALL IN THE DATABASE WORLD
Mobile and Embedded Analytics and ReportingOLTP and Decision Support
SQL Anywhere (Mobile Leader)
• Mobile database leader with 73% market share
• Mobile middleware market leader
• Leading embedded database vendor
• More than 20,000 corporate customers in
50 countries
• Over 80% of the Fortune 100
• Largest mobile application in the world with
more than 10 million users
Adaptive Server Enterprise –
Online Transaction Processing
• Worlds first programmable RDBMS
• More than 50% of worlds stock trades
run
on Sybase ASE
• More than 55% of Wall Street
• Awarded “Most Favorite”
Linux Database
by
Linux World Magazine
• Performance Leader
Sybase IQ – Column Based Analytics
• 10 to 100s times faster
than traditional RDBMS
• Data compression instead of data explosion
• Unique Architecture designed for analytics
• Over 2000 customers
world wide
• Easy
to implement and maintain
• Scalable
from gigabytes to petabytes and from
10s of users to 1000s of users
IQCPUCPUMemCPUCPUMem
IQCPUCPUCPUMemCPUMem
IQCPUCPUMemMemMemMem
IQCPUCPUCPUCPUMemMem
IQCPUMemCPUMemCPUMem
Fiber Channel
Dashboards
Database Management Requirements
Trading
Order Processing
Mobile WorkflowMobile SalesCustom Mobile App
Calendar
InBoxTimecardExpenses
4 – Company Confidential – September 21, 2011
Data ServicesData Services
Real-Time Report Server
Reporting /DashboardingReporting /
Dashboarding
ReplicationServer
Transactions Transactions Data Warehouse
SYBASE BI ARCHITECTURE & ECOSYSTEM
Mainframe
IBM DB2
Microsoft SQL Server
Oracle
Sybase ASE
Trading
Order Processing
Provisioning
Inventory
Financials
Enterprise Modeling
Analytics
PowerDesigner
5 – Company Confidential – September 21, 2011
SAP AND SYBASE ANALYTICS: DELIVERING A COMPLETE BI
STACK THAT INTEGRATES BOTH DATA PREPARATION AND DATA USAGE
Answers
SAP EIM SAP BI / Apps
IQ
Sybase IQ
Data Warehousing
Data preparation Data Usage
Data Modeling / Database Design Tool
Data Replication
Event Processing
Sybase IQColumn-based analytics server
IQ
Sybase PowerDesigner Leading Data Modeling Tool
Data Warehousing
Sybase Replication Server, Data movement & real-time synchronization
Sybase Aleri Streaming Platform (CEP), stream event data enabling Operational BI
+EIM
6 – Company Confidential – September 21, 2011
SYBASE IQ: THE MARKET LEADING COLUMN BASED ANALYTICS SERVER
What Is Sybase IQ• A highly optimized analytics server/engine
designed to
address challenging reporting and advanced analytics
infrastructure requirements
Business Challenges Addressed• System performance bottlenecks and unacceptable
delays in business analysis due to exploding data volumes
and increasing data complexity
Competitive Advantage• Speed and Flexibility
–
10x‐100x faster query
performance, Ad‐hoc queries, In‐DB Analytics
• Lower TCO
–
Compression, Data Partitioning, Commodity
Hardware, Better Price‐Performance
• Mature & Proven
– 1900 customers, 3300+ unique
deployments
Market Presence• Designated as a market leader in the data warehouse
space by Gartner and Forrester
–
BNP ParibasReporting (3x)
Transunion Analytics (6x)
Spice TelecomDisc savings (50%)
Wellmark(Health insurance)4x
5x
3x5x
All joint IQ
&
BOBJ customers
9 – Company Confidential – September 21, 2011
SYBASE IQ: DESIGNED FOR FAST ANALYTICS
Data is stored by columns•
Each column is stored separately–
Bit‐mapped index–
Index on every column–
Indexes are compressed•
Columns can be partitioned
Optimized storage •
Normal compression of 40%‐90%–
Database smaller than input data–
Includes indexes
Fast load and query•
Large 128K+ page sizes versus 2K
page‐size in traditional
•
Produces high performance even
with less expensive and larger
SATA disks
Engine only retrieves columnsused in the query•
Reduces system I/O dramatically–
Average 90% less than competition•
Permits better data manipulation–
Easy to alter and manage• VLDB Support
–
Partition data on any attribute
Unrestricted schema design•
Cost savings on warehouse design•
Can greatly reduce ETL workload
and latency
•
Design based on application use–
Flat, Star, Relational, Snowflake–
Any schema•
Also allows simplified, flattened
RCUBE‐type designs
10 – Company Confidential – September 21, 2011
THE SYBASE IQ DIFFERENCE:Column‐Based Architecture
Conventional Database
c 5
c 4
c 3
c 2
c 1 …c
9c 8
c 7
c 6
r1
r2
r3
r4
r5
SYBASE IQ
c 5
c 4
c 3
c 2
c 1 …c
9c 8
c 7
c 6
r1
r2
r3
r4
r5
Data is stored & retrieved vertically
Each column is stored separately – the data is the index
Retrieve only columns used in the specific query; no unnecessary processing
Dramatically reduce system I/O – dramatically increase query speed and accuracy
Data is stored & retrieved horizontally
Querying without indexes and views is extremely I/O intensive
Building indexes and views is a huge time and resource drain, and views/summaries do not give complete picture
Database footprint must be dramatically expanded to make the environment efficient for querying
11 – Company Confidential – September 21, 2011
SYBASE IQ: REDUCED DISK ‘HITS’= FASTER QUERY RESULTS
Example Query: How many MALES buy INSURANCE in OHIO?
800 Bytes/Row
Gender
M
M
F
M
M
‐
10M
ROWS
State
NY
TX
CT
OH
IL
VA
Row‐based
RDBMS
Insurance
Y
Y
N
Y
N
10M Rows x 800 Bytes4KB Page =
2,000,000 I/Os
Process large amounts of unused data
Often requires full table scan or space‐intensive indexes and materialized views
M Y VA
M N OH
F Y NY
M N OH
Gender Insured State
+ +1101
1101
10M
Bits
10M Bits x 3 column16KB Page
= 234 I/OsColumn‐based Sybase IQ
0101
10M
ROWS
12 – Company Confidential – September 21, 2011
EXAMPLE (ACTUAL CUSTOMER)
Sybase IQ improves query performance 50x over Oracle
Oracle Total:
519 minutes(31,196 seconds)
Sybase IQ Total:
10 minutes (603 seconds)
13 – Company Confidential – September 21, 2011
PROVEN DATA COMPRESSION SAVINGS WITH IQ
Sybase IQ Data Compression: saves money – saves time
Compression over Raw Data: 3.45x
Compression over Row‐Based (Oracle, DB2, etc.) Warehouses: 10x
14 – Company Confidential – September 21, 2011
LOAD
Conventional DBMS
Summaries Aggregates
1 – 2 TB
Indexes0.5 – 3 TB
Base table “RAW data” no indexes0.9 – 1.1 TB
2.4-6TB
Same INPUT data: “Conventional DW” is 3x-6x larger than
Sybase IQ DW
Base table: 0.2 - 0.5 TB
Indexes: 0.05 - 0.3 TB
Aggr/Summ: 0 - 0.1 TB0.25 - 0.9
TB
INPUT DATA:
1 TBSource: flat files,
ETL, replication, ODSLOAD
DATA COMPRESSION DRAMATICALLY REDUCES STORAGE AND
MAINTENANCE
15 – Company Confidential – September 21, 2011
MPP with Virtual Data Marts (VDM) — a deeper look
SYBASE IQ 15.3 PLEXQ®
SAN FabricSAN Fabric
Full Mesh High Speed Interconnect
VDM1
VDM2
Shared
Characteristics • VDM and works via login permission control
• VDM can isolate applications, workload, users
• DQP within VDM boundaries only
• Dynamic/scheduled resource reassignment b/w VDM
Characteristics • VDM and works via login permission control
• VDM can isolate applications, workload, users
• DQP within VDM boundaries only
• Dynamic/scheduled resource reassignment b/w VDM
Advantages• Effective user group assignment to resources
• Effective workload isolation and partitioning• Elastic, flexible resource provisioning/utilization• Economical scalability
• Ideal for MPP AND concurrent workload
Advantages• Effective user group assignment to resources
• Effective workload isolation and partitioning• Elastic, flexible resource provisioning/utilization• Economical scalability
• Ideal for MPP AND concurrent workload
16 – Company Confidential – September 21, 2011
Scenario 1Scenario 1
Application intent: delinquency trend balance by periodExecution characteristics: distributed scans, joins; near linear speed up ~35% per node
Beta program findings
DQP Response TimesDQP Response Times
Configuration: 8 machines with 8 cores eachConfiguration: 8 machines with 8 cores each
SYBASE IQ 15.3 PLEXQ®
18 – Company Confidential – September 21, 2011
SYBASE IQ: RECAP
How Sybase IQ leverages multiple approaches to scalability and performance
• Dynamic load balancing ability• Superior performance under heavy concurrent usage• Scale compute and storage separately
• Reasonable cost to scale out• Avoid being locked into a single vendor• Flexibility to adapt to technology trends
• Built from the ground up for analytic workloads• High query performance with minimal tuning• Efficient compression lowers storage requirement
Shared nothing MPP
Non‐proprietary hardware platform
Column‐based RDBMS
19 – Company Confidential – September 21, 2011
SAP AND SYBASEA complete BI analytics stack
Operational
Packaged Applications
SAP BusinessObjects EPM, SAP BusinessObjects BAS,SAP BusinessObjects GRC
Business Intelligence Tools
Web Intelligence, Crystal, Xcelsius, Explorer
SAP BusinessObjects
Business Intelligence
Platform
Event Processing Event InsightSybase Event Stream
Processing
Data Warehouse /
Analytics
SYBASE IQ / SAP HANA
Data IntegrationData Services(Integration &
Quality)
Master Data
Management
Information
Steward
20 – Company Confidential – September 21, 2011
IQ & SAP BUSINESSOBJECTS SYNERGIES
•Certified access with and without Semantic layer
•Faster query times• For ad‐hoc queries and analyses• For batch creation of reports (keeping time windows, SLAs, etc. ...)
•DBA support for the optimization of queries no longer necessary • Self‐service BI without IT bottleneck• More flexibility for query definition
•No aggregation tables, high data compression• More (detail) data available for analysis purposes
• Clear modeling
•More sophisticated queries and reports
In combination with SAP BusinessObjects front‐ends
21 – Company Confidential – September 21, 2011
SYBASE IQ –
SUMMARY
Sybase IQ is a proven, high‐performance, and scalable analytics server that can maximize
the performance of SAP BusinessObjects applications when analyzing large historical,
persistent data sets.
Optimize your SAP BusinessObjects
application performance and scalability with Sybase IQ.
LOBLOB ITITAUDIENCE
BUSINESS
NEEDS
KEY MESSAGE
• Just‐in‐time business decisions – near
real‐time, ad hoc analysis and data mining
• Predictive analytics (“windshield views”)
– enriched in‐database analytics
• Full text search and federated analytics• More analysis for more users in less time
– on‐demand, self‐service access
• Faster ROI –
leading best‐in‐class products
work together out‐of‐box
• Speed and flexibility – 10‐100x faster ad
hoc query performance
• Lowest TCO –
compression, data
partitioning, commodity hardware, price‐
performance, resource utilization
• Less burden on IT –
fewer resources,
faster implementations, less hardware
• Performance optimization – integration
across the BI stack
.• Mature and proven – 1,900 customers and 3,300+ unique deployments• Market Leader position in DW – Gartner Magic Quadrant and Forrester Wave
MARKET
PRESENCE
24 – Company Confidential – September 21, 2011
WHAT ANALYSTS SAY
Gartner
• “During the past few years, Sybase has shown increased ability to move from offering an analytic data mart to offering an enterprise data warehouse DBMS. “
• “The company has consistently won POCs with analytic applications, sometimes with performance 100 times greater than its competitors. “
• “We believe Sybase will achieve increased market share growth in 2011 and beyond. “
Gartner, Jan 2011 MQ for DW Database Mgt Systems
Forrester• “Sybase’s mature columnar EDW platform, coupled with a fresh injection of development capital from new parent SAP, gives it a competitive advantage.”
• “Expect SAP Sybase to leverage these strengths, plus intensify its focus on in-memory and in-database analytics, to grow its share of the EDW market among large enterprises requiring robust real-time analytics solutions.”
Forrester, Jan 2011 EDW Platform Wave
25 – Company Confidential – September 21, 2011
WHAT CUSTOMERS SAY
“We are able to deliver one data
warehouse for all of our applications at
one‐third the storage
of conventional
technologies, while seeing
performance gains as advertised.”
— Kim Ross
CIO, Nielsen Media Research
“We have seen remarkable
improvements
in productivity, and
big decreases in training, support,
and hardware costs."
— Jeff Butler
Director of Research Databases,
Office of Research, U.S. Internal
Revenue Service
“The IBM and Sybase IQ
combination offers unbeatable
cost/performance
and gives S&H a
competitive edge in retail
analytics."
— Steve Ritacco
CTO, S&H Solutions
"This is an across‐the‐board upgrade
that provides a whole new, entirely
different, experience for users
—
especially power users."
—Dan Feshbach
CEO, LoanPerformance
“The customers say it is incredible.
They are blown away
by what they
can do."
—Henri Asseily
Chief Technology Officer and Co‐
founder, Shopzilla
“This architecture is presented as a
model
for other divisions
and agencies for data
warehousing.."
— Sam Foster, President
FosterSoft, SEC contractor
“Our customers benefit directly as well. The speed improves our
ability to mine the data and produce results
for our customers
much more quickly. That helps them market more effectively and
generate more business.”
— Ric Elert
Vice President of Engineering, comScore
“Using Sybase IQ, we can deliver services we
never thought possible, and which will benefit our
public and private sector customers immeasurably
for years to come."
— Alison Mackenzie, manager of the HALOGEN
project, Mott Macdonald
27 – Company Confidential – September 21, 2011
JOINT SAP BOBJ AND SYBASE IQ CUSTOMERS
BNP was experiencing substantial growth in the use of their Web-based BOBJ reporting application and chose Sybase IQ to accelerate online transactions. RESULT: online customers generated twice as many reports 3x faster, freeing up OLTP resources.
Spice Telecom was able to populate a data warehouse with raw mobile and landline call data, analyzing data sets not previously possible, opening access to all departments. RESULT: achieved a 10x performance improvement with Sybase IQ.
REPORTING (3x) DISC SAVINGS (50%)TransUnion was able to process more than half a billion records each month with different data formats from different sources. RESULT: achieved enhanced data management, higher query speeds, lower costs, and a future-proofed system with Sybase IQ.
ANALYTICS (6x)
Sybase enables Business Objects users to dramatically improve their analytics
performance with higher query speeds and faster reporting.
Wellmark chose Sybase IQ as its analytics server to decrease reporting times and claims processing, as well as centralizing data silos at the enterprise level. RESULT: reduced software costs by 200- 300% and dramatically reduced people and hardware costs.
SOFTWARE SAVINGS (300%)
28 – Company Confidential – September 21, 2011
CHALLENGESpice Telecom, a large telecommunications company in India, needed to populate a data warehouse with raw mobile and landline call data.
SOLUTIONSybase IQ, ASE, Replication Server, PowerDesigner, SQL Anywhere
RESULTS10x improvement in performance Analyzes data sets not previously possible Decreases query times from hours and days to minutes Reduces storage needs by 50% Creates new ad hoc query possibilities Opens data warehouse access to all departments
CASE STUDY: SPICE TELECOM
29 – Company Confidential – September 21, 2011
CHALLENGEWellmark is a healthcare payer, processing millions of claims annually for its members, with increasing data volumes they needed to decrease reporting times and claims processing
SOLUTIONSybase IQ, PowerDesigner
RESULTSDB2 Required 3-4X more hardware5X less people to manageSoftware expenses reduced by 200-300%Hardware 5X less costly (compression) as compared to DB2. Centralizing existing data silos at the enterprise level, using PowerDesigner for
modeling into the data warehouse
CASE STUDY: WELLMARK
30 – Company Confidential – September 21, 2011
CHALLENGESourcing data from literally thousands of source systems at thousands of differentdata suppliers meant that TransUnion was processing more than half a billion records each month. Added to that was the complexity of the data, sourced in a variety of different formats, and containing distinctly different content such as financial, legal or public record information
SOLUTIONSybase IQ, PowerDesigner
RESULTS•Enhanced data management•Higher speeds•Future-proofed system•Lower costs
CASE STUDY: TRANSUNION
31 – Company Confidential – September 21, 2011
CHALLENGEBNP experienced substantial growth in the use of their web-based BusinessObjects reporting application for their investment customers wishing to monitor their portfolios, causing a dramatic drop in performance for report generation.
SOLUTIONSybase IQ, ASE, Replication Server, PowerDesigner
RESULTS•Allows online customers to generate twice as many ad hoc queries•Provides customized financial portfolio reports •Speeds report generation by 3X•Frees OLTP database resources to accelerate online transactions•Reduces stress on the system architecture by pushing rather than polling reports•BNP Paribas can now keep up to 10 years of data online for trend reporting and advanced analytics
CASE STUDY: BNP PARIBAS