Post on 24-Jan-2015
description
1
Sybase RAP&
Sybase CEP
Sybase
RAP & CEP 卢东明
技术总监, Sybase China
2
Sybase RAP&
Sybase CEP议题
▌ 金融行业面临的挑战
▌ 业务的响应
▌ 技术的解决方案 RAP
CEP
▌ 金融行业以外的应用
3
Sybase RAP&
Sybase CEP
Market Dynamics Create Extreme Pressures
Volumes, velocity, complexity of data are stressing systems
4
Sybase RAP&
Sybase CEPIt is all about Speed!
Near- or sub-millisecond latency times are needed to stay competitive in the trading markets, says Low*, “but at the same time, you’re expected to be rock-solid, bulletproof, and never fail as a system and never lose data.”
best-of-breed solution was able to support latency of
less than 1 millisecond (987 microseconds) at
6,000 orders per second, with full data replication
February 4, 2008
* Gideon Low, principal architect for business development and alliances at GemStone
5
Sybase RAP&
Sybase CEPNew Regulations
▌MiFID (Europe) and RegNMS (U.S.) Best execution
Must be able to prove that the trading decision was in the best interest of the customer
Need to be able to reconstitute a trade that is 5+ years old
▌Know Your Customer/Counterparty, etc Liable for business you conduct with nefarious people
Even legitimate business is not allowed
6
Sybase RAP&
Sybase CEP当前体系结构的挑战
MARKET DATA
Traders, Automated Trading Apps
Management & Compliance
Reporting
CORPORATE DATA
HISTORY (time series)RESEARCH (scenarios, post-trade analysis, etc)
Quants, Model Development
TRADE & RISK DATA
P&L Monitoring, Risk Manager, Others
问题 : 专门的系统产生数据孤岛,信息孤岛,增加系统延时
7
Sybase RAP&
Sybase CEP议题
▌ 金融行业面临的挑战
▌ 业务的响应
▌ 技术的解决方案 RAP
CEP
▌ 金融行业以外的应用
8
Sybase RAP&
Sybase CEPNew York Stock Exchange – Then
9
Sybase RAP&
Sybase CEPNew York Stock Exchange – Now
10
Sybase RAP&
Sybase CEP纽约最有价值的“地产”
▌The first thing Wall Street executives want to know when they tour the New York Stock Exchange's data centers to check on the servers they've co-located there is, who is hosted in the other cabinets and why aren't their own cabinets closer to the NYSE's servers, reports Steve Rubinow, the exchange's CTO.
January 22, 2008
11
Sybase RAP&
Sybase CEP创新者!
▌A challenge can also be an opportunity
▌A small hedge fund far from NY and London has figured out how to beat the markets
▌“Day trading” on steroids!
12
Sybase RAP&
Sybase CEP对冲基金策略
13
Sybase RAP&
Sybase CEP对冲基金策略
14
Sybase RAP&
Sybase CEP电子交易 vs. 传统交易
15
Sybase RAP&
Sybase CEP交易案例
16
Sybase RAP&
Sybase CEP议题
▌ 金融行业面临的挑战
▌ 业务的响应
▌ 技术的解决方案 RAP
CEP
▌ 金融行业以外的应用
17
Sybase RAP&
Sybase CEP关系型数据库 vs 事件处理模型
Stores data before processing
Optimized for business data processing
PollingQueries
Memory
Disk
UpdatesProcessing
Alerts Actions
Alerts Actions
CEP
Memory
Disk
Grabs, analyzes data as it flies by Different way of thinking:
Send data to the queries
Load the needles, not the haystack
Advantage: Minimal latency No waiting Results delivered in-flight
Updates
18
Sybase RAP&
Sybase CEP复杂事件处理 CEP
Human speed
(seconds to minutes)
Machine speed
(msec)
Simple
events
Complex
events
事件的复杂度
处理速度
Traditional Business
Intelligence
RDBMS
Complex Event
Processing
Messaging & RoutingSystems
20
Sybase RAP&
Sybase CEP最大的风险是“延迟”
More TIMEMore TIMEIt takes to know It takes to know
your position your position
More RISKMore RISK
Counterparty Risk?Liquidity Risk?
Enterprise Risk?
从事件发生到响应之间的延迟是你最大的风险
RiskTime
21
Sybase RAP&
Sybase CEP波动 = 机会
▌You must Separate opportunities from background noise
Measure opportunity – Risk vs. Reward
Act on opportunity
Act FasterAct Faster More More OpportunityOpportunity
RewardTime
22
Sybase RAP&
Sybase CEP
Need for complex event processing engine▌ Continuous analysis of high-speed data streams, prior to
storage▌ High-throughput ▌ Low-latency – sub-millisecond▌ Rapid development and deployment▌ Integration with ecosystem
Challenges in developing analytics apps on streaming data• Custom developed apps
• Time consuming • Expensive• Cannot be changed quickly
Solution
Complex event definitionA complex event is the discovery of a fact derived by correlating multiple events and applying rules to that set of events.
* IDC report #215785, December 2008
Financial Market DataTrading Systems
Risk SystemsApplication Data
DatabasesMessage Bus
Historical & Reference Data
Input Data Streams
CEPEngine
Output Events
Applications
Message Bus
Visualization Tools & Alerts
Databases
对 CEP 的需求
23
Sybase RAP&
Sybase CEP
Capital Markets Usage
Trading• Algorithmic trading• Real-time position and portfolio analysis• Smart order routing
Risk management• Real-time risk analysis• Trader compliance monitoring• Monitor activities of exchange, or Dark
Pool members for compliance• Real time P&L analysis
Market data support and pre-trade analytics• Data latency and data quality monitoring• Data cleansing• Computation of analytics for pre-trade
use
TechnologyRequirements
High speed data capture
Real time analytics and aggregation
Computations on event windows
Connectivity to multiple systems
Rapid development and deployment
Availability, reliability, scalability
资本市场上 CEP 的使用案例
24
Sybase RAP&
Sybase CEPCEP 体系结构
InternalOrder Book
Post Trade Analytics
Mar
ket D
ata
Mid
dlew
are
NYSE
AMEX
London Stock Exchange
…
Stock Exchanges
Database
Sybase CEP
Visualization Tools
Risk Management Systems
Order Management Systems
25
Sybase RAP&
Sybase CEPSYBASE CEP SERVER ARCHITECTURE
Input Streams
Sybase CEP Engine
Sybase CEP Studio & SDKs
External Databases & Applications
In-process Adapters
Out-of-process
Adapters
In-process Adapters
OutputStreams
Real-time Analytics
In-Memory Cache
Clustering & HA
Event-Driven Reads/Writes
SQL-like Language
Event Replay
Sybase RAP
Out-of-process
Adapters
26
Sybase RAP&
Sybase CEP
“CONTINUOUS COMPUTATION LANGUAGE – CCL”
▌ Sybase CEP is programmed using CCL, which is very “SQL Like” but also rich with intuitive streams processing extensions
▌ CCL queries are executed continuously▌ CCL language has support for stream processing, such as Windows,
which is key for analytics on event streams▌ Database SQL queries can be executed as part of the CCL language,
and joins between elements in the database and the data stream in CEP can be done
Example CCL Query
Calculation of 10 minute VWAPS
INSERT INTO VWAPEventStreamSELECT StockTickerSymbol, SUM(StockPrice*StockVolume)/SUM(StockVolume)FROM TradesEventStream KEEP 10 MINUTES GROUP BY StockTickerSymbolOUTPUT EVERY 10 MINUTES
27
Sybase RAP&
Sybase CEPSybase RAP – The Trading Edition
A powerful, flexible market analytics platform for capital markets
Enables better trading and portfolio decisions with less risk through timelier, more comprehensive market insight.
Shared access to consolidated data across the trade lifecycle:
High-performance quantitative analytics Real-time trade analytics Intra-day risk analytics Regulation-driven analytics
28
Sybase RAP&
Sybase CEP
Sybase RAP – The Trading Edition supports different trade lifecycle analytics requirements by layering various technologies to optimize performance as needed according to speed, computational complexity, or types of data.
Sybase RAP – The Trading Edition A Complete Platform for Optimized Performance
29
Sybase RAP&
Sybase CEPSYBASE RAP & CEP OPTION
完整的 RAP 平台 – 单一厂商Sybase’s CEP option enables you to quickly deploy live analytic applications that
provide up-to-the-moment intelligence on market and trade data
交易及算法支持Pre-trade analytics
Construct, test and implement real-time models and strategies for traders
Comprehensive views into years of granular market, trade and statistical data
Rapid deployment of strategies and models into a scalable, real-time infrastructure
风险分析Continuous computation of profit and loss on a
trade-by-trade basis, aggregating this information across multiple portfolios
Real-time calculation of risk at granular and summary levels
Delivery of risk information to the right people at the right time in the right format
业务定位
30
Sybase RAP&
Sybase CEP“ 速度”及“深度”
▌Speed – Know there is a material event happening
▌Depth – Know the risk/reward
▌Sybase CEP = Speed Event
▌Sybase RAP = Speed Analysis + Depth
Sybase RAP+ Sybase CEP Option
Speed Event + Speed Analysis + Depth
31
Sybase RAP&
Sybase CEP议题
▌ 金融行业面临的挑战
▌ 业务的响应
▌ 技术的解决方案 RAP
CEP
▌ 金融行业以外的应用
32
Sybase RAP&
Sybase CEPOpportunities Beyond Financial Services
▌电信 预付费电话授权,欺诈分析,网络监控,入侵检测 / 防
范,会话管理▌政府,安全及监控
情报监控,战地协调,国土安全▌实时供应链 / 物流 / 制造业
库存准确性,送货确认,质量跟踪
33
Sybase RAP&
Sybase CEP金融行业以外的应用
▌交通运输业 风险控制,调度,派送,路线规划
▌互联网 /Web 点击流分析,网络游戏,入侵监控 / 防范
▌RFID 库存管理,集装箱跟踪, ID 牌及安全控制,
34
Sybase RAP&
Sybase CEP金融行业以外的应用
▌零售业 产品关联分析, CRM, 库存管理,订单管理,脱
销, POS
▌媒体和娱乐业 内容及服务的动态包装,版权付费
▌医疗卫生 / 保险 资产管理,理赔操作,账单管理,欺诈分析
35
Sybase RAP&
Sybase CEP总结
▌Volume and Velocity are Overwhelming Financial Services
▌Challenges & Opportunities Compliance with RegNMS, MiFID
New trading strategies
▌New technology is changing the markets Market changes demand new technology, then the new
technology changes the markets
▌Coming to an IT Department Near You – SOON!
36
Sybase RAP&
Sybase CEP
www.sybase.com/rap