浅谈电商网站数据访问层(DAL)与 ORM 之适用性
-
Upload
xuefeng-zhang -
Category
Internet
-
view
671 -
download
14
description
Transcript of 浅谈电商网站数据访问层(DAL)与 ORM 之适用性
![Page 1: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/1.jpg)
浅谈电商网站数据访问层(DAL)与 ORM 之适用性
2012.11.2
中国,上海
![Page 2: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/2.jpg)
What’s Data ?
Data Files、Database、Data Service、 etc.SQL、NoSQL、AWS RDS、SQL Azure、etc.
![Page 3: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/3.jpg)
Why DAL ?
抽象不变的:DRY Don’t Repeat Yourself
封装变化的:SoC Separation of Concerns
![Page 4: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/4.jpg)
DAL 中的不变、变化?
不变的 or 一致的 (核心):Interface、SQL、etc.
变化的 - 1:Database、ORM Framework、 etc.变化的 - 2:Caching、Logging、Sharding、etc.
![Page 5: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/5.jpg)
电商网站 DAL 特点 (SoC)
• Simple Scalability 读写分离 DAL R/W Routing
• 10x Scalability DB Sharding DAL Routing + Aggregation
• Multi-targeting Logging Console/File/Log4Net/EntLib/Common.Logging
• Transparent Caching DAL 支持 + App 慎用实现难!
• Lazy Loading DAL 支持 + App 慎用用好难!
• 99% Availability Cache High Availability– 方案1:Dual-write Cache (Memcache) / Cache Replication (AppFabric)
– 方案2:Multi-level Cache L1 分布式 + L1.5 本地跨应用 + L2 本地应用内
![Page 6: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/6.jpg)
.NET 世界 ORM 现状
• 10x Scalability:仅 NH/MB 有限支持 Sharding– 通过 multi-db-session 实现 shards 结果聚合、in-memory 结果排序
– NH HQL 不支持 Sharding,如:排序,只能通过 ICriteria 间接实现
– 长时间未更新,不兼容最新 NH,不支持垂直分区及混合分区
• Multi-targeting Logging:– NH/MB 支持最好,L2S 也支持,但种类不丰富,需扩展
– EF 不支持,有开源爱好者提供了实现,较复杂、有风险
• Transparent Caching:此处略去 N 字…
• Lazy Loading:NH/L2S/EF/MB 都支持
• 99% Availability:此处略去 N 字…
![Page 7: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/7.jpg)
DAL Sharding Demo (ORM Extension) –NHibernate.Shards (ISessionFactory)
![Page 9: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/9.jpg)
DAL Sharding Demo (ADO.NET Extension) –Enzo SQL Shard (C# Extension Method)
![Page 11: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/11.jpg)
DB Sharding 策略
• Sharding is NOT SILVER BULLET for DB Scale-out
• 不适用场景– 处理事务型应用非常复杂,跨不同 DB 事务,很难保证一致性
– 不含 Sharding ID 查询较多的场景,即使并行处理,也要全扫描
• 大方向选择 (non-global tables)
– Sharding on every table primary separation child separation
– Sharding on every table group (sticky) primary + all children
– Sharding on every table group (non-sticky) primary + some children
• 小策略选择:Sharding ID、Aggregation、etc.
![Page 12: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/12.jpg)
DB Sharding 策略 (continued)
![Page 13: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/13.jpg)
DB Sharding 风险
• Sharded Data Migration
• Sharded Cluster Management
• Replication Management
• High-availability Management
• Load-balancing Co-existence
• Sharding Policy Change
![Page 14: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/14.jpg)
DAL Sharding 难点
• 切片策略– 精确定位:PK round-robin / PK hashing-modulus / attribute(s)-based
– 查询策略:all-shards iteration / 精确定位
– 遍历策略:sequential / parallel
• 结果聚合– 方案1:单库 SQL 查询 DAL 中完成聚合 (order, group, avg, etc.)
– 方案2:跨库 union 聚合 RDBMS 直接返回结果
– 方案3:读写分离多库实时写同步至单库单库非实时读
• 基于 ORM、自定义实现 (ADO.NET)?
• 其它:DB Proxy Sharding、In-mem Sharding?
![Page 15: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/15.jpg)
DAL Sharding 难点 (continued)
• DB Proxy Sharding:不太适合当前的 Ctrip性能问题、协议实现、etc.
• In-mem Sharding
–已在酒店查询服务优化项目架构设计中使用
– 1st-Level Sharding + 2nd-Level Load Balancing
–产品库可以这么做,订单库能这么做吗?
![Page 16: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/16.jpg)
DAL Sharding 设计参考
1、eBay Data Access Layer (QCon)
2、DAL Sharding 设计方案
![Page 17: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/17.jpg)
DAL Transparent Caching
• 现状– NH 二级缓存 Provider:NHibernate.Caches.MemCache
– MB 二级缓存 Module:mybatis-memcached-module
– EF:没有内建二级缓存机制,需通过 EFCachingProvider 扩展
– L2S:只有一级缓存,没有扩展机制,不推荐用于 scale-out 场景
• 难点– 对 Consumer 透明,如 Biz Layer 不需要知道缓存的“存在”
– 解决 Caching Policy 和 Data Consistency 间的矛盾与冲突
– 缓存刷新,拉模式 or 推模式?实时推 or 消息通知?
• 其它:Cache Provider 与 Cache-ware 区别
![Page 18: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/18.jpg)
DAL Transparent Caching (continued)
• High Availability 方案1:Multi-node– Dual-write Cache (Memcache, client-side, non-transactional)
– Cache Replication (AppFabric, server-side, transactional)
– 无论哪个 option,都要考虑 secondary node selection 问题!• 2 cache items 2 cache nodes 比较简单
• 3 cache items 3 cache nodes 有点复杂
• x cache items y cache nodes 如何确保 node unavailable 时造成的影响最小?
• High Availability 方案2:Multi-level– L1 分布式
– L1.5 本地跨应用
– L2 本地应用内
– 无论哪几个 options combination,都要考虑 sync coordination 问题!
![Page 19: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/19.jpg)
DAL 设计参考
1、NoSQL 数据库笔谈
2、James Hamilton
![Page 20: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/20.jpg)
这么多 ORMs,众口又难调,如何选择?
![Page 21: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/21.jpg)
1、One Size does NOT Fit ALL !
2、ORM Fx + Customized SQL + 其它
3、DAL 接口 + Base 抽象 + 自定义实现
![Page 22: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/22.jpg)
Demo for【 DAL 接口 + Base 抽象 + 自定义实现】
![Page 23: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/23.jpg)
介绍 DDFx 及相关架构
![Page 24: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/24.jpg)
What’s DDFx ?
• DDFx 的第一个 D 有两层含义:Data and Domain
• DDFx 的第二个 D == Driven,意为:…驱动的…
• DDFx 的 Fx == (Development / Design) Framework (and Tools)
• DDFx 理解 #1:Data-Driven (Development) Framework
• DDFx 理解 #2:Domain-Driven (Design) Framework
• DDFx 具体内容尚在不断完善中,但请记住:DDFx 不是万能的,所以,你要懂得:Context + Balance
• DDFx 中某些非关键技术将采用 Tech-Talk 方式进行
![Page 25: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/25.jpg)
什么是 Domain?
作为软件方面的专家(架构师、开发人员)和领域专家
(客户方业务人员、需求分析人员),所有人在一起创建
领域的模型,这个模型会统一体现两个专业领域的交汇
![Page 26: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/26.jpg)
什么是 Data?
Data Files、Database、Data Service、 etc.SQL、NoSQL、AWS RDS、SQL Azure、etc.
![Page 27: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/27.jpg)
DDFx 架构
![Page 28: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/28.jpg)
DDFx 中几个数字
• 1 座工厂:DDFx Factory
• 2 个车间:Domain Model (N 产品线) Repository (N 产品线)
• 3 种贯穿:DTO Domain Model Intf. Repository Intf.
• 6 层架构:Presentation Application/Controller
Service Domain Repository Resource
• 8 大前端:
– C/S:Console Windows Forms WPF
– B/S:ASP.NET Web Forms AjaxMVC / MVVM (ViewModel : DTO)
– RIA:Silverlight (.NET 子集)
– Device:Windows Phone 7 (Silverlight 子集)
![Page 29: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/29.jpg)
图表样例 - 1:Visual Studio 生成
![Page 30: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/30.jpg)
图表样例 - 2:Visual Studio 生成
![Page 31: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/31.jpg)
DDFx 架构层次
• 整体架构:6 层架构 + 1 座工厂 (DDFx Factory)
• Layer #1:用户表现层 (Presentation Layer)
• Layer #2:应用系统层 (Application Layer)
• Layer #3:服务表现层 (Service Layer, 可选)
• Layer #4:领域模型层 (Domain Layer)
• Layer #5:资源访问层 (Repository Layer)
• Layer #6:资源层 (关系数据库、外部服务、文件系统、etc.)
![Page 32: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/32.jpg)
DDFx 架构 + 开发曲线L1:UI -> VM/DTO (Requirement + UI Dev)
L2:-> Controller (Non-UI Dev) -> IDomain (Façade Intf.) (Arch + Dev Lead)
-> Domain Factory (Arch) -> Local/Service Client (Dev Lead)
L3:-> Local/Service Server -> IDomain (Façade Intf.) (Arch + Dev Lead)
L4:-> RealDomain (Real Façade) -> Biz. Rule + IRepository (Non-UI Dev)
L5:-> Repository Factory (Arch) -> Local/Service Client (Dev Lead)
-> RepositoryBase<T> : IRepository<T> (Dev Lead)
-> SpecialRepository : ISpecialRepository (Non-UI Dev)
-> Data Client / Service Client / etc. (Non-UI Dev)
L6:-> RDBMS / Service Server / File Server / etc. (Requirement + Arch)
![Page 33: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/33.jpg)
探讨:需求变更导致新增字段,你会怎么做?
![Page 34: 浅谈电商网站数据访问层(DAL)与 ORM 之适用性](https://reader030.fdocument.pub/reader030/viewer/2022012303/55941fbe1a28ab45128b461d/html5/thumbnails/34.jpg)
Q & A