Green Data CentersKen Hall, RCDD NTSSeptember 2008
August, 2008page 2 /
Green Initiatives: Materials
� Lead-Free, RoHS product set� Reusable products
� Modular Copper and Optical Fiber Preterminated Communications cabling
August, 2008page 3 /
What is a �Green� Data Center?
August, 2008page 4 /
Next Generation Ethernet Networks
Power Consumption
Cooling Requirements
Transceiver Size
~2W ~8-15W
Fiber Copper
Data Center Area $ $$$$
August, 2008page 5 /
Green Initiatives: Power
� High-density Cabling � Enables 7-27% power reduction on per port basis� Smaller cable than six 4-pair cables
� Better airflow = better cooling = less power
� Shielded Cat6A Cabling� Up to 40% smaller cable than Cat 6a U/UTP� Potential power savings on network electronics
� XG Optical Fiber Cabling� Optical ports draw less power than copper ports
August, 2008page 6 /
Data Center Standards
�TIA 942 Telecommunications Infrastructure Standard, April 2005
� Substantial, available, North America
�EN 50173-5 Information technology � Data Centers
� Smaller scope, Europe
�BICSI 942, �complementary� to TIA 942
� ISO/IEC Draft 24764 Information technology
� Global Generic Cabling for Data Centers
August, 2008page 7 /
Hierarchical Structure of Data Centers
August, 2008page 8 /
Different Tiers of Data Centers (TIA942)
August, 2008page 9 /
Annual Downtime
24 min
1 hour
36 min
22 hours
1 day
4 hours
48 min
Downtime/
Month
Downtime/
Year
2 min8 min
1 hour
50 min
2 hours
24 min
0,4 hours1,6 hours22,0 hours28,8 hours
Tier 4Tier 3Tier 2Tier 1
Not much time for Re-cabling!
August, 2008page 10 /
Tiering For Cabling Systems (TIA942)
Security/Availability
Structured Cabling Trends for SANs and Data Centers
August, 2008page 12 /
Data Center Dynamics Survey
August, 2008page 13 /
Power�.
� Demand for electricity is projected to grow 19% in next ten years� Generation capacity is projected to grow only 6% (Source: NA Electric Reliability Council)
� Virtualization & multi-core processors are driving dramatic increases in cabinet density� 2006: US servers consumed $14 B in power & cooling� 2009 Projection: $50 B (Source: IDC)
� 2011 Projection: $100 B � In 2008, half of the world�s data centers will not be able to utilize the
latest high-density equipment (Source: Gartner)
� Their power infrastructure simply won�t support it� 1W at the server takes ~ 2 watts at the meter
� Network power - Cooling of network equipment - utilization
August, 2008page 14 /
Availability, Space and Density
45%
52%
83%
0% 20% 40% 60% 80% 100%
Space
Availability
Power consumption
A survey of Data Centers owners and the biggest
concerns
Solution: flexible cabling solution, easy to migrate, just Place and Plug, high density
August, 2008page 15 /
Passive Cabling Solutions for Data Centers
� Data Center cabling requirements are different from the ones in Office Networks
� Dynamic vs. Static
� Easy deployment (weekend work/testing)
� High performance
� Structured scalability
� Fast & easy MAC�s
� Functional Security (less human errors)
� High density
� Effective cable management
� Availability (less planning, less downtime)
� Cooling support
August, 2008page 16 /
Passive Solutions for Data Centers
� Market situation� The market changed in 3 years to pre-terminated solutions� Copper
� High density copper 1 Gb/s systems� Integrated cable management
� Fiber� Multimode OM3 systems� MPO systems
Data Center Applications & TechnologiesToday & Tomorrow
August, 2008page 18 /
What applications are running in Data Centers ?� 10/100Base-T
� GigaBit Ethernet (1000Base-T, 1000Base-SX/LX)
� 10GigaBit Ethernet (10GBase-T, 10GBase-SR)
� Fibre Channel
� InfiniBand
� Coming: 40 & 100 GigaBit Ethernet
August, 2008page 19 /
0
5
10
15
20
25
3020
00
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
100M 1G 10G 40G 100G
mill
ions
ser
ver u
nits
x86 Servers by Ethernet Connection Speed (100 Mbps through 100 Gbps)
Source: Intel & Broadcom (April 2007)
10 year transition for 1G Ethernet 6 years for 10GBE 5 years for 40GBE
August, 2008page 20 /
GBE/10GBE
� GigaBit Ethernet
� OM1/OM2/OM3 for fiber (1000Base-SX/LX)� Cat.5e for copper (1000Base-T)
� 10GigaBit Ethernet
� OM3 recommended for fiber (10GBase-SR)� Cat.6A/Cat.7/Cat.7A for copper (10GBase-T)
August, 2008page 21 /
� Summary of power measurements
0
5
10
15
Link speed (Mb/sec)
Pow
er u
se (W
)
10 100 1000 10000
10G Base-T is a power concern
g00.xls
10G Base-T Application Cabling Power
20082008
August, 2008page 22 /
10G Copper Cabling
� 10G Copper Expectations� 36% expect to install 10G copper in the next 1-2 years� 61% expect to install 10G copper in the next 3 years
� Out of those expectations, 75% of those plan to choose shielded copper over unshielded
0%
5%
10%
15%
20%
25%
Cat 6 F/UTP Cat 6A U/UTP Cat 6A F/UTP Cat 7 S/FTP Cat 7A S/FTP UndecidedSource: BSRIA: U.S. Data Center Structured Cabling Study, January 2008
August, 2008page 23 /
10G Base-T Application Cabling Power
� Reason� Signal to noise (S/N) ratio of UTP cabling systems� DSP�s are required to improve the signal quality
� Shielded systems have a much better EMC performance� Better S/N ratio� Less calculation for DSP�s� Less power consumption
� Conclusion� Shielded cabling enables reduction of network power consumption
August, 2008page 24 /
� 10GB-T PHY optimized for 100m UTP Cabling� 15-20W (in 90nm) (1000B-T: 500mW)
� Problem for High Density Line Cards, PCI Cards, MBs, Modules
� 2.5usec latency (1000B-T: 250ns)
� Too high for some High Performance Applications, FC, IB
� Power Hungry AFE
� 50% of Power is in AFE
� Not likely to improve much over time� Large percentage of the power is in the AFE
� Power reductions in smaller process geometries are questionable
10G Base-T Application Cabling Power
August, 2008page 25 /
� Power Dissipation� 3W Typical, 4W Worst Case
� Latency� Less than 500ns� Goal of 250ns
� Low Cost� Significant SNR Margin
(1000B-T: 10dB)� Demonstrable Design of 1M
Gates (2x 1000B-T)
� Short Reach� 30m on STP/FTP Cable (field
configurable)� 2 Connector, 3 Links Segments� Goal of 45m
� Auto Negotiation Compatible� RJ-45 Connector� Backwards Compatible with 100B-
T / 1000B-T over UTP
10G Base-T Application Shielded Cabling Power
August, 2008page 26 /
Characteristics of Data Center Cabling
� 90% of Links are in Server Rooms Less Than 1,524 sq. meters(100% <= 30m)
� 5% of Links are in S. Rooms Between 1,524 � 6,096 sq. meters(80% <= 30m)
� 5% of Links are in Server Rooms Greater Than 6,096 sq. meters (55% <= 30m)
� => 97% Coverage with 30m
� => 99% Coverage with 45m
August, 2008page 27 /
Fibre Channel
August, 2008page 28 /
InfiniBand Bandwidth
2004 2005 2006 2007
Spee
d
40G
20G
10G8G4G2G
Infiniband10G EthernetFibre Channel1G Ethernet
120G QDR
DDR
SDR
Parallel lanes means parallel optics
August, 2008page 29 /
InfiniBand: High Speed Cabling Types
QSFP
August, 2008page 30 /
40/100 GBE Multimode Fiber (MMF) Solutions
!850nm 12x10G CWDM devices may support 100-300m
!Utilization of MPO connectors is an important factor for
economic feasibility of 100G MMF
!Pre-terminated OM3 ribbon fiber will be cost-effective
!90% data centre links are MMF today and within 150m!High Performance Computing requires links to 100m!Majority of corporate building backbones under 300m!IBM survey found 70 - 80% of above links under 100m!Corning survey of data centre links found 96% <200m
August, 2008page 31 /
40/100 Gb/s
� Official IEEE project � IEEE802.3ba� Current situation indicates an MPO solution� Reason: Parallel transmission needed
12-fiber MPO 72-fiber MPO
Examples of Tyco Electronics Parallel Optic connectors
August, 2008page 32 /
ISO/IEC 24764 data center standard
ZDMD LDP EQENI
ISO/IEC 24764 model for Data CentersENI
Network access cabling
subsystem
Main distributioncabling
subsystem
Zone distributioncabling subsystem
� Standard MPO systems allows mostly 2 cassettes for 10 Gb/s� Next Generation allows 6 cassettes
� Complete cabling from MD to EQ� Reduce number of actives� Allows more flexibility in DC design
QSFP-style Active Cable Assemblies for InfiniBand Applications
August, 2008page 34 /
Infiniband
� The situation:� SAN�s a essential part of Data
centers� More and more data will be
stored� Infiniband and Fiber channel
most common applications� Mostly Copper for Infiniband and
fiber for Fiber channel
August, 2008page 35 /
Why Active Optical Cable Assemblies?
!Up to 200mtr.!
!Bend radius 25mm!
!20 GB Bi- directional (Up to 40Gb)
!Multi Mode fiber optic cable
Copper compared to Fiber
!Limited lengths, max 20mtr.
!Large Bend radius
!28-26 AWG Copper cable
!Max. 10 GB Bi- directional
Longer lengthsare
not possible due to
Noise and Crosstalk
!28-26 AWG Copper cable
August, 2008page 36 /
Product Overview
� Products� DDR4 - 4 Channels of 5 Gb/s
� 20 Gb/s Up to 200 Meter Reach
� QDR4 - 4 Channels of 10 Gb/s� 40 Gb/s up to 120 Meter Reach
� Next steps to develop� Product DDR12 -12 Ch 5 Gb/s (60 Gb/s) Up to 120 Meter Reach� Product QDR12 � 12 Ch 10 Gb/s (120 Gb/s) Up to 120 Meter Reach
Data Centers Solutions
How to combine DC cabling infrastructure requirements with today�s and tomorrow�s
applications ?
August, 2008page 38 /
Recall: Passive Cabling Requirements for Data Centers
� Easy deployment (weekend work/testing)
� High performance
� Structured scalability
� Fast & easy MAC�s
� Functional Security (less human errors)
� High density
� Effective cable management
� Availability (less planning, less downtime)
� Cooling support
MRJ21 MPO 1�
August, 2008page 39 /
Why High Density Technology?
� Eco-Friendly� Assists in efficient power use at the switch� Assists in efficient power use in HVAC� Less material� Reusable
� Reduces need for expensive abandoned cable removal
� Modular� Speeds design & layout� Speeds installation / availability� Speeds training� Speeds MACs� Speeds recovery
August, 2008page 40 /
High Density Pre-terminated Solutions
� Pre-terminated MRJ21 Copper System
� High Density with 6 GBE Channels per
cable
� ≤ 48 ports in 1 rack unit
� Pre-terminated MPO Fiber System
� High Density with 12 or 72 fibers per
cable
MRJ21 cable is 30% smaller, 20% lighter than 6 4-pair cables
August, 2008page 41 /
Current GbE application benefits
� Density, modularity & performance on equipment and in infrastructure
� MRJ21� provides Hi-D Green GbE � Connector: 1/3 -1/4 RJ45 cassette
� Cable: 30% smaller, 20% lighter
� 1 cable vs. wrapped 6-cable bundle
� Eco-Friendly
� All components permanent link tested,
documented and serialized
� 7-27% less power than RJ45 @ 2x ports
� 10G Shielded Version in process
1.14�x.74�6 GbE ports
MRJ21�MPO
Euro
August, 2008page 42 /
High-density Pre-terminated Copper
� The MRJ21� 24-pair Connector� 1�x 5/8� Footprint� Being adopted directly on electronics
12~399 in2RJ-45
8~266 in2RJ-45 (High Density)
6~200 in2MRJ21
Rack UnitsArea288 4-pair Ports
1 MRJ21 6 RJ45s
August, 2008page 43 /
MRJ21 Energy Efficiency
4.88 Watts234 Watts481 GbE48 port MRJ21
5.25 Watts126 Watts241 GbE24 port RJ 45
3.56 Watts320 Watts901 GbE90 port MRJ21
4.75 Watts190 Watts401 GbE40 port RJ 45
Total power
per port
Max power
per card
# of
ports
Appl.Card
configuration
Combines high density and ease of maintenance with energy savings
August, 2008page 44 /
Network Vendors with MRJ21 Equipment
� Alcatel:� OmniSwitch� Alcatel TiMetra� Lucent / Riverstone
� Barco Xenia� EIT� EntriSphere � Foundry Networks� Force10 Networks � Fujitsu � IBM � Motorola � NMS Communications � Proworks
� Shanghai Electronics� Stratex Networks� Tellabs / Vivace� UT Starcom
Force10 E120090 GbE ports/card
Foundry BigIron RX-4,8, 16, & 3248 GbE ports/card
Note: The vendors listed and shown have publicly launched MRJ21 equipment into the marketplace. Those not listed should be contacted directly for program status updates.
IBM eServerBladeCenter Copper Pass-
thru module 15 GbE ports
Alcatel OmniSwitchNI48 card
August, 2008page 45 /
Content of One 7-foot Rack (40 RUs)
~$0.70
~$1.56
~$1.40
~$3.12
~$2.80
~$6.25
Op Cost per
Port (year)
1920960480Cu Ports per
Rack
0 RUs20 RUs20 RUsCable Mgt.
per Rack
484824# Copper
Ports per RU
AngledHigh-densityTraditional
low HIGH
9 sq. ft. Traditional Room � Avg $1,350/yr ops. cost20 sq. ft. Hot/Cold Aisle � Avg $3,000/yr ops. cost
August, 2008page 46 /
Content of One 7-foot Rack (40 RUs)
~$0.70
~$1.56
~$1.40
~$3.12
~$2.80
~$6.25
Op Cost per
Port (year)
1920960480Cu Ports per
Rack
0 RUs20 RUs20 RUsCable Mgt.
per Rack
484824# Copper
Ports per RU
AngledHigh-densityTraditional
low HIGH
9 sq. ft. Traditional Room � Avg $1,350/yr ops. cost20 sq. ft. Hot/Cold Aisle � Avg $3,000/yr ops. cost
August, 2008page 47 /
Content of One 7-foot Rack (40 RUs)
~$0.70
~$1.56
~$1.40
~$3.12
~$2.80
~$6.25
Op Cost per
Port (year)
1920960480Cu Ports per
Rack
0 RUs20 RUs20 RUsCable Mgt.
per Rack
484824# Copper
Ports per RU
AngledHigh-densityTraditional
low HIGH
9 sq. ft. Traditional Room � Avg $1,350/yr ops. cost20 sq. ft. Hot/Cold Aisle � Avg $3,000/yr ops. cost
August, 2008page 48 /
Installation Time of One 7-foot Rack (40 RUs)
256 Hours128 Hours64 hoursInstall Time per
Rack � RJ45
2 minsAvg. Install
Time � MRJ21
10 hours,
40 mins
5 hours,
20 mins
2 hours,
40 mins
Install Time per
Rack � MRJ21
8 minsAvg. Install
Time - RJ45
AngledHigh-densityTraditional
Cable prep, termination, dress, insert & label
August, 2008page 49 /
Installation Time of One 7-foot Rack (40 RUs)
480 ports 960 ports 1,920 ports
256 Hours128 Hours64 hoursInstall Time per
Rack � RJ45
2 minsAvg. Install
Time � MRJ21
10 hours,
40 mins
5 hours,
20 mins
2 hours,
40 mins
Install Time per
Rack � MRJ21
8 minsAvg. Install Time
- RJ45
AngledHigh-densityTraditional
Cable prep, termination, dress, insert & label
August, 2008page 50 /
Large Job (960 Ports) Cost Model Summary
~10 days less (E/P)~2 days less (NE/NP)
88% lessTotal
Time
~10% less (E/P)+15% more (NE/NP)
27% lessTotal
Cost
Fiber
12F MPO to Small Form
Factor
Copper
MRJ21 vs. RJ45
Product, placing cables, cable preparation, termination
The material referenced on this slide is for comparison purposes only and not for budgetary purposes. Actual costs may differ and may depend on project size, location or purchasing practice as well as other variables.
August, 2008page 51 /
High-density Pre-terminated Fiber
� The MPO� FOCIS 5, published in 1997
� 12 fibers in one connector� Being adopted directly on electronics
2~66 in212-fiber MPO Fiber
24~798 in2Duplex Fiber
12~399 in2Duplex Fiber (SFF)
<0.25~2 in272-fiber Para-Optix� MPO
Rack UnitsArea288 Duplex Ports
1 MPO
6 SC Duplex
1 Para-Optix
36 LC Duplex
August, 2008page 52 /
What is the value of time?
0 2 4 6 8 10 12 14 16
On-Site Person Hours
3) Para-Optix 36 DuplexPorts 1U
2) MPO Cassettes 1u (6x12Cable)
1) Field-terminated NE/NPDuplex LC
Installation Time 144 Fibers (72 OM3 fibers / end)
Time (Hrs)
� Installation Time� Uptime� Recovery Time
The Combination of a High Density/High Performance Cabling System Platform with Cable Management
August, 2008page 54 /
Size Matters�
� How does cable size really affect your pathway costs?� Conduit fill � based on 40% fill ratio� 40% more cables in a conduit
� Patch cord comparison � F/UTP compared to U/UTP
Conduit F/UTP 6A UTPSize 7.6mm (.300") 9.0 mm (0.354")
2 18 133 40 294 71 51
August, 2008page 55 /
10 Gigabit per Second � Space (Air Handling � Cooling � Power)
ShieldSpaceMitigation
TechniquesANEXT
Reduction
57
.25 inch
48
Cat 6 U/UTP
42
(15 < Cat 6)
28
(29 < Cat 6)3� Conduit Cable Fill
0.290 inch
(Up to 16% > Cat 6)
≤ 0.354 inch
(Up to 41% > Cat 6)Cable OD
4816 � 24Port Density
(1 RU)
Cat 6aF/UTP
Cat 6aU/UTP
August, 2008page 56 /
Cable Management
� Does this look familiar?
� Unmanaged and unstructured connections
� Poor documentation
� 30-40% abandoned patch cords after 1 year of operation
� Dead ports in switches and cable runs 30-40% unused capacity
� MAC time > 1h per port
� Adds up to 150% of operational cost
� Adds up to 200% of floorspace needed
� Residual cable mass blocks airflow in rack and under floor
August, 2008page 57 /
Recall: Passive Cabling Requirements
� Easy deployment (weekend work/testing)
� Structured scalability
� Fast & easy MAC�s
� Functional Security
(less human errors)
� High density
� Effective cable management
� Availability
(less planning, less downtime)
� Cooling support
August, 2008page 58 /
AMP Hi-D technology
� Patented Corner Brackets combined with Angled Patch Panels� Eliminates the need for horizontal cable managers� Uses unused valuable cabinet space - increase of cabinet density� Toolless cable management maintenance � increase of operational
speed
August, 2008page 59 /
End-to-End Data Center Solutions
Top Related