Data Center Cooling Study on Liquid Cooling
-
Upload
42u -
Category
Technology
-
view
3.041 -
download
5
description
Transcript of Data Center Cooling Study on Liquid Cooling
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
2007 Data Center Cooling Study:
Comparing
Conventional Raised Floors with Close Coupled Cooling Technology
November 14, 2007
Presented By:
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Speakers and Sponsor
• Daniel Kennedy, Solution Center Coordinator for Rittal Corp.– Degreed Electrical Engineer with a
concentration in power systems– Over 4 Years Experience with Design and
Implementation of data center facilities– Project Engineer for Rittal’s closed-coupled
liquid cooling systems in North America
• Jennifer Osborn, Data Center Infrastructure Consultant for DirectNET – 5 Years Experience in data center planning and
management – Professional portfolio includes over 250 clients,
including Fortune 100 organizations– Successfully managed over 25 enterprise
implementations, including deployment, training, and support
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Agenda
• Power: The Change Agent of Today’s Data Center • A Review of Your Cooling Options • An In-Depth Assessment of Close Coupled Cooling:
2007 Data Center Cooling Study • 7 Things to Consider before Making a Cooling
Investment• Q&A
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Data Center Development Trends
AutomationAutomation
AssetmanagementAssetmanagement
Energy-EfficiencyEnergy-Efficiency
VirtualisationVirtualisation
Physical / logical SecurityPhysical / logical Security
Adaptive InfrastructureAdaptive Infrastructure
Remote ManagementRemote Management
Fuel cell technologyFuel cell technology
Wireless SensorsWireless Sensors
Cooling / PowerCooling / Power
Co
mp
on
e nt s
So
luti
on
s Cost intensiveInfrastructure:
- None adaptive- High energy consumption- People-intensive- Island-character- Local managed- Lights-on- Wired
Cost intensiveInfrastructure:
- None adaptive- High energy consumption- People-intensive- Island-character- Local managed- Lights-on- Wired
Co
mp
on
e nt s
So
luti
on
s
Cost-effectiveInfrastructure:
- Adaptive- Low energy consumption- Automated- Shared IT- Remote managed- Lights-off- Wireless
Cost-effectiveInfrastructure:
- Adaptive- Low energy consumption- Automated- Shared IT- Remote managed- Lights-off- Wireless
Today Future
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Inside the Data Center
• Floor Systems• Cable Plant
– Structured Wiring• Cable Routing
– Ladder Rack– Cable Tray
• Climate Control– CRAC, CRAH
• Security– Access Control – Video & Biometrics
• Power – Direct, UPS, EDG‘s
• Outside Service – OC-XXX, Telco
• Enclosures
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
The Issue
• Optimized air circulation cannot be achieved without significant facility changes• The high costs for energy and climate management increase TCO. • New methods and technologies must be defined for the increased climate control requirements.
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Trends: Where is it Headed?
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Space Comparison: White Space vs. Mechanical Space
Floor Area Required in Data Center
- Rising load density of IT areas
- Solutions for High Density Areas needed
- Cooling solutions with air/water heat exchangers
TODAYTODAY TOMORROWTOMORROW
Mechanical Space 40%
White Space 60%
Mechanical Space 60%
White Space 40%
Source: Internal estimation Rittal 2007
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Trends: The Way Forward
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: A Tutorial
Close Coupled Cooling
Rear Door
Air Cooled
Supplemental In Row Solutions
Active Air
0 2 10 15 20 28 35 40 kW
Chip + Enclosure Cooling
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: Air Cooled
Ways to Cool – Where they are Used
Air Cooled
0 2 10 15 20 28 35 40 kW
• Air cooled Cabinet in a traditional hot aisle / cold aisle arrangement
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: Rear Door
Ways to Cool – Where they are Used
0 2 10 15 20 28 35 40 kW
• Rear Door type units are a lower capacity supplemental cooling system
• Allow user to bring air return temperatures from dense cabinets down to inlet temperatures (70F)
• Can be deployed along with CRAC units to bring the hot aisle temperatures down on struggling units
Rear Door
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: Active Air
Ways to Cool – Where they are Used
0 2 10 15 20 28 35 40 kW
• Active Air solutions
• Rely on current cooling capacity to handle the load, but more effectively return it to the CRAC units
• Provides no actual cooling capacity, utilizes drop ceiling if present as a return path to CRAC unit
Active Air
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: Supplemental In Row
Ways to Cool – Where they are Used
0 2 10 15 20 28 35 40 kW
• Supplemental In Row Solutions / Above Row
• Provide cold air and heat removal from typical hot aisle / cold aisle arrangement
• Supplements the CRAC unit much like rear door units, typically, but not always of greater capacity
Supplemental In Row Solutions
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: Close Coupled Cooling
Ways to Cool – Where they are Used
0 2 10 15 20 28 35 40 kW
• Close Coupled Cooling – Closed Loop
• Provides cooling regardless of room conditions
• Performs all the functions of the CRAC unit, but brings the cooling directly to the cabinet itself
• Creates microclimate that exist only inside the rack
Close Coupled Cooling
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: Chip-Level
Ways to Cool – Where they are Used
0 2 10 15 20 28 35 40 kW
• Chip + Enclosure cooling
• Deployed in ultra dense environments, using customized servers
• Typically not for commercial deployment
• Aimed at research environments that can benefit from ultra dense clusters
Chip + Enclosure Cooling
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling Options: Visuals
• Open Systsem (supplemental cooling)
• Closed System(less than 10% Heat to the room)
• CPU (Chip) Cooling • Combined Solution
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Determining an Approach
• Hot/Cold Aisle Configuration?– Is it possible in your data center
• If not rear door solutions maybe ideal
• What is the Current Density of your Rack? – < 15kW - Supplemental System
• Supplemental systems can provide cooling were needed on a hotspot basis
– > 15kW – Close Coupled System • Can remove all heat being produced by the system, leaving
nothing to chance in the ambient environment
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
The Concept: LCP Plus
Touch screen
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Differentiating Closed Couple Cooling
+ =
LCP Server Cabinet High-Performance Solution
HEX
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Differentiating Closed Couple Cooling
HEX
fan
alternative 2
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling: Redundancy Considerations
LCP
M1
M2
M3
LCP
LCP
LCP
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling: LCP Extend
For pre-deployed racks
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling: Supplemental Solutions
Cold aisle
Cold aisle
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
TCO: Real Estate Cost Analysis
Real Estate Savings $129,617$195, 946
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
TCO: Real Estate Cost Analysis
common
40KW
10 rack sp.
4KW 4KW 4KW 4KW 4KW 4KW 4KW 4KW 4KW 4KW4KW
Available
40KW, 3 rack space
with LCP
LCP
LCP
20KW 20KW
30 – 40 KW
Available
40+KW, 2 rack space
LCP
LCP
30 – 40 KW
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
TCO: Energy Cost Analysis
Energy Savings Excellent - $69KAverage - $128KPoor - $302K
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling: Total Cost of Ownership
$295,507.68$541,764.08
$1,280,533.27
Cumulative Annual Savings
Excellent ($/Year)Average ($/Year)
Poor ($/Year)
• Allowing for warmer water temperatures, while still maintaining high density loads could save a great deal of money using conventional chillers
• Depending on the operational condition of the chiller plant, the numbers below could be realized annually on an installation of less than 70 close couple cooled cabinets!
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling: Total Cost of Ownership
• Real Estate savings– Having the ability to cool dense loads allows for
server consolidation into a single rack, or adoption of denser server technologies such as blades
• Running costs– Closed coupled cooling allows for multiple savings in
regards to energy • Lower fan energy costs• Lower lighting cost for smaller, more dense data center• Lower chilled water plant costs• Evaporative and dry free coolers
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Cooling: Total Cost of Ownership
Freecooler
Low noise Chiller
PumpHeatrecovery
Pump-station
BuffertankEmergency
water
buildingairconditioning
Chiller options
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Evaporative or Dry Coolers vs. Chiller Plant
• Typical Chiller plants provide chilled water at <45F, which is then past to the CRAC units in the data center
• Raising this water temperature lowers operating cost, but the cooling solution must be able to handle the warmer water while providing the same amount of cooling
• Close coupled cooling allows for high density installations, while using water as warm as 70F!
• This warm water temperature allows for more hours each year where evaporative or dry air side economizers / coolers can be used lower operating cost, by reduced electricity usage
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Chiller Example – California
• With a load of 28kW per cabinet, and with an ASHRAE allowable intake temperature to the servers of 77F, we can use water as warm as 70F, which can be realized whenever the Wet Bulb temperature is below 63F.
• In Oakland this is the case over 97% of the year!
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Chiller Example: Major Metropolitan Areas
Water Side Economizer Potential
0102030405060708090
100
Atlant
a
Chica
go
Miam
i
New Y
ork
Oaklan
d
Phoen
ix
Pe
rce
nta
ge
of
Ye
arl
y H
ou
rs
72F Air -65F Water77F Air -70F Water
82F Air -75F Water
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Comparison with Traditional Methods - PUE
• Typical data centers range from 3.0-1.6, the lower the number the better– PUE = (Total input Power to Data
Center / Total IT Load)• Example:
– Total Mechanical Load including chillers, UPS units, etc. ~ 216 kW
– Total IT Load at 108 kW– PUE = 216 kW / 108 kW = 2.0
PUE
• Even small LCP+ systems, without economizers, result in a PUE of 1.54!
WattsTotal IT Load 216024
Chiller 87752UPS Input 229296LCP Fan Load 16640
Total Support Load 333688
Resultant PUE 1.54
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
The Green Impact
The EPA estimates that at the current data center growth rate that we will need 10 new power plants before 2012 just to support IT growth!
http://www.energystar.gov/index.cfm?c=prod_development.server_efficiency_study
The use of close coupled cooling results in less energy loss in moving the air through the data center, saving fan energy, as well as reduces the cost to produce chilled water. This can result in significant energy usage reduction to cool the IT load, the place where the biggest impact can be seen today.
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
A Look Back…
NowThen
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Drawbacks of Closed Couple
• If you are using water cooled CRAH units, then its already there!
• The technology has been around for better than 50 years.
• As densities rose in the 70s with TTL logic, water was required, with the advent of CMOS, it went away, but our densities are back where water is needed.
• Bringing the cooling to the rack offers major advantages!
Mainframes in these data centers were cooled via water, direct to the
chips!
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
7 Top Cooling Considerations
• Focus on complete data center cooling designs• Provide complete cooling redundancy
– Include a comprehensive monitoring and alarm system at all component levels
• Understand The Complete Facility– Hot/cold aisle – common aisle for exhaust– Placement of vented or cutout floor tiles– Air flow paths
• Consider alternate cabinet configurations– Supplemental Cooling Systems– Close Couple Cooling– Energy-efficient, cost –effective way
• Group common products together• Develop component installation standards• Plan for new cooling solutions
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
DirectNET Confidential © 2007 DirectNET, Inc. All rights reserved.
Q&A
To Receive a Copy of Today’s Presentation: