Analyzing server power consumption and costs

Measuring server power consumption

For quick and easy power measurement, we chose Watts Up Pro, a cheap power meter with USB output, which sells online for about $130.  There are others you can use as well, but be sure to investigate whether its possible to programatically consume power measurement data if you want to do any kind of serious analysis.

Watts Up Pro power meter

Plug the server into the meter, and you can immediately see the Wattage and a host of related metrics on the LCD screen (pictured above with one of our workstation machines).

Our main point of interest was the server itself – however, to extrapolate the power usage for your DC, you’ll likely want to measure a representative unit of equipment including the server(s) and related UPS/networking equipment.  You can plug all of these into a single power strip, and plug that into the meter to capture it all.

Monitoring power consumption

The next step is to be able to monitor the power use over a period of time, and across varying load conditions.  You can then use this information to more accurately predict the server power consumption in your environment, and gain insight into to control it by correlating it with many other operational and load factors.

To collect the power data automatically using the Watts Up Pro meter, we wrote WattsUpMon.exe. 

This tool captures the power samples in real-time from the meter’s USB interface, and publishes this information using Windows performance counters.  Doing so enabled us to monitor the power consumption using any standard performance counter collection tool (including PerfMon.exe), as well as correlate the data programmatically for analysis.

 
Using WattsUpMon.exe to monitor power data from a Watts Up Pro meter

If you are using the Watts Up Pro meter, download the WattsUpMon.exe tool.

To use:

  1.  Connect the meter’s USB cable to the monitoring computer
  2.  Install FTD’s virtual serial port driver: http://www.ftdichip.com/FTDrivers.htm
  3.  Double-click WattsUpMon.exe (or use the cmdline parameters to customize behavior)
  4.  Access the performance counters from the WattsUpPro performance object

Analyzing server power consumption

Using the meter and the monitoring tool, we could then monitor the power consumption of a server over time, and observe how it changes depending on the load.  Below is a PerfMon graph of power consumption for a DELL Optiplex 330 workstation machine, while running an example load scenario:

Analyzing power consumption patterns using PerfMon.exe

The machine is part of the new generation of power efficient hardware, and it shows with the average power consumption of 49 watts during the test.  This is impressive, compared to several much slower machines in our lab that draw 200+ watts.

An important thing to note that is that the power consumption of the server changes by as much as 20% of the maximum power consumption during the test run (45-57 watts).  This is due to the automatic power management, enabled by default in Windows Server 2008.

The default “Balanced” power policy uses demand-based switching (DBS) to automatically reduce the frequency of the processors to save power when the machine is under low load.  This leverages the P-state toggling support provided by the processors to the Windows ACPI layer. You can see this in action from the dashed green lines representing the % of maximum processor frequency: when the CPU utilization is low, the CPUs are working at roughly 66% of the maximum frequency.  When the utilization increases, the processor frequency is increased, and the power demand follows. 

Obviously, the processor is not the only power user in the server (although a major one).  Other key consumers include the internal disk drivers, and attached network devices e.g. NAS.  By correlating performance characteristics of the server and associated equipment for specific workloads together with power consumption, it is possible to more accurately model not only the overall power consumption – but also the power cost and power efficiency of specific workloads.  This can then help make better decisions at both the data center and application level to achieve optimal power efficiency, using intelligent workload allocation and techniques like resource consumption shaping (5).

The cost of power

Extrapolating from this scenario, our workstation might costs us 49 Watts / 1000 (KWH) * 24 hrs * 365 days * $.15 = $64/year (using average 15c/KWH commercial price for Middle Atlantic region in Sept 2008 (3)).

In a data center setting, we would use the average Power Usage Effectiveness (PUE) ratio of 2.5 to determine the real power consumption due to the server’s operation (4). 

This essentially means that the typical data center will consume 2.5 watts for each 1 watt consumed by a server in cooling, and power distribution overhead.  So, you might project that operating this server costs you $160/year in power costs.

The DELL Optiplex 330 is a new power-efficient workstation machine. Let’s say you had a more typical server that consumed 200 watts on average, and you had a 1000 of them.  Then, they might run you about $653,000/year.  If you had 2000 of them, your cost would be $1.3M/year.

In our case, Windows Power Management saves us about 14% of power consumed. This is a great start, and it’s to know that everyone using Windows Server 2008 will experience similar savings by default (you can also turn this on Windows Server 2003).  Windows 7 promises to deliver even more savings.

The even better news is that it is possible to do much better that that.

The technology that we are developing may be able to deliver much more considerable power savings for server environments.  Our goal is to reach anywhere from 30-80% depending on the environment and load patterns, while insuring quality of service and availability of the server environment.

We hope to realize very significant cost savings for data centers, without requiring expensive investments in new hardware, and application software changes – and provide added savings to those that are also looking to deploy server virtualization and more efficient server hardware to cut power costs.  This should help reduce the pressure of increasing energy prices, demand, and a growing imperative to reduce carbon footprint of DC environments.

More on this in the upcoming weeks.  In the meantime, happy measuring.

(1) Belady, Christian. 2007. “In the Data Center, Power and Cooling Costs More Than the IT Equipment it Supports.” Electronics Cooling. Vol. 23, No. 1, February 2007. 

(2) U.S. Environmental Protection Agency. 2007. “EPA Report to Congress on Server and Data Center Energy Efficiency”.

(3) U.S. Energy Information Administration. http://www.eia.doe.gov/cneaf/electricity/epm/table5_6_a.html

(4) The Uptime Institute. 2007. The Green Grid Data Center Power Efficiency Metrics: PUE and DCiE.

(5) James Hamilton’s blog. Resource Consumption Shaping.