Energy-Efficient Performance 2.0

Due to energy cost, availability and/or the environmental impact of producing electricity, energy efficiency has been an important point in enterprise/server computing for some time, and now client computers are starting to receive the same scrutiny. Intel has taken a leadership role in creating a meaningful measurement approach that encompasses both performance and energy efficiency.

PCs have changed the world, enabling people to work and play in ways previously thought unimaginable. And with so many desktop and notebook PCs in use, they have a measurable effect on the world’s energy use.

Aware of the increasing energy cost, environmental impact and overall load on the world’s electrical generation facilities, Intel is dedicated to delivering industry-leading performance, ever-expanding capabilities and energy efficiency. Intel calls this approach Energy-Efficient Performance, or EEP.

EEP Defined

Both desktop and mobile platforms obviously benefit from increased performance and capability, but the criteria and effects of increased energy efficiency differ for each platform. Mobile clients already deliver the lowest PC energy costs and are the preferred choice for users who are highly focused on energy costs. Mobile platforms benefit from EEP in a different way, too: battery life. When electricity is not readily available, as in working outdoors, in conference rooms, airports, or airplanes, energy-efficient processors deliver the required level of performance and capability for longer periods.

For the desktop PC, energy efficiency mainly translates into the energy cost required to deliver a specified level of performance and capability. EEP shouldn’t be confused with Thermal Design Power (TDP), which is the thermally-relevant power-draw of processor used to design PC platforms. TDP plays a crucial role in system design, determining elements of the cost and form factor of the PC, but conveys neither performance nor the energy cost at that performance level.

Energy efficiency will likely come into greater focus as an element when comparing PCs, and it is essential that the industry develop a meaningful and broadly adoptable metric that enhances the purchasing decision process. Energy efficiency and EEP pose some unique measurement challenges, however. While there are standard practices that have wide industry acceptance for gauging performance, such as SYSmark*, SPEC* CPU, and notebook-specific tests such as MobileMark* for battery life, there is no standard methodology for evaluating client performance coupled with its associated energy cost. This white paper outlines a proposed methodology for evaluating both the performance and AC energy efficiency of the client platform that may serve as an interim solution until an industry consortium-developed solution becomes available. In addition to this proposed methodology, Intel will fully participate with industry standards organizations to develop a consortium-based approach to this area of platform evaluation.

Modeling Energy-Efficient Performance

The first step in modeling energy-efficient performance is to define workloads that represent everyday PC activities. The thermal design power envelope is a design spec. Measurement under load is fine for maximum power consumption, but PCs don’t run at full power all the time. Idle power is useful information too, but neither do they run at idle all the time. A test workload must mirror the kinds of activities routinely done during everyday interactions with the PC. It must also report results that have meaning to the buyer/user. Intel’s EEP approach reports both the level of performance that can be achieved on the PC as well the annual energy cost to run the PC and achieve that performance.

In an office environment, much activity is centered on “office suite” applications such as word processors, spreadsheets, slide presentation, database, virus scanning, and file compression/decompression. Web content creators use a different set of applications, including HTML editors, image editors, media encoders, as well as video editing packages. An effective methodology will use these types of real-world applications as its workload.

In addition to the right workload, an effective methodology needs to define a usage context for that workload that simulates how the PC will be used over the course of an average day. This should be a reasonable mixture of executing the defined workload, some amount of time the system sits idle, and the system being put into standby or shut off during non-working/use hours.

The final results of an effective methodology should report the delivered performance at a given level of energy cost, communicating both performance and the energy cost of delivering that performance. Metrics that attempt to combine these measures are best avoided as they can obscure both the delivered performance and energy cost. This is especially true of taking ratios, as there is not a guaranteed linear correlation of the value of performance or energy cost. Platform choices are more easily made when both performance and its energy cost are clearly articulated.

Finally, the challenge of choosing the right workload is that one size does not fit all. Different usage models – digital office, digital home, gaming – require different representative workloads. Since much of the concern about PC energy use is found in the enterprise, it follows that a digital office workload is the logical starting point.

A Day in the Life

A typical workday in the United States encompasses nine hours: eight hours of work, with an hour for lunch. During this time, a PC will be actively used, will sit idle for some periods of time, and will be asleep for other periods.

In order to effectively model this typical behavior, the workloads must be based on real applications and the results of the applications must be meaningful, not just iterative loops or meaningless calculations. During the periods of active use, higher-performing systems will accomplish more work or finish more tasks. During brief periods of inactivity, such as “think time,” answering phone calls or interacting with coworkers, the system drops back to idle. For longer periods of inactivity, such as lunch or overnight, the system is assumed to be configured to drop into Standby (S3 ACPI state, suspend to memory) after 30 minutes of idle.

Intel is currently working with SYSmark* 2007 Preview, a performance benchmark suite designed and published by BAPCo*, a consortium of industry-leading companies with a common interest in consistent performance measurement. SYSmark* 2007 comprises office productivity, e-learning, 3D modeling and video content creation tasks. The application mix is based on research into emerging usage models and computing trends. It models realistic human-computer interactions, with think-time and human-level typing speeds.

The above chart illustrates that system level power usage can vary dramatically during each application type and that each application has a distinctive “fingerprint.” But the most significant result of the research is that the system is at or near idle a little over 40 percent of the time.

SYSmark* 2007 Preview takes approximately 50 minutes to complete, while the previous version, SYSmark* 2004SE took roughly 90 minutes. The shorter completion time affords greater flexibility in assigning the number of work sessions per day. The first iteration, EEP 1.0, broke the workday down into four two-hour work sessions, with the leftover time after the completion of the SYSmark* 2004 SE tasks defined as breaks:

EEP 1.0: The first EEP model had four SYSmark* 2004 SE runs in as many work periods.

The 90-minute run time allowed only four work sessions per day and one run per session. The 50-minute run time of SYSmark* 2007 permits two runs per work session—or just a single run—so that the overall number of tasks performed and amount of idle time can be tuned to more accurately reflect actual human workdays. Here is an example of four work sessions with six runs:

EEP 2.0: This new model achieves a better balance of work time, idle and sleep, while retaining essentially the same number of user tasks per day as EEP 1.0.

Measuring Energy Efficiency

System power-draw levels during a 9-hour workday. The percentages shown here are the power level amounts above the idle power level as compared to the idle power level. Intel® Core™2 Duo processor E6700 on the Intel® 965 Express chipset. See final page for system configuration details. Performance tests and ratings are measured using specific computer systems and/or components. Actual results may vary.

Each piece of this pie-chart represents the amount of time spent at a given system power-draw level. The percentages are the power level amount above the idle power level as compared to the idle power level. The main observation here is that over 34% of the time during the nine-hour working day— just over three hours—the system is at or near an idle power-draw level in this proposed methodology. More important however, is that all system power-draw characteristics – active, idle, and Standby – are in the context of a reasonable usage model. At the end of the day’s activities, the system is put to Standby for 15 hours, until it is awakened to start the next day. Accounting for this Standby time, the breakdown of system power-draw levels looks like this:

System power-draw levels during 24 hours. The percentages shown here are the power level amounts above the idle power level as compared to the idle power level. Intel® Core™2 Duo processor E6700 on the Intel® 965 Express chipset. See final page for system configuration details.

How the Methodology Works

As shown earlier, EEP 2.0 methodology is built on six SYSmark* 2007 Preview runs per day. Standard benchmarking practice runs each benchmark three times and uses the median of the three runs. For illustrative purposes, however, we will use a single run. The key metrics for EEP 2.0 are performance, as represented by the SYSmark* 2007 score, and the yearly system energy cost.

In order to calculate the energy cost, the PC must be monitored for actual energy consumption during the run. While this can be done with a simple, inexpensive plug-in wattmeter, a more sophisticated meter with data logging is preferable since it samples power consumption continuously and the amount of time spent at that power level can be calculated or imputed from the recorded data.

Intel’s measurements presented here used a sampling rate of 2Hz, or twice per second, with a wattmeter capable of automatic data logging. The testing protocol measure power during each of the SYSmark* 2007 Preview’s four component test. At the end of each test, the meter is stopped, since SYSmark* 2007 Preview reboots the system between sub-tests.

The information to be captured includes: * The time it takes each SYSmark* application segment to run (may be captured by stopwatch or the wattmeter). * The average power consumed during this time (captured by the wattmeter automatically or something that will have to be calculated manually from the captured sample). * To ensure stability of measurements, testers should perform five runs to capture Windows* Vista’s dynamic tuning capabilities in the test results, and then take the median of those five scores.

Next, measure the idle and Standby system power-draw levels.

For idle, allow the system sit for at least seven minutes with no active applications running on it, take samples over a three-minute period, and then calculate average idle power from those samples.

To measure Standby system power-draw, go to the Windows* Start Menu, and select Shut Down. Then select Standby and the system will go to Standby mode. Take samples over a three minute period, and then calculate average standby power from those samples. We recommend using average power value for idle and sleep states rather than median values, because the average captures the small fluctuations that occur in power-draw when in these two states.

This methodology can be applied to platform evaluation irrespective of the platform’s energy savings settings. Note however that for best energy efficiency, users should enable features like Enhanced Intel® SpeedStep™ Technology, whose benefits include achieving optimal idle power energy-efficiency. This technology exists to enable better energy efficiency, and should be used to minimize energy use when the computer is not being used. With that in mind, tested platforms presented here had Enhanced Intel® SpeedStep™ Technology enabled.

Mobile Platform Considerations

If the system being evaluated is a mobile platform, an industry-based benchmark is already available: MobileMark* 2007. The EEP 2.0 methodology is not meant to replace MobileMark* 2007, as that benchmark already conveys the relevant energy efficiency of a mobile platform, which is its battery life.

However, the EEP 2.0 methodology could be used to evaluate a mobile platform’s on-AC (plugged into the wall) energy-efficient performance. The system should be connected to an external monitor, and the system’s graphic subsystem configured to display only on the external monitor. The backlight in mobile platforms is the biggest consumer of power, and by removing it from the evaluation “equation,” a tester can get a clearer picture of the power-draw characteristics of the rest of the platform (CPU, chipset, RAM, disks, etc.). This approach will provide a better comparison between desktop and mobile platforms because monitor power is not included in the desktop calculations. Removing the LCD monitor relieves the notebook of the energy burden of its integrated display device just as the desktop measurement excludes the energy used by a display device. Alternatively, a tester could first evaluate the platform configured as described here, and then retest the system using the integrated display to understand the difference between the two configurations.

Finally, the laptop’s battery should be left in the system and should be fully charged prior to beginning the SYSmark* run, because the AC power adapters of some systems cannot handle the transient power demand spikes of the system, and the battery effectively augments the available power level to cover these spikes. Such a system will not run unless the battery is present. In addition, when charging the battery and powering the system, a laptop will draw additional power for both operations. At full charge, the only power-drawn from the wall is to power the system, and will provide a more accurate measurement.

Calculating Energy Cost

When a run is complete, we are interested in several data points: * The SYSmark* 2007 Preview overall score * Average system power-draw during the Video Content Creation test run * Average system power-draw during the Office Productivity test run * Average system power-draw during the E-Learning test run * Average system power-draw during the 3D Modeling test run * Completion time of the overall test run (sum of the four sub-test runs) * Idle system power-draw * Standby system power-draw

For example, Intel has measured the following performance and energy cost for the Intel® Core™2 Duo E4300 processor with integrated graphics on the Intel® Desktop Board DQ965GF motherboard. For full system configuration details, please see final page of this document. Note that both Enhanced Intel® SpeedStep™ Technology and Minimal Power Management are enabled.

Note that the kilowatt-hour figures will be small decimal numbers and will be eventually multiplied by much larger numbers. As such, it is important to avoid rounding off until after the final energy cost calculation has been made.

The energy cost amount will vary by country, and a localized energy cost value can easily be substituted. For this example we use 7.705 cents/KWh, which is the average of industrial and commercial energy prices in the United States as of May, 2007, according to the US Department of Energy.1

So, to calculate a daily energy cost, we first need to understand how much energy is consumed during the course of a single SYSmark* 2007 run.

We first take the average power during each of the four application segment runs:

	Average Power (watts)	Execution Time (minutes)	Energy Used (watt-minutes)
eLearning energy	71.9	11.12	799.528
video creation energy	74.2	14.83	1,100.386
office productivity energy	68.9	6.38	439.582
3D modeling energy	86.4	7.97	688.608
TOTALS		40.3	3,028.104

Then convert to kilowatt-hours, we first divide by 60, to make the units watt-hours, and then by 1,000 to make the units kilowatt-hours (KWh). So, we reduce this to a single operation by dividing by 60,000:

3,028.104 watt-minutes / 60,000 = 0.0504684 KWh

One note about SYSmark* 2007 Preview time: the EEP 2.0 model requires that each work period (morning and afternoon) contain at least three SYSmark* 2007 Preview Edition runs, and a 15-minute break. This means that a platform’s run-time on SYSmark* 2007 Preview cannot exceed 75 minutes, since three runs of that time-length plus the 15-minute break exactly equals 240 minutes, or four hours. If a system’s run-time on SYSmark* 2007 Preview does exceed 75 minutes, it is disqualified from being evaluated using this model.

Now calculate energy use during the morning period. First we determine the amount of time remaining at the end of the morning period. Recall that the first thirty minutes of this time the system will be idle, and then it goes to sleep:

Item	Time (minutes)
First SYSmark* 2007 run	40.3
Second SYSmark* 2007 run	40.3
Break	15
Third SYSmark* 2007 run	40.3
	135.9 minutes

So, we have 104.1 minutes remaining in the morning work period. The first 30 minutes of that will be at idle, so there will be 74.1 minutes of sleep during the morning work period.

Let’s next figure out the energy usage during the idle periods (15-minute break and 30-minute idle) and the sleep time of 74.1 minutes.

Total Idle time is 45 minutes, and the system idles at 59.6 watts. So:

59.6 watts x 45 minutes = 2,682 watt-minutes

Then convert to kilowatt-hours:

2,682 watt-minutes / 60,000 = 0.0447 KWh

Our example draws 3.2W of power when asleep. Now we figure out the energy consumed during the 74.1 minutes of sleep:

3.2 watts x 74.1 minutes = 237.12 watt-minutes

Then convert to kilowatt-hours:

237.12 watt-minutes / 60,000 = 0.003952 KWh

So, summing all the components of the morning work period, we have:

Item	Energy Used (KWh)
SYSmark* 2007 Run 1	0.0504684
SYSmark* 2007 Run 1	0.0504684
Break (15 minutes)	0.0149
SYSmark* 2007 Run 1	0.0504684
Idle Period (30 minutes)	0.0298
Sleep Period	0.003952
Total for Morning Session:	0.2000572

Next we calculate the energy used while the system is asleep during the lunch hour:

60 minutes x 3.2 watts = 192 watt-minutes

Then converting to kilowatt-hours:

192 watt-minutes / 60,000 = 0.0032 KWh

The afternoon work session is identical to the morning session, so the complete workday consists of:

Item	Energy Used (KWh)
Morning Session	0.2000572
Lunch	0.0032
Afternoon Session	0.2000572
Total For Workday	0.4033144

Finally, we calculate the energy used overnight while the system is sleeping. There are 15 hours between 5:00pm and 8:00am, so:

900 minutes x 3.2 watts = 2,880 watt-minutes

Converting to kilowatt-hours:

2,880 watt-minutes / 60,000 = 0.048 KWh

So the total for the 24-hour period of a workday is:

Item	Energy Used (KWh)
Workday	0.4033144
Overnight	0.048
24-Hour Total	0.4513144

Next, we calculate the energy used during non-workdays, such as weekends, holidays and vacation. The system is assumed to be asleep on these days, and there are 1,440 minutes in 24 hours, so:

1,440 minutes x 3.2 watts = 4,608 watt-minutes

Converting to kilowatt-hours:

4,608 watt-minutes / 60,000 = 0.0768 KWh

In the EEP 2.0 model, we assume there are 240 workdays, and 125 non-workdays, so the annual energy consumed is calculated:

(Workday energy used x 240) + (Non-Workday energy used x 125) = annual energy used

(0.4513144 x 240) + (0.0768 x 125) = 117.915456 KWh

Our model uses a per-KWh energy cost of $0.0705 /KWh, which is the average of the commercial and industrial rates paid in the United States, according to the US Department of Energy.2

So, the annual energy cost for our example system is calculated:

117.915456 KWh x $0.0705 = $9.09

The resulting yearly energy cost is $9.09/year.

This chart shows energy costs for other Intel processors tested in the same system used in the above example:

Energy-Efficient Performance: A well-managed desktop Intel® Core 2 Duo PC costs about $10 per year for its energy usage.

Variability Challenges

While performance and energy efficiency measurement methods have much in common, the latter has some unique considerations that must be accounted for to arrive at reasonably accurate measurements.

Inherent to the manufacture of integrated circuits (ICs) is that two ICs from even the same wafer of silicon can have different power consumption characteristics, which directly affect the energy efficiency of the platform built with these ICs. This variability affects all ICs to some degree, from CPUs, GPUs, RAM, I/O controllers, chipsets, and all others in the system.

In addition, power supplies, voltage regulators and fans have inherent variation in their power efficiency, and can also cause variations in power-draw measurements. For example, power supplies can vary by as much as 10 % from supplier to supplier or even within different models from the same supplier.

With so many moving variables in the equation, it might seem all but impossible to obtain repeatable measurements, since properly qualifying each component in the system would require thousands of test iterations for a single system. There is, however, a relatively simple solution.

At the outset of a project, the tester must decide what is to be the unit under test (UUT). For instance, if the UUT was to be pin-compatible CPUs, it is possible to hold all other system components exactly equal by using the exact same physical components to test all CPUs. This approach gives the tester a static snapshot of the collective variances of all system components. The principal downside of this approach is that it serializes the testing process, which can slow overall testing somewhat. But in order to achieve a reasonable degree of repeatability, this tradeoff is necessary.

Conclusion

It is clear that the industry needs to implement energy-efficient performance measurement tools that provide meaningful results to end-users. Although there are industry organizations that are working in this area, in the interim Intel is proposing a methodology for comparing Intel platforms. This methodology isn’t intended to be the final word, but rather the continuation of an industry discussion on a methodology to meaningfully gauge energy efficiency at a given performance level on the client PC. Intel continues to engage with industry consortia and government agencies to help foster the process of arriving at an industry-created methodology that can be broadly adopted and used across the industry.

An online calculator for the EEP 2.0 model can be found here:

System Configuration Details

Example system used in this paper had the following configuration:

Intel® Core™2 Duo E6700, Intel® DQ965GF motherboard with Intel® G965 Express chipset; graphics driver version: 7.14.10.1244; chipset driver version: 8.3.0.1013, 2x1GB Micron* DDR2-667 5-5-5-15, Seagate* Barracuda ST 3320620AS 320GB/16MB cache/7200rpm, Windows* Vista* Enterprise, DirectX* 10. Tests conducted with Enhanced Intel Speedstep® Technology power saving feature enabled.

Default Energy Cost: 7.705 cents/KWh, which is the U.S. average for industrial and commercial customers as of June, 2007. Source: U.S. Department of Energy, http://www.eia.doe.gov/cneaf/electricity/epm/table5_6_a.html

Performance tests and ratings are measured using specific systems and/or components and reflect approximate performance of Intel products as measured by those tests. Any difference in system hardware, software, or configuration may affect actual performance. Buyers should consult other sources of information to evaluate system or component performance they are considering purchasing. For information on performance tests and performance of Intel products, visit http://www.intel.com/performance/resources/limits.htm

Intel, and the Intel logo are trademarks of Intel Corporation in the United States and other countries.

*Other names and brands may be claimed as the property of others.

Copyright © 2007 Intel Corporation.