Variability Challenges

While performance and energy efficiency measurement methods have much in common, the latter has some unique considerations that must be accounted for to arrive at reasonably accurate measurements.

Inherent to the manufacture of integrated circuits (ICs) is that two ICs from even the same wafer of silicon can have different power consumption characteristics, which directly affect the energy efficiency of the platform built with these ICs. This variability affects all ICs to some degree, from CPUs, GPUs, RAM, I/O controllers, chipsets, and all others in the system.

In addition, power supplies, voltage regulators and fans have inherent variation in their power efficiency, and can also cause variations in power-draw measurements. For example, power supplies can vary by as much as 10 % from supplier to supplier or even within different models from the same supplier.

With so many moving variables in the equation, it might seem all but impossible to obtain repeatable measurements, since properly qualifying each component in the system would require thousands of test iterations for a single system. There is, however, a relatively simple solution.

At the outset of a project, the tester must decide what is to be the unit under test (UUT). For instance, if the UUT was to be pin-compatible CPUs, it is possible to hold all other system components exactly equal by using the exact same physical components to test all CPUs. This approach gives the tester a static snapshot of the collective variances of all system components. The principal downside of this approach is that it serializes the testing process, which can slow overall testing somewhat. But in order to achieve a reasonable degree of repeatability, this tradeoff is necessary.