We regularly publish a DVD full of application-based tests we call Software Evaluation Guides (SEGs). On it, we provide test scripts and input files you can use to evaluate PC platforms using real-world applications.

SEGs make a great compliment to industry-standard benchmarks since a complete test mix includes synthetic tests like PCMark*, application-based benchmarks like SYSmark*, and application-based tests you run on your own.

We normally provide our SEG collection to Intel customers, as well as members of the tech press. Now, we’re making it available to everyone. Here at ICF, you can download these SEG test scripts, as well as resource files (media input files, mostly) to run on your own system.

These SEG tests cover several usage areas including media encoding, office applications, as well as 3D, audio and video content creation. SEG tests are also compatible with both Windows* XP and Windows* Vista* if you want to compare performance on the two OSes. Some of these SEGs also make for good background workloads in multitasking scenarios. Today, we introduce you to three SEGs that cover photo editing, 3D rendering, and video effects.

Multitasking Workload Considerations

One way to think about multitasking performance is in terms of types of users, and what kinds of things they want to do simultaneously. Also, it’s important to consider the metric of interest. Is it: foreground task performance alone, background task completion time, or some combination of the two? Another way to approach it is as a “basket of tasks.” You have a set of things you want to get done, so the test is completed when all the tasks in the basket are completed.

To kick off the workload, start the background task first and then immediately start the foreground task. Allow all tasks to run, and stop your timer when the last task has completed. Whether it’s the foreground or background task that completes first will depend on the mix of tasks in flight.

Photo Editing (1/2)

Adobe* Photoshop Elements* 5.0 lets you organize, edit and enhance photos. With advance options, you can create composites using special effects, texts, and graphics. For more information go to http://www.adobe.com/products/photoshopelwin/main.html.

The workload is to run the auto smart fix feature from Adobe* Photoshop Elements* 5.0 to correct and enhance 103 jpeg images. The average size of the images is about 600KB.

1. Install Adobe* Photoshop Elements* 5.0 using default settings.

2. Reboot your system

3. Double-click the Photoshop Elements* 5.0 icon on your desktop to launch the application.

4. Click on the icon with the text View and Organize Photos.

5. From the File menu, select File->Get Photos->From Files and Folders… Navigate to where your photos are stored.

6. Select all the photos in your Photoshop Workload. Deselect the checkmark to Automatically Fix Red Eyes. Click the Get Photos button. Once the new photos have been added, select the new photos.

Photo Editing (2/2)

7. Enter Control-Alt-M to run “Auto Smart Fix Selected Photos.” The progress dialog below will appear.

8. When all photos have finished being fixed, the progress dialog will disappear.

9. Open the workload folder where the edited images have been created and stored. Sort the files according to Date Modified. Subtract the Created time of the first photo that was fixed from the Modified time of the last photo that was fixed. This is how long your system needed to auto-fix the files.

10. An alternate timing method is to start your stopwatch at the same time Control-Alt-M is entered to begin the auto-fix process and stop your stopwatch when the progress bar disappears. Be sure to measure all systems being compared with the same method.

You can download a PDF version of this SEG here.

You can also download some photos to use as input data here as well.

3D Rendering (1/2)

This procedure is designed to evaluate processor performance while creating 3D images using Autodesk* 3DS Max* 9. Run this test on a system running Windows* XP or Windows* Vista*.

Autodesk* 3DS Max* 9 is a popular animation modeling, and rendering solution for film, television, games and design visualization. It contains the essential high-productivity tools required for creating eye-catching film and television animation, cutting-edge games, and distinct design visualizations. For more information about Autodesk* 3DS Max* 9, visit http://www.Autodesk.com/

The workload used in this document is called Dragon_Character_Rig.max. The workload consists of a scene of a Dragon_Character_Rig.max rendered at 1920x1080. One frame is rendered. The render options set are Atmospherics, Effects and Displacement. The advanced lighting options are also set.

1. Install 3ds Max using default settings.

2. Select OpenGL if prompted for a software driver setting during the install process or if prompted when the application is first launched.

3. After installation is complete, reboot the computer.

4. Double click the icon for 3DS Max to launch the application. The window below should appear.

5. From the File menu, go to File  Open. This will open up the dialog below. Under c:\SEGs\3dsmax\Scenes\Characters\Complete, load the scene called Dragon_Character_Rig.max. You may need to copy the contents of the Scenes directory manually from your installation cd to c:\SEGs\3dsmax\Scenes\Characters\Complete prior to opening the file.

6. This will bring up the window below.

3D Rendering (2/2)

7. In the Render menu, choose the item for Render options. This will bring up the dialog below. Set the time output to Single and set the output size to be HDTV (video) which is 1920 x 1080. The options should appear as shown in the dialog below.

8. Next, Click the Render button in the bottom right corner of the dialog. The render progress dialog below will then appear.

9. Once rendering is finished, you will see the finished output in the window below.

10. Go to the File menu and click on Summary Info to get details of the scene just rendered. Look at the value under the heading Rendering where it says Last Animation Time. That is the time taken to render the scene. Record this value.

You can download a PDF version of this SEG here.

You can also download some photos to use as input data here as well. WARNING! This is a pretty big file (~188MB).

Video Effects

Adobe* After Effects* 7 software delivers a comprehensive set of tools to efficiently produce motion graphics and visual effects for film, video, multimedia, and the Web. The After Effects* 7 Production Bundle provides core compositing, animation and effects tools, 16-bit color support, vector paint capabilities, network rendering, and additional powerful keying, motion control, visual effects, 3D channel, and audio tools.

The workload applies different filters and effects to a variety of input file types including Photoshop* PSD, Illustrator AI and EPS, TIF, and QuickTime* MOV files. Some of the filters and effects include: blur, bulge, color key, frame blending, glow, motion blurring, fading, 2D and 3D manipulation, shadows, echo, median, radial blur, and invert. After each filter is applied, the composition is rendered to an uncompressed AVI movie file of the same resolution as the input file(s).

For this test, you will need to download and install Quicktime* from http://www.apple.com/quicktime/download/ and you'll also need a full commercial version of Adobe* After Effects* 7.

1. Purchase the Standard Bundle of Adobe* After Effects* 7.

2. Install After Effects* with the default settings.

3. Copy the supplied AE7_Workload-seg folder from the CD to your C:\ drive.

4. Reboot the computer.

5. When the system reboot finishes, start After Effects from the start menu,

6. Go to the file menu and select open project.

7. Type in the path for the file called test_all_AE70seg.aep in the file name box. Example: C:\AE7_workload\test_all_AE70seg.aep.

8. Click OK.

9. You should see the Render Queue window. Click on Render.

10. Click OK in the next 10 dialog boxes. The After Effects* 7 project will automatically generate footage files internally.

11. When the After Effects rendering has completed the time it took to do the test will be automatically written out to a text tile.

12. Close After Effects WITHOUT saving the project.

13. The rendering time text file is located in the folder AE Project Logs where your project file is located.

14. Open the text file and scroll down to the bottom. You will see Total Time Elapsed and then a time.

15. This is the total time it took to complete the After Effects* Workload.

16. Lower time means better performance.

17. To run the test again, delete all the files in the C:\AE7_workload-seg\Output directory.

18. Reboot the system and go back to step 5.

You can download a PDF version of this SEG here.

You can also download the input data for this SEG here as well.

Not the End, But the Beginning...

We're just getting started with bringing our collection of SEGs to you here at ICF. There are plenty more in the pipeline that use applications like XMPEG* with DivX*, Adobe* Premiere*, Windows* Media Encoder, iTunes*, and a whole bunch more.

We're always working on new SEGs in our labs, so if you have ideas about applications you think would make for interesting Software Evaluation Guides, please post in our forums. Stay tuned...

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.

*Other names and brands may be claimed as the property of others.