How We Test
How We Test Laptops
The laptop is set to run at full performance by setting the power scheme set to ‘High Performance’ with the AC plugged in for all benchmarking activities other than battery usage. This means that the CPU will not underclock while running the tests. For the battery performance test, the power scheme is set to ‘Power Saver’. The battery performance test is performed 3 times in varying levels (Full, Med, Low) to ensure accuracy of lifespan. Before each test is run, the laptop is rebooted, fully charged and its hard drive defragmented.System Performance Testing
PCMark Vantage / PCMark 05: For systems that run Microsoft Vista: PCMark Vantage is a hardware performance benchmark for PCs running 32 and 64 bit versions of Windows Vista®. PCMark Vantage is perfectly suited for benchmarking any type of Windows Vista PC from multi media home entertainment systems and laptops to dedicated workstations and hi-end gaming rigs.
For Systems that run Microsoft XP: PCMark 05 Advanced Build 1.2.0* is the Futuremark’s popular overall system benchmarking program. The 2005 version adds multithreading, DirectX 9, Windows Media Player 10, virus scanning, High Definition video playback (WMVHD), and a vast number of other tests to its suite. Testing your computer’s CPU, RAM, hard drive and graphics card, PCMark05 drives your computer to the max to determine its strengths and weaknesses.
PC World WorldBench 6 is the sixth revision of PC World’s well regarded full-system benchmarking suite. Worldbench runs a slew of benchmarking tests designed to simulate many common programs used by today’s typical user. Programs benchmarked are Adobe Photoshop CS2, Autodesk 3ds max 8.0, Firefox 2, Microsoft Office 2003 SP1, Microsoft Windows Media Encoder 9.0, Nero 7 Ultra Edition, Roxio VideoWave Movie Creator 1.5, and WinZip 10.0. WorldBench 6 scores are baselined against a test system consisting of a 2.4GHz Intel Core 2 Duo E6600, 2GB DDR2 RAM, nVIDIA GeForce 7900GS graphics card, and 2 x 320GB Western Digital hard drives in RAID-0, with Microsoft Windows Vista Home Premium installed as the OS. After each test has been run, the system clears temp files and reboots.
Adobe Photoshop CS2: A small, non-complex image opens and the software applies and undoes each set of artistic filter enhancements (11 sets in all). After this has completed, a much larger and complex image is loaded and the same filters are applied, at a much slower rate as the image takes longer to manipulate due to its size. After these tasks have been completed, the benchmark ends.
3ds max 8.0: A file is opened showing a wireframe of an underwater scene, which is rendered by the software, with a cartoon-looking lizard appearing in a second window in the background towards the very end. As soon as this rendering is complete, a three-dimensional mouse bearing more than a close resemblance to Stuart Little is quickly rendered, and as this completes, a dragon in its environment is rendered. Once these are all finished, the benchmark concludes.
Microsoft Office 2003 SP1: A new word document is opened, and an existing Access database is converted to Access ’97 format. After the conversion occurs, the tables are examined. An Excel spreadsheet showing multiple pages of financial information is opened and the data is manipulated by changing formulas and functions. The new Word document opened at the onset is then accessed, and converts an RTF document into Word format. Once the conversion has finished, a spell check is initiated on the 180-page document, and the benchmark ends after that has concluded.
Firefox / Windows Media Encoder 9.0 Multitasking: Windows Media Encoder is opened and a new project is started. As the project encodes, the PC World website opens and the benchmark begins navigating to numerous articles/links on the site. Once the encoding process completes, the benchmark concludes.
Roxio VideoWave Movie Creator 1.5: A new storyline is opened to make a video of a user’s vacation he recently filmed. Several clips are imported into the program and each one is rendered individually. After the fourth clip renders successfully, the benchmark ends.
• Nero 7 Ultra Edition: This benchmark simulates writing a DVD data disc and saving as an NRG image file. The test is repeated three times before the benchmark is finished.
WinZip 10.0: The system takes a group of files and compresses them within WinZip. Once this has been performed, the benchmark concludes.
3D Performance Testing
3DMark Vantage (Windows Vista)3DMark® Vantage is the new industry standard PC gaming performance benchmark from Futuremark, newly designed for Windows Vista and DirectX10. It includes two new graphics tests, two new CPU tests, several new feature tests, and support for the latest hardware. 3DMark® Vantage is based on a completely new rendering engine, developed specifically to take full advantage of DirectX10, the new graphics API from Microsoft. Newly Supported Tests: There are four main tests in 3DMark® Vantage, all completely new. The two graphics tests sport significantly advanced visuals, enabled by the new technology and performance available in the DirectX10 generation of graphics hardware. The two CPU tests have been re-designed from scratch, and now feature a more complete spectrum of Physics and Artificial Intelligence-related computation. CPU Test 2 (the Physics Test) features support for physics acceleration hardware, and a workload to match future generation game physics.
3DMark06 build 1.1.0 (Windows XP) uses advanced real-time 3D game workloads to measure PC performance using a suite of DirectX 9 3D graphics tests, CPU tests, and 3D feature tests, including HDR/SM3.0 graphics tests, advanced SM2.0 graphics tests, AI and physics-driven single and multiple core CPU tests.
Gaming Performance Testing
NOTE: Only Laptops with Dedicated GPU's will incur the Gaming Performance Testing section.
Each game benchmarked at Laptop Logic will be tested twice: first, at 800x600 resolution with default settings, and then at 1024x768 resolution with default settings.
Games will be played for 15 minutes at each resolution in predetermined in-game locales to best simulate a gaming experience, while running Fraps (http://www.fraps.com) in the background to measure the frames-per-second performance. At the end of the 15 minute period, minimum, maximum, and average frames-per-second (FPS) will be noted and displayed in a line graph.
F.E.A.R. is one of the most popular First Person Shooters in recent memory. A paramilitary force infiltrates a multibillion dollar aerospace compound taking hostages, but issuing no demands. With U.S. Special Forces in over their heads, the Commander in Chief turns to his final option: you. You've been trained to handle the inexplicable, armed with experimental high tech weaponry, and given full executive authority to end the crisis by any means necessary.
An advanced Special FX system showers you in sparks, smoke, and debris, making combat as intense and exhilarating as an action movie. The latest in DirectX 9.0 rendering technology uses real-time per pixel lighting, shadow volumes, normal mapping, and advanced shaders to create a world so convincing you'll forget you're playing a game. Employing Havok 2 Game Dynamics, unleashing hell on your enemies is visceral and satisfying. Bodies recoil and collapse accurately and objects react to being shot, pushed, or blown up.
Crysis SP Demo: A team of US scientists makes a frightening discovery on an island in the South China Sea. All contact with the team is lost when the North Korean Government quickly seals off the area. The US responds by dispatching an elite team of Delta Force Operators to recon the situation. As tension rises between the two nations, a massive alien ship reveals itself in the middle of the island. The ship generates an immense force sphere that freezes a vast portion of the island and drastically alters the global weather system. Now the US and North Korea must join forces to battle the alien menace. With hope rapidly fading, you must fight epic battles through tropical jungle, frozen landscapes, and finally into the heart of the alien ship itself for the ultimate Zero G showdown.
Benchmark Tool 1.05 Final: The tool provides a robust front-end to benchmark Crysis. It provides the ability to queue up many runs and will provide detailed results for each test as well as an overall summary with accurate averages. The first and foremost goals were simplicity and reliability in the tool and in the benchmark scores. Depending on hardware configuration, it will be set to 800x600, 1024x768, and 16XX x 10XX (dependant on monitor max resolution) - notable changes will be marked and recorded to provide accurate test results.
Unreal Tournament 3: Unreal Tournament 3 is a First person Shooter featuring intense action, graphics, and features a compelling single player experience that follows the adventures of Ronin, a team of mercenaries out for revenge on those who destroyed their home colony.
Benchmark Tool 0.2.1.0: The tool provides a robust front-end to benchmark Unreal Tournament 3. It provides the ability to queue up many runs and will provide detailed results for each test as well as an overall summary with accurate averages. The first and foremost goals were simplicity and reliability in the tool and in the benchmark scores. Depending on hardware configuration, it will be set to 800x600, 1024x768, and 16XX x 10XX (dependant on monitor max resolution) - notable changes will be marked and recorded to provide accurate test results.
Battery Performance Testing
Laptop Logic has chosen to take a “real-world” approach to battery testing, eschewing a more formal software-based benchmark. There are four components of Laptop Logic’s new battery performance testing: DVD playback, general use, gaming, and recharge time.
DVD Playback is just as the name suggests. A full length 3 hour DVD will be played on the system (in either Windows Media Player or PowerDVD or whatever DVD playback software ships with the laptop). The movie will be played on loop in case the laptop’s battery exceeds the film’s runtime. At the beginning of the test, the editor will record the start time with a PC-based stop watch program and note when the test concludes.
Blu-Ray Playback is just as the name suggests. A full length Blu-Ray movie will be played on the system (in either Windows Media Player or PowerDVD or whatever DVD playback software ships with the laptop). The movie will be played on loop in case the laptop’s battery exceeds the film’s runtime. At the beginning of the test, the editor will record the start time with a PC-based stop watch program and note when the test concludes.
Gaming: The editor will use one of the above games and mark the time it takes for the Laptop to power down while playing the game. The laptop is set to "High Performance" with Full-LCD brightness, 75% volume, wireless enabled, and no underclocking of the video card.
General Use will simulate a normal user experience, with tasks similar to those commonly performed by an average person. The test is broken down into 3 categories. The editor will set the following:
Full: Brightness, Volume, Wireless Enabled, Bluetooth Enabled (if available)
Medium: Brightness set to 75%, Volume at 75%, Wireless enabled, Bluetooth Disabled (if available)
Low: Brightness set to 35%, Volume muted, Wireless Enabled, Bluetooth Disabled
General Use will have several different tasks running simultaneously. They will have the Laptop Logic forums open in the background, and perform random web browsing, picture editing, file transfer and text editing. At the beginning of the test, the editor will record the start time with a PC-based stop watch program and note when the test concludes.
Recharge Time: will measure how long it takes for the system to recharge itself from 0% to 100%.
How We Test Netbooks
wPrime: Uses 100% of the CPU while in use. This is achieved by using CPUz to detect the CPU count and use exactly that many processing threads to avoid any performance losses due to multiple threads running on any single physical thread. Each thread is designed to do 1/n of the work, where n is the number of threads. For example, if you're calculating 16 roots on 4 CPU's, each CPU will calculate 4 roots. Some might argue that this style of threading is unrealistic in real-time performance, but in fact is quite indicative of performance in several real world tasks.