What we want to learn from benchmark-apps

What we want to learn from benchmark-apps

AnTuTu, Geekbench and Vellamo provide comparable values. But what really say benchmarks? We want to revise the performance section of our test. Now we invite you to join: help us in finding the best benchmark-apps and practices.

The fastest Smartphone processors compared
These are the fastest Android Smartphone

The computing power of modern smartphones is now sufficient for amazing items: 3D games in the VR Head mount, image processing in real time, video recordings in 4 k resolution, parallel analysis of several sensors, multi-tasking between complex applications without loading, downloading large amounts of data from the mobile data network with 300 Mbit / s.

However reliably to tell whether a Smartphone is faster than another, need standardized tests. These are issued in the form of benchmarks by various developers forging. AnTuTu is one of the better-known. The Chinese developer with stress tests, selling it to phone manufacturers make their money.

Moto x style antutu
The values from AnTuTu 5.x are now worthless. / © ANDROIDPIT
The normal user knows it from the AnTuTu benchmark. The play store app is the probably most well-known benchmark for comparing performance of Android and iOS devices. Unfortunately, we had to decide to remove them from the benchmark course. While the measurement data with the online database promised a good comparability, the frequent app updates ran counter to exactly this advantage. Time and again, the scale shifted so that the measured values of a test device differed from one day to the other 50 percent, because AnTuTu has been updated.

This put us in a tight spot and we went on the search for alternative methods for the performance of the test equipment. Basic questions to did up: what do we know? What makes a good benchmark app? Here, we determined the following main criteria:

The measured values must remain in the long term comparable
The app must be free readers available
In addition important subcriteria are characterized, which relate to benchmark details. I would like to explain this example PCMark for Android. For each benchmark app arrives on specific measures and thereby hides important other values.

PCMark for Android benchmark
The benchmark PCMark for Android will cover a wide range of realistic applications. It simulates in image processing, video playback, Web browsing and text editing. Each category is assessed individually with points and it is expected a comparable total score which you match with other devices in the online database.

PCMARK1
PC mark for Android provides standardized scores. / © ANDROIDPIT
PC mark includes a battery test. For this, the app runs the work test in loop until the battery level from 80 to 20 percent. The result is displayed in hours and minutes. Unfortunately you have to take care of then myself, that the display on a standardized brightness is regulated. Incoming calls can abort the test, messy background apps, temperature, age of the equipment or signals from Wi-Fi or LTE can influence the outcome.

PCMARK2
PC mark can stress test also your battery. / © ANDROIDPIT
A new part test, which you can download separately, is the memory benchmark. This measures the speed of the internal memory. Right here arise often significant differences in performance between two otherwise similar equipped smartphones. In particular the HTC 10 has disappointed here compared to the Galaxy S7 as slow memory installed HTC. So do you have a good performance in games while in both devices, you have previously but long loading times for the HTC 10.

PCMark for Android benchmark
Install on Google play
What we learn and what is missing
The PC market for Android delivers useful insights based on where smartphones can be compare with its components. Thanks to the points results results fast, easy comparability, which can easily understand each of you. The problem: The benchmark is too simple.

Not all the information we need can be found in the benchmark details: how many processor cores? How fast is the RAM? How fast does the graphics chip? Is the browsing benchmark also depends on your Internet speed?

Galaxy s7 nucleo performance
Vellamo provides excessive details about benchmarks. / © ANDROIDPIT
Can be found many of those details among other things the Vellamo benchmark of the processor producer Qualcomm. In turn, that app has the disadvantage that their tests only mathematical problems and theoretically deal with everyday tasks.

Gaming performance is more visible than measurable
Games benchmarks have a different concern. Their disadvantage is that they reflect not what happens in a real gaming apps and with real 3D engines. Because games like real racing do not scale with the frequency of frames drawn per second (fps). You scale the level of detail of the displayed material, to achieve a particular fps target.

real racing graphic details difference
The Galaxy S6 (o

Tweet