RightWare Launches Basemark GUI Free on Android Market - We Test It Out

Earlier this year RightWare launched a free version of the popular Basemark ES 2.0 benchmark free on the Android market in the form of Basemark ES 2.0 Taiji Free. The benchmark contains the Taiji subtest of that suite and offers benchmarking functionality at full resolution for Android smartphones and tablets. Today, RightWare is doing much the same thing with its Basemark GUI benchmark which we covered a year and a half ago, by releasing a slimmed down version called Basemark GUI Free on the Google Play Strore. 

Basemark GUI Free includes both the Vertex and Blend subtests, which are designed to test different areas of OpenGL ES 2.0 performance on Android devices. We're no stranger to this test, having seen it in original form a while ago. New and improved functionality in this version includes an offscreen version of each subtest which runs at 720p in addition to the native resolution run. The offscreen run displays small tiles of the currently rendered scene to verify that the scene is being drawn, just like GLBenchmark does. Results also get uploaded to RightWare's PowerBoard and presented (for example, see this HTC One X result page) alongside RightWare's other data. Check out the gallery for screenshots from the entire application. I should note that there's also a video of the full Basemark GUI run back in my original launch story. 

The Vertex and Blend subtests haven't changed at all since we originally saw them, but I'm reproducing RightWare's description of the tests in full below to give an idea what the workload is for each part. In general, Basemark GUI is a much more challenging and modern workload compared to Basemark ES 2.0, though it ends up being heavy on its geometry payload thanks to the Vertex test. 

Vertex Test:

The Vertex test loads vertex data from the disk, loads it into vertex buffers, displays the geometry and then discards unnecessary data after a frame has been rendered.

The Vertex test contains several load cycles of increasing complexity. A cycle contains a preprocessing step that is not scored, and the final scoring stage. The geometry is composed of 10 distinct city blocks which are each approximately 3000 triangles and 6000 vertices in size. In the preprocessing stage, a city layout is generated out of the blocks based on the complexity requirements of a specific cycle. The entire layout is then saved to disk into an indexed binary file. Although visually similar, each block has its own unique location in this file. Identical blocks are textured using the same texture which resides in the GPU memory throughout the test.

Once the actual scoring run starts, the city is moved towards the camera. Blocks that disappear beyond the plane of view are discarded from the memory. New blocks appearing at the visible horizon are read from the disk.

The test load is increased incrementally to a maximum load. The test load ranges from approximately 3k vertices per frame to 15k vertices per frame. The number of triangles on the screen ranges from 190k to 250k.

The scene geometry is rendered using a single texture material and fog. There are two lights. The OpenGL ES 2.0 shaders use per vertex lighting.

Blend Test:

The Blend test measures the efficiency of alpha blended rendering on the platform. Blending combines the incoming source fragment’s RGBA values with the RGBA values in the destination fragment stored in the frame buffer at the fragment’s x,y location. The active blend equation calculates the final RGBA value at the location from the input values. Overlay of graphical elements is a critical component of composition operations.

Overlay draw renders a large set of simple primitives. The number of overlain primitives will increase through the duration of the test up to a maximum specified count. The test is rendered twice: First with front-to-back ordering of the image planes and then with back-to-front ordering.

Results and Analysis:

I've gone ahead and run Basemark GUI Free on a fairly representative combination of devices, same as what I did in the GLBenchmark 2.5 piece, for comparison. 

Here we have two Samsung SoCs paired with the ARM Mali-400 MP4 GPU: the original dual-core Exynos 4210 as well as the Exynos 4 Quad (4412) from the international Galaxy S 3 which we're reviewing. Tegra 3 makes an appearance in the HTC One X International, as does TI's OMAP 4460. The latter shows up in two configurations, one with the SGX 540 running at full 384MHz clocks in the Huawei Ascend P1 and another with the SGX 540 running at 307MHz in the Galaxy Nexus (CPU clocks are 20% lower as well in the Galaxy Nexus). Finally we have Qualcomm's dual-core Snapdragon S4 with its Adreno 225 GPU in the HTC One X and the US Galaxy S 3, and Adreno 205 inside the HTC One V which I tossed in as well.

In the offscreen tests, both Adreno and Tegra 3 do very well, posting impressive numbers. In addition, SGX540 does surprisingly well. What's surprising is to see the Exynos based SoCs not do as well here, until you realize that the Vertex test is (hence the name) strongly dominated by vertex data, it isn't the kind of shader compute test we're used to seeing with GLBenchmark. Recall that Mali-400 isn't a unified shader architecture, and explicitly features more pixel shader hardware than vertex shader hardware. 

Because the end score (in FPS) for on and off screen is the average of both subtest scores, it's hard to be completely certain that the large vertex workload is the reason for Mali-400MP4 doing how it does in the above tests, but it's a logical conclusion. Of course, the licensed version of Basemark GUI not on the Play Store does expose data from individual subtests. 

The native resolution tests largely just show that a number of the 720p devices are actually at vsync for a large part of both test, though portions below 60 FPS end up bringing the average down slightly.

Unlike Basemark ES 2.0 Taiji Free, Basemark GUI Free includes that offscreen 720p test, which makes it a much more functional tool for end user comparison across devices. I'm told that future RightWare benchmarks in the Play Store will also include offscreen subtests, which is a great step in the right direction. 

Source: Google Play Store (Basemark GUI Free), PowerBoard (HTC One X)