Yesterday, Samsung was caught artificially boosting the Galaxy Note 3’s benchmark scores with a special, high-power CPU mode that kicks in only when the device runs popular benchmarking apps. Similar cheating was discovered by AnandTech for the Galaxy S4 back in July as well. Today, they have released a new report that reveals almost all Android smartphone makers are doing the same.
It seems that with the exception of Apple and Motorola, every single smartphone maker has shipped at least one device that runs CPU optimization for manipulating benchmark results. Looking at the list, popular Android smartphones like HTC’s One and One mini, LG G2, Samsung’s Galaxy Tab 10.1, Note 3 and S4, all cheat with popular benchmark apps. Google’s Nexus 4 and Nexus 7 smartphones are however not using this method.
We started piecing this data together back in July, and even had conversations with both silicon vendors and OEMs about getting it to stop. With the exception of Apple and Motorola, literally every single OEM we’ve worked with ships (or has shipped) at least one device that runs this silly CPU optimization.
The other funny thing is the list of optimized benchmarks changes over time. On the Galaxy S 4 (including the latest updates to the AT&T model), 3DMark and Geekbench 3 aren’t targets while on the Galaxy Note 3 both apps are. Due to the nature of the optimization, the benchmark whitelist has to be maintained (now you need your network operator to deliver updates quickly both for features and benchmark optimizations!).
AnandTech adds that unfortunately, this is all going to get a lot worse before it gets better. There are thankfully “ways of continuing to see and understand what’s going on” inside these closed platforms.
You can read the extensive report here.
