Leaks of new hardware and software are a staple of online technology publications, which includes XDA-Developers. Every leak should be treated with some skepticism, but leaks originating from benchmarking services even more so. There’s no way to tell if the device details listed in a benchmark are real, so it’s best to ignore benchmark leaks when there’s no corroborating evidence. Unfortunately, there are dozens of popular online publications that still publish articles based on benchmark results, whether it be to drive clicks or because they actually believe the result is really from an unreleased device. The latest benchmark leaks to fool dozens of online tech publications are two Geekbench listings that show off a “POCO F1 Lite” allegedly code-named “uranus.”
I’m not going to bother talking about the specifications of this “POCO F1 Lite” or whether it makes sense for POCO to launch a mid-range product, because I actually don’t know if POCO is preparing to launch a mid-range device. They very well might be, and we all know that Xiaomi loves releasing new devices at multiple price points. What I do know is that these Geekbench listings (the device details, not the scores) and the “uranus” code-name were fabricated. I’m in a few Telegram groups full of meme-loving custom ROM developers, and on March 13th one of them had the bright idea to “wreck the memebois with [a] Xiaomi uranus benchmark.” They wanted to see how easily they could fool people into believing a Xiaomi device code-named “uranus” was in the works, and that it would be the Xiaomi POCO F1 Lite. All they had to do was modify a few system build properties and run Geekbench. The Geekbench score itself is legitimate, but the device the benchmark was run on was most certainly not a “POCO F1 Lite.”
The problem of faked system properties showing up in benchmark results has long been known. This AndroidPit article from late 2016 talks about how easy it is, and you’ll often see more skeptical users call out obviously fake benchmarks. Yes, obviously that’s not a real POCO F2 result. That’s actually my rooted Google Pixel 2 XL running the Android Q beta. But even skeptical users can still fall for these fake device listings, as evidenced by the fact that the user who called out my fake POCO F2 benchmark is the same SlashLeaks contributor who submitted the fake POCO F1 Lite benchmark.
I spent a couple of minutes faking the above listings just to show how easily it can be done. I didn’t even bother properly faking the CPU info, because doing so requires a little more effort than just modifying a few system properties. My fake POCO F2 benchmark has already been picked up by two English-language and two German publications, and there’s probably more coverage out there that I haven’t spotted yet. If I really put effort into this, what’s stopping me from rooting a Xiaomi Mi 9 and passing it off as an LG V45 ThinQ, Xiaomi Mi Mix 3S, Google Pixel 4 XL, OnePlus 7, Samsung Galaxy Note X, or any other device that we expect to have the Snapdragon 855? Absolutely nothing. We even debated doing it on our review unit, but ultimately decided against it because we didn’t want to contribute to spreading fake news. (Sadly, my obviously fake POCO F2 benchmark already got picked up by some places.)
Does that mean all benchmark leaks can’t be trusted? No, it is possible for details of an unannounced device to leak through a benchmark listing if someone with the actual device ran the benchmark. However, there’s no way to verify that someone who has the device actually ran the benchmark producing the online listing. If a benchmark is the only evidence of a new device, then it should be disregarded. That’s why we ignored the Google Pixel 3a XL Geekbench listing that popped up in late January. We treated it as fake until that same marketing name appeared in a Google Camera library in the Android Q beta. It’s very unlikely for the Pixel 3a XL reference in Android Q to have been faked (unless Google is playing 4D chess at our expense), but it’s really easy to fake a Pixel 3a XL Geekbench listing as I’ve shown above.
When we leak details on a new device, we always have more reliable sources of information to back up our report. Persons with knowledge of or access to the device or unreleased firmware builds are how we usually obtain credible information. There’s no evidence corroborating the existence of a POCO F1 Lite, so this Geekbench listing should have been disregarded even if we didn’t literally watch a custom ROM developer plot to fake it. If we publish information on new Xiaomi devices, you’ll know that our information came from sources that can’t be faked.
Stop Trusting Benchmark Leaks
So what spurred me to write this article? As I’ve said, we usually ignore benchmark leaks because we don’t trust them. There’s a lot of uncertainty about if a Geekbench test was run on the hardware the listing claims it to be, but this time we knew it wasn’t so we’re using the widespread coverage of the fake POCO F1 Lite benchmark to remind you not to trust uncorroborated benchmark leaks. I wish this fake POCO benchmark was only picked up on smaller tech blogs, but sadly that’s not the case. There are dozens of publications and YouTubers with hundreds of thousands or millions of followers that have covered this “leak” cooked up by one bored custom ROM developer.
Alvin Tse, Head of POCOPHONE Global, gave a soft denial of the POCO F1 Lite on Twitter. Despite this, one publication said that “evidence suggests otherwise,” citing the Geekbench listing as their evidence. Several of India’s leading online technology publications fell for the post. A few publications and social media sites popular in Europe and America also fell for it. Many of these posts were couched in a careful manner, but how many people read beyond the headlines? Some of these sites know that these benchmark leaks might be fake, but still choose to cover them anyway. Doing so only hurts their credibility. How can we trust a future article claiming to have exclusive information if the publication is known to fall for such obvious fakes? This latest fiasco has certainly opened my eyes about how many publications still fall for these fake leaks, even if we’ve known for years how easy it is to fake the device details in Geekbench’s database.
The custom ROM developer who admitted to faking the POCO F1 Lite listing asked us not to name him because he doesn’t want to be flooded with messages asking him why he did what he did.
Refference - xdadevelopers.com
0 Comments