There’s an old joke in here about statistics, but I’m not going to be the one to resurrect it. What I will say is that what you see isn’t always what you get, and that seems to be the case with the much ballyhooed news that Google’s Chrome has now surpassed Microsoft’s Internet Explorer in worldwide use. The issue? It’s almost certainly not true.
When Ireland-based StatCounter in June released its attention-grabbing report about Chrome’s rise, U.S.-based competitor Net Applications challenged the findings. The company, which operates a tracking application called NetMarketShare, makes a convincing case that StatCounter’s methodology isn’t robust enough to give an accurate account of worldwide browser usage.
StatCounter reported that as of May Chrome is now in first place with 32.43% of desktop users, compared with IE’s 32.12%. But as Net Applications’ Vince Viccarazzo, EVP marketing and strategic alliances, points out, that really depends on how you count users, given a sample size. There are two key ways in which this matters. StatCounter counts page views rather than what in the analytics world is considered the more statistically accurate unique visitors. (The former counts the same person carrying out an action multiple times, while the latter counts that person once. For this reason, the page view metric generally inflates numbers.)
Second, StatCounter does not adjust its data for what’s called geo-weighting (in this case, population size). As Net Applications explains, the exclusion of geo-weighting results in the tiny country of Ireland being counted equally with the behemoth that is China. Microsoft, which has some pretty obvious skin in this game, also responded with a detailed online post (although this was before the May bombshell) showing how StatCounter’s numbers are flawed without the inclusion of geo-weighting.
If you adjust for country size it turns out that for May IE remains well ahead in the desktop market at 54.05% with Chrome trailing at 19.58%, according to NetMarketShare data.
StatCounter, which didn’t respond to a request for comment, does try to explain itself, but the reasoning seems shaky. For example, its web site says geo-weighting isn’t used because the CIA Factbook data used by others like Net Applications is from 2009 and therefore out of date. It further argues that its use of page views is more accurate than using unique visitors because the former method tracks “how frequently browsers are used. …”
But as Net Applications points out, you don’t have to use the CIA’s numbers—you can use any other more up-to-date numbers. At the very least, you can project the growth rate and use those numbers instead. As for page views, that metric has long fallen out of favor in business because of the aforementioned overcounting and its susceptibility to manipulation such as from bots generating phantom traffic. (Viccarazzo notes that Net Applications previously used only page views up until 2008 when it changed its methodology to foil bot attacks.)
So why has StatCounter’s news been greeted relatively uncritically in the media? Schadenfreude, perhaps. While many might not admit it, some in the tech world have been angered by Microsoft’s actions on issues around competition and innovation, and take a certain delight in watching the company suffer a kind of comeuppance. And it makes for sexy headlines, too.
Viccarazzo gives his competition credit though. “It’s intelligent timing on their part to provide this story that’s going to nationally carry a lot of weight and a lot of intrigue, and then at the same time come out with a comparative analysis of us and try to discredit our numbers. It was well done.”
But when you step back, the idea that Chrome has surpassed IE seems dubious on its face. Windows still commands an over 80% share of PC operating systems and IE ships pre-installed with each copy of Windows (although not exclusively in the European market as of Windows 7). This makes for a strong disincentive for the casual user to switch browsers. Certainly that’s what the European Union and Mozilla Foundation (makers of Firefox) chief Mitchell Baker suggested, and not without reasonable cause. In that context, for Chrome to catch and surpass IE within the four years since its 2008 release is almost completely implausible.