The slide whizzed by during a Google Cloud Next keynote address last month in San Francisco, perhaps a little too quickly for the more than 30,000 in attendance to process the bold statement it contained: that Google’s cloud computing service was the most reliable in the business during 2018. That was probably the idea.
The two analyst firms cited by Google representatives in defense of its claim that its cloud service was out of commission less than Amazon Web Services in 2018 — industry stalwart Gartner and newcomer Krystallize — were unwilling to fully support that conclusion in recent interviews with GeekWire. During Google Cloud Next in April, Google’s Urs Hölzle claimed its cloud service was down for 208 minutes and that AWS was down for 312 minutes in 2018, attributing that claim to “two leading third-party research firms” and going on to say “this research was neither sponsored nor influenced by Google.”
Watch for yourself here:
However, when later contacted by GeekWire, Gartner’s Raj Bala said that his company’s data showed the opposite: AWS was the leader in global uptime during 2018, available 99.9987 percent of the time, compared to Google’s uptime score of 99.9982 percent. That is the tiniest of gaps between the two companies, to be sure, but Google still trailed the cloud leader by a hair, contrary to Hölzle’s claim that “GCP demonstrated, by a substantial margin, the highest reliability of the three major clouds. It’s not even close.”
In the very, very, very small print that accompanied its slides, Google said the data was “compiled based on downtime data from multiple industry sources.” Krystallize was the other “leading third-party research firm” cited by Hölzle, according to Google representatives, but Krystallize director of partnerships and strategic alliances Caitlin Rice said in an email response to an inquiry that “the data presented in the slide you sent over does not reflect the source data we have shared with Google.”
Google declined multiple requests to explain the methodology behind the slide.
The marketing departments of Silicon Valley have been working magic with statistics since, well, forever, and this is not the first time Google has tried to make an argument it runs the tightest ship in cloud computing. Back in 2017, then-Google Cloud CEO Diane Greene tried a similar tactic in the wake of a pretty bad AWS outage, citing a firm called Cloud Harmony in defense of the notion that Google had the best uptime metrics.
In an interview, Krystallize founder and CEO Clinton France said that real-world evaluations of cloud computing performance are trying to move beyond the zero-sum notion of “uptime” in favor of metrics that emphasize consistent performance. To that effect, Google topped (by a slim margin over AWS) a recent Krystallize assessment of “service capability measurement,” which gauges how cloud service providers “continue (to) provide continuous, high quality, cost predictable service regardless of location, maintenance, patching, noisy neighbors, outages, upgrades, lifecycle management and other service changes outside of the enterprise customers hands.”
Gartner’s Bala also noted that Google tends to perform well in tests of networking availability. “They tend to have the most capacity of any of the providers worldwide,” he said, citing years of investments in undersea cables and data centers to support Google’s search traffic.
However, it would appear Hölzle was not allotted enough time in his keynote to get into such distinctions. The slide containing the data also mentioned in the fine print that “average downtime data includes planned and unplanned downtime,” which would appear to be a plug for Google Cloud’s live migration update service that patches systems without requiring a reboot, but the company declined to answer several specific questions regarding whether that service factored into its data.