AUSTIN – Pinterest was one of the first clear cloud-computing success stories: now valued at over $12 billion, the massive site launched on Amazon Web Services and has never looked back. But the cloud has changed since Pinterest first put into place the technologies and policies that allowed to scale to over 200 million monthly users, and the company is now in the middle of a two-year project to reorient itself around containers and Kubernetes.
Micheal Benedict, product manager for cloud and data infrastructure at Pinterest, walked a packed session at CloudNativeCon and KubeCon Wednesday through Pinterest’s decision to shift its infrastructure from virtual machines to containers. The company also officially joined the Cloud Native Computing Foundation as an end-user member, alongside other big web operations like Box, Twitter, and Github.
Pinterest is definitely one of the quieter big tech companies, perhaps in part because the average Silicon Valley bro probably doesn’t use it. But Pinterest has seen huge growth since it was founded in 2010, and it’s now using all the data that users have pinned to its service to power a visual search engine, which brand advertisers are certainly watching.
In 2016, Pinterest decided — after a fair amount of evaluation — that it needed to move its production infrastructure to containers, Benedict said. One of the biggest developments in enterprise computing this decade, containers allow software developers to package their apps in small lightweight packages that can be run across different environments and squeeze even more performance out of hardware than virtual machines.
The company had built a lot of custom tools in order to improve the performance and reliability of its site, and that starts to get really hard at scale, Benedict said. Those custom tools are often one-off projects that don’t necessarily work well with modern concepts, and given the degree of engineering turnover in Silicon Valley, companies can run into problems down the road when they need to fix something written by an engineer that’s been gone for years.
“It’s hard to migrate those workloads off these systems because we lack the context of how they work,” Benedict said. Containers give Pinterest a lot of the same benefits as those custom tools in a standard format, and while he said the company has no plans to move any workloads off AWS, containerizing one’s applications at least allows the company to have the discussion.
That shift to containers is nearly complete, and right now Pinterest is putting its Kubernetes strategy into play. By the middle of next year, Pinterest hopes to be using Kubernetes to manage clusters of containers in production, starting with some non-critical workloads that tend to be predictable, Benedict said.
When it’s all complete, the project will simplify life for Pinterest’s software development teams and give the company better tools for controlling and managing its infrastructure, Benedict said. As I talked about with Slack’s Julia Grace a few weeks back, this two-year process is the new normal for any big software company: figuring out how to rebuild the engine while still flying the plane.
