Statsig is moving fast to keep pace with the latest advancements in generative artificial intelligence and data storage technology, rolling out new features that help developers test and evaluate products with live customers.
The Seattle-area startup was founded in 2021 by former Facebook engineering executive Vijaye Raji, inspired by the social network’s internal tools for experimenting with new features.
The company announced Wednesday the release of Statsig Warehouse Native, a platform that runs on an organization’s data warehouse. The tool helps product teams run A/B testing and product analytics, with insights based on the company’s internal metrics. It integrates with data platforms like Snowflake, BigQuery, Redshift, and Databricks.
The launch is based on the insight that specific customers, such as those in finance and health-related sectors, prioritize privacy and prefer to keep their data on-premises, Raji told GeekWire.
He said data warehouse capabilities have significantly improved in recent years, offering a wider range of capabilities and functions. Statsig released its new warehouse experimentation tool in order to keep pace with these advancements.
“We want to be where people are gravitating toward,” said Raji.
The launch of Statsig’s warehouse functions come on the heels of its release of generative AI experimentation features. The tools help companies test AI model costs, latency, and other performance metrics. Raji said that “top AI companies” are already using the platform, but did not disclose specific company names.
The startup said it now has “hundreds of paying customers,” as well as “thousands of active users” enrolled in its free tier. Notable clients include Microsoft, Notion, Brex, Vanta, Flipkart, Cruise, Univision, Bolt, Headspace, and others.
Statsig has grown to a workforce of 65, up from about 30 in 2022. The startup raised $43 million last year from Silicon Valley’s Sequoia Capital, with participation from Seattle’s Madrona Venture Group. Statsig raised a $10.4 million Series A round in 2021. Total funding is $53 million.
Raji previously led Facebook’s engineering outpost in Seattle. His tenure at Facebook included stints as vice president of gaming and vice president of entertainment. He previously worked at Microsoft for nearly a decade and was a principal software design engineer.
In a recent profile on Sequoia’s blog, Sequoia partner Mike Vernal called Raji “the most impactful engineer that I’ve ever worked with.”
We caught up with Raji to chat about Statsig’s growth, how it balances enterprise and startup customers, new features for generative AI, and his contrarian take on AI hallucinations. Read on for takeaways from our conversation.
Statsig initially focused on smaller customers, but the product gained a reputation as a sophisticated experimentation tool among developers, opening its customer base to larger clients.
- Raji said much of the startup’s journey has been “peeling back layers,” finding new use cases for its tech and serving those customers.
Both startups and large companies can use Statsig to find areas to make product changes.
- “Imagine that you launched a new feature,” Raji said about early-stage companies. “You don’t need millions of samples to tell you something’s broken, you just need tens of samples.”
- Large companies, equipped with vast data sets, can use Statsig to identify opportunities for making incremental changes, Raji said. Companies like Amazon operate with millions of samples and seek even the slightest improvements — a 0.1% revenue enhancement can be a “huge deal,” he said.
To test AI models, Statsig lets companies control the level of randomness in generated text and apply frequency penalties to manage repetition of words and phrases.
- The idea is to help companies identify the optimal settings that align with their internal objectives, such as boosting engagement or revenue, Raji said.
- Statsig aims to develop prompt engineering experimentation tools that let companies assess the most effective combination of words for achieving their desired outcomes.
Raji recently wrote a post on LinkedIn outlining how AI “hallucinations” can actually be a good thing.
- “We say AI is hallucinating when it makes up something that it’s never explicitly encountered or seen before,” Raji wrote in the post.
- These random synapses can actually generate new and elegant insights that may not have been apparent in theory before, Raji said.
- Statsig is experiencing this first-hand. AI is able to analyze documentation from various engineering teams and come up with inferences that the engineers would have missed.
- “I wanted the post to be a little bit of a contrary take,” he said. “Everyone thinks hallucinations are bad, but I don’t think that way.”
