Organizations are increasingly turning to Artificial Intelligence to solve business problems and make workflows more efficient. However, integrating and scaling AI comes with various challenges. While building a proof of concept is a comparatively simple task, scaling it to production is another thing altogether — not to mention maintaining it once it’s live. When you combine that with the growing complexity of an AI system and the naturally occurring entropy of data drift that compromises performance over time, the result is less than optimal.
However, industry experts like Chhaya Methani, Principal Applied Science manager at Microsoft, have found success in scaling AI systems without hitting these pitfalls. Armed with a few key strategies, Chhaya is adapting AI systems across a variety of enterprise applications while minimizing risk and keeping customers happy.
Learn more about the challenges of successfully scaling Artificial Intelligence and how Chhaya is overcoming them.
The Challenges of Scaling AI for Enterprises
Most organizations want to adopt Artificial Intelligence in some form or another, but they often find it hard to go from proof of concept to a production-ready system. One reason for this is the sheer plethora of models to choose from, making it difficult to choose the right one for your needs. To further complicate things, each component in the planned system may need a different model. The eventual system architecture must also be flexible enough to support AI’s rapid innovation.
Another challenge is the fact that many AI platforms (especially LLMs) are black-box systems — that is, their inner workings are buried behind layers of complex and proprietary code, making it difficult for even seasoned professionals to fully control them, trust their outputs, or customize them for a specific use case. This lack of transparency can lead to various business risks, like compliance issues or errors in prompt responses.
Finally, these models (whether for text generation, image recognition, or data prediction) must be able to handle diverse inputs. But given their probabilistic nature — meaning they make predictions based on likelihoods rather than fixed rules — they may produce inconsistent or incorrect results, with errors and biases quickly spreading without careful oversight. Thorough testing can alleviate these issues, but exhaustively testing every scenario at such a large scale is unfeasible.
This is where AI experts like Chhaya Methani come in, providing key insights for scaling these systems without sacrificing quality, productivity, or efficiency.
Strategies to Scale AI Systems
Chhaya Methani has over 14 years of experience in Artificial Intelligence and Machine Learning, and she’s intimately familiar with the challenges of effectively scaling AI. She’s contributed to the orchestration engine that powers Microsoft Copilot Studio’s ability to create autonomous agents, and she led the development of natural language to code generation models, empowering users to create sophisticated automated workflows with plain language.
Chhaya has authored multiple patents in Machine Learning, and she’s developed recommendation systems for search engine giant Bing. While at Microsoft, she also created tools to perform functions like sentiment analysis, which helps businesses understand customer opinions so they can serve them better. During the COVID-19 pandemic, she designed a smart news retrieval model to help supply chain managers prepare for disruptions by identifying and ranking global risks.
“What excites me most is building tools that turn information into action,” Chhaya says. “But to make AI scalable, you need models that are transparent, adaptable, and ready for real-world challenges. That means fine-tuning models with the right data and building monitoring systems that catch issues early.”
Over the years, she’s honed a distinct ability for scaling AI. In her opinion, successful scaling comes down to three key areas: understanding your goals and defining the problem you’re solving, balancing development speed with cost and flexibility, and defining key metrics and tracking them as you scale.
Understand Your Goals and Define the Business Problem
When building AI systems, organizations should start by understanding exactly which business scenarios would benefit from using AI at scale, whether it’s for product development, customer experience refinement, or data analysis.
“This clarity will help you determine which models to use, rather than arbitrarily picking models to fit vague goals,” Chhaya explains. “For code generation, for example, you might use Claude Sonnet, while using something like GPT-4o for conversation. Meanwhile, a smaller, fine-tuned model like classic BERT would be appropriate for fast computation or supplementing GPT output with relevant data. You might also consider fine-tuning the model for certain domain-specific scenarios to improve model quality.”
Creating a culture of open communication and knowledge-sharing across teams is also vital for building scalable AI solutions. This allows teams to learn from each other’s successes and mistakes, speeding up problem-solving and keeping business goals front and center: “Encouraging a growth mindset where experiments (even failures) are seen as learning opportunities helps build a resilient team.”
Balance Development Speed with Cost and Flexibility
When scaling AI, it’s essential to use systems where models can be easily swapped in and out across components. This kind of flexibility means enterprises can always use the best model for each specific task, resulting in a seamless system of complementary models where output from one model feeds into the next.
This way, businesses can more effectively balance quality with speed and cost. A flexible system also allows enterprises to adapt individual components of the system as new models and technologies emerge — without having to waste significant time and money overhauling their existing systems.
Define and Measure Key Metrics When Scaling
Finally, it’s important to evaluate the model and measure its effectiveness before attempting to scale. Here, it’s important to have a robust testing framework (including developing datasets that reflect customer usage) to fully evaluate and validate the various components of the system.
To accomplish this, Chhaya recommends developing a robust and clear workflow for testing new models, what she calls an “experimentation pipeline.” This helps enterprises track the results of a particular AI model and compare it to different versions.
“Breaking down model parts based on their functions, versioning any changes to prompts, and tracking updates or changes to the model will make the whole system more stable and easier to manage,” Chhaya says.
While it’s tempting to take your AI to the next level after internal testing, real-world customer data can always shed new light on any remaining issues. Therefore, it’s vital to continue measuring the model throughout the scaling process. A good way to do this is to incrementally deploy to randomized cohorts of users and test the system again to evaluate its accuracy. This includes rigorous hypothesis testing backed by data-driven decision-making. You should also consider A/B testing any future changes and measure their impact on various metrics.
As a final note of caution, Chhaya warns against scaling without robust ethical guardrails, emphasizing the importance of making responsible AI use a key metric. To do so, make sure via testing that models are not biased and outputs don’t contain any harmful content.
Empowering Employees with Scalable AI
Chhaya Methani believes these strategies are key to making sure AI stays effective and adaptable as it scales, setting organizations up for success in the future.
For more of Chhaya’s insights, follow her on LinkedIn or learn more about her work at Microsoft.
Disclaimer: GeekWire newsroom and editorial staff were not involved in the creation of this content..
