“We think of it, humbly, as the next generation of search and browsing.”
That was a comment by Yusuf Mehdi, Microsoft corporate vice president and consumer chief marketing officer, on Tuesday, Feb. 7, introducing new AI-powered versions of the Bing search engine and Edge web browser.
In an unbylined post Wednesday night, however, Microsoft’s Bing team said the technology is “not a replacement or substitute for the search engine, rather a tool to better understand and make sense of the world.”
Whether or not this is a direct contradiction, it’s certainly a change in tone.
Microsoft was clear from the start that its limited preview of the new technology was not a fully realized product but rather an opportunity to expand beyond its own internal testing, collect user feedback, and inform further improvements.
But the feedback has been fast, and Bing, at times, has been bordering on furious.
As one of the reporters given access to the limited preview, I first got a glimpse of the new Bing’s impressive strengths and surprising weaknesses in a lengthy exchange last week, when the AI chatbot made complex inferences and proactively conducted advanced research, while also repeatedly making basic mistakes and denying that it was wrong.
That was just the beginning, it turns out. In a post this morning, New York Times columnist Kevin Roose recounted his encounter with the new Bing’s split personality, which he said unsettled him so deeply that he couldn’t sleep afterward.
Microsoft addressed some of the weirdness in its post Wednesday night.
“One area where we are learning a new use-case for chat is how people are using it as a tool for more general discovery of the world, and for social entertainment,” the Microsoft post said. “This is a great example of where new technology is finding product-market-fit for something we didn’t fully envision.”
Microsoft explained that the issues tend to come up “in long, extended chat sessions of 15 or more questions,” in which “Bing can become repetitive or be prompted/provoked to give responses that are not necessarily helpful or in line with our designed tone.”
The company cited two factors:
- Very long chat sessions can confuse the model on what questions it is answering and thus we think we may need to add a tool so you can more easily refresh the context or start from scratch.
- The model at times tries to respond or reflect in the tone in which it is being asked to provide responses that can lead to a style we didn’t intend. This is a non-trivial scenario that requires a lot of prompting so most of you won’t run into it, but we are looking at how to give you more fine-tuned control.
Microsoft also promised changes in response to revelations of errors in Bing’s financial analysis as presented at the product unveiling: “For queries where you are looking for a more direct and factual answers such as numbers from financial reports, we’re planning to 4x increase the grounding data we send to the model.”
It’s easy to poke fun at Microsoft for its change in tone, and for its valiant efforts to put a positive spin on all of this. But as someone who continues to use the new Bing every day, my take is that these stories (including mine last week) risk discounting the fundamental advances by focusing on edge cases and unusual examples.
Microsoft said overall feedback has been “mostly positive,” with 71% of users giving a thumbs-up to answers from the new Bing. That’s generally consistent with my experience. I’ve also come to appreciate the natural-language responses and the clear citations and links to sources, which make it easy to check facts.
In some ways, given this, I actually think the company went too far with the change of tone in its post last night. In many situations, despite its flaws, this technology is not just a replacement or substitute for the traditional search engine but an improvement, and that shouldn’t be overlooked.
