The winning team of Amazon’s inaugural Alexa Prize competition sits right in the company’s backyard.
The Seattle-based tech giant today gave a $500,000 first place prize to a team from the University of Washington that developed a conversational “socialbot” for Amazon’s Alexa platform.
The Alexa Prize, first announced last year, challenged students to build a “socialbot” that can have intelligent conversations through the company’s digital brain, Alexa, about pop culture and news events.
Using the Alexa Skills Kit, 12 different university teams supported by Amazon had access to numerous content sources to build their bots’ knowledge base. Owners of Alexa-powered devices like the Echo and Echo Dot could test the bots by saying “Alexa, let’s chat about,” and then picking a topic like baseball, celebrity gossip or scientific breakthroughs.
The competing bots were graded based on a number of criteria, including ratings from Alexa customers, ratings from judges, depth and breadth of topics covered, and appropriateness and accuracy of responses.
The UW team, which included students from the electrical engineering department and the computer science school, won first place with its Sounding Board bot that earned an average score of 3.17 on a 5-point scale from the panel of judges and achieved an average conversation duration of 10:22.
“The philosophy behind developing Sounding Board is bringing a variety of relevant content into a natural conversation,” Hao Fang, the UW team leader, said in a statement. “Ultimately, we hope Sounding Board can become a conversational gateway to online information that users enjoy talking with.”
Amazon gave $100,000 to a second place team from Czech Technical University in Prague, team Alquist, and $50,000 to the third place team, What’s Up Bot, built by students from Heriot-Watt University in Edinburgh, Scotland.
“This competition was particularly interesting compared to other research competitions in that it let us see how people interacted with our system, outside of the lab,” Noah Smith, a faculty advisor who helped the UW team, said in a statement. “It’s pretty unusual for academics to get to observe that kind of real-world interaction.”
No team was able to meet Amazon’s Grand Challenge of maintaining a conversation for 20 minutes — a $1 million prize would have been awarded to the winning team’s university.
Ashwin Ram, a senior manager for Alexa who leads the Alexa Prize team, noted that “judging a conversational AI competition is super hard because conversation is inherently subjective; there isn’t a clear right or wrong response at each turn in a dialog, nor a precise definition of what makes a conversation ‘coherent’ or ‘engaging.'”
From his blog post:
Every team involved helped us advance speech science on several dimensions, from significantly advancing the custom language model (LM) we developed for the competition, to creating numerous natural language understanding components which addressed conversational AI challenges that arise when a conversation can be on any topic, and the content of the dialog can change rapidly. Moreover, the teams made important advances related to dialog management and response generation and selection. For example, several teams created ensemble approaches to dialog modeling by employing hierarchical architectures that included a main dialog manager (DM) and multiple, smaller DMs related to specific tasks, topics or conversational contexts. For generating and selecting responses, several Alexa Prize teams developed novel hybrid approaches that combined generative models with sequence-to-sequence approaches incorporating variants. Other teams utilized a reinforcement learning approach that maximized the tradeoff between satisfying the customer immediately versus taking into account the long-term reward of selecting a particular response.
Amazon plans to publish technical papers from all the Alexa Prize teams, which retain ownership of their social bots, though Amazon does have a non-exclusive license to any technology or software developed in connection with the competition. The company will open applications for its 2018 Alexa Prize competition on Dec. 4.
The competition is another example of Amazon crowdsourcing artificial intelligence on Alexa. Alexa Voice Service lets manufacturers integrate Alexa into their products. The Alexa Skills Kit encourages third-party developers to build skills for Alexa. Developers who want to add to Alexa’s abilities can write code that works with Alexa in the cloud, letting the smart assistant do the heavy lifting of understanding and deciphering spoken commands.
Alexa, which currently has more than 25,000 skills, powers Amazon’s Echo smart devices and can be integrated into other third-party devices. Both the Alexa virtual assistant and Echo devices are two of the company’s most successful products. Amazon is battling other tech giants like Google, Microsoft, and Apple in the competitive voice assistant market, though it has a big lead as the first mover. A forecast from eMarketer estimates that Amazon will have a 71 percent market share of all voice-enabled speakers in the U.S. this year.
Amazon, which has more than 5,000 employees working on Alexa-related teams, also recently graduated the first cohort of its Alexa Accelerator, a new Seattle-based program supporting early-stage companies that are working on B2C and B2B technologies related to Alexa. The accelerator is part of Amazon’s $100 million Alexa Fund, which launched in 2015 year and is used by Amazon to invest in companies that will push the boundaries of voice-based interaction.