If the winners of the Zillow Prize decide to spend their $1 million winnings on a new house, at least they’ll know that the Zestimate on whatever they’re looking at is a little more accurate thanks to their hard work.
Seattle-based real estate technology company Zillow awarded “Team ChaNJestimate” the big prize Wednesday in the company’s two-year competition to improve upon its tool for home valuation. Jordan Meyer of the United States, Chahhou Mohamed of Morocco, and Nima Shahbazi of Canada bested more than 3,800 teams representing 91 countries with an algorithm that beat Zillow’s benchmark model (evaluated against real-time home sales between August and October 2018) by approximately 13 percent.
According to the company, the teams’s improvements will help push the Zestimate’s current nationwide error rate of 4.5 percent to below 4 percent. The tool, created in 2006, automates valuations on 110 million homes across the U.S.
“People are incredibly passionate about their home and understanding its value, and we are amazed by the winning team’s hard work the past two years to make the Zestimate even more precise,” said Stan Humphries, chief analytics officer and creator of the Zestimate, in a news release. “We’ve been on a 13-year journey making the Zestimate more accurate, and hosting Zillow Prize allowed us to invite thousands of brilliant data scientists from around the world to join us on this journey. We’re so proud that the winning team’s huge achievement, and the work of all the teams in the competition, will provide millions of homeowners with a better understanding of one of their biggest life investments.”
In the video below, Zillow CEO Spencer Rascoff called the competition one of the largest computer science contests in the history of technology. “To be the winning team … that’s big bragging rights,” he said.
Shabazi is CEO and co-founder of Mindle.ai, a machine learning financial tech company in Toronto. He and Mohamed, a professor at the Faculty of Science Dhar Mahraz in Morocco, competed together in the Zillow Prize’s initial qualifying round. Meyer, A Raleigh, N.C.-based customer facing data scientist at DataRobot, competed on his own.
The three recognized the potential power of combining their Zestimate models and joined forces in the final round. They worked across two continents and multiple time zones, and have yet to meet in person.
“I worked about five hours a day, seven days a week for two months,” Shabazi told GeekWire via email. “Our team had a bit of a late start but once we got going, we were obsessed. I got excited every time we made a small improvement and even thought about the problem in the shower.”
Meyer said he spent well over 400 hours of free time on the competition. “My poor computer was running 24/7 during both rounds,” he said.
According to Zillow, the team’s algorithm incorporated several sophisticated machine learning techniques, including using deep neural networks to directly estimate home values and remove outlier data points that fed into their algorithm. They also leveraged publicly-available, external data including rental rates, commute times, and home prices, among other types of contextual information, such as road noise — all variables that factor into a home’s estimated value.
“Not long before the competition started, my wife and I moved across the country,” Meyer said. “We relied heavily on Zillow to preview and short-list the houses we would see in person. That worked out very well for us and gave me some insights into the differences between house listings and the houses themselves.”
Shabazi said going against a worldwide field of top data scientists required novel ideas and problem solving. Each team member worked on a different model and feature set and communicated actively on Slack to share insights.
“For every idea that worked, there were a hundred that didn’t work. But we kept going,” he said.
Zillow said its own team of data scientists has already begun to incorporate parts of the winning team’s algorithm — as well as other ideas inspired by top Zillow Prize competitors — into the Zestimate. The company stresses that it will never be perfect and is intended as another tool to help home buyers and home sellers. Incremental improvements in the error rate can move home valuations potentially by thousands of dollars. On average, Zillow said, the Zestimate is $10,000 off of the actual sale price for a typical home, and with the learnings from Zillow Prize, future Zestimates could be approximately $1,300 closer to the sale price.
There are no Zestimates in Toronto, but Shabazi called the real estate market very competitive and he said he is always curious about what factors drive housing prices.
“It’s amazing to know that millions of people will benefit from our ideas,” he said.
Zillow also awarded $100,000 to the second-place team, Team Silogram-2, and $50,000 to the third-place team, Team Zensemble. In the qualifying round, Team Zensemble placed first and Team Silogram-2 placed second.
As for how two of the three ChaNJestimate team members plan to spend their share of the $1 million, Shabazi said it’s hard to imagine. He’ll probably invest in his startup, buy some gifts for family and take a trip somewhere hot to get away from snowy Toronto.
Meyer might be keeping his eye on the (Zillow) prize.
“I feel obligated to invest it in real estate,” he said.
