The Zillow Zestimate, that much-debated, computer-generated home valuation tool, has been a fixture of the Seattle-based real estate media company for 11 years. And now Zillow is putting a price tag on whether the algorithm that assigns a price tag can be improved upon.
Zillow announced Wednesday that it was launching a competition aimed at awarding $1 million to the first person or team who could most improve the Zestimate algorithm. “Zillow Prize” hopes to attract data scientists everywhere to try their hand at tinkering with “one of the highest-profile, most accurate and sophisticated examples of machine learning.”
Zillow consistently refers to the Zestimate as just one data point that consumers have access to when considering buying or selling a home — along with information such as recent home sales and guidance from real estate professionals. Launched in 2006, it marked the first time that homeowners gained access to estimated home values — data that was previously only available to real estate agents, appraisers and mortgage lenders.
But over the years the tool has served as a source of contention among everyone from home sellers expecting to get more, home buyers expecting to pay less, and real estate professionals wishing they weren’t caught in the middle. Zillow co-founder and executive chairman Rich Barton called the Zestimate “very provocative and personal and a little voyeuristic” in a 2016 GeekWire interview discussing how the company came up with the tool.
Barton also happens to sit on the board of directors for Netflix, and the DVD-rental and video-streaming service famously conducted its own $1 million competition, awarding the prize in 2009. But a Forbes report in 2012 mentions how Netflix never bothered to implement the algorithm-improving solution.
Zillow said Wednesday that its own data science team is continually working to improve the accuracy of the Zestimate, which is measured by how close that figure is to the eventual sale price of a home. The U.S. median absolute percent error currently stands at 5 percent, improved from 14 percent in 2006, Zillow said.
The contest marks the first time that anyone outside of Zillow will get a look at a portion of the proprietary data that powers the Zestimate.
“We still spend enormous resources on improving the Zestimate, and are proud that with advancements in machine learning and cloud computing, we’ve brought the error rate down to 5 percent nationwide,” said Stan Humphries, Zillow Group’s chief analytics officer and creator of the Zestimate. “While that error rate is incredibly low, we know the next round of innovation will come from imaginative solutions involving everything from deep learning to hyperlocal data sets — the type of work perfect for crowdsourcing within a competitive environment.”
Here are more details from Zillow on how the competition will function:
The contest is being administered by Kaggle, a platform designed to connect data scientists with complex machine learning problems. It will be staggered into two rounds, the public qualifying round which opens May 24 and concludes Jan. 17, 2018, and a private final round that kicks off Feb. 1, 2018 and ends Jan. 15, 2019.
Contest participants have until Oct. 16, 2017, to register for the qualifying round, download and explore the competition data set, and develop a model to improve the Zestimate residual error. The top 100 teams from the qualifying round, those whose solutions most reduce the difference between the Zestimate home valuation and the actual sale price of the homes within the dataset, will be invited to participate in the final round and compete for the $1 million dollar prize.
In the final round, the winning team must build an algorithm to predict the actual sale price itself, using innovative data sources to engineer new features that will give the model an edge over other competitors. The home value predictions from each algorithm submission will be evaluated against real-time home sales in August through October 2018. To take home the $1 million dollar grand prize, the winning algorithm must beat Zillow’s benchmark accuracy on the final round competition data set and enhance the accuracy further than any other competitor. A $100,000 second place prize and $50,000 third place prize will also be awarded in the final round. A total of $50,000 will also be awarded to the top three ranking teams in the qualifying round.