Researchers from the University of Washington have developed a face-modeling algorithm that turns photos from the Internet into a digital “persona” that can say and do things that were never actually done by the person behind the persona.
The software can put George W. Bush’s words in the mouths of, say, Barack Obama or Tom Hanks. But it can also take pictures of a loved one – for instance, a dearly departed grandparent or a friend who lives far away – and create an avatar good enough for virtual interactions.
That means, someday, you could have a real conversation with Grandma on your couch even though she lives 1,000 miles away.
The University of Washington research project is showing that machine-learning algorithms can capture a “persona” and create a digital version of a well-photographed person from the many images available on the Internet.
In the fascinatingly creepy video posted (above) by one of the UW study’s lead authors and a grad student in computer science and engineering, Supasorn Suwajanakorn, you’ll see former President George W. Bush’s answer to a question being transferred onto several other well-known people, including Hillary Clinton, Daniel Craig and Neil Patrick Harris.
“It’s one step toward a grand goal shared by the UW computer vision researchers: creating fully interactive, three-dimensional digital personas from family photo albums and videos, historic collections or other existing visuals,” states the University of Washington post about the research.
“As virtual and augmented reality technologies develop, they envision using family photographs and videos to create an interactive model of a relative living overseas or a far-away grandparent, rather than simply Skyping in two dimensions.”
The UW researchers don’t want to just stop at Grandma. They say that the technology could be used to create conversations with almost anyone you can’t meet in person — such as “LeBron James, Barack Obama, Charlie Chaplin.”
The technology uses advances in 3-D face reconstruction, tracking, alignment, multi-texture modeling and puppeteering. It was developed over five years by a research group led by Ira Kemelmacher-Shlizerman, a UW assistant professor of computer science and engineering, and is also funded by Samsung, Google and Intel.
The UW group will present its paper at the International Conference on Computer Vision in Chile on Dec. 16.
Watch a video that Suwajanakorn posted about the process, “What Makes Tom Hanks Look Like Tom Hanks” below:
Update for 3:45 p.m. PT Dec. 7: We asked Suwajanakorn and Kemelmacher-Shlizerman a couple of follow-up questions about their research, and here’s what they had to say via email (edited for context):
It seems as if the first application of this technology would be in movies, ranging from creating characters like Gollum to letting “young Tom Hanks” appear as a character in future movies. Would you agree?
Suwajanakorn: “Yes. Also, the transfer / reconstruction technique could be used to synthesize a face for a person wearing a head-mounted display, to give a ‘face’ to a virtual assistance system like Siri, to synthesize one’s own face for low-bandwidth communication that otherwise would only permit voice, or to digitize a person and his or her persona, e.g., the way the person speaks, moves and interacts.”
It also seems as if there’s the potential for bringing “Photoshopping” to a new level — for example, creating a malware-type of video that makes it look as if Obama is admitting he’s a Muslim terrorist. Have you thought about those sorts of identity concerns?
Suwajanakorn: “I don’t know a good solution to that. I think people will start to treat video with caution and possibly come up with a technology that analyzes certain ‘Photoshop’ artifacts, as has been done with photos.”
Kemelmacher-Shlizerman: “An interesting direction would be to use our technology to identify fake videos from real ones.”
GeekWire’s Alan Boyle contributed to this piece.
