As the sun set over the Olympic Mountains on a warm and muggy Sunday, I trekked from the manicured suburbia of Sammamish, Wash., to Seattle’s historic Fort Lawton district, where an eclectic home, known as the Bird House, sits among low-hanging branches at the end of a quiet street.
Unlike VR hackathons where coders sit at screens encouraging their routines and algorithms to execute in alignment with their concepts and expectations, the Cinematic VR Challenge teams primarily consist of creatives coaxing a new form of video to tell their stories.
But there is a problem. As mentor David Feuillatre explained, “VR is the first real digital medium. Digital film was just a change in technology; it didn’t change the way people told stories, just how they edited them. With VR, we don’t know any more how to tell stories. We are reinventing everything.”
And that is the crux of the issue those participating in the challenge must explore: what does Cinematic VR mean?
VR Changes Everything About Linear Storytelling
Much of the content available to VR enthusiasts comes in the form of immersive videos. But most would be hard pressed to call these videos “cinematic,” and VR purists wouldn’t label them as virtual reality. Most current “VR” video consist of often random clips of places or experiences haphazardly edited together. You are on the ocean. You are on a boat. You are swimming. You are looking down from a crow’s nest. You are under water with fish. You are surfing. There is no story, just a jumble of loosely connected 360 videos.
But that, to some degree, is to be expected, because much of what exists on YouTube is poorly edited, haphazard and story-challenged. Cinema implies telling a story, or at least creating a consistent tone that moves an audience.
While a linear approach to filmmaking can be applied to VR, it underserves the technology’s capabilities. The canvas is too large for even a director skilled in IMAX. In VR, what happens behind a participant, or in their peripheral vision, may be as important to the story as what happens in front of their eyes.
As co-organizer Julian Peña points out, “If we are making a cinematic VR film, we don’t know where the director should be…and then there is engagement where the audience is in the film. You have to think about how to make the whole world interesting, not just the part you presume is in from of the viewer’s eyes. But you also have to balance what might be considered distracting to the actual subject.”
There is also that subtle shift from viewer to participant. Not only do 360 videos surround the viewer, but the viewer becomes participant because they are free to look wherever they want while the video plays. In a movie theater, if you look at your feet, you will miss something on the screen. In 360-degree video, you will also miss something on the screen if you look at your feet, but you may also see something relevant to the narrative.
“Part of it is being able to go back to the same material and experience a different thing,” says co-organizer Budi Mulyo, “And you aren’t getting bored. You can walk through an area in a cursory way, and the next time you see it you explore it in more detail. And that is what happens to us in real life when we visit places multiple times. In VR, people will probably want visit VR multiple times if it is interesting.”
Which is true of a film as well, but in a traditional film you learn very little if you watch the entire film facing away from the screen. In VR, facing away from what the director purportedly chose as the focus of attention creates an entirely new experience.
Challenge mentor and Sprawly CEO Eric Neuman explains the problem: “Storytelling tools of VR have leaped forward our capabilities for communicating visuals, but they are sending us back in time in our ability to communicate story, because we’ve lost all the techniques on how to direct people’s vision, to use auditory cues to get people riled up into certain states.”
Neuman references a story made by the “One Way Trip” team in the Cinematic VR Challenge. “That fake blood, when you are watching it on a screen, is totally campy. Inside the VR headset it’s scary. When people jump into the car window, it’s scary.”
I point out, “Unless you are looking in the wrong direction.”
Neuman’s company, Sprawly, seeks to address some of these challenges with editing tools that, for instance, put the video in a loop until the viewer looks at the place the director want him or her to look. Those seeking to explore VR video editing in Sprawly can access their community level for free.
They want to put storytelling back into the hand of the director, “with interaction,” Neuman says, “or else it isn’t VR.”
Neuman believes the divergence between self-driven narrative of VR and passive VR has already happened, and it’s going to keep going forward. “Some people sit at home and passively watch television, other sit home and play Grand Theft Auto, which is a giant sandbox where the game player makes choices about how the story evolves within constraints.”
The Cinematic VR Challenge
Cinematic VR Challenges help participants explore these emergent issues. They combine boot camp with participation. Storytelling mentors and technologists share their insights while the creatives absorb and apply them in real-time to create full 360-degree film experiences they can share with guests at the final evening reception.
It was at this reception that I was tossed into the deep end of the Bird House, navigating tiled walkways in the soft loam, finding enclaves of creatives hidden in basements and among groves of giant rhododendrons. Monitors sat among wine bottles and cutting boards laden with cheese and sausage.
At about 8 p.m., one of the organizers announced the end of work … and the beginning of dinner—an interlude of chaos and consolidation that would lead to the projects being shown for the first time. There was an ever-growing influx of people entering the Bird House to see the films.
Four teams were competing for a chance to have their work entered into a VR film festival. As with any emergent technology, a few minor glitches set back the start for most projects for a few minutes as sound poured through the wrong port, or the experience started up too late in the experience.
The one flaw in a reception for a “VR” film with only one HTC Vive on hand means that only one person gets to “drive” the narrative as a participant. What the guests see on the monitor is the driver’s view on the film, not all of the film. And that is yet another major disruption in assumptions about narrative. Had more headsets and computers been available, each person donning a headset would experience a different version of the film depending on where he or she decided to look. People watching monitors of what different drivers looked at could conceivably see entirely different films with no overlap.
The film still creates constraint and limits options, but there are thousands of options within those constraints. Imagine watching the Blue Angels at Seafair from the I-90 Bridge. You are looking at Mount Rainer and you never change your perspective past 45-degrees to your left or right. Someone with their back to you is looking at the 520 Bridge, and they too only look 45-degrees to the left or right. The two experiences exist within a shared space, but they are completely different except for some aircraft that occasionally cross the planes within the respective fields of view.
Each of the demonstration films — be it the horror short “One Way Trip,” which puts the viewer in the passenger side of a car on some undescribed mission; or a hot tub full people waxing poetic, visually and verbally about the meaning of life; or a mediation on oneness in “VR Tai Chi” — offered a unique perspective on the transformation of 360-degree video capture and the application of storytelling techniques. The one project that didn’t get completed combined live-action with 3D animation. The failure of the team to complete the film proved instructive on its own, helping the participants understand the difficulties of combining 3D elements with captured video in a short period of time.
Finding and Inventing Theory
Many conversations took place among the contestants and the guests. In the yellow glow of analog string lights, a wooden Buddha and a crown molding of books that made up the Bird House’s great room, I could image the kinds of gatherings held in Paris during the 1920s, with Ezra Pound, T.S. Eliot, James Joyce, Ernest Hemingway and Ford Madox Ford bantering about the new forms of literature they were crafting and unleashing. One night at Sylvia Beach’s Shakespeare and Company, the next in Gertrude Stein’s salon where they might also be joined by Picasso or Thornton Wilder.
That’s the vibe that emanated from the Bird House. Emergent experiences, derivative on one level, but completely unshackled on another. A new kind of freedom for storytelling, with little theory to guide it. The task is not to apply theory but to discover, perhaps even invent it. These kinds of gathering encourage experiments, and they force filmmakers to ask questions about not only how to craft films in the new medium, but what it will mean to the participants who dive into them.
For Neuman this is all about balance: “We want to walk this line between storytelling and engagement. What we produce isn’t quite a game, and it isn’t quite a movie. You can be engaged if you want to.”
The approach proves different from traditional VR where the entire point is to be engaged. Perhaps the recognition of these emergent modalities will spread the desire for VR hardware as people realize that VR doesn’t always imply room-sized investments to house the virtual spaces. Meaningful experiences can be had in the comfort of a favorite swivel chair. Neuman says the message should be, “You can get on this ride and have a lot of fun.”
Feuillatre punctuates the evening by asserting that, in the end, “presence” is the essence of VR filmmaking. Immersive video allows us to be someplace else, and when we are there, we get to decide what is important. “In traditional film you are present in the theater. In VR you are present in the film, and that changes everything.”
The work continues
The “One Way Trip “team continues to work on their film. Since the event they have added three people the visual effects team, are recording new audio material in a sound studio, and have added an additional sequence ahead of the Sundance Film Festival where they plan to enter it.