Science

6 things you need to know about virtual reality (from the guy who’s doing it right)

Jun 12, 2015 /

For all the hoo-hah and fanfare around next-generation technologies like Oculus Rift and Google Cardboard, it’s still early days for virtual reality. Sure, we’ve come a long way from clunky simulators (even ones with wind machines that blew back your hair), but the field is still pretty much open for all comers. Filmmaker Chris Milk (TED Talk: How virtual reality can create the ultimate empathy machine) is bullish about the possibilities. Here, the man who melted the Internet with “have you seen it yet?” projects blending technology and music for the likes of Arcade Fire and Johnny Cash mulls the possibilities of the medium he’s betting on in a big way.

“I’m interested in how ones and zeros dance around inside of your heart.”

The key to VR? It’s the sound. When you think of virtual reality, you probably picture it: a 360-degree environment full of immersive vistas. But it’s the sound that makes it real, Milk says. He first grasped this while watching Hunger in Los Angeles, an early VR film by Nonny de la Peña, which places the audience in the middle of a street scene. It grabbed him immediately, but not for the reasons he expected: “The visuals were not photorealistic; she’d recreated the scene with blocky CGI characters,” he says. “But what really affected me the most was the audio, because it was real. Basically, she’d recorded a food bank line where somebody had a diabetic seizure and everybody was talking. You could walk around the characters and the scene, and it was really affecting.”

The old rules don’t apply. Like many VR pioneers, Milk has brought years of filmmaking experience to his work inside the goggles — and now he’s setting it all aside. “As a director, the main tools I use are framing, performance and editing,” Milk explains. “Through framing I show you exactly what I want you to see in the screen; I capture the performances using different angles of framing, and then I edit it all together into something. Those tools go completely out the window with virtual reality.” One by one, all the long-held rules of filmmaking fall away. “Everything’s a total experiment,” he says.

Non-fiction is a good place to start. Although the idea of creating an epic virtual reality narrative film is alluring, Milk and his team are focused on nonfiction, from documentary stories to musical performances. “It’s a little bit easier because people understand their context within a scene,” he says. “With a narrative, if you’ve got two people in bed having a conversation, well, now you’re in the room with them. Your presence in the scene changes the scene somewhat. What do you represent? Are you an invisible observer or part of the story? We’re still not sure.”

“It’s got to be about something human, because that’s what humans care about.”

A live-action virtual reality shoot will require some ingenuity. One of Milk’s first experiments was for the musician Beck: a monumental performance, filmed in February 2013. “I got a 200-person orchestra playing every imaginable instrument. It was seated in the round. Beck was in the center; the audience was around him and the musicians were around the audience,” he says. “I shot it with both traditional Alexa movie cameras for a linear, rectangular cut, and with three 360 spherical cameras made of GoPros mounted together with a 3D-printed piece of plastic.” Fair to say, it all looked a bit odd. But for a first-time experiment, it worked. “I heard person after person saying they felt so connected to the musicians,” says Milk. “It showed humanity in a way that they’d never seen before.”

TED_Ideas_Chris_Milk

The power of technology is not about technology: For years, Milk has created immersive experiences from emerging technologies; he clearly revels in pushing at the possible. But technology, in the end, is just another tool. “I think there’s a perception of technology that it’s cold and hard, it’s ones and zeros,” he says. “I’m not interested in that; I’m interested in how the ones and zeros dance around inside of your heart. What can you do with those ones and zeros that can make a person ball up and cry when they watch it? It’s got to be about something human, because that’s what humans care about.”

Why virtual reality is the next great medium: “Right now it’s this bulky thing that goes over your head and projects pixels, but eventually we’ll get to a system that has no extra weight, you can’t feel it at all, and the resolution is so sharp that you can’t differentiate between it and reality,” says Milk. “If you have that, and you have an AI computer that can adapt and tell a story in a million different ways depending on how you react inside it, then you have Total Recall. Then you have The Matrix or Neuromancer. That’s where this goes.” Not surprising from one who’s started a company around it, but Milk’s betting the farm on VR. “It will be the next great medium,” he says. “If I can put a chip in your head and sunglasses over your face and give you whatever experience you want, whether it’s to be on a tropical island swimming and sipping on mai tais, or with some beautiful woman who says you have to help her save Mars and there’s a spaceship waiting out on the street … That’s what this medium will eventually be. How long it takes to become that, of course, is quite another question.”

Photo courtesy Chris Milk.