Chris Hegstrom

Uncategorized

The Importance of 3D Audio for VR

Written for Designing Sound:

Audio for VR is important, especially for audio people. For us it’s really important, but how important will it be to your everyday developer?

Before we dive into that, a quick back-story:

VR has been around in one form or another for decades, but computing power has only recently caught up to our dreams & allowed people (not at technology institutes) access to head mounted displays & experience fabricated worlds in stereoscopic 3D.

& now…the arms race.

It’s an exciting time for technology right now because it feels like everything from camera technology, to story-telling techniques to graphics cards have a new goal: Be the VR standard. Be first, be the best & most importantly: be the one that everyone uses.

[Read the complete text at Designing Sound]

VR Theater

A few months ago,  Variety had an article all about Broadway’s new love of VR. This made sense the more I thought about it. An old-school entertainment industry embracing a new-school technology. Theater folks would love to introduce folks unwilling or unable to travel to a specific square mile in Manhattan to experience their creations. Plays on TV or DVD don’t translate quite as well once the whole element of ‘being there’ goes away.

But capturing / broadcasting a theatrical performance form Orchestra seat 5b is not only brilliant from an immersive standpoint, it’s also quite economical. Most of your interesting content is fixed to the front 45 degrees, although you’re free to look around at the surrounding environment. It’s pretty easy to capture with VR camera & 3D audio arrays (what’s more binearal than sitting in a seat & letting the theater sound system spoon feed your audio?) & finally, you don’t need to concern yourself with playspace because people watching plays don’t generally climb around onto other seats during the performance.

There’s a social-economic implication with broadcasting Broadway plays to VR as well. Theater was originally imagined as a cultural destination for anyone affluent enough to consider ‘dinner and a show’ reason enough to brave the bridge and tunnel traffic.

The first musical I saw was Grand Hotel when I was 17. My dad drove us in from Moristown New Jersey. We parked, grabbed dinner near the theater, saw the 2 hour production & had drinks (soda for me) after. This was an amazing experience but even then I thought how foreign this was for a suburban high school student. The fact that I didn’t summer in the Hamptons put me in the minority of patrons but a cultural shift was already under way to make Broadway more accessible.

As entertainment became more available through tv, megaplex movie theaters & video rentals, Broadway’s brow began to lower, the ties & evening gowns replaced with baseball caps and jean shorts. Gum chewing tourists (like me) would grab discount matinee tickets from third party vendors or add-ons to Greyhound bus tours.The content shifted as well. Ask Sonheim or Loyd Weber what they think of Spiderman & Jerry Springer the Opera. But what to do? People have to come to your show. Theater never really translated to recorded medium. It sort of felt like a low budget soap opera with no cinematography. Besides, the whole allure was physically being there, part of a live performance.

That’s where VR comes in. While it’s been more challenging to adapt movies or TV to VR, theater & live sports seem to be a natural fit. Transformative media works best when you actually want to be there. Watching Shark Tank on my couch with a bigger screen? Meh. Being in seat B10 at the Gershwin Theater without going to Midtown? That’s pretty cool.

Despite what you might think, Theater isn’t a passive media. From 1995 to 1998 I did live sound for Blue Man Group. In addition to being exposed to about a dozen or so other Broadway & off-Broadway shows I also got to see a couple hundred performances of the production I worked on.

Wasn’t that torture, seeing the same show over & over? Not really. Every performance of every show has an energy feedback loop between the audience and the performers. The sleepy Sundaymatinees were a much different energy than the 10pm Saturday shows and audience participation (both wanted & unwanted) can influence the cast & crew to react differently.

Feeling like you’re a part of that energy loop (even if you’re not really) can add a layer of realism in a VR setting. It’s that missing link that  DVD of Phantom doesn’t really have. In that case, you’re watching an audience watch the play. Not so in VR.

As VR defines itself in its infancy & Broadway reinvents itself in its twilight, it makes perfect sense for these two formats to intersect. Both embrace the excitement of presence with the depth of narrative & both coax you into an experience that seems maybe a bit less linear than it actually is.

Maybe theater & VR will take off into a beautiful mutually beneficial relationship, maybe it will die on the vine, or maybe, just maybe it will encourage new ways to write / perform and consume theatrical content. I anticipate this future with the same wide-eyed enthusiasm I had before curtain rise at that showing of Grand Hotel twenty something years ago.

The VR Revolution Will Not Be Televised

There’s lots of excitement about VR right now. Everywhere you turn from the New York Times to pretty much every conference out there, the VR bell is being rung by everyone everywhere.  But let me ask you this: How much content have you actually digested? My guess is not much & here’s why:

VR content doesn’t translate very well to Screen Based Technology (SBT).

Those of us lucky few who have been involved in VR developement know it’s difficult, it’s time consuming & it’s worth it. Once you have a full VR experience that tricks your brain with 360 degree images, sound and motion, it’s easy to understand what the hype is all about.

For the vast majority who haven’t experienced it, the effort donning a Head Mounted Display (HMD) hardly seems worth the effort when the only press seems to be one of three categories:

  1. Awkward looking people, eyes obscured, staring into a black box
  2. A 2d video representation of mediocre looking graphics / animation
  3. Photoshopped / doctored combination footage that comes across as amateur green-screen

The rub is you need to experience VR IN VR in order to truly get the intention & that’s not going to happen any time soon for those who don’t actively seek it.

This is much different than established industries like video games and mobile. Displaying a video of said app or game gives the viewer a good idea of what it would be like as a spectator to someone else playing or navigating it.

With the recent release of the Oculus & Vive, this is changing slowly but the early adaptors are most likely already converts. Because VR is a singular experience, even in “multiplayer”, it doesn’t matter if the other player is in the same room with you or in Guam, your experience is yours alone.

In VR there is no spectator mode or passive viewing, at least not yet.

The accidental by-product of this is that security is much less of an issue. You go though a top secret VR project in the same room as a hundred reporters and they will have no insight into what you’re experiencing, even if they’re standing right next to you.

Is this a problem? Well, yes and no. Yes, it’s a problem if you need to reach a wide audience, go viral or communicate the depth of VR immersion on a screen. No, it’s not a problem if you embrace this adolescent phase of VR with all it’s inherent awkwardness, potential and limited user-base.

Despite the hype, this industry is just getting started and the next year or two of early adaptor feedback is critical in determining the path of future VR development.

Soon enough we’ll be experiencing VR demos in VR while waiting for a download or seamlessly integrated into virtual social network but for now, understand that watching a video of a VR experience is like hearing a radio announcer describe art or reading a record review.

Take it with a grain of SBT salt, embracing the knowledge that gradually people will be exposed to this alternate universe at least enough to apply the filter themselves & understand that VR is more than just a bigger, closer screen.