October 27, 2019
Behind the Scenes: The Sound of Terrarium

My name is Colin Northway, I work on the Museum of Other Realities, and one of my roles is sound implementation. Today I'm going to talk about sound implementation for Durk van der Meer's piece, Terrarium.

Like many pieces we show in the MOR , Terrarium didn't come with a full sound-scape. Few visual artists are also confident sound designers but sound adds tremendously to the VR experience. We like to add audio whenever the artist is open to it.

We work with A Shell in the Pit Audio who does all the sound design for the museum and also the pieces we add audio to. They jumped into VR audio design very early and are an amazing team to work with. For Durk's piece, A Shell In The Pit had very little direction so Em Halberstadt had to make decisions about what needed sound, what to highlight, and what everything would sound like.

Every art piece in the MOR goes through a draft staging period where we have the rough layout of the room and the artwork's position in it. From there we get feedback from the artist as we move towards final staging. This drafting process also gives A Shell in the Pit a chance to drink in the piece and get a feel for what the artist is trying to convey and how to compliment it.

In this case, Em decided to break Terrarium into two ambiences, a spot sound and one stereo-split music track:

  • Ambience for the top floor
  • Ambience for the bottom floor
  • Spot sound for the Amp on the bottom floor
  • Track for the left wall-mounted speaker
  • Track for the right wall-mounted speaker

The Museum of Other Realities uses Moona to manage all of our audio. Moona is a Shell in the Pit tool written by Maris Tammik and Nicholas Zhang which is especially good at managing sound for VR projects. It manages all our disparate sounds, sounds being on the other side of portals, walls occluding sound, a bunch of tricks for making conversation more natural, and a lot more.

Audio placement and colliders in  Terrarium
Audio placement and colliders in Terrarium

Usually, my job of audio implementation is really easy, I just have to drag some Moona assets into the right place in Unity and let Moona do its magic. For Terrarium, we had a slightly more technical challenge implementing the ambiences between the top and bottom floors.

In Moona, sounds radiate from a point and can be heard anywhere within hearing distance. This means sounds are "spherical shaped". But the bottom and top of Terrarium meet in a line and are right next to each other. Meaning spherical sounds are hard to place to cover one floor without bleeding into the other.

The solution to this was to use box-shaped colliders to bound in the sounds. Moona lets us turn sounds on and off when the visitor's head is inside of certain boxes, for Terrarium I made one for the top and one for the bottom. These let the visitor hear the ambience slowly rise as they appear the piece but also flip back and forth more suddenly as they move up and down the stairs. To make the transition between top and bottom smooth, Nicholas Zhang added a fade transition between the two sounds.

Terrarium also has one of my favorite strange VR audio tricks in it with the left, right speaker tracks. Durk put a video preview of his personal project Maranga to the bottom room and added two speakers on the wall. Em decided to add audio to the Maranga video but to make it in stereo with different tracks for each speaker. This is a strange, wonderful thing. All the environment audio in the MOR is spatialized. This means it's mathematically transformed to sound as if it's coming from behind or beside or above you. Stereo sound is part of spatializing but not the whole equation. In the bottom floor of Terrarium, you are hearing two stereo tracks further spatialized into three-dimensional space. To my surprise, it works amazingly well and adds such a strange verisimilitude.

Read more
You might also be interested in these
News
We're excited to host this year's Tribeca Immersive selections in the museum! Tribeca Immersive showcases cutting-edge virtual, augmented and mixed reality experiences by top artists pushing the boundaries of storytelling with technology through impactful and emotionally engaging stories.
MOR x Tribeca Immersive 2022
We're excited to host this year's Tribeca Immersive selections in the museum! Tribeca Immersive showcases cutting-edge virtual, augmented and mixed reality experiences by top artists pushing the boundaries of storytelling with technology through impactful and emotionally engaging stories.
News
The Museum of Other Realities presents its first original collection of narrative experiences curated across the Canadian immersive ecosystem. The Canadian Collection, a project co-created with Kaleidoscope and produced in partnership with the National Film Board of Canada and Canada Media Fund will be held in the virtual gallery from December 17, 2021 to April 29, 2022.
Announcing The Canadian Collection
The Museum of Other Realities presents its first original collection of narrative experiences curated across the Canadian immersive ecosystem. The Canadian Collection, a project co-created with Kaleidoscope and produced in partnership with the National Film Board of Canada and Canada Media Fund will be held in the virtual gallery from December 17, 2021 to April 29, 2022.
Stay up to date with new exhibits and events.
Thank you! You've been added to our mailing list
Oops! Something went wrong while submitting the form.