Sound and AR – “Watch the Breakdown”

 In Branding

Sound and AR – “Watch the Breakdown”

By Ross Forsyth – CTO, Overworld

Our last blog discussed 5 considerations for a great AR experience. This month’s will focus on one of the most important: Sound. To remind you what the highlights were…

Sound – Recap

Sound. Would a rose by any other name sound as sweet? We think not! We think Shakespeare would be proud of AR. The right sound can add an unbelievable amount of depth to a video, as well as to the overall experience. The wrong audio, the wrong mix of the audio, or a blasting of sound when the video starts can be as jarring as an ice water bath. We are exploring transitions of sound and the right mix depending on the content (spoken word vs. music, etc.). Sound has the ability to take an AR experience up another level and should be considered when thinking about the video presentation.

 

Watching the Breakdown – Spatial Sound

If you’ve heard the Drake song you will know what I mean, if not, go listen! The idea is to have the user “watch” and “feel” with more intensity and reality and lead them on a journey of interaction. Easier said than done with a “scene” but it can be done. After experimenting with spatial sound, especially using Sennheiser’s Ambeo Headphones and their soon to be released dev kit, several observations start to surface as we begin to explore sound and AR. For the most part these experiences have been “gamey”, read, door creaking open to add drama to a portal or birds chirping to simulate an “outside” environment. While these are incrementally better than no sound, it still feels like a black and white movie compared to surround sound from a Star Wars film.

Spatial sound is the idea of capturing sound in 360 degrees and then using the user’s location, where they are looking in a scene, to amplify or enhance certain sounds from those directions. Sennheiser makes this easy with their head tracking unit and Ambeo headphones to record and translate this into usable audio.

The real question to ask yourself when designing these experiences is “What is the user’s expectation of sound and how do you guide them on the journey of the experience using those expectations?” Every sound has a certain expectation, big or small, that we consciously or subconsciously associate with something. Have you ever heard a crack of a baseball bat…the first thing I’m looking for is a baseball coming my way – “Heads Up!” It’s these kind of expectations that we should be focusing on as we build out sound into a scene or experience. Sure you can add birds and the sound of trees swaying in the wind to try to make a “real” environment. This is first order type potential and we should really be in the business of enriching the experience and not just simulating it. Strategically placed birds, trees, car sounds added subtly to the scene will help re-enforce these environmental effects while letting the scene breathe and create the experience.

 

 

Have questions or comments? Leave them below and we promise to comment back. What are your favorite AR experiences with great sound?

About Overworld

An Oklahoma based Technology Company. We are narrowing the gap between our digital and physical lives. Creating content for today while building a platform for tomorrow in order to connect fans of all sports with their teams and brands. Contact us at info@overworldapp.com for more information or a demo.

Recent Posts

Leave a Comment