AR Audio On iOS ARKit


Augmented reality on mobile devices is only a few years old, but already it’s being used or considered for a lot of different purposes. And for the most part, these revolve around largely visual experiences. Consider a few examples:

Gaming – Unquestionably the biggest development since Apple and Android phones embraced augmented reality is the introduction of brand new games. The technology allows traditional mobile games to blend with the real world in unprecedented ways, and as a result there are all sorts of new interactive experiences available. It’s widely assumed also that as mobile AR is further enhanced by forthcoming smart glasses from companies like Apple, the games are only going to get better at putting us in the midst of virtual characters and effects.

Splice Rent-to-Own

ShoppingPioneered by companies like Shuttershock, the idea of shopping through AR has also taken off, and again, is primarily a visual experience. Shuttershock’s app, as an example, allows users to view digital representations of products as if they’ve appeared in rooms, and then to make shopping decisions accordingly. Functionality like this has largely served to set up mobile furniture markets thus far, but could have significant implications across the retail industry.

Betting – Right now this is more of a hypothetical, but it’s an example of how even simpler visual applications can showcase AR. Betting tips and picks are easy to find online these days for those interested, and typically concern events that are about to happen. But a lot of online bookmarks are also pitching “live betting” lately, which is to say wagering during sporting events. The expectation is that AR will allow people to see options for this sort of betting even as they watch live sports in person. In this way, even a few lines of text flickering across a user’s AR glasses during a basketball game represent AR advancement.

The above examples are wildly different applications, but share a common thread of being based on visuals. However, this shouldn’t leave anyone with the impression that mobile AR is exclusively a visual technology, because there are also fascinating implications for audio recording. This is something developers are still essentially experimenting with, but it’s already becoming clear that there are going to be various ways in which we can treat AR creative platforms like ARKit and ARCore (from Apple and Google, respectively) as audio tools.

Splice Sounds

Even back in 2017, when AR was brand new, there were talks of recording spatial sounds with ARKit, which still seem to hold promise for interesting audio endeavors. What essentially happened in the 2017 experiment referenced is that an innovative developer recorded sounds in physical space, such that it could be played back by moving through a digital representation of that sound in space. It doesn’t seem to have been quite so refined as this, but imagine a favorite song made visual in the form of a trail of clouds winding its way through open space. Then imagine moving physically through that space in order to play the song, and you have the idea.

This is just a single concept we know to have been explored, but it’s also worth mentioning that Apple has made tools available for others to work on immersive, AR-based audio experiences as well. It’s something we don’t talk about as much because virtually all of the significant AR apps to date revolve around visuals. But it appears that AR shouldn’t be dismissed as an audio medium.

There are only going to be more programs and pieces of equipment moving forward, and it stands to reason that some of them will be used to create incredible audio AR applications.

Add a Comment

Your email address will not be published. Required fields are marked *