Where We Are Now And What's Next

As my time in the URO program comes to a close, and I have a poster to present that shows what I've been up to all summer, let's reflect a bit on where this project has taken me and where it could continue in the future.

Things I've Learned About Spatial Audio

• How to capture the motion data from Airpods Pro with the HeadphoneMotion plugin on Unity

• How to use said plugin to create an interactive soundscape that is responsive to changes in head motion with 3 degrees of freedom

• How to use Swift to compile the code from Unity and create a portable application that can run on an iPhone

• Sometimes, if nothing seems to work, the solution is just to run a slightly older version of the software

Things I've Learned About The Brain And The Ears

• The brain is incredible, and it works in really interesting ways with the ears. You're doing these things constantly, but largely subconsciously. We take in a lot of information from what we hear, and we can tell things like the size of a room and the materials it's made of, the approximate height and location of an object making sound, and if it's being obstructed by anything. A lot of what we do in audio post-production is try and reverse-engineer these responses to make media more immersive, to expand the 16:9 frame into a full sphere of context.

• There are a lot of concepts that seem like common sense to us now with easy access to mass music and audio libraries, in high fidelity, at our fingertips. It's been really interesting going back and looking at early uses of music therapy, or the Iso principle (something a lot of us do nowadays with personalized playlists!) in its experimental phases. I like to imagine what things that we're experimenting with now will be the commonplace solutions of the future (and I like to think object-based sound engineering will be one of them).

Things I've Learned About Myself

(maybe this is less interesting but the point of the blog is to humanize this, so I'm going to talk about it)

• I had a difficult summer. My first summer out of undergrad, in an apartment with constantly breaking AC, also juggling several other projects where I was not given much oversight. I have learned that I am capable of diving headfirst into an interesting topic and going hogwild on it, provided that life isn't getting in the way.

• I found speaking to people who are actively working on these technologies to be both more compelling and more insightful than reading published literature. Is this because I come from a media background rather than STEM? Probably. Regardless, I am pleased to have had so many of my emails answered, even though some of them never got replies.

For Those Who Come After Me

It's pretty likely that someone who is a better programmer than I will be able to pick up where I'm leaving off with this project, working to implement more variability and an actual plotted-out program of gradual change, using GPS data for 6DoF, putting some fancy UI on it all. I am excited to be able to pass the mantle onto whoever comes next, and here is some advice for them from my perspective:

• Ask a lot of questions. Ask the professors in your department, in other departments, at other universities. Ask your advisor for their opinions and feedback as often as you can (remind them you exist!). Ask the people who are making things that are similar to this what their strategies are. It is very fun to work alone, until you start chewing on the walls of your own brain. Talk to other people about what you're doing and get their take on it all.

• Know your audience. I love writing, but I found myself struggling to communicate my work and findings in a way that felt "professional enough" to fit in with the academics whose papers I was reading. In my work in undergrad, I was often working in a weird liminal space with regards to academic-style writing. The standards are not typically as rigorous as they would be in STEM or post-grad work, but when everything is grouped into the category of "research", it's hard to know what's expected of you. If you need to be putting out documentation that can be cited by other academics, try to write like a grad student. If you're showing other people your process, it's a bit easier to get it across like this.

• Familiarize yourself with the software you're using consistently. It is very annoying to know that you knew how to do x in program y at some point, but have since forgotten it. In these cases it's also very helpful to document when you learn something new so that you can go back to it later, with the extra layer of memory to reinforce it. Also, follow along with the software yourself WHILE you're watching or reading the tutorial.

• A lot of this tech is moving really quickly, so things become out of date fast, or they break, or they get easier and you won't know it because people are still doing it the hard way out of habit. There is a lot of troubleshooting involved here.

There's a references page on the website that has a MLA citations for the academic sources (and some general webpages) that informed my research and my process. If that's your thing, go and check them out! If there's a source that you want a direct link to, feel free to reach out to me and I'll help you get sorted with it.

Thanks for reading my silly little blog, trying out my silly little demo, or reading my silly little poster!