This project focused on crafting an authentic, immersive audio-visual experience of a live horn performance by integrating spatial audio with 3D video in a virtual reality environment. We recorded two horn players in the Davis Studio using a combination of a tetrahedral microphone array, a centrally positioned Sennheiser Ambeo mic, and dedicated stereo pairs for each performer. The captured media was then edited and implemented in Unity, where interactive mixing controls and spatialized audio were integrated into a 3D environment.
In this project, I played a key role in both the recording and post-production phases. I was responsible for setting up the studio, configuring microphone placements, and managing the signal routing to ensure clean, high-quality recordings. During post-production, I collaborated closely with Zhengdong to align audio sources within Unity’s spatial environment and resolve issues related to audio bleed. My main focus was ensuring that each performer’s audio corresponded precisely with their physical location in the VR space, contributing to a believable and immersive sonic experience.
The final product delivered an engaging VR experience with dynamic, user-controlled audio. Through this project, I gained hands-on experience in spatial audio workflows and learned the importance of precision, collaboration, and problem-solving in immersive media production.
SongSmith is a prototype web app designed to help novice singers practice solo vocal performance through accessible, AI-assisted feedback. Built with Python, React.js, and Flask, the tool provides visual and text-based feedback on pitch and volume by analyzing user recordings and aligning them to reference melodies.
The initial version included basic pitch tracking, volume visualization, and LLM-generated vocal coaching. While early in development, SongSmith demonstrated strong potential for expansion—particularly in areas like rhythm tracking, group feedback, and guided practice based on deliberate learning principles.
This project laid the groundwork for a more comprehensive, scalable system and deepened my experience with audio processing, user-centered design, and the integration of generative AI in creative tools.