2d-3d Audio Converter
A python application that takes a regular (2d) audio file (in mp3, wav), manipulates its channels, and exports a 3d sounding file.
Python + Pydub, Pysfx libraries | *Songs below belong to Drake
The Idea
I have always been a fan of music and recently went to my first concert where I got to experience it in a completely new way. Listening to music at a concert with powerful bass and reverb was an experience like no other and inspired me to recreate it using code.
Development
1. Figuring out how to manipulate audio
Initially, I attemted to open up mp3/wav datafiles. With mp3's compression, that was a nightmare. Wav files are uncompressed, but reading their data was very difficult. The next plan was to create a plug-in that could be applied to audio files in audio manipulation softwares such as Audacity. I decided not to pursue this becasue of the limited documentation and instead found python and the pydub library.
2. Creating the proper effect
To create a concert experience, it had to seem like the audio was coming from all directions. Headphones only provide 2 directions, so this had to be virtually done by mainpulating the gain of left and right channels.
Left Ear
| n (3s int) | Gain(applied) | Gain(total) |
|---|---|---|
| 0 | 0 (initial) | 3 |
| 1 | -3 | 0 |
| 2 | -3 | -3 |
| 3 | -3 | -6 |
| 3 | 0 (switch) | -6 |
| 4 | 3 | -3 |
| 5 | 3 | 0 |
| 6 | 3 | 3 |
| 6 | 0 (switch) | 3 |
| length-1 | ... | ... |
The right ear table is inverse; it starts at -3 and at n=3, switches at +6 total gain
*Original snippet
Snippet with code applied
*Original waveform of song
Same waveform with code applied
The code fades the left and right channels inverse of each other, creating a soundform resembling a sinusoidal graph
3. Adding effects
Using Pysfx python library, effects and filters like reverb, delay, and lowpass were added to achieve the concert sound experience.
Highpass and reverb applied in python
4. Optimization
The process of applying the effects was initially performed recursively; however, with large files, that was CPU-demanding. The algorithm was switched to be performed interatively which noticeably reduced CPU demand and the time it took to apply the conversion and filter algorithm.
*Another reminder that all songs above belong to Drake