Saturday, September 11, 2010

5.1 Converted to Binaural

NOTE: Before listening to any examples, know that earbuds are best for listening to binaural audio, as the sound goes directly to your eardrum. Binaural audio is processed in a way that effects it the same way your pinna would, and if you use headphones, the audio is technically going through two pinnae. I use headphones most of the time anyways. It doesn't fuck with it that bad, but there is a difference. Also, binaural audio doesn't work for everyone. Everyone's ears are different, and this works based on the "average" person's ear and head shape, so if you have a really fucked up skull or weird ears, then this might not work for you.

     Ok. Here's a little something I've been working on. I was wanting to make a jimmy-rigged version of a BRS system. BRS is binaural room scanning, and it allows you to do 5.1 mixes in headphones since it converts 5.1 to binaural. It even lets you hear the mix in different types of room through different types of surround systems.
 
     Well, doing that for real obviously requires some serious programming skills and electric/mechanical engineering (or some shit) skills that I don't have. You'd have to make some type of software, special headphones, and a bunch of other crap that I probably don't even care to know about.

     So, I figured I'd go the super-easy route and just try to do it with Logic. Since I don't have headphones that do head tracking, I've been using a Wiimote for that. No, I don't tape it to my head, I just hold it in my hand and move it the same way my head moves.

     Well, in order to do all this shit, the first thing that I had to do was convert a 5.1 mix to a binaural mix. Well that'd be pretty easy if you just had 6 mono tracks to work with, but I wanted it to work instantly without having to demix it first. I wanted to be able to pull in the track and just let it be ready to go. So I used a bunch of routing that would demix it in real-time.

     Let it be known that this doesn't work so well. It kind of, sort of has a binaural effect, but there are problems. One problem is that field recordings are done using many different types of surround micing techniques. It makes the most sense to set up the binaural mix so that it would simulate a regular 5.1 setup (speaker placement), but in some cases, that completely screws up the Ls and Rs and makes everything that should be behind you sound like it's right be-fucking-side you.

     To combat this, the Ls and Rs are moved further back, which takes them away from your ears, but it also stretches the whole mix further than it should be. It removes some of the "whoa! That shit is behind me" factor that should be there (for some mixes anyways. Like I said, it depends on how it was recorded)

     Really, the main problem is that this is "blurrier" than a real 5.1 mix being played on a 5.1 system, and it's also "blurrier" than a regular binaural mix. It's hard to close your eyes and pin-point exactly where something is. I can't complain though, because I'm not doing this with software or hardware that was designed for this purpose. Also, this isn't the same as BRS. BRS was made so you could do a 5.1 mix without needing a 5.1 setup. What I'm doing is taking 5.1 mixes and converting them to 5.1. You could use do a few extra steps and be able to do a 5.1 mix using this method, but since it's so inaccurate, there's really no point in doing it (although I do give an example of how this is done)

     So, at the end of the day, the only use I get out out this is being able to pull in a 5.1 Apple Loop and go "coooool.... it almost sorta sounds like a shitty version of real life". Either way, I figure it's a good starting point, and this is something I'm constantly messing with and changing, so it may end up working pretty well one day. Anyways, if here's how it's done.

First, I take the source audio and buss it out to 6 different aux channels


This is where it starts. I'm basically splitting up the source into 6 separate channels, which will each become the corresponding "speaker"

Here's the most important step. These are the aux channels that the source channel has been bussed to. They are all set to be 5.1 channels


On each of these channels, I inserted the Gain plug-in. (Notice the labeling of the channels... but ignore the fact that they are out of order)
Using the gain plug-in, the necessary channel (depending on what "speaker" is being worked with) is routed to away from all the other channels (this is for the Left speaker, so L has been routed to B, while everything else is routed to A, and LFE is by itself anyways


Now everything other than L is turned all the way down. LFE must be turned down too.
                    


Now, each channel will only have the channel information it needs. This has been done for each channel (L, R, C, Ls, Rs, and LFE)

Now, each of these 5.1 aux channels are now bussed to mono aux's

This step allows each 5.1 signal (which only contains one mono signal anyways) to be isolated in its own channel strip.

Now, each mono channel is bussed to a stereo channel


Again, excuse the fact they they are out of order. Anyways, each of these channels are set to have a binaural output, which brings in the binaural panner. They are each panned according to the placement of each speaker they represent. You can see on the Ls and Rs that they aren't where they should be. They should be at -114 degrees, but they are at -150 here. This was one of those cases where it sounded wrong when it was panned normally.

Now, these channels, having been panned (and diffuse-field being turned off) are bussed to a single aux

This aux contains the Binaural Post-Processing plug-in. Now everything should be totally binaural.
The next step is redundant and I have no idea why I did it, but I bussed that aux to another aux...


So that is how you split a 5.1 track into mono signals and make it binaural.
Here is how you do 5.1 mixing using this method. It's pretty much the same process, but you have to have a 5.1 channel strip for each instrument you plan on using.


You can see that each channel (Test Audio and Test MIDI) have been bussed to a "splitter", which is pretty much the same as the Source channel (the very first picture) except this one has a send going to a reverb. All the routing after this step is the same as above.
This way, you can use the surround panner, and it acts like a binaural panner.

This is where you can really notice the problems. When something is behind you, it still sort of sounds like it's in front of you. Also, there's a huge dead space in the very very back where the sound almost disappears. This dead spot gets bigger when you pan the Ls and Rs to -114 (where they should be if you're trying to replicate a 5.1 setup). Plus, when swinging the sound from back to front, it's hard to tell when exactly it stops being behind your ears to being right beside your ears.

Here's a video with Ls and Rs being at -114 (worst dead spot, but most realistic). I used an Apple Loop of footsteps for this video. Also, note that these may even sound less believable due to the sound quality (not sure if Blogger reduces sound quality on uploaded videos the same way Youtube does)

Now, here's a video with Ls and Rs panned at -150 (smaller dead spot, less realistic)

Here's Ls and Rs being panned at -165 (smallest dead spot, least realistic)

This video is a straight up binaural version. There's no funky routing and no conversion. It's purely binaural. Notice how much better this one is. You can close you eyes and pinpoint where the footsteps are.

Ok, here's a sample of the first process I showed you. It's an Apple Loop of rain. First, you hear the whole mix and see all the routing. Then the Ls and Rs are solo'd (everything else is muted. Soloing isn't effective due to the routing), and I switch the panning from -150 to -165. You can hear that they sound more "behind" you like this (but this panning doesn't work with every 5.1 mix recording. It depends on the mic placement when recorded). Then everything is unmuted and you hear it all together with the -165 panning. Then the LFE (which contains all the subwoofer info) is muted


Sadly, I don't have a normal binaural version of this, as it's not just one signal being panned around. It's a full 5.1 mix being split and converted to binaural, so this is all I can give you. This would have been recorded with a binaural dummy head.

I would give you a video where I use the Wiimote for headtracking, but it's pretty unimpressive and confusing unless it's your own head moving. But know the when doing that, you can totally hear that dead space I've been talking about. You turn all the way around hear a big gap in front of you.

I hope this was semi-cool. Enjoy


20 comments:

  1. oh god the effort to do that! Good post though sir :]

    ReplyDelete
  2. hey man if you could do one of these for FruityLoops studio that would be boss!

    ReplyDelete
  3. that was quite a read, and I learned a lot

    ReplyDelete
  4. nice one! I am really into sound engineering and stuff, so thanks for the great post!

    ReplyDelete
  5. Thanks for sharing this mate...
    Will check on you daily :)

    ReplyDelete
  6. Sir, you posted a vast amount of information. I will have to reread the whole thing again tomorrow. Great post though. :) Keep it up and I will check you out daily.

    ReplyDelete
  7. cool i really like this!!
    come check out my blog too if you have time :)

    ReplyDelete
  8. great guide, especially appreciate that you used so much videos.

    could you make a guide for FL studios?

    ReplyDelete
  9. Way more in depth than I'd ever need, thanks for the pictures though, those will be extreamly useful.

    ReplyDelete
  10. Thats surprising, i always thought that headphones were supposed to be better for binural

    ReplyDelete
  11. I'm in the market for new headphones.. this is helpful

    ReplyDelete
  12. it looks like you have a good thing going here... keep up the great work on your blog!

    ReplyDelete