Hello! I am wondering if any of the FMOD products provide 3D audio spatialization and/or localization? If so, how do you consider echoes, reverberation, and attenuation? I’d really appreciate any details or links to specifics!
- mcdermscott asked 4 years ago
This is a big topic so I’ll try to summarize how FMOD goes about spatialization before we dive into any details. Each product, FMOD Ex, FMOD Designer and FMOD Studio have various in-the-box solutions to spatialization but can all be augmented with the use of custom DSPs or plugins.
FMOD Ex spatializes 3D sounds by applying amplitude panning, distance attenuation and sending to a reverb unit who’s parameters are blended between different settings based on the listener’s location. FMOD Ex also has an in-built low-pass that is employed for sounds occluded by geometry and can be set up additionally as a very basic HRTF filter. This low-pass and another built-in high-pass can automatically band-pass the signal based on distance as well. Other panning options include blending between the automatic 3D panning and fixed speaker levels and controlling the spread of multi-channel 3D sounds.
FMOD Designer utilises FMOD Ex’s functionality but also provides automation of any effect parameter or internal parameter based on distance and angle. Plugin automation allows for custom spatialization algorithms.
FMOD Studio has a new 3D panner which has an additional section controlling the spread of the sound based on distance for sound-size simulation. The algorithm has also been improved to spatialize multi-channel audio for use in creating 3D ambiences from multichannel field recordings. In FMOD Ex/Designer, panning happens on each individual audio asset whereas Studio has a panner effect which sits on the output (sub-mix) of the event. Reverb is routed via reverb sends which can be placed pre- or post-panning and automated based on 3D position. There can be as many sends as you like sending to different reverbs, delays or any other effect chains in the main mixer. Mixer Snapshots allow effect settings to be blended or different simultaneous reverbs to be mixed based on 3D position. In Studio, 3D audio is as much about mixing as realistic localisation.
FMOD Studio’s panner effect module can be swapped out with custom spatialization plugins which will also listen to the position of the event. Plugins can therefore be used to perform HRTFs, geometry-based early reflections, distance filtering, etc… There are many features being introduced into the plugin architecture to allow game studios and third-party developers to easily implement their own advanced spatialization algorithms into their projects.
We’re happy to discuss any of those things in detail so please let us know what interests you.
Thanks for the write-up, Gino. I’ve been interested to know how things like Studio’s equivalent of the ‘Occlusion’ effect was going to work. Sorry if this is a bit of a thread hijack:
My understanding is that the LP/HP filtering in FMOD Ex/Designer (via Occlusion) is very light on CPU when compared to adding an LP and HP filter to every 3D event. If so, is there a similar effect coming for Studio, or are the filter’s in Studio light enough on CPU that they can be added to the master out of every 3D event?
As for reverb, it sounds like reverb presets have been replaced with creating a distinct reverb unit for each desired preset. Assuming that’s the case, is the bypassing of effects part of mixer snapshots (since I assume reverbs are fairly CPU heavy)? Also, is there already a system in place for reverb ’emitters’ (i.e. as the listener moves into the reverb emitter’s sphere, a specific reverb activates and fades-in as the listener gets closer to the center of the sphere)? Is this handled via mixer snap shots too?
Lastly, if you’re trying to gauge interest in 3rd party plugins, I’d say that if the licenses for those cost on the order of several thousand dollars per platform, then it may be a tough sell for studios like ours. We tend to develop $10-15 PSN/XBLA/Steam games, and it can be tough enough to convince the budget gatekeepers of the merit of spending money on new VST plugins, updates to Cubase/Wavelab or better hardware. But maybe these plugins aren’t geared towards lower budget games.
The occlusion effect in FMOD Ex/Designer uses the built-in low-pass effect which lives on each channel. The distance filter uses the same low-pass and an additional high-pass filter on each channel. Studio does not use the built-in effects on each channel, but those effects can be found in the ‘Legacy’ sub-menu as FMOD Low-pass Simple and FMOD High-pass Simple effects. While we haven’t added occlusion support to Studio yet, one way we are planning to support it is to have an Occlusion parameter with which you can automate filters. This parameter will be automatically calculated from game geometry, as it already is, but you’ll be able to create your own occlusion effects from any of the effects available in Studio. This could also lead to setting up different filter automations for different materials.
In most cases, using a Low-pass Simple effect on the master track will be equivalent, if not lighter on CPU then Designer’s way. This is because both built-in and added effects are bypassed if their cut-offs are at max (or min for high-pass); so in Studio, the built-in low-pass will always be bypassed. Possibly lighter because there will only ever be one low-pass applied per event instance rather than one on each channel if you had more than one sound playing simultaneously in your event.
Reverb presets will be done via snapshots in Studio and you would only need the one reverb effect on your mixer’s reverb return bus unless you wanted to mix between multiple simultaneous reverbs on different return buses! Yes, reverbs are heavy on CPU so best keep simultaneously-running reverbs to a minimum if you do plan to mix between reverb units. By saving your reverb settings in a snapshot, you can either simply trigger or automate them inside events based on game logic or you could automate the snapshot’s intensity over distance to achieve a similar result to reverb spheres in FMOD Ex/Designer. This will work somewhat with the current snapshot system but will be fully functional once blending snapshots are introduced later this year.
The plug-in architecture is as much about making it easier to implement and design with custom in-house DSPs as it is about providing for 3rd-party plug-ins. An example of an in-house system would be an early-reflections effect which uses the geometry from the game engine, or in-house sound synthesis or dialogue tech. As for third-party plug-ins, I can’t comment on the pricing or models that companies will decide on over time, or about what will be available for low-budget games, but I’m sure there will be companies looking to make affordable products for smaller developers. The focus on plug-ins is really about deeper and easier integration of external audio tech and Studio’s modular nature is well suited for this.
Thanks for the info Sean. We should be able to provide a migration process for the automation for your set up. I will have a look into it over this week. Would you mind sending me the fdp again? You can email it to firstname.lastname@example.org. Thanks.
Great! Thanks, Gino and Thuan.
I’m using the Occlusion effect in Designer on the 3D Distance parameter, and I have the programmer set the flag that enables Low pass filtering for Occlusion. We haven’t been factoring geometry into occlusion; distance has been our only consideration so far.
Although we haven’t used reverb very much to date, I do set up the automation curve for the reverb aspect of the Occlusion effect (in case an area/level ends up needing a reverb).
Let me know if you need any other info. A month or so ago I sent Brett an fdp for the project in question, in case that’s useful for you to see.
A belated thanks for all the info, Gino! One thing that’s holding us back from switching over from Designer to Studio is that (at last check) the importer would discard Occlusion automation. Is it feasible to have the importer translate Occlusion automation to FMOD Low-pass Simple, Volume and Reverb send automation (or whatever is most appropriate)? And have 3D Auto Distance Filtering translated to FMOD High-pass Simple (or whatever is most appropriate)? Having to re-implement this manually after importing seems like a pretty big job.
[quote:heqqxnib]Is it feasible to have the importer translate Occlusion automation to FMOD Low-pass Simple, Volume and Reverb send automation (or whatever is most appropriate)? And have 3D Auto Distance Filtering translated to FMOD High-pass Simple (or whatever is most appropriate)?[/quote:heqqxnib]
We should be able to sort this out… I’ll get back to you soon.
I just had a discussion with Gino regarding the Occlusion automation and 3D Auto Distance Filtering. We can definitely update the Designer importer to convert 3D Auto Distance Filtering (I will add it to my TODO list).
As for Occlusion automation, just wondering what sort of setup are you guys using for your project? Are you setting the occlusion and reverb using the programmer API, or are you using the FMOD geometry and reverb sphere, or the Occlusion effect in Designer (if so, which parameter are you automating it on, Distance parameter or a game parmeter), or a combination of them all?
Please login first to submit.