I’m putting together a mini-DAW (digital audio workstation) for iPhone as a personal project and I’ve suddenly been faced with some performance issues. Basically, the program needs to be able to play multiple files at once. I have my sounds being initialized with "createStream" (I’ve also tried createSound) and I’ve got a channelGroup for each to enable some features. The sounds right now can be compressed audio files of a few minutes long each. Also, as you might imagine, there are lots of sliders and vu meters and timers that are all getting updated.
When I launch the app on the iPhone, everything is working fine. As soon as I begin playing a song though, I experience a minor performance hit. The buttons and sliders aren’t as reliable, but all the vu meters and time indicators update fine.
When I play a second song, the app’s interface becomes almost non-responsive. Again, every vu meter and time indicator update, but hitting a button or slider has very little effect, and if it does have an effect it is extremely slow and choppy. Interestingly enough, the audio output is fine.
If I pause one or both of the songs, the app comes back to its snappy self.
In the simulator, everything works fine.
Do you have any suggestions on how to troubleshoot performance or increase it?
I’ve played around with XCODE Instruments, but I’m not getting much out of it right now. I plan on delving into how to really use it. I’m hoping that someone here could shed some light first.
- CuriousG asked 8 years ago
Ok here is some additional info from the FMOD Profiler, which is a pretty cool tool I must say.
When no channels are playing, I get the following values (approximate because they change frequently):
DSP: 1.30 / 100%
Stream: 1.54 / 100%
Update: .08 / 100%
Total: 1.97 / 100%
With 1 channel playing I get the following:
With 2 channels playing I get the following:
So, it looks like the DSPs and Streams are adding load. I have multiple channelGroups set up so I can apply effects to them separately. Also, I have multiple versions of the same DSP (really just instances of the same DSP as part of the class instance) because I will have different DSP parameters for each channel. For example, if I could want a bass boost for both channels but a different amount of boost for each.
Can you see a problem with my network here?
Thanks for all your help.
From a performance stand point, I can see you are probably playing stereo sounds, maybe MP3 or Ogg Vorbis. These can be quite expensive to runtime decode. Depending on the requirements of your app (memory usage, cpu usage, download size) you can probably make some savings.
Using a less expensive format could help, like ADPCM for instance, that would probably halve your stream usage but your source files would be larger (increase in download size).
You could also ship your sounds as MP3 and decode them to PCM on first run, then stream the PCM from disk (this keeps download size small, but costs space on the device). See the offline decoding example.
Also remember if you write your own DSPs, the cost of float operations is quite high, a good optimization is to make sure when you compile you disable "compiling for thumb", you will get a nice performance boost when working with floats.
As for the DSP network, you can make it as simple or as complex as you need, but obviously each point that requires adding effects or mixing will cost precious CPU (as you can see with the CPU numbers), so it is a delicate balancing game of functionality vs performance.
Yes, I was testing with the "wave.mp3" file that is included with FMOD. It is a 96kbps stereo MP3 file. It’s actually a pretty small file. Does that seem right, that even with that small file there should be a performance lag?
I converted it to a WAV file (16 bit, stereo, 44.1khz), and indeed, I did get a major performance increase. Below are the numbers. Could you take a look?
1 Channel Playing:
2 Channels Playing:
I’ve started taking a look at the offlinedecoding example. Any chance there is one already made for iPhone? How long does the processing take?
I’ve also experimented with createSound and FMOD_CREATESAMPLE, which provides a comparable boost in performance, even with MP3s, but I’m sure at a cost to memory.
Any other thoughts? Thanks!
The length of the file will not affect the performance, since you are streaming from the disk you take in a chunk then decode it, if the file is longer you just keep decoding for longer, the processing at any one time is not greater or smaller.
Those numbers you are getting when playing a wav file look normal, since no decompression is required for WAV the CPU overhead is much lower.
There should be an offline decoding example for iPhone in versions after (and including) 4.25.04.
When you do FMOD_CREATESAMPLE, FMOD decompresses the entire file to PCM at load time, then plays it as PCM, so the cost is once upfront. You should see the same runtime CPU performance as playing a WAV file. Although the entire file is decoded in memory, so if the file is large there will be a significant memory cost compared with streaming.
I’ve implemented the decodeSound function into my app, but FMOD is not able to load the file using createStream. It says:
FMOD error! (25) Unsupported file or audio format.
I’ve tried to load the file into Audacity and it can’t read it either. Looks like I need to specify the format using FMOD_CREATESOUNDEXINFO. Would you happen to have any examples of that?
PS: Thanks for all your help. I truly appreciate it!
If you want to load raw PCM data, you need to pass in the FMOD_OPENRAW flag to System::createSound() and you need to provide an FMOD_CREATESOUNDEXINFO structure.
In the structure you need to specify cbsize, length, defaultfrequency, numchannels and format. Check out the CreateSound() and FMOD_MODE sections of the docs for more details.
You should be able to load it in Audacity by choosing File -> Import -> Raw PCM.
Alternatively instead of outputting raw PCM data, you could output a simple WAV file. Unfortunately we don’t have an iPhone example for it yet, but the Mac version has some code you could use in the recordtodisk example. (Just ignore all the big endian stuff since iPhone is only little endian).
Firstly, the simulator isn’t useful for testing performance. It only simulates the iPhone by giving you comparable APIs, it does not emulate the hardware of the device or the performance of the device (so you are executing on 2Ghz Core2Duo processor instead of the much slower phone ARM processor).
To get an idea of what is using up all your CPU you can do a couple of things. Firstly you can try System::getCPUUsage(), this will give you CPU usage figures for the DSP (mixing and effects), Stream (decompressing streamed files) and Update (properties, voice management, etc that happens on the main thread).
If you need finer grain usage, you can connect to your app with the FMOD Profiler (running on your Mac) over the network (to the device Wi-Fi). With the profiler you can show CPU usage for each element of the DSP tree. Note that the overall costs will go up when using the profiler but the relative costs between the items will be the same (which is what is important).
You should be able to see where the costs are, which effects are hitting the processor hard, and how much you are paying for streaming highly compressed formats, etc. If you have any questions about the usage costs you see let me know and I’ll help you out.
Please login first to submit.