0
0

I am writing a Unity3D plugin that reads data from an MP3 file, feeds the PCM data to Unity so that it can play it inside the engine. On iOS, I use the AVAssetReaderAudioMixOutput class to decode and read the data, and on Android/Windows, I use FMOD.
I have set up a program on both Windows and iOS which uses FMOD to play back the music, just like Unity3D does.
I am having trouble getting the same results on both iOS and windows, and I cant seem to find the difference in audio output settings/format that would cause the difference.

So first, these are the settings that I set for my output audio stream, which are the same settings as Unity3D uses:

FMOD_CREATESOUNDEXINFO   exinfo2;
memset(&exinfo2, 0, sizeof(FMOD_CREATESOUNDEXINFO));
exinfo2.cbsize            = sizeof(FMOD_CREATESOUNDEXINFO);             
exinfo2.decodebuffersize  = 44100;                                  
exinfo2.length            = 44100 * 1 * sizeof(float) * 100;            
exinfo2.numchannels       = 1;                                          
exinfo2.defaultfrequency  = 44100;                                  
exinfo2.format            = FMOD_SOUND_FORMAT_PCMFLOAT;         
exinfo2.pcmreadcallback   = pcmreadcallback;                          
result = system_->createStream("./1.mp3", FMOD_LOOP_NORMAL | FMOD_SOFTWARE | FMOD_OPENUSER | FMOD_CREATESTREAM, &exinfo2, &sound2_);
ERRCHECK(result);
result = system_->playSound(FMOD_CHANNEL_FREE,sound2_,false,0);

Basically, 1 channel, 32-bit floating point PCM data. This is set on both the iOS and windows playback programs.
Now, on iOS, I set the AVAssetReaderAudioMixOutput audio settings like this:

NSDictionary *audioSetting = [NSDictionary dictionaryWithObjectsAndKeys:
                                  [NSNumber numberWithFloat:44100.0],AVSampleRateKey,
                                  [NSNumber numberWithInt:1],AVNumberOfChannelsKey,    //how many channels has original?
                                  [NSNumber numberWithInt:32],AVLinearPCMBitDepthKey, //was 16
                                  [NSNumber numberWithInt:kAudioFormatLinearPCM], AVFormatIDKey,
                                  [NSNumber numberWithBool:YES], AVLinearPCMIsFloatKey,  //was NO
                                  [NSNumber numberWithBool:0], AVLinearPCMIsBigEndianKey,
                                  [NSNumber numberWithBool:NO], AVLinearPCMIsNonInterleaved,
                                  [NSData data], AVChannelLayoutKey, nil];

I set the PCMIsFloatKey to 1 so that the PCM data is a floating point, I set the bit-depth to 32, 1 channel, so that everything matches the FMOD output settings.

I read the data and write it into a circular buffer:

        float* convertedBuffer = (float * ) audioBufferList.mBuffers[0].mData;
        //We don't need the audioconverter on iOS
        //Fill up the circular buffer
        for(int i = 0; i < numSamples; i++)
        {
            circularAudioBuffer_[bufferWritePosition_] = convertedBuffer[i];
            bufferWritePosition_++;
            if(bufferWritePosition_ >= circularBufferSize_)
                bufferWritePosition_ = 0;

        }

Then read the data from the buffer and write it into the audio stream in the pcmreadcallback:

float *writeBuffer = (float *)data;
for(int i = 0; i < dataLength; i++)
{
    sampleBuffer[i] = circularAudioBuffer_[bufferReadPosition_];
    bufferReadPosition_++;
    if(bufferReadPosition_ >= circularBufferSize_)
        bufferReadPosition_ = 0;
}

With this, the audio plays perfectly, and the range of values inside the circular buffer is 0.0-1.0f

Now, on windows, I initialize the sound from which I read the data like this:

exinfo.cbsize            = sizeof(FMOD_CREATESOUNDEXINFO);             
exinfo.decodebuffersize  = 44100;
exinfo.numchannels       = 1;
exinfo.defaultfrequency  = 44100;            
exinfo.format            = FMOD_SOUND_FORMAT_PCMFLOAT;  

Setting the same parameters: 1channel, 32bit floating point. I read the data and write the data in the buffer:

    FMOD_RESULT result = sound_->readData(rawBuffer, N4, &bytesRead);

    float* floatBuffer = (float*) rawBuffer;
    for(int j = 0; j < N; j++)
    {
        circularAudioBuffer_[bufferWritePosition_++] = floatBuffer[j];
        if(bufferWritePosition_ >= circularBufferSize_)
            bufferWritePosition_ = 0;
    }

Now, when I read the data, I get very high or very low floating-point values (about 1e34 or -1e33). In the test program, I can’t hear anything in the output.

I can switch the input and output sound format to PCM32 and it plays fine in the test program, but can`t be read properly in Unity3D (it screeches a lot, but I can make out the song).

Can anyone help me figure this out and make it works properly using the PCMFLOAT format? thanks!

TL;DR: I can’t read data from FMOD sound with PCMFLOAT format!

  • You must to post comments
0
0

FMOD_CREATESOUNDEXINFO::format doesn’t control the output format of the codec, but tells the codec what the input format in cases where it can’t work it out from other sources.

You need to use Sound::getFormat() to query what the codec will return when Sound::readData() is called. Our MP3 codec will always return PCM16.

  • You must to post comments
0
0

I see, so the PCM data coming in is a 16bit short.
I try converting it to a float to be able to play it back:
for(int j = 0; j < N4; j+=4)
{
short l = pcmBuffer[j]; //left channel
short r = pcmBuffer[j+2]; //right channel
circularAudioBuffer_[bufferWritePosition_++] = ((float)r + (float)l)/32768.0f;
if(bufferWritePosition_ >= circularBufferSize_)
bufferWritePosition_ = 0;
}

But I still only hear static. How would I go about converting the data for playback as a float buffer?

  • You must to post comments
0
0

Try this code:

[code:2w61chrd]static const int convertBlockSize = 1024;
short pcm16in[convertBlockSize];
float pcmfloatout[convertBlockSize];
unsigned bytesread;
FMOD_RESULT result = sound->read((void*)pcm16in, convertBlockSize * sizeof(short), &bytesread);
ERROR_CHECK(result);
unsigned samplesread = bytesread / sizeof(short);
for (int sample = 0; sample < samplesread; sample++)
{
pcmfloatout[sample] = (float)pcm16in[sample] / 32768.0f;
}[/code:2w61chrd]

  • You must to post comments
Showing 3 results
Your Answer

Please first to submit.