0
0

Hello,

I’ve done a VoIP in C# using NAudio library, and am now doing a c++ win32 VoIP application. FMOD looks absolutely awesome I must say and I can’t wait to get started.
I am to record voice into a buffer which I then will send over the network and then directly play the received data.
I’ve been fiddling with examples and reference sheet, but there’s a few things I’m not sure about;

1) How does one declare a buffer? How do you know when the buffer gets filled (does a msg/event get fired?) For example I want a recording to save all PCM data into the buffer and notify me when its appropiate to send.

2) I assume sys->recordStart is used to initiate recording, but what are the steps before that if I just want to record to buffer? All the tutorials use files to save to / listen on!

3) how relevant is update() to a VoIP application? what role does it fulfill?

Cheers!

  • You must to post comments
0
0

1) If the record position catches up to what you have locked it will simply overwrite the data, so you have to ensure this doesn’t happen by having an overall buffer of sufficient size.

2) This is up to you depending on buffer size, so if sending over the network is a fast operation then it’s probably okay. But if sending is slow you increase the chance of the record position overrunning the lock position.

3) I would recommend taking a look at the usercreatedsound example for this. If you have multiple streams of network voice data, you would need multiple sounds, one for each voice.

  • You must to post comments
0
0

I would recommend taking a look at the recording example, it demonstrates how to do all these things. For send data you would need to keep an eye on System::getRecordPosition and use Sound::lock to get at the recorded data instead of playing the sound on a channel.

You should always call System::update regularly regardless of the application type as it does a lot of house keeping things in FMOD. That being said you could possibly get away with not calling it for your app. If you want a list of what it does check System::update in the fmodex.chm docs.

  • You must to post comments
0
0

Hello,

1) How does one feed data into a DSP? I realise there’s a readCallback, but it appears to me to be more of a filter (when inserting, or playing it?).

2) How is memory management done with DSPs? Will Played() data be removed automatically afterwards, or how do you clean played() data? I want to feed DSPs chunks of data (16bit 16khz PCM), play them and then remove them. I had a look at the usercreatedsounds example, but it used an already existing file, and I’m not sure how to do it with raw PCM data

Thanks in advance

  • You must to post comments
0
0

To continously send data over the network, do I have to have a loop which continously checks if getRecordPosition > 0? I guess the loop’d function will have to run on a separate thread then aside from the main program?

Once I see recordposition > 0, call lock() to get the data; but won’t this create ‘jumps’ in the recording, if the user is talking and the memory gets locked() at every now and then, so some of his recordings wont get saved?

Also what about the memory management, for example here’s how I initiate the recording:
exinfo->length = exinfo->defaultfrequency * sizeof(short) * exinfo->numchannels * 5;
sys->createSound(0,FMOD_2D | FMOD_HARDWARE | FMOD_OPENUSER,exinfo,&rec);
Does the exinfo->length indicte the buffersize?

More importantly; When I got stuff to send, do I empty the buffer or does it get rewritten(meaning while recording it simply starts at the beginning if buffer full)? How does this interact with a socket for example which sends the buffer and is therefore reliant on the audio not getting overwritten?

  • You must to post comments
0
0

I think you need to take another look at the usercreatedsound example, it should do what you are looking for. It doesn’t use any existing files, you can see the createSound call has ‘0’ for the name_or_data parameter. That example generates a tone using ‘sin’ in the read callback, which is where you could substitute your data from the network.

  • You must to post comments
0
0

[quote:1w9ad1qo]do I have to have a loop which continously checks if getRecordPosition > 0?[/quote:1w9ad1qo]
If you want to continuously send data you would need a looping record buffer (you specify loop in recordStart). So for example if your record buffer was 2 seconds, you would wait until the record position is past the 1 second mark, then lock the 1st second of data, then you would wait until the record position has passed the 2 second mark (this this case it would wrap to 0), then lock that 2nd second of data.

Obviously you can make the buffers smaller so there is less delay between recording and sending data over the network.

Doing this wont halt the recording, you can be locking / reading from the first chunk of data while FMOD is writing to the second chunk (this is double buffering).

[quote:1w9ad1qo]I guess the loop’d function will have to run on a separate thread then aside from the main program? [/quote:1w9ad1qo]
Not necessarily, you could just poll the record position regularly, FMOD isn’t thread safe so take care.

[quote:1w9ad1qo]Does the exinfo->length indicte the buffersize? [/quote:1w9ad1qo]
Yes

[quote:1w9ad1qo]More importantly; When I got stuff to send, do I empty the buffer or does it get rewritten(meaning while recording it simply starts at the beginning if buffer full)? How does this interact with a socket for example which sends the buffer and is therefore reliant on the audio not getting overwritten?[/quote:1w9ad1qo]
You would send in chunks, and yes the recording will go back to the start (overwriting data as it goes). Therefore you need to send or copy the recording data before FMOD overwrites it, adjusting buffer sizes to ensure you never lose data.

  • You must to post comments
0
0

[quote="mathew":y9gh35nk][quote:y9gh35nk]do I have to have a loop which continously checks if getRecordPosition > 0?[/quote:y9gh35nk]
If you want to continuously send data you would need a looping record buffer (you specify loop in recordStart). So for example if your record buffer was 2 seconds, you would wait until the record position is past the 1 second mark, then lock the 1st second of data, then you would wait until the record position has passed the 2 second mark (this this case it would wrap to 0), then lock that 2nd second of data.

Obviously you can make the buffers smaller so there is less delay between recording and sending data over the network.

Doing this wont halt the recording, you can be locking / reading from the first chunk of data while FMOD is writing to the second chunk (this is double buffering).

[quote:y9gh35nk]I guess the loop’d function will have to run on a separate thread then aside from the main program? [/quote:y9gh35nk]
Not necessarily, you could just poll the record position regularly, FMOD isn’t thread safe so take care.

[quote:y9gh35nk]Does the exinfo->length indicte the buffersize? [/quote:y9gh35nk]
Yes

[quote:y9gh35nk]More importantly; When I got stuff to send, do I empty the buffer or does it get rewritten(meaning while recording it simply starts at the beginning if buffer full)? How does this interact with a socket for example which sends the buffer and is therefore reliant on the audio not getting overwritten?[/quote:y9gh35nk]
You would send in chunks, and yes the recording will go back to the start (overwriting data as it goes). Therefore you need to send or copy the recording data before FMOD overwrites it, adjusting buffer sizes to ensure you never lose data.[/quote:y9gh35nk]

Hello,

By making the buffer smaller, I assume you mean locking and sending in smaller intervals? Is ‘seconds’ the best unit to use; getRecordPosition returns number of PCM samples(2 bytes per PCM sample?), would it not be best to send as soon as a sample is detected?

Cheers!

  • You must to post comments
0
0

[quote:3t6eay3i]By making the buffer smaller, I assume you mean locking and sending in smaller intervals?[/quote:3t6eay3i]
Yes, the closer you are reading to the record position the less latency.

[quote:3t6eay3i]Is ‘seconds’ the best unit to use[/quote:3t6eay3i]
No not really :p it was just to convey the idea, x number of samples is probably more appropriate.

[quote:3t6eay3i]getRecordPosition returns number of PCM samples(2 bytes per PCM sample?)[/quote:3t6eay3i]
Yes, for mono that is correct, stereo would be twice as much.

[quote:3t6eay3i]would it not be best to send as soon as a sample is detected?[/quote:3t6eay3i]
This is up to how frequently you want to send data, and regularly you poll etc. You’ll probably notice that the record position updates in blocks anyway, 512 or 1024 samples at a time.

  • You must to post comments
0
0

I see.

I have yet another question if you don’t mind :) This time about the looping. In all the examples, you have a loop which sleeps() for a certain amount of time. I was thinking of using a more event-driven approach, to process data when getRecordPosition > 0 or something, instead of continously polling, like this;

while (true) {
if (getRecordPosition>0)
process data
sleep(1)
}

Sleep() is abit wasteful (sleep(1) is more like 40ms too in practice and I try to strive for performance) so I wanted to try something else.

I made a thread on it on Stackoverflow.com (http://stackoverflow.com/questions/4648 … hout-sleep) . The first suggestion means altering the source code, to trigger an event where you write into the buffer.

1) Is altering the source code available/allowed? It’s a non-commercial project, just something for the education/fun.
2) I am not familiar with the actuall source code; where would one set the event so I can make it event-driven?
3) In practice you probably wouldn’t notice the difference between the two solutions to the looping recording, but in theory this would provide lower latency and more efficient cpu-usage right?
4) If source code is not available, is sleep() the most efficent way?

Thanks in advance!

  • You must to post comments
0
0

Firstly it should be made clear that getRecordPosition > 0 doesn’t mean there is data you should copy. When doing loop recording FMOD will write into a ring buffer, you need to remember the position you locked up to so your next read is from that point (taking any wrapping into account).

Now onto answering your questions: you really don’t want to be sleeping for 1ms since FMOD will be updating the record buffer in chunks anyway. Also since you are sending over the internet you will incur latency there too, so sleeping for 1ms or 40ms won’t impact overall latency significantly.

1) Altering the source is allowed, but we only provide source to licensees.
2) See above.
3) Sleep certainly isn’t wasteful as the code will yield to other threads, we use it in all of FMOD, events are useful for precision though (however unavailable in this case).
4) Yes, although you don’t have to be sleeping while you wait, you could be doing other work in your application perhaps?

  • You must to post comments
0
0

I see. The main reason I would use sleep() for would be to throttle the CPU usage, otherwise it would run at full speed with CPU temp and fans screaming (and draining batteries faster), which is not the experience I want for the user.

I guess game applications have a continous update event called in which they could do the audio polling, I am making a win32 application so its event-driven so there’s nothing that gets called frequently for me to use :)

Would there be another way to slow down CPU usage?

Thanks in advance

  • You must to post comments
0
0

[quote:19gnpfqs]Would there be another way to slow down CPU usage? [/quote:19gnpfqs]
Simply don’t poll the audio as frequently, it’s a simple trade off of CPU vs latency. If you are using win32 events you can probably use a timer of some description to have it do the job of calling your poll code regularly.

  • You must to post comments
0
0

Alright, I understand. I got a good setup working right now, and even more questions!

1) After I have a segment of the recording buffer to send, I lock() that segment. What happens if the recording is trying to write to that segment, will it simply jump to the next available non-locked position in the buffer and write it there?

2) Would it be possible to send the data in the locked() segment directly over the network or should you need to copy the data segment to a temporary buffer before you send it? The protocol in question is RTP.

3) Whenever I receive an audio packet, I want to play it directly. I am not sure what type of Sound to use for this. I realise a FMOD_CREATESTREAM is logical since I am getting a data stream from the internet, but how would that deal with playing multiple voices at the same time? Afaik, in the Stream you feed all incomming sound into a buffer which you play, so all incomming audio is played in a synchronised way, or am I wrong?
I am simply looking for a way to play a number of bytes directly.

Thanks in advance!

  • You must to post comments
Showing 13 results
Your Answer

Please first to submit.