0
0

Hi Brett,

I’ve looked into the options of putting our own streamer to replace FMOD’s one, but I’m out of luck.

I first thought (blindly) that I can use the PCMREADCALLBACK, PCMSETPOSITIONCALLBACK, and NONBLOCKING one to wrap our asynchronous file system and stream XMA files.

I was kind of successfull with the PCM16 format (but my streaming double buffered code was inferior to what FMOD already had, i really wrote it in half an hour), but I was never able to use that for XMA.

Now I guess there comes the name – it’s pcmreadcallback, so it’s only intended for PCM’s…

I’ve looked into the SetFileSystem stuff, but it would not work for us, as there is no way to prebuffer (there is only READ). While with the pcmreadcallback I can prebuffer (as I know what to prebuffer in advance) then with the filesystem read wrapper I can’t do that, as it’s blocking.

So my question is: How does FMOD reads data on WIN32 and similiar platforms: Does it use ReadFile OVERLAPPED (asynchronous) or just ReadFile (non-asynchronous) but in a separate thread? I’ve probably missed that somewhere in the documentation.

Second question: What would be the correct way, If I want to use my own file system (which supports asynchronous reading, and checking for end of reading) and non-PCM type of format.

Thanks,
malkia

  • You must to post comments
0
0

Wow this thread is over a year old, i’m not sure why you bumped it.
Anyway, considering i’ve had 16 seperate streams running from a ps2 cd and from all parts of the cd (especially on the inner tracks where it is slower to read), i’m pretty sceptical of your claim of ‘several seconds’. I’ve never experienced anything like that in years of coding ps2.
You’d be doing something wrong to get it taking that long, or the disk would be scratched / corrupted.

  • You must to post comments
0
0

[quote="brett":2dzkw8hj]Wow this thread is over a year old, i’m not sure why you bumped it.[/quote:2dzkw8hj]
Wait, it’s not 2006?

  • You must to post comments
0
0

Aha, I understand. It’s just that we already have our own streamer, which for example sorts all the file i/o requests, in future it’ll try to combine some of them into one, etc.

We also are doing streaming-type of a game for the consoles, and we want to have as much as possible full control of what is streamed off the media.

Probably we should issue FMOD source code license, and then we can hack it our own way (if the FMOD license allows us to do that).

There is also TRC on various platforms, which we have to follow when comes to I/O (fortunately when comes to sound, most of them are just saying even if no sound plays that is fine, but we’ll see).

I’ll write you next week a mail from work, and talk more about the source-code license.

Cheers,
malkia

  • You must to post comments
0
0

[quote="brett":cy7fmabr]Why would you want to use your own streamer?[/quote:cy7fmabr]

I’ve got the same problem. I would like to use our engines existing file I/O system to handle the streaming of audio data. When working on consoles in the past, having multiple threads trying to access the CD\DVD drive at the same time was bad as it lead to lots of disc thrashing, and seek times on optical drives are terrible. I’d really like to only have one thread talking to the drive, so I need to figure out how to use my own streamer, or how to read non-audio data through FMOD.

Am I just being paranoid about the disc thrashing problem? I can’t be the only person working on a project that does non-audio file I/O while sounds are streaming?

-Chris

  • You must to post comments
0
0

[quote="csakanai":3lwncdlh][quote="brett":3lwncdlh]Why would you want to use your own streamer?[/quote:3lwncdlh]

I’ve got the same problem. I would like to use our engines existing file I/O system to handle the streaming of audio data. When working on consoles in the past, having multiple threads trying to access the CD\DVD drive at the same time was bad as it lead to lots of disc thrashing, and seek times on optical drives are terrible. I’d really like to only have one thread talking to the drive, so I need to figure out how to use my own streamer, or how to read non-audio data through FMOD.

Am I just being paranoid about the disc thrashing problem? I can’t be the only person working on a project that does non-audio file I/O while sounds are streaming?

-Chris[/quote:3lwncdlh]

We are the same, our games were in the past and now were always streaming (streaming a big city, or levels) and it’s important for us to be able to control the streamer our own way.

For example we do sorting of all I/O requests by sector position, or offset in the big packed file (on consoles where you don’t have sector position), then we have priorities for some of the reads, and we are planning on doing scattering (or gathering?) reads (read from one place, distribute to different memory regions).

We’ll probably license FMOD for source-code version, as we can then hack it to go through our file-system (as much as it’s possible).

  • You must to post comments
0
0

For one of our upcoming games, we would like to be able to LOAD the LEVEL data, while streaming a MOVIE.

Which means, we have to interleave MOVIE (mpeg probably), SOUND and GAME-DATA.

For example the CRI SDK allows us to do that, and even has a filesystem that supports interleaved files. Unfortunately the latter does not have the mixing capabilities of FMOD, and only support their own software-only formats (ADX and AHX).

And yes, it’s really important that the loading times are decreased to a minimum even if the cost of doing that is more time spend coding, and testing different things.

  • You must to post comments
0
0

Just doing the math based on generic DVD drives gives about one meg of data that could be read insted of one seek. (if the data were properly interleaved)

When talking about a streaming game (such as the one I am working on) adding a second stream is a huge hit. Easily 10% of possible bandwidth is consumed just moving the head around. (that stat is for relativly high granularity loads even) That is 10% more geometric detail or audio content or texture resolution that could enhance the game.

DVD bandwidth is the number one bottleneck for a streaming game and is something that deeply impacts all of our subsystems. We will be using every last byte of bandwidth and have and will spend significant time optimizing this path specificly for each platform.

  • You must to post comments
0
0

Have you considered using your own streamer to read the audio data in advance in large chunks and then using a FMod file callback to read out of that data buffer? On the XBox and XBox 360 you should have more than enough RAM to do that, and as long as you perform reads using your streamer often enough you should be able to get good performance.

  • You must to post comments
0
0

[quote="Janus":1v4nzj4u]Have you considered using your own streamer to read the audio data in advance in large chunks and then using a FMod file callback to read out of that data buffer? On the XBox and XBox 360 you should have more than enough RAM to do that, and as long as you perform reads using your streamer often enough you should be able to get good performance.[/quote:1v4nzj4u]

You can do that for PCM data, through the PCMREADCALLBACK. I was able to do that, and it worked just fine (simple double buffer).

But PCMREADCALLBACK is not being called, if your data is not PCM (and in the case of XMA … of course it’s not).

So we have to override the Global File System, but at that level you are losing the context – e.g. you can’t prebuffer, as you don’t know what to pre-buffer. In the case of PCMREADCALLBACK you know what would be read in advance, and you can do that, but with global file callbacks (setFileSystem) you don’t know.

So as you can’t prebuffer, the only option is if the filesystem layer was written in asynchronous interface, but that’s not the way it is.

As far as my understanding goes, FMOD, internally does not use asynchornous reading (e.g. OVERLAPPED, or aio_read, or whatever the consoles have (NDA here, can’t talk)) but uses a separate thread where it reads in a blocking manner. Now doing the latter is probably fine, and probably I can wrap that, but I would be called from that thread, not from the one I’m expecting… Then again, here is a problem with using blocking reads – you can’t cancel them – you have to wait for them to finish, or abruplty kill the thread, and hope that the file system would stop the read. And if your buffers are big, you’ll have lots of time spent waiting for that read to finish, if you decide to cancel it.

Now i’m just theorizing, I don’t have the source code to know, probably Brett can tell us more.

But we have to stress it again:

  1. We need to wrap FMOD through our file system
  2. We need to make sure that if a streaming sound is being interrupted, or bank which is currently loading must be unloaded, then the latter is done as soon as possible (e.g. CancelIO on Win32, aio_cancel on unix, and whatever other function on the other consoles)
  3. On DVD-R’s, the fastest way to read is to always read from 32768 bytes location, with 32768 bytes sizes. Why? It’s because of the ECC checksum which is made for every 16 sectors (16*2048=32768). If you don’t keep that, then you’ll always end up with the same speeds in the inner and outer sides of the disc, and that’s not good.
  4. Interleaving the sound data with custom game data. Say for example that I wan’t to play streamed geometry animation (not-movie, but geometry animation streamed off the disk) and the same time soundtrack for it. Those two needs to be interleaved, otherwise you’ll end up seeking back and forth between the two pieces.

Here is more on the DVD ECC:

[url:1v4nzj4u]http://www.ecma-international.org/publications/files/ECMA-ST/Ecma-267.pdf[/url:1v4nzj4u]

[quote:1v4nzj4u]18
ECC Blocks
An ECC Block is formed by arranging 16 consecutive Scrambled Frames in an array of 192 rows of 172 bytes each
(figure 20). To each of the 172 columns, 16 bytes of Parity of Outer Code are added, then, to each of the resulting 208
rows, 10 byte of Parity of Inner Code are added. Thus a complete ECC Block comprises 208 rows of 182 bytes each.
The bytes of this array are identified as B
i,j
as follows, where i is the row number and j the column number.
B
i,j
for i = 0 to 191 and j = 0 to 171 are bytes from the Scrambled Frames
B
i,j
for i = 192 to 207 and j = 0 to 171 are bytes of the Parity of Outer Code
B
i,j
for i = 0 to 207 and j = 172 to 181 are bytes of the Parity of Inner Code”[/quote:1v4nzj4u]

  • You must to post comments
0
0

[quote="brett":2bqst9lu]Seeking once every second or 2 is not going to impact your data read that much, and is a lot simpler.[/quote:2bqst9lu]
Unfortunately, that’s not true. A seek on PS2, for example, can take several [b:2bqst9lu]seconds[/b:2bqst9lu]. Anything you can do to avoid seeks, no matter how complex, is usually worth it, in a streaming environment.

  • You must to post comments
Showing 10 results
Your Answer

Please first to submit.