So, Casey mentions here:
https://youtu.be/qGC3xiliJW8?t=4266
that we use a 2 second sound buffer, because it sounds better when we drop frames. i didn't really get his explanation though.
First, he brought up that there are 2 ways to actually approach a audio dropout:
first, you could write way ahead of the actual game. he said the negatives with this is that it creates latency (i understand this) and that it can cause overwriting (i have no idea what this means)
second, you could write one frame ahead, but if the buffer was too small, then you would hear the contents of the buffer repeating again and again. if this is the case, how is a 2 second buffer better? is it because, for music, for example, you would not hear a 100ms clip repeating again and again, and would instead hear a 2-second clip which sounds a lot more normal than a very short clip when put on loop? According to Casey, when using a 2 second buffer, "even if your game gets hammered for some reason... you'll still actually be fine. it can play forever on that 2 second buffer, and there'll only be one skip point." I don't really get this.
i guess to summarize my main 2 questions are:
- what is overwriting and how would writing way ahead of the game cause it?
- how does having a 2 second buffer, lead to having only one "skip point" as Casey put it / having more normal sounding audio during a audio dropout?
Thanks!
https://youtu.be/qGC3xiliJW8?t=4266
that we use a 2 second sound buffer, because it sounds better when we drop frames. i didn't really get his explanation though.
First, he brought up that there are 2 ways to actually approach a audio dropout:
first, you could write way ahead of the actual game. he said the negatives with this is that it creates latency (i understand this) and that it can cause overwriting (i have no idea what this means)
second, you could write one frame ahead, but if the buffer was too small, then you would hear the contents of the buffer repeating again and again. if this is the case, how is a 2 second buffer better? is it because, for music, for example, you would not hear a 100ms clip repeating again and again, and would instead hear a 2-second clip which sounds a lot more normal than a very short clip when put on loop? According to Casey, when using a 2 second buffer, "even if your game gets hammered for some reason... you'll still actually be fine. it can play forever on that 2 second buffer, and there'll only be one skip point." I don't really get this.
i guess to summarize my main 2 questions are:
- what is overwriting and how would writing way ahead of the game cause it?
- how does having a 2 second buffer, lead to having only one "skip point" as Casey put it / having more normal sounding audio during a audio dropout?
Thanks!
Edited by Draos
on
- Overwriting is just the fact that if you write some sample in advance (let's say 1 second worth), and need to add another sound after after 0.5 second, you'll need to overwrite 0.5 second of samples that are already in the buffer if you want to have low(ish) latency. It's a little complicated to do because you need to keep track of what is in the buffer since you can't just add to the existing sample values in the buffer.
- Having a buffer of 2 seconds means that if for some reason the frame time is too long, you'll only have one glitch in audio every 2 seconds. In practice, since we write the audio for the next frame or two, if the game stop responding, you'll hear a audio glitch followed by about two seconds of old audio (what was written in the buffer the previous 2 seconds) and that will repeat until the game start writing audio again. If you have a small buffer, lets say 4 frames worth of sample, you'll hear a glitch, followed by only 2 or 3 frames of old audio, and that will loop. This will be far worse to listen to.
So in both case there is only one skip point, but with small buffer the skip point will loop far more often.
- Having a buffer of 2 seconds means that if for some reason the frame time is too long, you'll only have one glitch in audio every 2 seconds. In practice, since we write the audio for the next frame or two, if the game stop responding, you'll hear a audio glitch followed by about two seconds of old audio (what was written in the buffer the previous 2 seconds) and that will repeat until the game start writing audio again. If you have a small buffer, lets say 4 frames worth of sample, you'll hear a glitch, followed by only 2 or 3 frames of old audio, and that will loop. This will be far worse to listen to.
So in both case there is only one skip point, but with small buffer the skip point will loop far more often.
Ok,so I got the other question, but overwriting still confuses me. Casey said that if we write much further than where we are playing we either get lots of latency or overwrite. I still dont get how writing to a point in the buffer far away causes overwrite whereas writing one or two frames ahead of the play cursor would not.
Edited by Draos
on
When Casey says "you can write way ahead", he means adding a lot of valid sample in the buffer.
When you overwrite, you actually overwrite starting as close as you can from the play cursor to reduce the latency.
To avoid sound glitches you try to always have some valid samples in the buffer. To do that, in this case we chose to always have (for example) 1 second of audio ahead of the play cursor. If you don't overwrite (you only add audio after what has already been written), you've a minimum latency of 1 second, which is bad because because you will notice the delay.
If you want to have a lower latency, you can chose to replace (overwrite) the samples that are in the buffer.
But with overwriting you'll write several time in the same part of the buffer (the amount of data is not really an issue) and you need to make sure you keep things continuous to avoid audio glitches (if you overwrite part of a sound, you need to make sure you overwrite with the same sample as the first upload). Also overwriting is not possible in all audio API, for example I think you can't overwrite if you use WASAPI.
When you overwrite, you actually overwrite starting as close as you can from the play cursor to reduce the latency.
To avoid sound glitches you try to always have some valid samples in the buffer. To do that, in this case we chose to always have (for example) 1 second of audio ahead of the play cursor. If you don't overwrite (you only add audio after what has already been written), you've a minimum latency of 1 second, which is bad because because you will notice the delay.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | ^ is the play cursor + is latency - what was previously in the buffer Before we start, the buffer is empty: |----------| ^ The first time we write 1 second |xxxxx-----| ^++++ After that, we write 1 frame worth of sample each frame |xxxxxy----| ^++++ |xxxxxyz---| ^++++ |
If you want to have a lower latency, you can chose to replace (overwrite) the samples that are in the buffer.
1 2 3 4 5 6 7 8 9 | Initial audio |xxxxx-----| ^++++ Something happened (e.g. an explosion), we need to replace the audio (we keep one frame after the play cursor), latency is small |xxxyyy----| ^+ Another thing happened |xxxyzzz---| ^+ |
But with overwriting you'll write several time in the same part of the buffer (the amount of data is not really an issue) and you need to make sure you keep things continuous to avoid audio glitches (if you overwrite part of a sound, you need to make sure you overwrite with the same sample as the first upload). Also overwriting is not possible in all audio API, for example I think you can't overwrite if you use WASAPI.
Edited by Simon Anciaux
on
sorry, if i'm picking this up super slow, but could you just explain this line
you need to make sure you keep things continuous to avoid audio glitches (if you overwrite part of a sound, you need to make sure you overwrite with the same sample as the first upload).
i don't get what you mean by same sample as the first upload, and continuous.
amazing answers btw. i really appreciate it!
you need to make sure you keep things continuous to avoid audio glitches (if you overwrite part of a sound, you need to make sure you overwrite with the same sample as the first upload).
i don't get what you mean by same sample as the first upload, and continuous.
amazing answers btw. i really appreciate it!
If you played 100 samples from a sound, and need to overwrite, you need to start overwriting with the 101th sample of that sound. You don't know this number in advance and computing it wrong would make you hear a glitch in audio.
It's nothing hard, just a little bit harder than not using overwriting and keeping track of the last sample you uploaded.
It's nothing hard, just a little bit harder than not using overwriting and keeping track of the last sample you uploaded.
Edited by Simon Anciaux
on
Btw with WASAPI what you typically do is to have high priority thread that submits audio buffers with as small as possible increments (to reduce latency) from data you already prepared ahead (second or two). This way you can still "overwrite" samples that should be submitted in future - because "overwriting" will happen in buffer your application owns. And in case of some delay in main thread of application, the audio data will still be submitted and there won't be any glitch.
Only annoying part is to correctly synchronize submission thread with new data uploads from main/mixer thread, but this can be done pretty efficiently with "lock-free" primitives.
I'm pretty sure DirectSound is doing something similar behind the scenes - because it uses WASAPI as backend.
Only annoying part is to correctly synchronize submission thread with new data uploads from main/mixer thread, but this can be done pretty efficiently with "lock-free" primitives.
I'm pretty sure DirectSound is doing something similar behind the scenes - because it uses WASAPI as backend.
Edited by Mārtiņš Možeiko
on