Hi everyone,
I am currently trying to get sound working on Linux, and I chose Alsa as the common denominator between most Linux distros.
The way Alsa works is that there is no concept of a "write" or "play" cursor. It seems that you simply feed data to the sound buffer and it plays it. If you don't write enough data in a certain amount of time, you get a buffer underrun.
I am unsure the way to model my game sound model which will both a) not cause a buffer underrun, or b) be in sync with the game state. I have looked at nxsy's early Linux platform layer of handmade hero, but it looks like it suffers from some buffer underruns itself.
I have tried a few things:
1) Only write the amount of samples produced in a single frame into the Alsa sound buffer. This causes a buffer underrun.
2) Start off with writing 60ms of silence (idea from nxsy's). This doesn't cause an underrun, but now I am now 60 ms out of sync. Lowering this time will eventually cause a buffer underrun.
3) Buffer a bunch of frames worth of samples and then write them all out to Alsa. This will eventually cause a buffer overrun on the game sound buffer. I know what I could do is create a thread that will smartly feed the game buffer sample into alsa, but I don't want to have to deal with threads just yet. I feel there is an obvious single threaded solution I'm not thinking of.
Does anyone have any experience with Alsa with any single threaded suggestions?
Thanks very much!
I am currently trying to get sound working on Linux, and I chose Alsa as the common denominator between most Linux distros.
The way Alsa works is that there is no concept of a "write" or "play" cursor. It seems that you simply feed data to the sound buffer and it plays it. If you don't write enough data in a certain amount of time, you get a buffer underrun.
I am unsure the way to model my game sound model which will both a) not cause a buffer underrun, or b) be in sync with the game state. I have looked at nxsy's early Linux platform layer of handmade hero, but it looks like it suffers from some buffer underruns itself.
I have tried a few things:
1) Only write the amount of samples produced in a single frame into the Alsa sound buffer. This causes a buffer underrun.
2) Start off with writing 60ms of silence (idea from nxsy's). This doesn't cause an underrun, but now I am now 60 ms out of sync. Lowering this time will eventually cause a buffer underrun.
3) Buffer a bunch of frames worth of samples and then write them all out to Alsa. This will eventually cause a buffer overrun on the game sound buffer. I know what I could do is create a thread that will smartly feed the game buffer sample into alsa, but I don't want to have to deal with threads just yet. I feel there is an obvious single threaded solution I'm not thinking of.
Does anyone have any experience with Alsa with any single threaded suggestions?
Thanks very much!
I'm also using ALSA and right now I'm accepting 50ms of delay and I still get buffer underruns when switching to another window and back.
The single-threaded solution that I will eventually implement is:
- Start by feeding ALSA several video frames worth of audio data (snd_pcm_writei)
- On every frame, top the buffer off to the desired buffer fill level (to find out the current buffer fill level, subtract the value returned by snd_pcm_avail from the one returned by snd_pcm_hw_params_get_buffer_size)
- Whenever I need to correct the contents of the buffer (because of user input, collisions between objects, etc.), I check how much of the buffer I can safely modify (snd_pcm_rewindable), rewind (snd_pcm_rewind) and rewrite it (snd_pcm_writei)
These safe ALSA subset guidelines advise against assuming that snd_pcm_rewind will be available or work as expected, but also recommend using it. I guess that speaks both to the quality of the ALSA subsystem and to the quality of the guidelines themselves.
The single-threaded solution that I will eventually implement is:
- Start by feeding ALSA several video frames worth of audio data (snd_pcm_writei)
- On every frame, top the buffer off to the desired buffer fill level (to find out the current buffer fill level, subtract the value returned by snd_pcm_avail from the one returned by snd_pcm_hw_params_get_buffer_size)
- Whenever I need to correct the contents of the buffer (because of user input, collisions between objects, etc.), I check how much of the buffer I can safely modify (snd_pcm_rewindable), rewind (snd_pcm_rewind) and rewrite it (snd_pcm_writei)
These safe ALSA subset guidelines advise against assuming that snd_pcm_rewind will be available or work as expected, but also recommend using it. I guess that speaks both to the quality of the ALSA subsystem and to the quality of the guidelines themselves.
Edited by Miguel Lechón
on
debiatan
- Whenever I need to correct the contents of the buffer (because of user input, collisions between objects, etc.), I check how much of the buffer I can safely modify (snd_pcm_rewindable), rewind (snd_pcm_rewind) and rewrite it (snd_pcm_writei)
Oh man, that sounds a nightmare. I feel it would almost be easier to use the multi-threaded solution. Please correct me if I am wrong.
I haven't use snd_pcm_rewind before, but it looks like Pulse Audio (which I think most modern Linux distros have, right?) doesn't support this:
http://www.spinics.net/lists/alsa-devel/msg31536.html
And it seems that rewinding is broken in general:
http://mailman.alsa-project.org/p...sa-devel/2011-October/045485.html
Granted, these are from 2010-2011, but it sounds like in general, the concept of rewinding is broken on Alsa, especially on versions from these years.
Regardless, I'll debate whether to use your solution, or just do a multi threaded solution. It really sounds like the Alsa API just isn't well designed for games in general. Thanks for your reply.
Also, I realized that I accidentally posted in the wrong forum. I meant to post this in Code discussion, so if any moderator sees this thread, feel free to move it to Code Discussion, if it's possible.
Edited by Dan
on
I can't think of any other way around the latency/buffer underrun issue using just one thread and ALSA so, if we can trust snd_pcm_rewind to work reliably, using multiple threads sounds like a better option.
I don't use pulseaudio myself, but yes, you want to take its presence into account. SteamOS ships with pulseaudio enabled.
I don't use pulseaudio myself, but yes, you want to take its presence into account. SteamOS ships with pulseaudio enabled.
At the moment I use SDL2 for the sound.
I don't think it is a good idea to use ALSA nowadays.
Even Linux from Scratch (blfs) takes PA into account.
But...
lately I was looking on a shadertoy standalone implementation
https://github.com/simon-budig/shadertoy | shadertoy.com
and tried to implement sound reactive shaders with libmpg and libao.
http://hzqtc.github.io/2012/05/play-mp3-with-libmpg123-and-libao.html
My patches aren't pushed but I solved the timing resolution issues.
I am pretty close to make that work. Already playing sound, but atm
I stopped, cause it needs a complete SDL2 based rewrite and I want to add a shader manager.
I think audio responsive parts of a game are a great Idea,
even if it is just the menu or a demo mode or any other game state, wheter affects lightning, animation or whatever.
https://github.com/jimbo00000/kinderegg solved the sound issue.
Might be worth checking the PA Code:
https://github.com/spurious/SDL-m...audio/pulseaudio/SDL_pulseaudio.c
For the GPU Code I'll focus on github.com/grimfang4/sdl-gpu
I think it is important to understand the ongoings on the local machine, but if I want to contribute or improve "real world code", I would submit it, as a patch to the SDL-Team.
I don't know much about the Linux sound stack, maybe this site helps?
http://moi.vonos.net/linux/sound-stack/
I don't think it is a good idea to use ALSA nowadays.
Even Linux from Scratch (blfs) takes PA into account.
But...
lately I was looking on a shadertoy standalone implementation
https://github.com/simon-budig/shadertoy | shadertoy.com
and tried to implement sound reactive shaders with libmpg and libao.
http://hzqtc.github.io/2012/05/play-mp3-with-libmpg123-and-libao.html
My patches aren't pushed but I solved the timing resolution issues.
I am pretty close to make that work. Already playing sound, but atm
I stopped, cause it needs a complete SDL2 based rewrite and I want to add a shader manager.
I think audio responsive parts of a game are a great Idea,
even if it is just the menu or a demo mode or any other game state, wheter affects lightning, animation or whatever.
https://github.com/jimbo00000/kinderegg solved the sound issue.
Might be worth checking the PA Code:
https://github.com/spurious/SDL-m...audio/pulseaudio/SDL_pulseaudio.c
For the GPU Code I'll focus on github.com/grimfang4/sdl-gpu
I think it is important to understand the ongoings on the local machine, but if I want to contribute or improve "real world code", I would submit it, as a patch to the SDL-Team.
I don't know much about the Linux sound stack, maybe this site helps?
http://moi.vonos.net/linux/sound-stack/
Edited by Carsten Holtkamp
on
Reason: small addition
Using just pulse audio isn't ideal. People with only alsa available on their systems won't have sound.
But if you use alsa, then people who use pulse can have a sound. Pulse audio can provide alsa virtual device. So all audio that goes to alsa, gets routed back to pulse.
But if you use alsa, then people who use pulse can have a sound. Pulse audio can provide alsa virtual device. So all audio that goes to alsa, gets routed back to pulse.
I actually was able to get Alsa to work in a single-threaded way. The issue I was having is that I was writing a fixed amount samples to Alsa every frame. I needed to make sure that I was writing the number of samples based on the delta time between each frame. That is, I changed:
to
It also seemed important that I batch together about 200ms of samples on initial startup before writing to alsa regularly every frame: That is:
It seems to okay insofar as that there are not really any buffer underruns. Seems to be pretty free of delay on my gaming desktop, but not so much on the Linux VM on my laptop.
Now I am trying to do audio sync debugging with the hash marks that Casey went over. Unfortunately, there seems to be no concept of a PlayCursor in Alsa, but I'm looking into snd_pcm_status_get_audio_htstamp to see if that will help....
1 | u32 samplesToWrite = gameSoundState->sampleRate/60; |
to
1 | u32 samplesToWrite = gameSoundState->sampleRate*deltaTimeInSeconds; |
It also seemed important that I batch together about 200ms of samples on initial startup before writing to alsa regularly every frame: That is:
1 2 3 | if (secondsElapsedSinceStartOfProgam > .2) {
playSamplesToAlsa(&alsaSoundState, &gameSoundState);
}
|
It seems to okay insofar as that there are not really any buffer underruns. Seems to be pretty free of delay on my gaming desktop, but not so much on the Linux VM on my laptop.
Now I am trying to do audio sync debugging with the hash marks that Casey went over. Unfortunately, there seems to be no concept of a PlayCursor in Alsa, but I'm looking into snd_pcm_status_get_audio_htstamp to see if that will help....
Hey @DanB91, if possible can u share the code? I am having latency issues with ALSA and it wld help a lot
I opened a PCM device and the hardware params after initialization are as follows:
I have set the hardware params such that period time = seconds per game update(33ms for 30fps and 16ms for 60fps), buffer size is 1 sec long and from these two quantities I set periods per buffer = (buffer size/period size)
The logic to fill the buffer is as follows:
Initially expectedSamplesPerFrame is the 1 frame(i.e game update) worth of samples.
Then I query ALSA for availableSamples and pendingSamples.
Given pendingSamples, I need to write (expectedSamplesPerFrame - pendingSamples) samples to line up with the frame flip.
Finally the amount of frames to write is the minimum of (expectedSamplesPerFrame - pendingSamples) and availableSamples.
When expectedSamplesPerFrame is exactly 33ms worth of samples, I constantly get underruns.
So I tried increasing expectedSamplesPerFrame to 2 frames worth of samples. Here after first few frames the Pending gets stuck at 0.067s, due to which Expected is set to 0s and thus the program doesn't write anything to the buffer for the rest of its execution. The write doesn't throw underrun or any other errors.
Finally I tried filling whatever samples were available i.e avaiableSamples, which resulted in lesser pop sounds but latency was very high.
How do I decide the amount of samples to write so as to have minimum latency and lesser sound drops?
Cause I tried increasing expectedSamplesPerFrame from 33ms to whatever was the max available in buffer but the issue persisted.
I opened a PCM device and the hardware params after initialization are as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | PCM handle name = 'default' PCM state = PREPARED access type = RW_INTERLEAVED format = 'S16_LE' (Signed 16 bit Little Endian) subformat = 'STD' (Standard) channels = 2 rate = 48000 bps period time = 33333 us period size = 1600 frames buffer time = 1000000 us buffer size = 48000 frames periods per buffer = 30 frames exact rate = 48000/1 bps significant bits = 16 |
I have set the hardware params such that period time = seconds per game update(33ms for 30fps and 16ms for 60fps), buffer size is 1 sec long and from these two quantities I set periods per buffer = (buffer size/period size)
The logic to fill the buffer is as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | local_persist int runningSample = 0; local_persist int toneVolume = 3000; local_persist int hz = 480; int bytesPerWave = (bytesPerSample * samplesPerSecond)/hz; int samplesPerWave = bytesPerWave/bytesPerSample; int16 *currentSample = (int16*)soundBuffer; currentSample = (int16*)soundBuffer; snd_pcm_sframes_t expectedSamplesPerFrame = (int32)((real32)samplesPerSecond/(real32)framesPerSecond); snd_pcm_sframes_t availableSamples; snd_pcm_sframes_t pendingSamples; snd_pcm_avail_delay(soundDeviceHandle, &availableSamples, &pendingSamples); expectedSamplesPerFrame -= pendingSamples; int frames = MINIMUM(expectedSamplesPerFrame, availableSamples); for(int currentFrame = 0; currentFrame < frames; currentFrame++) { int16 sampleValue = toneVolume * Sine((real32)runningSample/(real32)samplesPerWave); *currentSample++ = sampleValue; *currentSample++ = sampleValue; runningSample++; } char debugSound[1024] = {}; snprintf(debugSound, 1024, "\nPending: %.3fs, Avail: %.3fs, Expected: %.3fs Filling: frames: %.3fs\n---------\n", (real32)pendingSamples/(real32)samplesPerSecond, (real32)availableSamples/(real32)samplesPerSecond,(real32)expectedSamplesPerFrame/(real32)samplesPerSecond,(real32)frames/(real32)samplesPerSecond); while((alsaReturnCode = snd_pcm_writei(soundDeviceHandle, soundBuffer, frames)) < 0) { if(alsaReturnCode == -EPIPE) { snprintf(debugSound, 1024, "Underflow: Preparing PCM. %s", snd_strerror(alsaReturnCode)); snd_pcm_prepare(soundDeviceHandle); } else { snprintf(debugSound, 1024, "%s", snd_strerror(alsaReturnCode)); snd_pcm_recover(soundDeviceHandle, alsaReturnCode, 0); } } if (alsaReturnCode != (int)frames) { snprintf(debugSound,1024, "short write, write %d frames\n", alsaReturnCode); } |
Initially expectedSamplesPerFrame is the 1 frame(i.e game update) worth of samples.
Then I query ALSA for availableSamples and pendingSamples.
Given pendingSamples, I need to write (expectedSamplesPerFrame - pendingSamples) samples to line up with the frame flip.
Finally the amount of frames to write is the minimum of (expectedSamplesPerFrame - pendingSamples) and availableSamples.
When expectedSamplesPerFrame is exactly 33ms worth of samples, I constantly get underruns.
1 2 3 4 5 6 7 8 | Pending: 0.000s, Avail: 1.000s, Expected: 0.033s, Filling: 0.033s -> (this happens for first few frames) --------- Pending: 0.024s, Avail: 0.967s, Expected: 0.009s, Filling: 0.009s --------- Pending: 0.009s, Avail: 0.991s, Expected: 0.024s, Filling: 0.024s -> (this happens for the rest of the program) --------- |
So I tried increasing expectedSamplesPerFrame to 2 frames worth of samples. Here after first few frames the Pending gets stuck at 0.067s, due to which Expected is set to 0s and thus the program doesn't write anything to the buffer for the rest of its execution. The write doesn't throw underrun or any other errors.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | Pending: 0.000s, Avail: 1.000s, Expected: 0.067s, Filling: 0.067s --------- Pending: 0.040s, Avail: 0.977s, Expected: 0.027s, Filling: 0.027s --------- Pending: 0.033s, Avail: 0.995s, Expected: 0.034s, Filling: 0.034s --------- Pending: 0.033s, Avail: 0.961s, Expected: 0.033s, Filling: 0.033s --------- Pending: 0.065s, Avail: 0.967s, Expected: 0.001s, Filling: 0.001s --------- Pending: 0.067s, Avail: 0.999s, Expected: 0.000s, Filling: 0.000s -> (this happens for the rest of the program) --------- |
Finally I tried filling whatever samples were available i.e avaiableSamples, which resulted in lesser pop sounds but latency was very high.
1 2 3 4 5 6 7 8 9 10 11 12 13 | Pending: 0.000s, Avail: 1.000s, Expected: 1.000s Filling: frames: 1.000s --------- Pending: 0.969s, Avail: 0.042s, Expected: 0.042s Filling: frames: 0.042s --------- Pending: 0.978s, Avail: 0.044s, Expected: 0.044s Filling: frames: 0.044s --------- Pending: 0.989s, Avail: 0.000s, Expected: 0.000s Filling: frames: 0.000s --------- Pending: 0.956s, Avail: 0.046s, Expected: 0.046s Filling: frames: 0.046s |
How do I decide the amount of samples to write so as to have minimum latency and lesser sound drops?
Cause I tried increasing expectedSamplesPerFrame from 33ms to whatever was the max available in buffer but the issue persisted.
Edited by Karan Joisher
on
Reason: Added details about the latency issues