Hi,
I was wondering how premultiplied alpha and gamma work together. Particularly if the alpha might be influencing the outcome of the sRGBToLinear function.
I was thinking that the outcome might not be correctly linear when not undoing the alpha premultiplication before the sRGBToLinear part. But maybe this is not a problem for some reason that I might be aware of?
I see it like this: c is color, a is alpha
So a color with 1.0 red and alpha 0.5 would be r'=a*r = 1.0*0.5 = 0.5
Doing the sRGBToLinear would then get 0.5*0.5 = 0.25
But you might want
or
Which would still give you 0.5, which in my eyes is the expected representation of 50% transparent red in linear space. Or should the alpha component already have effect on the color? Casey kept the alpha the same in the sRGBToLinear function, so that gives me the idea that alpha should already be linear.
LinearTosRGB would probably also need to involve unpremultiplying.
Premultiplied alpha already had my head hurting, so gamma on top of this really makes my world spin. Anybody have any thoughts on this?
Mox
I was wondering how premultiplied alpha and gamma work together. Particularly if the alpha might be influencing the outcome of the sRGBToLinear function.
I was thinking that the outcome might not be correctly linear when not undoing the alpha premultiplication before the sRGBToLinear part. But maybe this is not a problem for some reason that I might be aware of?
I see it like this: c is color, a is alpha
1 2 | c' = a*c sRGBToLinear(c') = c' * c' = (a*c) * (a*c) = a*a * c*c |
So a color with 1.0 red and alpha 0.5 would be r'=a*r = 1.0*0.5 = 0.5
Doing the sRGBToLinear would then get 0.5*0.5 = 0.25
But you might want
1 | sRGBToLinear(c, a) = a * c*c |
or
1 | sRGBToLinear(c', a) = c'*c' / a |
Which would still give you 0.5, which in my eyes is the expected representation of 50% transparent red in linear space. Or should the alpha component already have effect on the color? Casey kept the alpha the same in the sRGBToLinear function, so that gives me the idea that alpha should already be linear.
LinearTosRGB would probably also need to involve unpremultiplying.
Premultiplied alpha already had my head hurting, so gamma on top of this really makes my world spin. Anybody have any thoughts on this?
Mox
I think you may be right, but don't know, for sure. Anyways, I found the material in this link pretty enlightening.
http://dvd-hq.info/alpha_matting.php
I have tried to understand alphablending, and I found it useful to consider that
1. C = A + (B-A)*k can also be expressed as:
2. C = A*(1-k)+B*k
or that C is a "100% color" that contain a X% of A and 100-x % of B. You can do a "crossfade" by moving k the entire way from 0.0 to 1.0 having A and B come from two different bitmaps.
Some "oldschool" link that may be useful.
http://www.flipcode.com/archives/...sue_03_Timer_Related_Issues.shtml
The way I see it, one must also consider that an artwork can have premultiplied gamma.
But the way I see it, if it looks good already why through cycles at it?
http://dvd-hq.info/alpha_matting.php
I have tried to understand alphablending, and I found it useful to consider that
1. C = A + (B-A)*k can also be expressed as:
2. C = A*(1-k)+B*k
or that C is a "100% color" that contain a X% of A and 100-x % of B. You can do a "crossfade" by moving k the entire way from 0.0 to 1.0 having A and B come from two different bitmaps.
Some "oldschool" link that may be useful.
http://www.flipcode.com/archives/...sue_03_Timer_Related_Issues.shtml
The way I see it, one must also consider that an artwork can have premultiplied gamma.
But the way I see it, if it looks good already why through cycles at it?
Yeah, I know, throwing cycles away isn't my hobby either. But I was merely thinking theoretically. The correct way is good to know before you decide whether you want to use it or not.
I actually found a great link that does support my theory: http://ssp.impulsetrain.com/gamma-premult.html
Searching on google should have been my first stop ;)
It states (if I read it correctly) that the colors should not be premultiplied with a, but with pow(a, 1/2.2) or in our case sqrtf(a), which would make the sRGBToLinear square work again:
I actually found a great link that does support my theory: http://ssp.impulsetrain.com/gamma-premult.html
Searching on google should have been my first stop ;)
It states (if I read it correctly) that the colors should not be premultiplied with a, but with pow(a, 1/2.2) or in our case sqrtf(a), which would make the sRGBToLinear square work again:
1 2 3 4 5 | c' = sqrtf(a)*c
sRGBToLinear(c') = c'*c'
= (sqrtf(a)*c) * (sqrtf(a)*c)
= sqrtf(a)*sqrtf(a) * c*c
= a * c*c
|
Edited by Mox
on
Mostly what I'd say about this is that I'm the wrong one to ask :( I don't really know very much about gamma and blending operations. They are admittedly simple, but I just don't spend any time thinking about them because I haven't ever really had to, and I don't actually experience any artifacts that I'm aware of that make me want to look deeper at them.
My assumption is that you want linear color values, premulitplied by linear alpha, everywhere, if you can get it. So in the ideal world, you would always take your sRGB source data and turn it into linear floating point premulitplied values immediately, and all the math would be done thereafter that way. But when you start doing things like having intermediate buffers that are sRGB, I haven't really thought through what the best things to do are.
It's worth noting that as recently as 1998, Jim Blinn still considered things like this "unsolved problems" because they are traditionally handled so sloppily:
http://www.siggraph.org/s98/conference/keynote/blinn.html
More recently, I feel like both film and game rendering guys have become more careful about these things, even if other software domains haven't. But I don't know if anyone has definitively said "here is the best way to do this always, here you go" in a way that we can point to and say "there is the bible of gamma and alpha". I feel like, for the most part, we probably did a pretty good job on the stream of doing alpha the correct way, but honestly, it would take someone with a lot more experience working through everything to say for sure.
- Casey
My assumption is that you want linear color values, premulitplied by linear alpha, everywhere, if you can get it. So in the ideal world, you would always take your sRGB source data and turn it into linear floating point premulitplied values immediately, and all the math would be done thereafter that way. But when you start doing things like having intermediate buffers that are sRGB, I haven't really thought through what the best things to do are.
It's worth noting that as recently as 1998, Jim Blinn still considered things like this "unsolved problems" because they are traditionally handled so sloppily:
http://www.siggraph.org/s98/conference/keynote/blinn.html
More recently, I feel like both film and game rendering guys have become more careful about these things, even if other software domains haven't. But I don't know if anyone has definitively said "here is the best way to do this always, here you go" in a way that we can point to and say "there is the bible of gamma and alpha". I feel like, for the most part, we probably did a pretty good job on the stream of doing alpha the correct way, but honestly, it would take someone with a lot more experience working through everything to say for sure.
- Casey
It was cool to see that after my quick question pre-stream, you actually devoted half the episode to it :) I guess it had quite some impact, not to mention on the performance...
I do believe the code for loading the bitmap could be optimized by using the following code, which conforms to the online source I posted earlier. It won't be that important of an optimization, but at least it was good for me to see that my thoughts where correct.
I tested this with the following code (Abs is needed as extra function in math) to see if it indeed results in the same values, which it does (within some floating point margin).
If only the actual pixel operations in the Draw calls could be optimized like this :P But I guess -O2 or #ifdefs could come in handy there.
But thanks for spending the time going over this! And I do love seeing you speed through the pixel operations cleaning up the code and everything.
I will try to think about scenarios to test whether your (dare I say "our") solution now works correctly. But I guess it is not as easy as in 3D space, where the diffuse shading for example will be greatly different if done in linear color space.
Mox
PS Sorry for slowing down the main drawing routing :oops:
But I guess that was inevitable
I do believe the code for loading the bitmap could be optimized by using the following code, which conforms to the online source I posted earlier. It won't be that important of an optimization, but at least it was good for me to see that my thoughts where correct.
1 2 3 4 5 6 7 8 | v4 Texel = {(real32)((C & RedMask) >> RedShiftDown),
(real32)((C & GreenMask) >> GreenShiftDown),
(real32)((C & BlueMask) >> BlueShiftDown),
(real32)((C & AlphaMask) >> AlphaShiftDown)};
#if 1
Texel.rgb *= SquareRoot(Texel.a*(1.0f/255.0f));
#endif
|
I tested this with the following code (Abs is needed as extra function in math) to see if it indeed results in the same values, which it does (within some floating point margin).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | v4 Texel = {(real32)((C & RedMask) >> RedShiftDown),
(real32)((C & GreenMask) >> GreenShiftDown),
(real32)((C & BlueMask) >> BlueShiftDown),
(real32)((C & AlphaMask) >> AlphaShiftDown)};
v4 TexelSlow = SRGB255ToLinear1(Texel);
#if 1
Texel.rgb *= SquareRoot(Texel.a*(1.0f/255.0f));
TexelSlow.rgb *= TexelSlow.a;
#endif
TexelSlow = Linear1ToSRGB255(TexelSlow);
real32 Epsilon = 0.001;
Assert(Abs(Texel.r - TexelSlow.r) < Epsilon && Abs(Texel.g - TexelSlow.g) < Epsilon && Abs(Texel.b - TexelSlow.b) < Epsilon);
|
If only the actual pixel operations in the Draw calls could be optimized like this :P But I guess -O2 or #ifdefs could come in handy there.
But thanks for spending the time going over this! And I do love seeing you speed through the pixel operations cleaning up the code and everything.
I will try to think about scenarios to test whether your (dare I say "our") solution now works correctly. But I guess it is not as easy as in 3D space, where the diffuse shading for example will be greatly different if done in linear color space.
Mox
PS Sorry for slowing down the main drawing routing :oops:
But I guess that was inevitable
cmuratori
My assumption is that you want linear color values, premulitplied by linear alpha, everywhere, if you can get it. So in the ideal world, you would always take your sRGB source data and turn it into linear floating point premulitplied values immediately, and all the math would be done thereafter that way.
This is exactly right. As Alvy Ray Smith pointed out, all of our algorithms (including blending) assume linear-space data. ARS noted that gamma is a display device issue, and there are no known display devices which display partially-covered pixels (although AR headsets which do may be coming soon...).
But that isn't quite true. Gamma is also a compression issue, and that's why we can't just assume everything is linear until you splat it onto the screen.
cmuratori
More recently, I feel like both film and game rendering guys have become more careful about these things, even if other software domains haven't.
Certainly film and video, yes. Almost all film/video rendering and compositing systems use 32-bit floating point internal data paths for everything these days both for quality and performance (you can use SIMD instructions for pretty much everything). If you have raw footage (e.g. from a camera), then you correct for the camera's colour space at import time.
Well, that's assuming that your system supports the camera's colour space. A friend of mine, Chris Zwar, is a motion graphics designer and compositor (he does some very beautiful work). He also does tutorials, and he did a brilliant one last year on why colour spaces matter in that particular domain, as part of a tutorial on how to work with Arri Log C in After Effects (which at the time did not support it). If you're interested in what compositors have to put up with, here are part 1 and part 2.
Man, film/video industry compositing must be a huge headache... it's basically the nexus point where everyone dumps whatever the hell format they happened to use in your lap and then you have to deal with it :/
- Casey
- Casey
It is indeed a huge headache. If you want to see how deep the rabbit hole goes, check out ampasCTL:
http://ampasctl.sourceforge.net/
This is an attempt to create a way to express transforms between different color spaces. But there's so much complexity that they couldn't just specify it with a matrix, or a giant LUT, or something like that. They had to create an arbitrary interpreted bytecode to cover all the cases. :)
in film, there's also the additional layer of "filmlook". You have to approximate how the result will look on a given projector or film stock. Often this is hugely different, especially in the darks where people can notice very tiny differences.
In practice, most pipelines standardize on what format and color space they use internally for everything, and you just have to convert everything on the way in, and then on the way out, and also have show-specific settings to match the target filmlook. But it gets messy in practice, artists trying to keep track of these colorspaces might just go through all the options and assign the one that looks "best" on their monitor. :)
http://ampasctl.sourceforge.net/
This is an attempt to create a way to express transforms between different color spaces. But there's so much complexity that they couldn't just specify it with a matrix, or a giant LUT, or something like that. They had to create an arbitrary interpreted bytecode to cover all the cases. :)
in film, there's also the additional layer of "filmlook". You have to approximate how the result will look on a given projector or film stock. Often this is hugely different, especially in the darks where people can notice very tiny differences.
In practice, most pipelines standardize on what format and color space they use internally for everything, and you just have to convert everything on the way in, and then on the way out, and also have show-specific settings to match the target filmlook. But it gets messy in practice, artists trying to keep track of these colorspaces might just go through all the options and assign the one that looks "best" on their monitor. :)