What i can't understand from the Friday video, was why a bitmap blend on premultiplied alpha bitmaps is still a premultiplied alpha. i can understand if you want to declare it as such, but i thought you still want to be aligned with the normal argb blend. but maybe you don't.
Maybe we can do a little more talking about premultiplied alpha on Monday? Now that we've done it, I can do a little more high-level overview stuff, perhaps.
- Casey
- Casey
I'm still thinking my way through this stuff as well, so you have been forewarned...
There is a technical memo that discusses this and derives the same formula that Casey talked about. In that memo he talks about the "over" operation (Source is drawn over Destination) not being "closed" for the non-premultiplied alpha formulation which is a fancy way of saying non-premultiplied colors go in but premultipled alpha colors come out. So, why is that?
My current mental model is that the color that is ultimately drawn on the screen is, in fact, a premultiplied alpha color. The pixels that get drawn on the screen are just ( R, G, B ) in the end - the alpha is just used to figure those final values. So what should the final pixel value be for ( R, G, B, A ) drawn over black ( 0, 0, 0, 1 )? There are various ways to interpret what alpha means but if we think of it as either opacity or fraction of the pixel occupied to be drawn, the drawn pixel would be ( A*R, A*G, A*B ), which is the premultiplied alpha color and the output of non-premultiplied alpha formula (i.e. non-premultiplied colors go in but premultipilied colors come out and, ultimate, get drawn on the screen).
Anyway, that's the best I've got at the moment. I'm going to ramble a bit more in the hopes that what I'm going to say is helpful or generates a discussion that is helpful.
One interpretation of alpha is that it is the fraction of a pixel filled by a surface. So, if we randomly sample a point in a pixel the probability that we hit the source surface is SA and probability that we miss the source surface is 1 - SA. So, thinking about the Source over Destination case again, what is the alpha of the resulting pixel given that we started with DA (destination alpha) and we "painted" a surface over it with SA (source alpha)?
In below, read "Probability" to mean "Probability a random point sample of the pixel":
Which is the formula Casey derived. In this interpretation, alpha is the probability that a random point sample would hit either the Source or Destination. We can't simply add SA and DA since that would double count the case where a point sample would hit a Source point over a Destination point.
To find the color of the pixel we just apply the probabilities:
or, using premultiplied alphas,
There is a technical memo that discusses this and derives the same formula that Casey talked about. In that memo he talks about the "over" operation (Source is drawn over Destination) not being "closed" for the non-premultiplied alpha formulation which is a fancy way of saying non-premultiplied colors go in but premultipled alpha colors come out. So, why is that?
My current mental model is that the color that is ultimately drawn on the screen is, in fact, a premultiplied alpha color. The pixels that get drawn on the screen are just ( R, G, B ) in the end - the alpha is just used to figure those final values. So what should the final pixel value be for ( R, G, B, A ) drawn over black ( 0, 0, 0, 1 )? There are various ways to interpret what alpha means but if we think of it as either opacity or fraction of the pixel occupied to be drawn, the drawn pixel would be ( A*R, A*G, A*B ), which is the premultiplied alpha color and the output of non-premultiplied alpha formula (i.e. non-premultiplied colors go in but premultipilied colors come out and, ultimate, get drawn on the screen).
Anyway, that's the best I've got at the moment. I'm going to ramble a bit more in the hopes that what I'm going to say is helpful or generates a discussion that is helpful.
One interpretation of alpha is that it is the fraction of a pixel filled by a surface. So, if we randomly sample a point in a pixel the probability that we hit the source surface is SA and probability that we miss the source surface is 1 - SA. So, thinking about the Source over Destination case again, what is the alpha of the resulting pixel given that we started with DA (destination alpha) and we "painted" a surface over it with SA (source alpha)?
In below, read "Probability" to mean "Probability a random point sample of the pixel":
1 2 3 4 5 6 7 8 9 | SA = PS = Probability hits the Source DA = PD = Probability hits the Destination (before Source applied over) A = P = Probability hits either Source or Destination after Source has been applied over P = PS + (1 - PS)*PD or A = SA + (1 - SA)*DA |
Which is the formula Casey derived. In this interpretation, alpha is the probability that a random point sample would hit either the Source or Destination. We can't simply add SA and DA since that would double count the case where a point sample would hit a Source point over a Destination point.
To find the color of the pixel we just apply the probabilities:
1 | C = SA*S + (1 - SA)*DA*D |
or, using premultiplied alphas,
1 | C = S + (1 - SA)*D |
Edited by Patrick Lahey
on
BTW, thanks for the shout-out in the show, Casey! You'll excuse my absence since it was 3am local time...
That's not bad as oversimplifications go, but I suspect that the reason why you're thinking this way is because you're visualising the image composited onto a solid black background. I think that it's better to think in terms of alpha as coverage (as Casey mentioned in the stream, and you mentioned below) or alpha as probability (a thought you developed below).
First, let's think about the physics of this for a moment. We'll assume a single colour channel which represents a pure wavelength of light.
Physically, what the eye perceives as "brightness" is proportional to the number of photons per unit time, which is also proportional to energy per unit time. This quantity is known as the "radiant flux", or "flux" for short.
(Technically, the energy is wavelength-dependent, but we'll ignore this complication. Look up the Planck-Einstein relation if you're curious. Also, technically, the perceptual part of the vision system is nonlinear, but we'll ignore this too. Photometry, the physics of light transport and the human visual perception system, is a very important topic in computer graphics, but it's not going to come up in HMH very much.)
If you take a small area which is emitting photons, and then cover half of that area, you get only half the photons. The flux is halved, and so the apparent brightness of the area is correspondingly halved. The non-premultiplied colour is the light emitted by the surface, and the premultiplied colour is the light received by the eye.
Again, using the coverage interpretation, this is saying that if you put S over D, the flux reaching the eye is the flux coming from S, plus the proportion of the flux from D which isn't covered by S.
rathersleepy
My current mental model is that the color that is ultimately drawn on the screen is, in fact, a premultiplied alpha color.
That's not bad as oversimplifications go, but I suspect that the reason why you're thinking this way is because you're visualising the image composited onto a solid black background. I think that it's better to think in terms of alpha as coverage (as Casey mentioned in the stream, and you mentioned below) or alpha as probability (a thought you developed below).
First, let's think about the physics of this for a moment. We'll assume a single colour channel which represents a pure wavelength of light.
Physically, what the eye perceives as "brightness" is proportional to the number of photons per unit time, which is also proportional to energy per unit time. This quantity is known as the "radiant flux", or "flux" for short.
(Technically, the energy is wavelength-dependent, but we'll ignore this complication. Look up the Planck-Einstein relation if you're curious. Also, technically, the perceptual part of the vision system is nonlinear, but we'll ignore this too. Photometry, the physics of light transport and the human visual perception system, is a very important topic in computer graphics, but it's not going to come up in HMH very much.)
If you take a small area which is emitting photons, and then cover half of that area, you get only half the photons. The flux is halved, and so the apparent brightness of the area is correspondingly halved. The non-premultiplied colour is the light emitted by the surface, and the premultiplied colour is the light received by the eye.
Incidentally, people who have done some high school-level probability theory or set theory may recognise this as the inclusion-exclusion principle:
1 | P = PS + PD - PS * PD |
or, using premultiplied alphas,
Again, using the coverage interpretation, this is saying that if you put S over D, the flux reaching the eye is the flux coming from S, plus the proportion of the flux from D which isn't covered by S.