Alpha Blending in Software

June 14th 2021, 8:45 am

For the past couple of months I've been reverse engineering Konami's TXP2/IFS container formats and AFP animation format out of curiosity. The latter is a Flash-based animation file format that appears to have been forked and extended by Konami starting around 15 years or so ago. It retains the same general concepts as SWF. Animation files are composed of a list of tags, some of which are acted on each frame in order to place, update and remove objects from the animation canvas. Bytecode can be executed that has access to the placed objects and current execution engine. Color blending and placed object masking is available. The root animation is known as a clip, and an animation can include additional clips which are embedded animations complete with their own set of tags. These are treated the same as any other placed object and can be transformed and color blended at will as well as contain their own embedded clips. Animations can import and export tags from other animations so AFP files can be used as libraries or embed animations from other files in their own animations. The full format is in use in The*BishiBashi which implements all levels and most of the menus as AFP files with a ton of bytecode for the level logic and many AFP files acting as libraries for things such as displaying ready/go/finish animations and common functions. If we fast forward to current games, many of the original features have been stubbed out and no longer have code to function. The format has been extended to allow for 3D transforms and a camera system instead of SWF's simple affine transforms. As of writing this blog post AFP files appear to be used in virtually all Konami games for animations in menus, character displays, background videos and the like. Bytecode use is limited to setting properties on placed clips such as looping animations, requesting masks and other simple playback features.

What started as a dive into the way The*BishiBashi stored and executed level data turned into a full-on AFP rendering engine. This is, of course, written in Python 3 with some equivalent C++ code that can be loaded in for performance-critical sections such as blending pixels. My goal wasn't to provide a real-time viewer of files, but to provide as accurate a documentation of the format as possible. Because of this I implemented all of the pixel blending in software instead of relying on GPU functions. If you are curious, all of the code that implements the pixel blending discussed here can be found in the blend directory on GitHub. I maintain two equivalent blending engines, one in pure Python 3 and one in C++. You can examine whichever one is closer to your preferred language. AFP, much like SWF, supports several different blending modes which are effectively identical to the ones found in Adobe products such as Photoshop. While additive, subtractive and other blending modes are interesting they do not contain nearly the amount of gotchas that "normal" blending has. So, I'm focusing on the normal blending mode which works out to alpha blending a source pixel onto a destination pixel.

Let's start out with the simplest iteration of alpha blending. A RGB pixel is esentially a group of 3 integers that represent the intensity of light (or brightness) for red, green and blue color that gets mixed together to form the final color. Many software packages choose to use 8-bit pixels, meaning there are 2^8 possible numbers that can be stored for each color, from 0 through 255. A 0 in a particular color's integer bucket means there is absolutely none of that color mixed into the final color, and 255 means that as much of that color as possible is mixed into the final color. Any number in between corresponds to a brightness that is proportional to the number itself. If you divide each number by the maximum possible number (255) you can visualize the color as a percentage instead. So, a RGB color of "128, 0, 0" can be thought of as "50% of possible red, 0% of possible green, 0% of possible blue".

With RGB you can represent every single color that is possible to display on a computer screen. RGB colors are missing something, however. There's no way to represent how see-through a pixel is. This means that if all you have is RGB you can accurately store an image, but not how it would interact with another image. Blending is boring in this case because each image is fully opaque. If you had an image already placed on a canvas (the destination) and wanted to blend a new image with it (the source), you would simply replace each pixel in the destination with the corresponding pixel from the source. We can fix this by adding a new integer "alpha" to the RGB pixel, creating an RGBA color instead. The alpha number works very similar to the red, green and blue numbers. It can hold any number from 0 through 255. However, instead of representing the brightness for a particular color, it instead represents the amount that the RGBA pixel modifies a destination pixel when it is blended. It means nothing on its own but it allows us to store how each pixel should interact with another image if blended.

For ease of discussion, let's assign a variable to each part of the source and destination RGBA colors:

  • Sr = The red component of the source image.
  • Sg = The green component of the source image.
  • Sb = The blue component of the source image.
  • Sa = The alpha component of the source image.
  • Dr = The red component of the destination image.
  • Dg = The green component of the destination image.
  • Db = The blue component of the destination image.
  • Da = The alpha component of the destionation image.

Now blending a source image with a destination image can get interesting. Your source image can have some fully or partially transparent pixels and some fully opaque pixels. You can now represent things like stained glass windows, sunglasses, chain link fences and any other thing in the real world that allows part or all of the thing behind it to be visible. We can come up with some simple code to blend each primary color in each pixel based on the source image's alpha which dictates how much of the source and destination color we mix together. That code looks like the following:

source_percent = Sa / 255
source_remainder = 1 - source_percent

Dr = (Sr * source_percent) + (Dr * source_remainder)
Dg = (Sg * source_percent) + (Dg * source_remainder)
Db = (Sb * source_percent) + (Db * source_remainder)

Effectively, the code is figuring out what ratio of each color to include when mixing. If you had a source RGBA color of "255, 0, 0, 64", that works out to a source percent of 0.25 (25% of the final color should be the source) and a source remainder of 0.75 (75% of the final color should be the original destination). If you were blending a source image representing red stained glass onto your destination you would expect the resulting image to be tinted red. That's exactly what the code ends up doing! If you work out the math for a source alpha of 0 (completely transparent), you can see that the color component equations simplify to leaving the destination colors unchanged. If you set the source alpha to 255 (completely opaque), you can see that the equations simplify to setting the destination colors equal to the source colors. Anything in between and your destination pixel ends up being a mix of the source and destination colors with the appropriate ratio.

You might notice that the above code does not handle one particular thing. It does not update (or even use) the destination alpha. This is because we assumed that the destination canvas was fully opaque. That makes sense in many cases because usually the destination canvas is an image that we are going to display on a computer screen or print out later. That means we can consider the destination to be the the final image and thus we can assume that the alpha component for each pixel in the destination is 255 (fully opaque). If all you want to do is alpha blend a source image onto a destination and then look at it, you're done! However, what if you want part or all of the destination canvas to be transparent? What if the final canvas is meant to have transparency so it can be placed onto another canvas? In that case, we need to update the code a little bit:

source_percent = Sa / 255
destination_percent = Da / 255
source_remainder = 1 - source_percent

Dr = (Sr * source_percent) + ((Dr * destination_percent) * source_remainder)
Dg = (Sg * source_percent) + ((Dg * destination_percent) * source_remainder)
Db = (Sb * source_percent) + ((Db * destination_percent) * source_remainder)
Da = (255 * source_percent) + ((255 * destination_percent) * source_remainder)

Okay, this got a little complicated! Let's dissect the new code a little bit. First, you'll notice the introduction of destination percent. I placed parenthesis around where its used so its easier to see that we are now doing the same thing to the destination colors as we are to the source colors! Now, both the source and destination colors are being scaled down by their respective alpha percentages. Essentially, we are using the alpha component to figure out how much of each color should be mixed into the final color for both the source and destination image now. If you work out the math for a destination alpha of 255, you'll see that this code simplifies down to the original code for the three color components! We're still computing a ratio of the two colors based on the source alpha since it is the one being blended onto the destination. We just added scaling the destination color components by the destination alpha. The second addition is updating the destination alpha. We treat it almost the same as the color components, except that we already have a percentage so we multiply by the maximum number (255) instead of the alpha component itself. Again, if you plug in 255 for the destination alpha, you'll see that the equation turns into "Da = 255" which matches the assumption we had previously made! And again, if you have a source pixel that's fully opaque (alpha component of 255) or fully transparent (alpha component of 0), the equations simplify to setting the destination RGBA components equal to either the source or destination as you would expect. Cool!

But wait! The above code has a subtle bug. Imagine we are blending a source pixel with an alpha component of 64 onto a destination pixel with an alpha component of 128. The source percent works out to 0.25 and the destination percent works out to 0.50. That means we are blending 25% of the source pixel and 50% of the destination pixel together. To do this, we scale the source and destination color components by 0.25 and 0.50 respectively before blending them together. We then compute the new alpha component which works out to 159. That means nothing by itself, but remember, the whole point of keeping around a destination alpha is because we want a new image suitable for blending with another canvas. The RGB components have already been scaled down by their respective alpha components when we mixed the colors. But we also computed a new alpha component that was not 255. When we blend the image on this canvas with another image, we will multiply thse RGB color components by the new alpha percentage that we computed (which works out to about .623) meaning we will have scaled our colors down twice! Our colors will end up darker than we wanted because of this! This is easiest to see if you assume a source pixel that's fully transparent (alpha component of 0) and a destination pixel that's partially transparent. The destination RGB components get multiplied by the destination percent, and the destination alpha gets left alone, meaning we just accidentally darkened the colors.

The key to understanding this bug is this: In our original code, we assumed the final alpha component was always 255. We didn't premultiply the destination color by its alpha percentage. We only took the ratio of the two colors, computed by figuring out the alpha percentage of the source. In these new equations we are still computing the ratio based on the source alpha, but we are also scaling the destination by its alpha percentage as well. You'll notice that when the destination alpha is 255, these equations work out. Its no coincidence that the bug does not appear in these scenarios! That's because the ratio of colors we compute is not out of 255, but out of the final alpha component! Its only when the final alpha percentage is 1.0 that we have actually calculated the colors correctly. The fix then, is to perform the inverse scaling on the destination RGB components so that when we later scale based on the alpha component we computed we get the correct colors:

source_percent = Sa / 255
destination_percent = Da / 255
source_remainder = 1 - source_percent
final_percent = source_percent + destination_percent * source_remainder

Dr = ((Sr * source_percent) + ((Dr * destination_percent) * source_remainder)) / final_percent
Dg = ((Sg * source_percent) + ((Dg * destination_percent) * source_remainder)) / final_percent
Db = ((Sb * source_percent) + ((Db * destination_percent) * source_remainder)) / final_percent
Da = 255 * final_percent

Finally! You'll note that the computed destination RGBA pixel, if inserted into the same code as a source image in the future, get multiplied by the same final percent. So we've effectively un-scaled the colors so they can be correctly scaled again in the future. The destination alpha component equation is factored out but remains the same. We were always calculating the alpha component correctly, we just weren't taking into consideration that the RGB components were going to be scaled by the alpha component when the final image was blended onto a new canvas. If we take the previous example of a source pixel that is fully transparent and a destination pixel that is partially transparent, you can see that final percent reduces to the destination percent, meaning that the color component equations do indeed reduce to "Dr = Dr, Dg = Dg, Db = Db, and Da = Da". Aside from some clamping to make sure color components always stay in the range of 0 through 255, this is the code that appears in the AFP renderer. It was necessary in order to create PNG and WEBP files that could be placed on top of other graphics after they were rendered.