Skip to content

October 10, 2011

Chroma Key Adventures Part 1: ImageMagick

In my last post, I mentioned the Photobot 3000, a portable photobooth-without-the-booth project I’ve been working on. After renting it out for a few events, we decided to investigate the possibility of doing in-software chroma key to add fun backgrounds to the photos before they get displayed on the projector.

We were already using the amazing and versatile ImageMagick to add a custom logo for each event to the images, so it made sense to see if ImageMagick could do something like chroma key. I was hoping it’d be as simple as calling ImageMagick with the right arguments, but it turned out to be a bit of a rabbit hole.

Before I started looking into this, I had a skewed idea about how chroma key works. I thought that you just set your key color and software removes every pixel with that color, with some fudge factor so that if a pixel is close enough to be under a certain threshold, it will be removed as well. That mental model is only partially true, and it’s only one way of implementing chroma key. The point that my brain was glossing over is the definition of the word “close enough.” Since I’d only ever seen images with color information represented as either RGB or CMYK, I assumed that “close enough” meant nearby in RGB color space.

In RGB color space however, similar colors aren’t really next to each other. The lightest green is #00FF00, but the darkest green of the same shade is #000000. In fact, the darkest green is so dark that you can’t even tell that it’s green. It’s just black like every other black. The darkest red is the exact same color as the darkest green in RGB. Because of this, just replacing colors that are close in RGB is not enough.


Bright Green                           Dark Green

Prove it

To demonstrate this, we’re going to do some ImageMagick acrobatics. Basically what we have to do is create a mask image based on closeness to our key color in RGB space, then overlay our background image, but mask out the areas that are not close to our key color.

Step 1 is creating the mask image. If we start with a JPG or other compressed image, we need to convert it to a bitmap so that the compare command will work. Similar images in a compressed format may have differences that have nothing to do with the image itself, but are the result of compression artifacts.

The first part of creating the mask is changing every instance of a pixel with a color close to our key color to white and saving the results.

Next, we compare the original image to the mask and then re-save it as the completed mask.

convert IMG_7938.JPG IMG_7938.BMP
convert IMG_7938.BMP \
   -fuzz 20% -fill white -opaque \#467c46 IMG_7938_Mask.BMP
compare IMG_7938.BMP IMG_7938_Mask.BMP \
   -compose SRC IMG_7938_Mask.BMP

Now that we have the mask, we do some smoothing to try and remove some of the noise

convert IMG_7938_Mask.BMP \
   -morphology smooth square IMG_7938_Mask.BMP

Then we sandwich the background, the mask, and the original JPG together

composite IMG_7938.JPG Lasers.gif IMG_7938_Mask.BMP \
   IMG_7938_Chromakeyed.JPG

and get the resulting images:


Original
The Original Photo

Masked in RGB Space
The Mask

Chromakey in RGB Space
The Composited Result


This sucks. It can’t tell the difference between differently colored shadows because it’s in RGB. We need to account for the difference between things like the green-black in the folds of the background and the red-black in the folds of my shirt. This is where hue comes in.

Hue

If we look at that color map a little more closely, we will find the answer to our problem:


Dark Green Hue                           Bright Green Hue

That’s right, the brightest green and the darkest green have the same exact hues! That’s because hue refers to distinct colors, “…without tint or shade (added white or black pigment, respectively).”

Well OK color scientist, but how can we use this? Well, if we just change the colorspace we’re working with to something that has hue as a component, such as HSL or HSV, we can identify pixels of our image that have the same hue (or are close within a threshold) regardless of how luminous or saturated they are.

Luckily, we can do this pretty easily with ImageMagick. The ImageMagick documentation on chroma key gives us a hint as to how to proceed. We can use the -separate option on convert to separate the image into it’s distinct hues.

convert IMG_7938.JPG \
   -colorspace HSL -channel Hue -separate IMG_7938_Hues.JPG

If we combine that with what we already have, we get

convert IMG_7938.JPG \
   -colorspace HSL -channel Hue -separate IMG_7938_Hues.JPG

convert IMG_7938_Hues.JPG IMG_7938_Hues.BMP

convert IMG_7938_Hues.BMP \
   -fuzz 20% -fill white -opaque \#606060 IMG_7938_Mask.BMP
compare IMG_7938_Hues.BMP IMG_7938_Mask.BMP \
   -compose SRC IMG_7938_Mask.BMP
convert IMG_7938_Mask.BMP \
   -morphology smooth square IMG_7938_Mask.BMP

composite IMG_7938.JPG Lasers.gif IMG_7938_Mask.BMP \
   IMG_7938_Chromakeyed.JPG

#606060 is the specific grey that our background’s hue turns out to be in the separated file:


Original
The Original Photo

Original Hue Separated
Hue Separated Image (notice that the background is #606060)

Hue Mask
HSL Mask

Chromakey in HSL Space
HSL Chroma Key


This is much better. There are still some problems though, which I will go over in my next post. I think this is the best result we can reasonably get short of writing our own chroma key functionality and operating on the pixels individually. That turns out to not be that scary though…

Leave a Reply

Your email address will not be published.