Friday, October 26, 2012

Perfect alpha cutouts

Getting something cut out of a background in SL perfectly can be difficult and frustrating- unless you know the trick of it.  Many people use the green screen trick, but you can have slight acid green bleed in your semi transparent areas, like hair whisps.

First requirement: you need to be able to take at least 2 completely identical pictures, aside from background colour (if there are flexis, you might want to turn them off after you've gotten everything into position, including your camera- Advanced menu>Rendering Features>Flexible Objects.  This CAN be optional, your flexis might not move so long as nothing moves...but then again, they might, depending on settings and conditions)- and you need to be able to change that background colour.

Take one picture with your background solid fullbright white.  Take a second picture identical but for the background being solid black.  Optionally, take a third (or more) with the colour a general match for the touchy alpha issues- a vague match for your hair, any semi transparent spots, etc.

In Photoshop, one of the layer styles is "Difference"- copy paste the black or white background over the white or black background, and set the top layer to that.  The image will be all black and white- the background will be white, everything else in the shot will be black, with perfect alphaed edges for cutouts.  Select all and shift-copy, paste that into a channel mask for the image, and invert it, so it's a white outline on a black background.  If your picture has shadows, you probably WILL still have to clean this up in minor ways, due to lines and shadows rendering differently on different backgrounds- make sure the places that should be solid are white, with no grey spots from shadows not rendering quite the same, and if you're taking larger than screen size, the line problem will have to be fixed.  If you're not taking it with shadows on, your mask is already perfect- aside from occasional minor shiny glitches, so you might want to zoom in on those and make sure they're all white!

If your white and black aren't quite 100% pure (which can happen, even with fullbright- especially with shadows and Windlight), you may need to adjust the levels of your mask ever so slightly pulling the end arrow in to the center a tad.  You should not need much, if you need this step at all- but you can have a slightly messy background mask that is mostly, but not 100%, transparent, under some circumstances (i.e. SL having a bad day).

Now, you can use this mask on your white background picture, your black background picture, or the picture(s) with the alpha colours.  This last option works best- if you have multiple problem spots, layer them all only showing the bits you need where to prevent the colour bleed (if you're scaling the picture down, the colour bleed may disappear, depends on how much of a stickler you are as to how much effort to put in here- it's going to be minor even if you just use the white background version).  If you have antialiasing on, there will be VERY slight bleed over the entire outline, so if you're wearing black, might want to do your cutout on the black background copy (black halo can also be less immediately noticeable than white, "shadows" rather than "halo").  But whatever you decide to cut out, use that channel mask for your selection or mask for your cutout!