Using STMap based deformations for Lens Distortion workflow

This post is about the full workflow for creating, modifying and applying STMap deformation images for lens undistortion and redistortion. It is meant for Nuke <> SynthEyes exchange but works with other matchmoving apps too (if you know how to apply and remove distortion there ofcourse).

Before getting to the "real thing", some background info about STMaps and their use for (un)distorting images.

For applying or removing lens distortion there are basically two ways:

• using a matching function and coefficients that descibe the lens distortion characteristics mathematically
• using a deformation map that describes where every input pixel should end up on output image

Mathematical lens analysis is usually the first step because it is more precise. It could also be possible to make an interpolated deformation map by simply analysing test grid image but it is not that good method and as higher-order fitting is quite fast these days, lens is usually first described with functions and coefficients.

If it is easy to describe lens with distortion function, why would one bother to use a deformation map at all? Answer is simple - as each matchmoving program uses different lens models and compositing softwares have their own models and filters, it becomes hard to exchange lens distortion info. For example, we can matchmove our undistorted plate in SynthEyes or 3DEqualizer or wherever and get a good track, but if we are not able to distort our cg elements to match the original plates then we can't do the composite. It would be highly impractical and often impossible to push all our elements through matchmoving software simply to distort the images. It is much easier to do distorting in compositing application but how to get our genius lens model in there?

STMap comes to resque! As it is simply an image that tells us where our pixels end up, it is basically software agnostic and can be made to work in almost any compositing software that has image-based deformation filter. How crazy our lens model might be, in the end it simply shifts pixels and thus can be baked into deformation image.

All this goodness does not come without price though... It is very cumbersome (or should I say practically useless) to use deformation maps for fixing or applying changing distortion. Zooming and focusing can change the lens distortion parameters and interpolation between two maps is very hard to do. Function based lens models are much easier to manipulate if one needs to animate distortion changes.

With all this info we can start getting into stuff.

Basic order of operations for this is:

• film the test pattern for SynthEyes Lens Grid auto-calibration
• create new lens preset
• generate STMap base image in Nuke
• apply undistortion (with padding) to generated base image
• generate STMap base image for redistortion
• apply distortion to generated redistortion base image

After this is done, you can:

• undistort original distorted plates using undistortion map
• apply distortion to cg elements using redistortion map

Using STMap based distortion in one step (undistort or redistort) should be more or less the same quality as for example LensDistort node. Still bear in mind that errors in choosing high enough bit depth for distortion map images or some out-of-nowhere applied gamma correction can make your life miserable and introduce ugly artefacts.

Test pattern and lens preset creation in SynthEyes

Download the test pattern for SynthEyes Lens Grid auto-calibration and follow instructions in this video:
Lens Grid auto-calibration by Russ Andersson from SynthEyes.

Save lens preset and don't forget to name it so that it points to both the lens and zoom level if necessary.

STMap base image generation in Nuke

Create a new Constant image with the size that matches your plate photography (it can also be bigger but it is easier for description purposes to use the same size).

Apply Color > Math > Expression node with these equations:
r = (x+0.5) / width
g = (y+0.5) / height

This creates an image where red and green channel contain normalised x and y coordinate values. Adding 0.5 to coordinate value is important because it fixes the offset introduced by interpreting the pixel position as centered in STMap node (thanks to Jarrod Avalos for pointing it out).

Save this coordinate image as either a float / half-float exr or 16-bit tif. For tif, you must click "raw data" checkbox in colorspace settings or choose "Linear", otherwise Nuke applies gamma correction and image gets messed up.

Apply undistortion to generated base image

Load the generated base image in SynthEyes, don't forget to set bit depth according to format - for exr choose float for example.

In Image Preparation dialog choose the correct lens preset from dropdown in Lens panel.

In overall Summary panel click the Lens Workflow button and choose Redistorted(2). Base image gets "undistorted" and overscanned - it gets bigger to fit all the "undistorted" pixels.

Save the "undistorted" image by clicking the Save Sequence button. Choose exr or tif as format and follow the rules described earlier.

Generate base image for redistortion

Create a new Constant image with the size that matches the "undistorted" base image. It is larger than original plate photography.

Apply Color > Math > Expression node with these equations:
r = (x+0.5) / width
g = (y+0.5) / height

Save this image as either a float / half-float exr or 16-bit tif. For tif, you must click "raw data" checkbox in colorspace settings or choose "Linear", otherwise Nuke applies gamma correction and image gets messed up.

Apply distortion to redistortion base image

Load the redistortion base image in SynthEyes, don't forget to set bit depth according to format - for exr choose float for example.

In Image Preparation dialog choose the correct lens preset from dropdown in Lens panel and click the Apply Distortion checkbox. Image gets distorted as if projected through the lens.

Save the "distorted" image by clicking the Save Sequence button. Choose exr or tif as format and follow the rules described earlier.

---------------------------------------------------------------------------------------------

After all this is done, we have a map that helps to undistort plate photography and a map that helps to distort cg elements to match the plates.

Undistorting plate photography with STMap

To undistort original plates one simply has to apply the STMap operation to plate and connect the undistortion map to it's stmap input. Resulting image is as big as the undistortion map (larger than original image).

Distorting cg elements and whatnot with STMap

To distort elements one has to apply the STMap operation to element and connect the distortion map to its stmap input. Resulting image is as big as the undistorted image. To make it match plate photography for example, simply add a Reformat node with "type - to format" and "resize type - none, centered" settings.

EDIT: cropping to center with Reformat only works when lens is not decentered. When there is offset, make sure that you shift the image properly! This method does not compensate for decentering. Another way is to choose redistort cg and select the larger stmap base image. Results are very similar to original but for some reason do not match exactly.

Please note that all your cg elements must be rendered out larger than original plate photography - their size must match the distortion map. As matchmove is done on undistorted plates, camera fov and other settings are also based on this larger undistorted image size.

End of story

I hope this description makes it a bit more clear how to apply and remove distortion using STMap node and deformation maps and how to generate ones to begin with. There are not that many descriptions of this technique in the internet and although it is rather easy, it can take some fiddling to get it working.

Some sites discussing the same topic:
Nukepedia tutorial by Florian Gellinger from RiseFX
VFXTalk forum talk about PFTrack > Nuke

PS. if someone knows an easy way how to generate distortion maps from undistortion maps and vice-versa, it would be nice to know! STMap does not have the reverse button as LensDistortion node has although it could be quite useful. It should be pretty straightforward pixel math but as with everything, it takes some moments of thought. If I find the solution I will post it here.