Using PanoTools projects within Natron

Those are preliminary notes on using PanoTools or Hugin projects from within Natron.

Creating a PanoTools (pto) project

Using the Hugin GUI

Using command-line tools

Full details on using command-line tools are given in the Panorama scripting in a nutshell documentation.

On macOS, if Hugin was installed using homebrew, the tools are available in various directories, and they can be added to the PATH using:

PATH="$PATH:/Applications/Hugin/tools_mac:/Applications/Hugin/Hugin.app/Contents/MacOS:/Applications/Hugin/HuginStitchProject.app/Contents/MacOS"

Here is an example of running the panorama tools from a set of JPEG images (with suffix .jpg) placed in the current directory to generate e perspective panorama (other options are described in the documentation):

pto_gen -o project.pto *.JPG
cpfind --multirow -o project.pto project.pto

celeste_standalone -i project.pto -o project.pto
cpclean -v --output project.pto project.pto

autooptimiser -a -l -s -m -o project.pto project.pto

nona -m TIFF_m -o project project.pto

enblend --save-masks -o panorama.tif project*.tif

The result is:

  • A project.pto file, which is the Hugin project itself

  • The project*.tif images, which are images warped to the reference projection.

  • The mask-*.tif images, which are masks generated by enblend.

  • The panorama.tif image, which is the final panorama.

Using Hugin/PanoTools parameters in Natron

The pto file syntax is described is the PTOptimizer and PTStitcher docs.

This file can be viewed and edited in any text editor.

Project size

The project size, or format, should be set to the panorama size, which is given in pixels on the p line (at the start of the file).

Distortion correction

The distortion correction parameters for the input images are given in the i lines (one for each image).

The values for the a, b, c, d, e, g, t parameters should be entered in a LensDistortion node, with Model/model=PanoTools and Direction/direction=Undistort, placed after the Read node for each input image or video. The script name for these values are pt_a, pt_b, pt_c, pt_d, pt_e, pt_g, pt_t.

Note that if all images share the same distortion parameters (this is written as a=0 b=0 c=0 d=0 e=0 g=0 t=0 on the i line), the LensDistortion node can be cloned in Natron (right-click on node, Edit, Clone Nodes).

Projection

The projection (which can be performed by the Nona tool) can be done using Card3D nodes placed after each LensDistortion node.

Each Card3D node must have its Transform Order/cardXFormOrder set to STR, and the Rotation Order/cardRotOrder should be left to the default value (ZXY). Be careful, these are neither the Axis nor the Cam Transform Order, which are in the two first groups of the Card3Dnode.

Set the Output Format to Project.

The r p and y values from the i line are roll, yaw, pitch angles. Their values should be put in the Rotate parameter of the Card3D (script name is cardRotate), using the following convention: cardRotate.x = p, cardRotate.y = -y, cardRotate.z = -r.

The v value from the i line corresponds to the horizontal field of view, and has to be converted to an aperture value. The Lens-In H.Aperture/lensInHAperture param should be set to 2*tan(v*pi/360). This expression can be directly typed in the value field, with v replaced by the actual value from the corresponding i line of the pto file.

Setup all the Card3D nodes for each input image that way.

Camera

In Natron 2, the camera used by the Card3D node is set in the Cam group at the top of the parameters list.

In the Card3D node for the first image, unfold this group, and unfold the Cam Projection group.

The v value on the p line (usually at the top of the pto file) gives the horizontal field of view of the output panorama.

Set the Focal Length/camfocal to 1., and set the Horiz. Aperture/camhaperture to 2*tan(v*pi/360), replacing v with its actual value.

You can then copy these two parameters to all the Card3D nodes, or - even better - link these parameters, so that the output camera for the panorama can then be modified: Right-click on the parameter from the first Card3D, Copy Link, then right-click on the same parameter of every other Card3D node, Paste Link.

If you intend to modify the camera orientation later, you can also link the same way the Rotate/camRotate parameter (those for the camera, not the card).

Note that when the Natron Card3D node is used in Nuke (where it appears as Card3DOFX), the node has an external Cam input, to which a Camera node may be connected. There may be a similar concept in future versions of Natron.

Building the panorama in Natron

First sketch: overlap the images

Make sure that the Output Components parameter in all readers is set to RGBA, so that images have a transparent value outside of their domain.

Now, connect the outpput of the first Card3D to the B input of a Merge node, connect the secont to the A input, the third to the A2 input, etc…

The output of the Merge node should show a first panorama, obtained by overlapping all images.

Drawing the masks

Add a Roto node after each LensDistortion, before each Card3D.

Only leave connected the A and B inputs to the Merge node to the two first Card3D nodes.

Check Premultiply in each Roto node, check that only the A channel is affected by Roto in its parameters, and start editing the roto mask on the second image, for which the LensDistortion output is connected to the A input of the Merge, while viewing the output of the Merge.

Set the compositing operator of each roto shape to “min” instead of “over”, so that the original image alpha gets masked by the roto shape.

Do not forget to add feather, especially in the overlap area.

Then, reconnect the A2 input to the third Card3D, and start editing its roto mask, always in “min” compositing mode.

Do the same with A3, A4, etc… and you should end up with a full panorama.

Future work

Camera response

See Camera response curve and Vig_optimize.

The values Ra Rb Rc Rd Re on the i line encode a color response curve (EMoR). See EMoRParamsin the hugin source code.

Exposure and color

See Vignetting and Vig_optimize.

Eev encodes the exposure, see ExposureValue in the hugin source code.

Er and Eb encode the red and blue multipliers, see WhiteBalanceRed and WhiteBalanceBlue in the hugin source code.

Vignetting

See Vignetting and Vig_optimize.

Va is always 1, see VigCorrMode in the hugin source code.

Vb Vc Vd encode the degree 2, 4 and 6 coefficients for vignette correction, see RadialVigCorrCoeff in the hugin source code.

Vx and Vy encode the vignetting center shift, see RadialVigCorrCenterShift in the hugin source code.

Blending

  • importing enblend masks, using them as roto masks

  • executing enblend externally (using RunScript maybe?)