Using PanoTools projects within Natron ====================================== Those are preliminary notes on using PanoTools or Hugin projects from within Natron. Creating a PanoTools (pto) project ---------------------------------- Using the Hugin GUI ~~~~~~~~~~~~~~~~~~~ - download `Hugin `__ - on macOS this can be done using `homebrew `__, by typing the command ``brew cask install hugin`` in a terminal. - read the `Hugin documentation `__ or start with a `Hugin tutorial `__ - make your panorama Using command-line tools ~~~~~~~~~~~~~~~~~~~~~~~~ Full details on using command-line tools are given in the `Panorama scripting in a nutshell `__ documentation. On macOS, if Hugin was installed using homebrew, the tools are available in various directories, and they can be added to the PATH using: :: PATH="$PATH:/Applications/Hugin/tools_mac:/Applications/Hugin/Hugin.app/Contents/MacOS:/Applications/Hugin/HuginStitchProject.app/Contents/MacOS" Here is an example of running the panorama tools from a set of JPEG images (with suffix ``.jpg``) placed in the current directory to generate e perspective panorama (other options are described in the documentation): :: pto_gen -o project.pto *.JPG cpfind --multirow -o project.pto project.pto celeste_standalone -i project.pto -o project.pto cpclean -v --output project.pto project.pto autooptimiser -a -l -s -m -o project.pto project.pto nona -m TIFF_m -o project project.pto enblend --save-masks -o panorama.tif project*.tif The result is: - A ``project.pto`` file, which is the Hugin project itself - The ``project*.tif`` images, which are images warped to the reference projection. - The ``mask-*.tif`` images, which are masks generated by ``enblend``. - The ``panorama.tif`` image, which is the final panorama. Using Hugin/PanoTools parameters in Natron ------------------------------------------ The ``pto`` file syntax is described is the `PTOptimizer `__ and `PTStitcher `__ docs. This file can be viewed and edited in any text editor. Project size ~~~~~~~~~~~~ The project size, or format, should be set to the panorama size, which is given in pixels on the ``p`` line (at the start of the file). Distortion correction ~~~~~~~~~~~~~~~~~~~~~ The distortion correction parameters for the input images are given in the ``i`` lines (one for each image). The values for the ``a``, ``b``, ``c``, ``d``, ``e``, ``g``, ``t`` parameters should be entered in a **LensDistortion** node, with *Model*/``model``\ =PanoTools and *Direction*/``direction``\ =Undistort, placed after the **Read** node for each input image or video. The script name for these values are ``pt_a``, ``pt_b``, ``pt_c``, ``pt_d``, ``pt_e``, ``pt_g``, ``pt_t``. Note that if all images share the same distortion parameters (this is written as ``a=0 b=0 c=0 d=0 e=0 g=0 t=0`` on the ``i`` line), the **LensDistortion** node can be cloned in Natron (right-click on node, *Edit*, *Clone Nodes*). Projection ~~~~~~~~~~ The projection (which can be performed by the `Nona `__ tool) can be done using **Card3D** nodes placed after each **LensDistortion** node. Each **Card3D** node must have its *Transform Order*/``cardXFormOrder`` set to STR, and the *Rotation Order*/``cardRotOrder`` should be left to the default value (ZXY). Be careful, these are neither the Axis nor the Cam Transform Order, which are in the two first groups of the **Card3D**\ node. Set the *Output Format* to Project. The ``r`` ``p`` and ``y`` values from the ``i`` line are roll, yaw, pitch angles. Their values should be put in the *Rotate* parameter of the **Card3D** (script name is ``cardRotate``), using the following convention: ``cardRotate.x = p``, ``cardRotate.y = -y``, ``cardRotate.z = -r``. The ``v`` value from the ``i`` line corresponds to the horizontal field of view, and has to be converted to an aperture value. The **Lens-In H.Aperture**/``lensInHAperture`` param should be set to ``2*tan(v*pi/360)``. This expression can be directly typed in the value field, with ``v`` replaced by the actual value from the corresponding ``i`` line of the ``pto`` file. Setup all the Card3D nodes for each input image that way. Camera ~~~~~~ In Natron 2, the camera used by the **Card3D** node is set in the *Cam* group at the top of the parameters list. In the **Card3D** node for the first image, unfold this group, and unfold the *Cam Projection* group. The ``v`` value on the ``p`` line (usually at the top of the ``pto`` file) gives the horizontal field of view of the output panorama. Set the *Focal Length*/``camfocal`` to 1., and set the *Horiz. Aperture*/``camhaperture`` to ``2*tan(v*pi/360)``, replacing ``v`` with its actual value. You can then copy these two parameters to all the **Card3D** nodes, or - even better - link these parameters, so that the output camera for the panorama can then be modified: Right-click on the parameter from the first **Card3D**, Copy Link, then right-click on the same parameter of every other **Card3D** node, Paste Link. If you intend to modify the camera orientation later, you can also link the same way the *Rotate*/``camRotate`` parameter (those for the camera, not the card). Note that when the Natron **Card3D** node is used in Nuke (where it appears as **Card3DOFX**), the node has an external Cam input, to which a Camera node may be connected. There may be a similar concept in future versions of Natron. Building the panorama in Natron ------------------------------- First sketch: overlap the images ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Make sure that the *Output Components* parameter in all readers is set to RGBA, so that images have a transparent value outside of their domain. Now, connect the outpput of the first *Card3D* to the B input of a **Merge** node, connect the secont to the A input, the third to the A2 input, etc... The output of the **Merge** node should show a first panorama, obtained by overlapping all images. Drawing the masks ~~~~~~~~~~~~~~~~~ Add a **Roto** node after each **LensDistortion**, before each **Card3D**. Only leave connected the A and B inputs to the **Merge** node to the two first **Card3D** nodes. Check *Premultiply* in each Roto node, check that only the A channel is affected by **Roto** in its parameters, and start editing the roto mask on the second image, for which the **LensDistortion** output is connected to the A input of the **Merge**, while viewing the output of the **Merge**. Set the compositing operator of each roto shape to "min" instead of "over", so that the original image alpha gets masked by the roto shape. Do not forget to add feather, especially in the overlap area. Then, reconnect the A2 input to the third **Card3D**, and start editing its roto mask, always in "min" compositing mode. Do the same with A3, A4, etc... and you should end up with a full panorama. Future work ----------- Camera response ~~~~~~~~~~~~~~~ See `Camera response curve `__ and `Vig\_optimize `__. The values ``Ra`` ``Rb`` ``Rc`` ``Rd`` ``Re`` on the ``i`` line encode a color response curve (EMoR). See ``EMoRParams``\ in the hugin source code. Exposure and color ~~~~~~~~~~~~~~~~~~ See `Vignetting `__ and `Vig\_optimize `__. ``Eev`` encodes the exposure, see ``ExposureValue`` in the hugin source code. ``Er`` and ``Eb`` encode the red and blue multipliers, see ``WhiteBalanceRed`` and ``WhiteBalanceBlue`` in the hugin source code. Vignetting ~~~~~~~~~~ See `Vignetting `__ and `Vig\_optimize `__. ``Va`` is always 1, see ``VigCorrMode`` in the hugin source code. ``Vb`` ``Vc`` ``Vd`` encode the degree 2, 4 and 6 coefficients for vignette correction, see ``RadialVigCorrCoeff`` in the hugin source code. ``Vx`` and ``Vy`` encode the vignetting center shift, see ``RadialVigCorrCenterShift`` in the hugin source code. Blending ~~~~~~~~ - importing enblend masks, using them as roto masks - executing enblend externally (using RunScript maybe?)