DenoiseSharpen node

This documentation is for version 1.0 of DenoiseSharpen (net.sf.openfx.DenoiseSharpen).

Description

Denoise and/or sharpen images using wavelet-based algorithms.

Description

This plugin allows the separate denoising of image channels in multiple color spaces using wavelets, using the BayesShrink algorithm, and can also sharpen the image details.

Noise levels for each channel may be either set manually, or analyzed from the image data in each wavelet subband using the MAD (median absolute deviation) estimator. Noise analysis is based on the assumption that the noise is Gaussian and additive (it is not intensity-dependent). If there is speckle or salt-and-pepper noise in the images, the Median or SmoothPatchBased filters may be more appropriate. The color model specifies the channels and the transforms used. Noise levels have to be re-adjusted or re-analyzed when changing the color model.

Basic Usage

The input image should be in linear RGB.

For most footage, the effect works best by keeping the default Y’CbCr color model. The color models are made to work with Rec.709 data, but DenoiseSharpen will still work if the input is in another colorspace, as long as the input is linear RGB:

  • The Y’CbCr color model uses the Rec.709 opto-electronic transfer function to convert from RGB to R’G’B’ and the the Rec.709 primaries to convert from R’G’B’ to Y’CbCr.

  • The L * a * b color model uses the Rec.709 RGB primaries to convert from RGB to L * a * b.

  • The R’G’B’ color model uses the Rec.709 opto-electronic transfer function to convert from RGB to R’G’B’.

  • The RGB color model (linear) makes no assumption about the RGB color space, and works directly on the RGB components, assuming additive noise. This is the only option if the noisy source contains negative values. If, say, the noise is known to be multiplicative, one can convert the images to Log before denoising, use this option, and convert back to linear after denoising.

  • The Alpha channel, if processed, is always considered to be linear.

The simplest way to use this plugin is to leave the noise analysis area to the whole image, and click “Analyze Noise Levels”. Once the analysis is done, “Lock Noise Analysis” is checked in order to avoid modifying the essential parameters by mistake.

If the image has many textured areas, it may be preferable to select an analysis area with flat colors, free from any details, shadows or highlights, to avoid considering texture as noise. The AnalysisMask input can be used to mask the analysis, if the rectangular area is not appropriate. Any non-zero pixels in the mask are taken into account. A good option for the AnalysisMask would be to take the inverse of the output of an edge detector and clamp it correctly so that all pixels near the edges have a value of zero..

If the sequence to be denoised does not have enough flat areas, you can also connect a reference footage with the same kind of noise to the AnalysisSource input: that source will be used for the analysis only. If no source with flat areas is available, and noise analysis can only be performed on areas which also contain details, it is often preferable to disable very low, low, and sometimes medium frequencies in the “Frequency Tuning” parameters group, or at least to lower their gain, since they may be misestimated by the noise analysis process. If the noise is IID (independent and identically distributed), such as digital sensor noise, only “Denoise High Frequencies” should be checked. If the noise has some grain (i.e. it commes from lossy compression of noisy images by a camera, or it is scanned film), then you may want to enable medium frequencies as well. If low and very low frequencies are enabled, but the analysis area is not a flat zone, the signal itself (i.e. the noise-free image) could be considered as noise, and the result may exhibit low contrast and blur.

To check what details have been kept after denoising, you can raise the Sharpen Amount to something like 10, and then adjust the Noise Level Gain to get the desired denoising amount, until no noise is left and only image details remain in the sharpened image. You can then reset the Sharpen Amount to zero, unless you actually want to enhance the contrast of your denoised footage.

You can also check what was actually removed from the original image by selecting the “Noise” Output mode (instead of “Result”). If too many image details are visible in the noise, noise parameters may need to be tuned.

This plugin was compiled with OpenMP support.

Inputs

Input

Description

Optional

Source

The footage to be denoised. If nothing is connected to the AnalysisSource input, this is also used for noise analysis.

No

Mask

An optional image to use as a mask. By default, the effect is limited to the non-black areas of the mask.

Yes

AnalysisSource

An optional noise source. If connected, this is used instead of the Source input for the noise analysis. This is used to analyse noise from some footage by apply it on another footage, in case the footage to be denoised does not have enough flat areas.

Yes

AnalysisMask

An optional mask for the analysis area. This mask is intersected with the Analysis Rectangle. Non-zero pixels are taken into account in the noise analysis phase.

Yes

Controls

Parameter / script name

Type

Default

Function

Output / outputMode

Choice

Result

Select which image is output when analysis is locked. When analysis is not locked, the effect does nothing (the output is the source image).
Result (result): The result of denoising and sharpening the Source image.
Noise (noise): An image containing what would be added to the image to denoise it. If ‘Denoise Amount’ is zero, this image should be black. Only noise should be visible in this image. If you can see a lot of picture detail in the noise output, it means the current settings are denoising too hard and remove too much of the image, which leads to a smoothed result. Try to lower the noise levels or the noise level gain.
Sharpen (sharpen): An image containing what would be added to the image to sharpen it. If ‘Sharpen Amount’ is zero, this image should be black. Only image details should be visible in this image. If you can see a lot of noise in the sharpen output, it means the current settings are denoising not enough, which leads to a noisy result. Try to raise the noise levels or the noise level gain.

Color Model / colorModel

Choice

Y’CbCr(A)

The colorspace where denoising is performed. These colorspaces assume that input and output use the Rec.709/sRGB chromaticities and the D65 illuminant, but should tolerate other input colorspaces (the output colorspace will always be the same as the input colorspace). Noise levels are reset when the color model is changed.
Y’CbCr(A) (ycbcr): The YCbCr color model has one luminance channel (Y) which contains most of the detail information of an image (such as brightness and contrast) and two chroma channels (Cb = blueness, Cr = reddness) that hold the color information. Note that this choice drastically affects the result. Uses the Rec.709 opto-electronic transfer function to convert from RGB to R’G’B’ and the the Rec.709 primaries to convert from R’G’B’ to Y’CbCr.
CIE L*a*b(A) (cielab): CIE L*a*b* is a color model in which chrominance is separated from lightness and color distances are perceptually uniform. Note that this choice drastically affects the result. Uses the Rec.709 primaries to convert from RGB to L*a*b.
R’G’B’(A) (gammargb): The R’G’B’ color model (gamma-corrected RGB) separates an image into channels of red, green, and blue. Note that this choice drastically affects the result. Uses the Rec.709 opto-electronic transfer function to convert from RGB to R’G’B’.
RGB(A) (linearrgb): The Linear RGB color model processes the raw linear components. Usually a bad choice, except when denoising non-color data (e.g. depth or motion vectors). No assumption is made about the RGB color space.

Lock Analysis and Apply / analysisLock

Boolean

Off

Lock all noise analysis parameters and apply denoising. When the analysis is not locked, the source image is output.

Bottom Left / bottomLeft

Double

x: 0.1 y: 0.1

Coordinates of the bottom left corner of the analysis rectangle. This rectangle is intersected with the AnalysisMask input, if connected.

Size / size

Double

w: 0.8 h: 0.8

Width and height of the analysis rectangle. This rectangle is intersected with the AnalysisMask input, if connected.

HiDPI / hidpi

Boolean

Off

Should be checked when the display area is High-DPI (a.k.a Retina). Draws OpenGL overlays twice larger.

B3 Spline Interpolation / useB3Spline

Boolean

On

For wavelet decomposition, use a 5x5 filter based on B3 spline interpolation rather than a 3x3 Lagrange linear filter. Noise levels are reset when this setting is changed. The influence of this parameter is minimal, and it should not be changed.

Analysis Frame / analysisFrame

Integer

-1

The frame number where the noise levels were analyzed.

Analyze Noise Levels / analyzeNoiseLevels

Button

Computes the noise levels from the current frame and current color model. To use the same settings for the whole sequence, analyze a frame that is representative of the sequence. If a mask is set, it is used to compute the noise levels from areas where the mask is non-zero. If there are keyframes on the noise level parameters, this sets a keyframe at the current frame. The noise levels can then be fine-tuned.

Y Level (High) / ylrNoiseLevelHigh

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cb Level (High) / cbagNoiseLevelHigh

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cr Level (High) / crbbNoiseLevelHigh

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Alpha Level (High) / alphaNoiseLevelHigh

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Y Level (Medium) / ylrNoiseLevelMedium

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cb Level (Medium) / cbagNoiseLevelMedium

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cr Level (Medium) / crbbNoiseLevelMedium

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Alpha Level (Medium) / alphaNoiseLevelMedium

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Y Level (Low) / ylrNoiseLevelLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cb Level (Low) / cbagNoiseLevelLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cr Level (Low) / crbbNoiseLevelLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Alpha Level (Low) / alphaNoiseLevelLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Y Level (Very Low) / ylrNoiseLevelVeryLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cb Level (Very Low) / cbagNoiseLevelVeryLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Cr Level (Very Low) / crbbNoiseLevelVeryLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Alpha Level (Very Low) / alphaNoiseLevelVeryLow

Double

0

Adjusts the noise variance of the selected channel for the given noise frequency. May be estimated from image data by pressing the “Analyze Noise” button.

Noise Level Gain / noiseLevelGain

Double

1

Global gain to apply to the noise level thresholds. 0 means no denoising, 1 means use the estimated thresholds multiplied by the per-frequency gain and the channel gain. The default value (1.0) is rather conservative (it does not destroy any kind of signal). Values around 1.1 or 1.2 usually give more pleasing results.

Denoise Amount / denoiseAmount

Double

1

The amount of denoising to apply. 0 means no denoising (which may be useful to sharpen without denoising), between 0 and 1 does a soft thresholding of below the thresholds, thus keeping some noise, and 1 applies the threshold strictly and removes everything below the thresholds. This should be used only if you want to keep some noise, for example for noise matching. This value is multiplied by the per-channel amount se in the ‘Channel Tuning’ group. Remember that the thresholds are multiplied by the per-frequency gain, the channel gain, and the Noise Level Gain first.

Denoise High Frequencies / enableFreqHigh

Boolean

On

Check to enable the high frequency noise level thresholds. It is recommended to always leave this checked.

High Gain / gainFreqHigh

Double

1

Gain to apply to the high frequency noise level thresholds. 0 means no denoising, 1 means use the estimated thresholds multiplied by the channel Gain and the Noise Level Gain.

Denoise Medium Frequencies / enableFreqMedium

Boolean

On

Check to enable the medium frequency noise level thresholds. Can be disabled if the analysis area contains high frequency texture, or if the the noise is known to be IID (independent and identically distributed), for example if this is only sensor noise and lossless compression is used, and not grain or compression noise.

Medium Gain / gainFreqMedium

Double

1

Gain to apply to the medium frequency noise level thresholds. 0 means no denoising, 1 means use the estimated thresholds multiplied by the channel Gain and the Noise Level Gain.

Denoise Low Frequencies / enableFreqLow

Boolean

On

Check to enable the low frequency noise level thresholds. Must be disabled if the analysis area contains texture, or if the noise is known to be IID (independent and identically distributed), for example if this is only sensor noise and lossless compression is used, and not grain or compression noise.

Low Gain / gainFreqLow

Double

1

Gain to apply to the low frequency noise level thresholds. 0 means no denoising, 1 means use the estimated thresholds multiplied by the channel Gain and the Noise Level Gain.

Denoise Very Low Frequencies / enableFreqVeryLow

Boolean

On

Check to enable the very low frequency noise level thresholds. Can be disabled in most cases. Must be disabled if the analysis area contains texture, or if the noise is known to be IID (independent and identically distributed), for example if this is only sensor noise and lossless compression is used, and not grain or compression noise.

Very Low Gain / gainFreqVeryLow

Double

1

Gain to apply to the very low frequency noise level thresholds. 0 means no denoising, 1 means use the estimated thresholds multiplied by the channel Gain and the global Noise Level Gain.

Adaptive Radius / adaptiveRadius

Integer

4

Radius of the window where the signal level is analyzed at each scale. If zero, the signal level is computed from the whole image, which may excessively blur the edges if the image has many flat color areas. A reasonable value should to be in the range 2-4.

Y Gain / ylrGain

Double

1

Gain to apply to the thresholds for this channel. 0 means no denoising, 1 means use the estimated thresholds multiplied by the per-frequency gain and the global Noise Level Gain.

Y Amount / ylrAmount

Double

1

The amount of denoising to apply to the specified channel. 0 means no denoising, between 0 and 1 does a soft thresholding of below the thresholds, thus keeping some noise, and 1 applies the threshold strictly and removes everything below the thresholds. This should be used only if you want to keep some noise, for example for noise matching. This value is multiplied by the global Denoise Amount. Remember that the thresholds are multiplied by the per-frequency gain, the channel gain, and the Noise Level Gain first.

Cb Gain / cbagGain

Double

1

Gain to apply to the thresholds for this channel. 0 means no denoising, 1 means use the estimated thresholds multiplied by the per-frequency gain and the global Noise Level Gain.

Cb Amount / cbagAmount

Double

1

The amount of denoising to apply to the specified channel. 0 means no denoising, between 0 and 1 does a soft thresholding of below the thresholds, thus keeping some noise, and 1 applies the threshold strictly and removes everything below the thresholds. This should be used only if you want to keep some noise, for example for noise matching. This value is multiplied by the global Denoise Amount. Remember that the thresholds are multiplied by the per-frequency gain, the channel gain, and the Noise Level Gain first.

Cr Gain / crbbGain

Double

1

Gain to apply to the thresholds for this channel. 0 means no denoising, 1 means use the estimated thresholds multiplied by the per-frequency gain and the global Noise Level Gain.

Cr Amount / crbbAmount

Double

1

The amount of denoising to apply to the specified channel. 0 means no denoising, between 0 and 1 does a soft thresholding of below the thresholds, thus keeping some noise, and 1 applies the threshold strictly and removes everything below the thresholds. This should be used only if you want to keep some noise, for example for noise matching. This value is multiplied by the global Denoise Amount. Remember that the thresholds are multiplied by the per-frequency gain, the channel gain, and the Noise Level Gain first.

Alpha Gain / alphaGain

Double

1

Gain to apply to the thresholds for this channel. 0 means no denoising, 1 means use the estimated thresholds multiplied by the per-frequency gain and the global Noise Level Gain.

Alpha Amount / alphaAmount

Double

1

The amount of denoising to apply to the specified channel. 0 means no denoising, between 0 and 1 does a soft thresholding of below the thresholds, thus keeping some noise, and 1 applies the threshold strictly and removes everything below the thresholds. This should be used only if you want to keep some noise, for example for noise matching. This value is multiplied by the global Denoise Amount. Remember that the thresholds are multiplied by the per-frequency gain, the channel gain, and the Noise Level Gain first.

Sharpen Amount / sharpenAmount

Double

0

Adjusts the amount of sharpening applied. Be careful that only components that are above the noise levels are enhanced, so the noise level gain parameters are very important for proper sharpening. For example, if ‘Noise Level Gain’ is set to zero (0), then noise is sharpened as well as signal. If the ‘Noise Level Gain’ is set to one (1), only signal is sharpened. In order to sharpen without denoising, set the ‘Denoise Amount’ parameter to zero (0).

Sharpen Size / sharpenSize

Double

10

Adjusts the size of the sharpening. For very unsharp images it is recommended to use higher values. Default is 10.

Sharpen Y Only / sharpenLuminance

Boolean

On

Sharpens luminance only (if colormodel is R’G’B’, sharpen only RGB). This avoids color artifacts to appear. Colour sharpness in natural images is not critical for the human eye.

(Un)premult / premult

Boolean

Off

Divide the image by the alpha channel before processing, and re-multiply it afterwards. Use if the input images are premultiplied.

Invert Mask / maskInvert

Boolean

Off

When checked, the effect is fully applied where the mask is 0.

Mix / mix

Double

1

Mix factor between the original and the transformed image.